A biological process multi-missing mechanism data filling system based on incomplete information game

Abstract

The application discloses a kind of based on incomplete information game's biological process multiple deletion mechanism data filling system, the system includes: original data preprocessing module, data missing mechanism identification module, game model construction module, data filling algorithm module, evaluation and verification module and display and man-machine interaction module.The application realizes the identification, filling and verification of multiple missing mechanisms in structured data, can realize the high-confidence data filling of multi-field structured data, and has the advantages of strong feature extraction capability, fast identification speed, high filling confidence, wide application range, high intelligent degree, etc., solves the shortcomings of traditional data filling system, such as manual judgment of missing mechanism, low identification accuracy, slow speed, algorithm only suitable for single missing mechanism.

Description

Technical Field

[0001] This invention relates to the field of missing value imputation in structured data, and more particularly to a data imputation system for multiple missing mechanisms in biological processes based on incomplete information game theory. Background Technology

[0002] In real-world applications across various fields, data loss frequently occurs due to various reasons, such as limitations in sensor hardware, privacy concerns, and human error in original data collection. Therefore, data filling tasks are crucial for downstream data-using businesses.

[0003] Research on missing data has existed since the beginning of the field. Rubin first proposed a basic reasoning framework for missing data and three missing mechanisms: Completely Random Missing (MCAR), Random Missing (MAR), and Non-Random Missing (MNAR). Building on this, many scholars have further explored the problem of missing data, improving the accuracy and efficiency of data completion. The simplest way to handle missing data is to discard samples with missing values. However, this can introduce bias or affect the representativeness of the results, and even make the data more incomplete. Therefore, a series of imputation methods have been proposed, including simple statistical imputation methods, traditional machine learning imputation methods, and modern deep learning imputation algorithms. One approach is to replace missing components with the most similar methods from the training data, such as Mean imputation, hot deck imputation (HDI), cold deck imputation (CDI), and Knearest neighbor imputation (KNNI). The second type of imputation method involves building predictive models in machine learning to predict missing values, such as XGBoost imputation (XGBI), MissForest imputation (MissFI), Chain Equation Multiple Imputation (MICE), Individual Model Imputation (IIM), Soft Imputation (SI), Matrix Factorization Imputation (MFI), Principal Component Analysis Imputation (PCAI), Multilayer Perceptron Imputation (MLPI), and Recurrent Sinkhorn Imputation (RRSI). In contrast, modern deep learning algorithms are inspired by the powerful capabilities of deep generative models. They either enrich the prior or posterior of explicit generative models, such as deep autoencoders (AEs), or utilize log-likelihood as an alternative training objective to generate implicit generative models, such as generative adversarial networks (GANs). Early statistical inference methods used statistics (e.g., mean, median, or pattern values), and although many imputation algorithms have been proposed for handling missing data, applying them to specific real-world scenarios remains challenging.

[0004] Compared to traditional generative schemes, the proposed scheme in this invention, through the introduction of an adaptive reconstruction function and a Hessian matrix iteration scheme, is suitable for the data generation needs of multimodal biomass energy manufacturing industry. While reducing the risk of mode collapse that is easy to cause in specific engineering practice, it effectively improves the diversity of generated data and significantly improves the stability, robustness and sensitivity of data generation in the bio-intelligent manufacturing process.

[0005] Building upon previous work, this invention focuses on generative algorithms and, based on practical application scenarios, designs a data imputation system for biological processes with multiple missing mechanisms, grounded in incomplete information game theory. Traditional imputation methods suffer from drawbacks such as requiring manual identification of missing mechanisms, low accuracy, slow speed, and applicability only to single missing mechanisms. Their effectiveness in practical applications is limited. Therefore, there is an urgent need to establish a data imputation system capable of autonomously identifying missing mechanisms, possessing high imputation accuracy and confidence, and applicable to various real-world scenarios. Summary of the Invention

[0006] The purpose of this invention is to address the shortcomings of existing technologies by providing a data imputation system for biological processes with multiple missing mechanisms based on incomplete information game theory.

[0007] The objective of this invention is achieved through the following technical solution: a data imputation system for multiple missing mechanisms in biological processes based on incomplete information game theory, the system comprising: a raw data preprocessing module, a data missing mechanism identification module, a game model construction module, a data imputation algorithm module, an evaluation and verification module, and a display and human-computer interaction module;

[0008] The raw data preprocessing module is responsible for receiving and processing the raw industrial dataset from the biomanufacturing process, and then preprocessing the raw industrial dataset to obtain the preprocessed dataset.

[0009] The data missing mechanism identification module is used to identify data based on the preprocessed dataset. A Bayesian inference clustering model is used to identify missing data mechanisms, resulting in a missing mechanism matrix {X}. miss This provides a basis for constructing targeted game theory models;

[0010] The game model construction module is used to construct the missing mechanism matrix {X}. miss We will construct a data imputation game model to guide the selection of the optimal strategy for data imputation, and model the game participants, strategy space and payoff function.

[0011] The data imputation algorithm module is used to intelligently select or combine imputation methods based on the guidance of the game model construction module, taking into account biological principles and the temporal characteristics of the data, so as to achieve biological rationality and data continuity of the imputation results.

[0012] The evaluation and verification module is used to verify the effectiveness of the filling strategy by comparing the statistical characteristics of the data before and after filling, the model prediction performance, and the consistency of biological indicators. This module should also provide visualization tools to help users intuitively understand the filling effect.

[0013] The display and human-computer interaction module provides data overview, visualization of missing mechanisms, comparison before and after filling, and parameter configuration and filling process management functions. It allows users to upload data, configure filling parameters, monitor the filling process, view filling reports and evaluation results, and retains a flexible customization interface to meet the professional needs and preferences of different users.

[0014] Furthermore, the preprocessing of the original industrial dataset to obtain the preprocessed dataset specifically involves:

[0015] First, we examine the original industrial dataset {X} = {x1,…,x...}. i ,…,x N Data cleaning was performed using the Z-score method.

[0016] z i =(x i -μ) / σ (1);

[0017] Where, x i Let represent any data point in the original industrial dataset {X}; μ represents the mean of all data points in the original industrial dataset {X}; σ represents the standard deviation of all data points in the original industrial dataset {X}.

[0018] Define the normal range as [-3, +3], and then define z values ​​that do not belong to the range [-3, +3]. i Corresponding x i It is considered an outlier;

[0019] Subsequently, programming logic was used to unify the date format and units of all data in the original industrial dataset {X}, resulting in a formatted original industrial dataset.

[0020] Then, the original industrial dataset after format standardization was processed. Each data point is scaled to the [0,1] interval to obtain the standardized industrial dataset {X′}:

[0021]

[0022] in, The original industrial dataset after standardization of format Any data in the dataset; The original industrial dataset after standardization of format The maximum value in the middle; The original industrial dataset after standardization of format Minimum value;

[0023] Finally, through logic programming, all missing values, outliers, and non-numeric items in the standardized industrial dataset {X′} are labeled as NaN, resulting in the preprocessed dataset.

[0024] Furthermore, the data missing mechanism identification module is based on the preprocessed dataset. A Bayesian inference clustering model is used to identify missing data mechanisms, resulting in a missing mechanism matrix {X}. miss This provides a basis for constructing targeted game theory models, specifically:

[0025] First, we introduce the missing mechanism type parameter. Where k∈{1,2,3} corresponds to completely random missing mechanism, random missing mechanism, and non-random missing mechanism, respectively; assuming the missing mechanism type This is implicitly reflected through clustering attributes, let's assume... Representing data The index variable belonging to the k-th cluster; and the prior probability assigned to the k-th cluster. initial mean and the initial covariance matrix

[0026] Then iteration is performed, and in the t-th iteration, the preprocessed dataset is calculated. Each data Formula for calculating the expected posterior probability of belonging to each cluster k:

[0027]

[0028] in, This represents the cluster weight of the k-th cluster after the (t-1)-th iteration; Let represent the mean of the k-th cluster after the (t-1)-th iteration; This represents the covariance matrix of the k-th cluster after the (t-1)-th iteration. This represents the data after the (t-1)th iteration. The indicator variable belonging to the k-th cluster; Representing data The probability density function of a Gaussian distribution; Represents given data The posterior probability of the missing mechanism type when belonging to the kth cluster;

[0029] Then update the cluster weights, mean, and covariance matrix:

[0030]

[0031] After the update is complete, check if the following conditions are met:

[0032] If satisfied If the condition is met or the set number of iterations is reached, the iteration ends, and the expected value is determined based on the posterior probability. Output each data Corresponding missing mechanism y i The missing mechanism matrix {X} is obtained. miss};in, Indicates the given information in the t-th iteration. back The posterior probability; express Approximate function; express The prior probability, ∝ represents a direct proportional relationship;

[0033] If the conditions are not met, proceed to the next iteration;

[0034] This module constructs a Bayesian inference clustering missing mechanism identification model to make high-confidence inferences about the actual situation of the data. The model combines posterior probability inferences, which significantly improves the identification accuracy and makes subsequent data imputation strategies more targeted.

[0035] Furthermore, the game model construction module is based on the missing mechanism matrix {X} miss A data imputation game model is constructed to guide the selection of optimal strategies for data imputation. This model models the game participants, strategy space, and payoff function. Specifically, the game model construction module constructs the model based on the missing mechanism matrix {X}. miss A data imputation game model is constructed to guide the selection of optimal strategies for data imputation. The model models the game participants, strategy space, and payoff function. There are two main participants: the imputer and environmental variables or data characteristics. These data characteristics are quantified by the likelihood function, expected posterior probability, cluster weights, mean, and covariance provided by the data missing mechanism identification module. Assuming the imputer's strategy space has n imputation strategies, the model can be:

[0036]

[0037] in, This represents any imputation strategy, which can be a mean imputation strategy, a KNN imputation strategy, a model prediction imputation strategy, or an environment strategy.

[0038] The payoff function for the filler is U F (S F ,θ k The effectiveness of imputation strategies is evaluated by a comprehensive score, including the statistical properties of the imputed data, model predictive performance, and biological plausibility indicators. The payoff function can be formalized as follows:

[0039] U F (S F ,θ k )=w1f stat (X filled )+w2f model (X filled )+w3f bio (X filled (8);

[0040] Where Xfilled represents filling in the data; f stat (·) represents the statistical evaluation function, and w1 represents the weight of the statistical evaluation; f model (·) represents the function for evaluating model performance, and w2 represents the weights for evaluating model performance; f bio (·) represents the function for evaluating biological rationality, and w1 represents the weight of the evaluation of biological rationality;

[0041] This module constructs a new weighting system and comprehensive evaluation system based on game theory, optimizes the decision-making optimization space, effectively balances the adaptability and reliability of the strategy in the decision-making process, and provides an objective function space for the subsequent application of the filling algorithm.

[0042] Furthermore, the data imputation algorithm module is designed with three different working modes for the three missing data mechanisms: Mode 1 is the MCAR working mode, which uses a generative algorithm based on statistics, machine learning, and deep learning to impute missing data; Mode 2 is the MAR working mode, which uses a generative algorithm based on machine learning to impute missing data; and Mode 3 is the MNAR working mode, which uses a generative algorithm based on decision trees and the XGBOOST model to impute missing values ​​non-parametrically. The switching between Mode 1, Mode 2, and Mode 3 is done through conditional judgment.

[0043] The generative algorithms improved based on statistics, machine learning, and deep learning, as well as those improved based on machine learning and decision trees and the XGBoost model, are unified in an iterative network. The basic theoretical framework of this iterative network involves training two deep learning network models: a data generator G and a data discriminator D. The data generator G corresponds to the filler in the data imputation game model, and the data discriminator D corresponds to the environment variable in the data imputation game model. The algorithm used in the iterative network is any strategy in the data imputation game model. Furthermore, improvements are made to these two deep learning network models based on different machine learning theoretical methods. The objective functions of the two deep learning network models are as follows:

[0044]

[0045] Where V is the objective function; G(l) is the output of the generator; G is the data generator, which receives a random noise signal l, which is sampled from a Gaussian distribution, and attempts to generate samples similar to the real data distribution. The data generated by this information is denoted as G(l); D is the data discriminator, which receives any sample from a standardized industrial dataset {X′}. And output the sample The probability of being true This represents the expected value function that the generated data samples satisfy the distribution of the original data samples; V(G,D) represents the expected value function satisfied by the data sample; V(G,D) represents the value function of an adversarial game between the data discriminator and the data generator.

[0046] This module integrates multiple types of data imputation algorithms and uses conditional judgment to automatically select the most suitable imputation working mode, ensuring the flexibility and efficiency of the processing, reducing the need for manual intervention, making data preprocessing more automated, and reserving an access interface for strategy space update and iteration, providing a standardized process for the integration of new algorithms.

[0047] Furthermore, the evaluation and verification module verifies the effectiveness of the filling strategy by comparing the statistical characteristics of the data before and after filling, the model's predictive performance, and the consistency with biological indicators. This module should also provide visualization tools to help users intuitively understand the filling effect, specifically:

[0048] The evaluation and verification module uses statistical measures such as mean, variance, skewness, or kurtosis to compare the statistical characteristics of the data before and after imputation to verify the effectiveness of the imputation strategy: by comparing the overall characteristics of the dataset before and after imputation, if the imputed dataset is reasonably close to the complete dataset in these statistical measures, then the imputation strategy is considered to be effective in maintaining the original distribution characteristics of the data.

[0049] The effectiveness of the imputation strategy is verified by comparing the prediction performance of the model before and after imputation, where the prediction model is a mean squared error or R-squared error. 2 Fraction;

[0050] The effectiveness of the imputation strategy is verified by calculating the degree of agreement between the imputation data and known biological facts or theoretical expectations.

[0051] Furthermore, the display and human-computer interaction module provides data overview, visualization of missing mechanisms, before-and-after comparison of filling, and parameter configuration and filling process management functions. It allows users to upload data, configure filling parameters, monitor the filling process, view filling reports and evaluation results, and retains a flexible customization interface to meet the professional needs and preferences of different users. Specifically:

[0052] The display and human-computer interaction module provides a data overview function, which allows users to quickly view the basic statistical information, missing value distribution and data structure of the data after uploading it. The data overview is displayed intuitively in the form of charts and summary tables, which helps users to have a preliminary understanding of the data quality and missing value situation.

[0053] The display and human-computer interaction module provides a missing mechanism visualization function. The missing mechanism visualization function is based on the results of the data missing mechanism identification module. The interface provides a graphical display of the proportion of different missing types and their distribution in the dataset, so as to help users intuitively understand the data missing pattern and provide a basis for selecting imputation strategies.

[0054] The display and human-computer interaction module provides a before-and-after comparison function, which provides a series of visualization tools: box plots, density plots or time series plots, so that users can intuitively compare the distribution, statistical characteristics and patterns of data before and after filling, and confirm the effectiveness of the filling strategy. The visualization tools also include difference plots, which highlight the specific changes caused by filling, so that users can evaluate the potential impact of the filling effect on the analysis results.

[0055] The display and human-computer interaction module provides parameter configuration and filling process management functions. These functions allow users to customize filling parameters, select filling methods, adjust model hyperparameters, and set missing mechanism assumptions through the interface. At the same time, it provides a process monitoring tool to track the status, progress, and resource consumption of the filling task in real time, ensuring that users can flexibly control and adjust the filling process.

[0056] The beneficial effects of this invention are:

[0057] 1) It has high confidence in identifying various incomplete data missing situations and missing mechanisms; it innovatively constructs a Bayesian inference clustering missing mechanism identification model, which can make reasonable inferences about data missing situations, has strong interpretability, high identification accuracy, and makes subsequent data imputation strategies more targeted.

[0058] 2) It possesses high robustness based on incomplete information data filling; a new weighting system and comprehensive evaluation system are constructed based on game theory, which optimizes the decision-making optimization space and effectively balances the adaptability and reliability of the strategy in the decision-making process.

[0059] 3) It has high agility in strategy switching and system iteration updates; it integrates multiple types of data imputation algorithms and uses conditional judgment to automatically select the most suitable imputation working mode, ensuring the flexibility and efficiency of the processing, reducing the need for manual intervention, making data preprocessing more automated, and reserving access interfaces for strategy space updates and iterations, providing a standardized process for the access of new algorithms. Attached Figure Description

[0060] Figure 1 A result diagram for a data imputation system of multiple missing mechanisms in biological processes based on incomplete information game theory;

[0061] Figure 2 A functional module diagram for filling in data on multiple missing mechanisms of biological processes based on incomplete information game theory;

[0062] In the diagram, 1-raw data preprocessing module; 2-data missing mechanism identification module; 3-game model construction module; 4-data imputation algorithm module; 5-evaluation and verification module; 6-display and human-computer interaction module. Detailed Implementation

[0063] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are within the scope of protection of this invention.

[0064] Example 1

[0065] like Figure 1 and Figure 2 As shown, the present invention provides a data imputation system for multiple missing mechanisms in biological processes based on incomplete information game theory. The system includes: a raw data preprocessing module 1, a data missing mechanism identification module 2, a game model construction module 3, a data imputation algorithm module 4, an evaluation and verification module 5, and a display and human-computer interaction module 6.

[0066] The raw data preprocessing module 1 is responsible for receiving and processing the raw industrial dataset from the biomanufacturing process, and then preprocessing the raw industrial dataset to obtain the preprocessed dataset.

[0067] The preprocessing of the original industrial dataset to obtain the preprocessed dataset is specifically as follows:

[0068] First, we examine the original industrial dataset {X} = {x1,…,x...}. i ,…,x N Data cleaning was performed using the Z-score method.

[0069] z i =(x i -μ) / σ (1);

[0070] Where, x i Let represent any data point in the original industrial dataset {X}; μ represents the mean of all data points in the original industrial dataset {X}; σ represents the standard deviation of all data points in the original industrial dataset {X}.

[0071] Define the normal range as [-3, +3], and then define z values ​​that do not belong to the range [-3, +3]. i Corresponding x i It is considered an outlier;

[0072] Subsequently, programming logic was used to unify the date format and units of all data in the original industrial dataset {X}, resulting in a formatted original industrial dataset.

[0073] Then, the original industrial dataset after format standardization was processed. Each data point is scaled to the [0,1] interval to obtain the standardized industrial dataset {X′}:

[0074]

[0075] in, The original industrial dataset after standardization of format Any data in the dataset; The original industrial dataset after standardization of format The maximum value in the middle; The original industrial dataset after standardization of format Minimum value;

[0076] Finally, through logic programming, all missing values, outliers, and non-numeric items in the standardized industrial dataset {X′} are labeled as NaN, resulting in the preprocessed dataset.

[0077] The data missing mechanism identification module 2 is used to identify data based on the preprocessed dataset. A Bayesian inference clustering model is used to identify missing data mechanisms, resulting in a missing mechanism matrix {X}. miss This provides a basis for constructing targeted game theory models, specifically:

[0078] First, we introduce the missing mechanism type parameter. Where k∈{1,2,3} corresponds to the completely random missing mechanism (MCAR), the random missing mechanism (MAR), and the non-random missing mechanism (MNAR), respectively; assuming the missing mechanism type This is implicitly reflected through clustering attributes, let's assume... Representing data The index variable belonging to the k-th cluster; and the prior probability assigned to the k-th cluster. initial mean and the initial covariance matrix

[0079] Then iteration is performed, and in the t-th iteration, the preprocessed dataset is calculated. Each data Formula for calculating the expected posterior probability of belonging to each cluster k:

[0080]

[0081] in, This represents the cluster weight of the k-th cluster after the (t-1)-th iteration; Let represent the mean of the k-th cluster after the (t-1)-th iteration; This represents the covariance matrix of the k-th cluster after the (t-1)-th iteration. This represents the data after the (t-1)th iteration. The indicator variable belonging to the k-th cluster; Representing data The probability density function of a Gaussian distribution; Represents given data The posterior probability of the missing mechanism type when belonging to the kth cluster;

[0082] Then update the cluster weights, mean, and covariance matrix:

[0083]

[0084] After the update is complete, check if the following conditions are met:

[0085] If satisfied If the condition is met or the set number of iterations is reached, the iteration ends, and the expected value is determined based on the posterior probability. Output each data Corresponding missing mechanism y i The missing mechanism matrix {X} is obtained. miss};in, Indicates the given information in the t-th iteration. back The posterior probability; express Approximate function; express The prior probability, cc, indicates a proportional relationship;

[0086] If the conditions are not met, the next iteration will proceed. This module constructs a Bayesian inference clustering missing data mechanism identification model to make high-confidence inferences about the actual situation of the data. The model, combined with posterior probability inferences, significantly improves the identification accuracy and makes subsequent data imputation strategies more targeted.

[0087] The game model construction module 3 is used to construct the missing mechanism matrix {X}. miss To construct a data imputation game model to guide the selection of optimal strategies for data imputation, we will model the game participants, strategy space, and payoff function, specifically as follows:

[0088] The game model construction module is based on the missing mechanism matrix {X} miss A data imputation game model is constructed to guide the selection of optimal strategies for data imputation. The model models the game participants, strategy space, and payoff function. There are two main participants: the imputer and environmental variables or data characteristics. These data characteristics are quantified by the likelihood function, expected posterior probability, cluster weights, mean, and covariance provided by the data missing mechanism identification module. Assuming the imputer's strategy space has n imputation strategies, the model can be:

[0089]

[0090] in, This represents any imputation strategy, which can be a mean imputation strategy, a KNN imputation strategy, a model prediction imputation strategy, or an environment strategy.

[0091] The payoff function for the filler is U F (S F ,θ kThe effectiveness of imputation strategies is evaluated by a comprehensive score that includes the statistical properties of the imputed data (such as the degree of restoration of the mean and variance), model predictive performance (such as prediction accuracy, AIC, BIC, etc.), and biological rationality (such as whether it conforms to known biological pathways and mechanisms). The payoff function can be formalized as follows:

[0092] U F (S F ,θ k )=w1f stat (X filled )+w2f model (X filled )+w3f bio (X filled (8);

[0093] Where Xfilled represents filling in the data; f stat (·) represents the statistical evaluation function, and w1 represents the weight of the statistical evaluation; f model (·) represents the function for evaluating model performance, and w2 represents the weights for evaluating model performance; f bio (·) represents the function for evaluating biological rationality, and w1 represents the weight of the evaluation. This module constructs a new weighting system and a comprehensive evaluation system based on game theory, optimizes the decision-making optimization space, effectively balances the adaptability and reliability of the strategy in the decision-making process, and provides the objective function space for the subsequent application of the filling algorithm.

[0094] The data imputation algorithm module 4 is used to intelligently select or combine imputation methods based on the guidance of the game model construction module, taking into account biological principles and the temporal characteristics of the data, so as to achieve biological rationality and data continuity of the imputation results.

[0095] The data imputation algorithm module is designed with three different working modes for the three missing data mechanisms: Mode 1 is the MCAR working mode, which uses a generative algorithm based on statistics, machine learning, and deep learning to impute missing data; Mode 2 is the MAR working mode, which uses a generative algorithm based on machine learning to impute missing data; and Mode 3 is the MNAR working mode, which uses a generative algorithm based on decision trees, XGBOOST, and other tree models to impute missing values ​​non-parametrically. The switching between Mode 1, Mode 2, and Mode 3 is done through conditional judgment.

[0096] The generative algorithms improved based on statistics, machine learning, and deep learning, as well as those improved based on machine learning and tree models such as decision trees and XGBoost, are unified in an iterative network. The basic theoretical framework of this iterative network involves training two deep learning network models: a data generator G and a data discriminator D. The data generator G corresponds to the filler in the data imputation game model, and the data discriminator D corresponds to the environment variable in the data imputation game model. The algorithm used in the iterative network is any strategy in the data imputation game model. Furthermore, improvements are made based on different machine learning theoretical methods on these two deep learning network models. The objective functions of the two deep learning network models are as follows:

[0097]

[0098] Where V is the objective function; G(l) is the output of the generator; G is the data generator, which receives a random noise signal l, which is sampled from a Gaussian distribution, and attempts to generate samples similar to the real data distribution. The data generated by this information is denoted as G(l); D is the data discriminator, which receives any sample from a standardized industrial dataset {X′}. And output the sample The probability of being true This represents the expected value function that the generated data samples satisfy the distribution of the original data samples; V(G,D) represents the expected value function that the data sample satisfies; V(G,D) represents the value function of an adversarial game between the data discriminator and the data generator.

[0099] This module integrates multiple types of data imputation algorithms and uses conditional judgment to automatically select the most suitable imputation working mode, ensuring the flexibility and efficiency of the processing, reducing the need for manual intervention, making data preprocessing more automated, and reserving an access interface for strategy space update and iteration, providing a standardized process for the integration of new algorithms.

[0100] The evaluation and verification module 5 is used to verify the effectiveness of the filling strategy by comparing the statistical characteristics of the data before and after filling, the model prediction performance, and the consistency of biological indicators. This module should also provide visualization tools to help users intuitively understand the filling effect, specifically:

[0101] The evaluation and verification module uses statistical measures such as mean, variance, skewness, or kurtosis to compare the statistical characteristics of the data before and after imputation to verify the effectiveness of the imputation strategy: by comparing the overall characteristics of the dataset before and after imputation, if the imputed dataset is reasonably close to the complete dataset in these statistical measures, then the imputation strategy is considered to be effective in maintaining the original distribution characteristics of the data.

[0102] The effectiveness of the imputation strategy is verified by comparing the prediction performance of the model before and after imputation, where the prediction model is a mean squared error or R-squared error. 2 Fraction;

[0103] The effectiveness of the imputation strategy is verified by calculating the degree of agreement between the imputation data and known biological facts or theoretical expectations.

[0104] The display and human-computer interaction module 6 provides data overview, visualization of missing mechanisms, comparison before and after filling, and parameter configuration and filling process management functions. It allows users to upload data, configure filling parameters, monitor the filling process, view filling reports and evaluation results, and retains a flexible customization interface to meet the professional needs and preferences of different users. Specifically:

[0105] The display and human-computer interaction module provides a data overview function, which allows users to quickly view the basic statistical information, missing value distribution and data structure of the data after uploading it. The data overview is displayed intuitively through charts (such as bar charts, histograms, heatmaps) and summary tables, helping users to have a preliminary understanding of the data quality and missing value situation.

[0106] The display and human-computer interaction module provides a missing mechanism visualization function. The missing mechanism visualization function is based on the results of the data missing mechanism identification module. The interface provides a graphical display of the proportion of different missing types and their distribution in the dataset, so as to help users intuitively understand the data missing pattern and provide a basis for selecting imputation strategies.

[0107] The display and human-computer interaction module provides a before-and-after comparison function, which provides a series of visualization tools: box plots, density plots or time series plots, so that users can intuitively compare the distribution, statistical characteristics and patterns of data before and after filling, and confirm the effectiveness of the filling strategy. The visualization tools also include difference plots, which highlight the specific changes caused by filling, so that users can evaluate the potential impact of the filling effect on the analysis results.

[0108] The display and human-computer interaction module provides parameter configuration and missing process management functions. These functions allow users to customize missing parameters through the interface, select missing methods (mean imputation, interpolation, model prediction, etc.), adjust model hyperparameters, and set missing mechanism assumptions. At the same time, it provides a process monitoring tool to track the status, progress, and resource consumption of the missing task in real time, ensuring that users can flexibly control and adjust the missing process.

[0109] This example illustrates the beneficial effects of the present invention:

[0110] 1) Possesses high confidence in identifying various types of incomplete data missing situations and missing mechanisms. An innovative Bayesian inference clustering missing mechanism identification model has been constructed, capable of making reasonable inferences about missing data situations. It has strong interpretability, high identification accuracy, and enables subsequent data imputation strategies to be more targeted.

[0111] 2) It possesses high robustness based on incomplete information data completion. A new weighting system and comprehensive evaluation system are constructed based on game theory, optimizing the decision-making optimization space and effectively balancing the adaptability and reliability of the strategy during the decision-making process;

[0112] 3) It possesses high agility in strategy switching and system iteration updates. It integrates multiple types of data imputation algorithms and uses conditional judgment to automatically select the most suitable imputation working mode, ensuring the flexibility and efficiency of the processing, reducing the need for manual intervention, making data preprocessing more automated, and reserving access interfaces for strategy space updates and iterations, providing a standardized process for the integration of new algorithms.

[0113] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A data imputation system for multiple missing mechanisms in biological processes based on incomplete information game theory, characterized in that, The system includes: a raw data preprocessing module, a data missing mechanism identification module, a game model construction module, a data imputation algorithm module, an evaluation and verification module, and a display and human-computer interaction module; The raw data preprocessing module is responsible for receiving and processing the raw industrial dataset from the biomanufacturing process, and then preprocessing the raw industrial dataset to obtain the preprocessed dataset. ; The data missing mechanism identification module is used to identify data based on the preprocessed dataset. A Bayesian inference clustering model is used to identify missing data mechanisms, resulting in a missing mechanism matrix. This provides a basis for constructing targeted game theory models; The game model construction module is used to construct the model based on the missing mechanism matrix. A data imputation game model is constructed to guide the selection of optimal strategies for data imputation. This model models the game participants, strategy space, and payoff function. The participants are primarily two types: the imputer and environmental variables or inherent data characteristics. These inherent data characteristics are quantified using the likelihood function, posterior probability expectation, cluster weights, mean, and covariance provided by the data missing mechanism identification module. It is assumed that the imputer's strategy space has... A filling strategy can then be modeled as: （7）； in, This represents any imputation strategy, which can be a mean imputation strategy, a KNN imputation strategy, a model prediction imputation strategy, or an environment strategy. The payoff function for the filler is The effectiveness of imputation strategies is evaluated by a comprehensive score, including the statistical characteristics of the imputed data, model predictive performance, and biological plausibility indicators. The payoff function can be formalized as follows: （8）； in, Indicates filling in data; Functions representing statistical evaluation Indicates the weight of the statistical evaluation; A function representing the evaluation of model performance. The weights represent the evaluation of model performance. A function representing the evaluation of biological rationality. The weights representing the biological rationale assessment; The game model construction module constructs a new weight system and comprehensive evaluation system based on game theory, optimizes the decision optimization space, effectively balances the adaptability and reliability of the strategy in the decision-making process, and provides an objective function space for the subsequent application of the filling algorithm. The data imputation algorithm module is used to intelligently select or combine imputation methods based on the guidance of the game model construction module, taking into account biological principles and the temporal characteristics of the data, so as to achieve biological rationality and data continuity of the imputation results. The evaluation and verification module is used to verify the effectiveness of the filling strategy by comparing the statistical characteristics of the data before and after filling, the model prediction performance, and the consistency of biological indicators. The evaluation and verification module should also provide visualization tools to help users intuitively understand the filling effect. The display and human-computer interaction module provides data overview, visualization of missing mechanisms, comparison before and after filling, and parameter configuration and filling process management functions. It allows users to upload data, configure filling parameters, monitor the filling process, view filling reports and evaluation results, and retains a flexible customization interface to meet the professional needs and preferences of different users.

2. The data imputation system for multiple missing mechanisms in biological processes based on incomplete information game theory according to claim 1, characterized in that, The original industrial dataset is preprocessed to obtain the preprocessed dataset. Specifically: First, analyze the original industrial dataset. Data cleaning was performed using the Z-score method. （1）； in, Represents the original industrial dataset Any data in the dataset; Represents the original industrial dataset The average of all data in the data; Represents the original industrial dataset The standard deviation of all data in the dataset; Set the normal range as The scope does not belong to of corresponding Considered an outlier; Subsequently, relying on programming logic, the original industrial dataset was... The date format and units of all data are standardized to obtain the original industrial dataset with a standardized format. ; Then, the original industrial dataset after format standardization was processed. Each data point is scaled to the [0, 1] interval to obtain the standardized industrial dataset. : （2）； in, The original industrial dataset after standardization of format Any data in the dataset; The original industrial dataset after standardization of format The maximum value in the middle; The original industrial dataset after standardization of format Minimum value; Finally, logic programming was used to process the standardized industrial dataset. All missing values, outliers, and non-numeric items are marked as The preprocessed dataset is obtained. .

3. The data imputation system for multiple missing mechanisms in biological processes based on incomplete information game theory according to claim 1, characterized in that, The data missing mechanism identification module is used to identify data based on the preprocessed dataset. A Bayesian inference clustering model is used to identify missing data mechanisms, resulting in a missing mechanism matrix. This provides a basis for constructing targeted game theory models, specifically: First, we introduce the missing mechanism type parameter. ,in, These correspond to completely random missing mechanisms, random missing mechanisms, and non-random missing mechanisms, respectively; assuming the missing mechanism type... This is implicitly reflected through clustering attributes, let's assume... Representing data Belongs to the The index variable for the first cluster; and assign it to the first cluster index variable. Cluster weights of each cluster Initial mean and the initial covariance matrix ; Then, iterations are performed, and during the t-th iteration, the preprocessed dataset is calculated according to the following formula. Each data Belonging to various clusters Posterior probability: （3）； in, Indicates the first After the first iteration Cluster weights of each cluster; Indicates the first After the first iteration The mean of each cluster; Indicates the first After the first iteration The covariance matrix of each cluster; Indicates the first Data after round of iteration Belongs to the The index variables for each cluster; Representing data The probability density function of a Gaussian distribution; Represents given data Belongs to the When performing clustering, the posterior probability of the missing mechanism type; Then update the cluster weights, mean, and covariance matrix: （4）； （5）； （6）； After the update is complete, check if the following conditions are met: If satisfied If the condition is met or the set number of iterations is reached, the iteration ends, and the expected value is determined based on the posterior probability. Output each data point Corresponding missing mechanism The missing mechanism matrix is ​​obtained. ;in, Indicates the first Round iteration given back The posterior probability; express Approximate function; express The prior probability, Indicates a direct proportional relationship; If the conditions are not met, proceed to the next iteration. The missing data mechanism identification module constructs a Bayesian inference clustering missing data mechanism identification model to make high-confidence inferences about missing data. The combination of the Bayesian inference clustering missing data mechanism identification model with posterior probability inference significantly improves the identification accuracy and makes subsequent data imputation strategies more targeted.

4. The data imputation system for multiple missing mechanisms of biological processes based on incomplete information game theory according to claim 1, characterized in that, The data imputation algorithm module is designed with three different working modes for the three missing data mechanisms: Mode 1 is the MCAR working mode, which uses a generative algorithm based on statistics, machine learning, and deep learning to impute missing data; Mode 2 is the MAR working mode, which uses a generative algorithm based on machine learning to impute missing data; and Mode 3 is the MNAR working mode, which uses a generative algorithm based on decision trees and the XGBOOST model to impute missing values ​​non-parametrically. The switching between Mode 1, Mode 2, and Mode 3 is done through conditional judgment. Generative algorithms improved based on statistics, machine learning, and deep learning, as well as generative algorithms improved based on machine learning and decision trees and XGBoost models, are unified in an iterative network. The basic theoretical framework of this iterative network is to train two deep learning network models: a data generator and a data generator. and data discriminator The data generator The data discriminator corresponds to the filler in the data filling game model. The corresponding data is used to impute environmental variables in the game model. The algorithm used in the iterative network is any strategy in the data imputation game model. Furthermore, improvements are made to these two deep learning network models based on different machine learning theories and methods. The objective functions of the two deep learning network models are as follows: （9）； in, The objective function is... This is the output of the generator; This is a data generator used to receive a random noise signal. The random noise signal Data is sampled from a Gaussian distribution, and an attempt is made to generate samples similar to the real data distribution. The data generated using this information is denoted as... ; This is a data discriminator used to receive a standardized industrial dataset. any sample And output the sample The probability of being true ; This represents the expected value function that the generated data samples satisfy the distribution of the original data samples; Represents the expected value function satisfied by the data sample; The value function represents an adversarial game between a data discriminator and a data generator; The data imputation algorithm module integrates multiple types of data imputation algorithms and uses conditional judgment to automatically select the most suitable imputation working mode, ensuring the flexibility and efficiency of the processing, reducing the need for manual intervention, making data preprocessing more automated, and reserving an access interface for strategy space update and iteration, providing a standardized process for the access of new algorithms.

5. A data imputation system for multiple missing mechanisms in biological processes based on incomplete information game theory according to claim 1, characterized in that, The evaluation and validation module is used to verify the effectiveness of the filling strategy by comparing the statistical characteristics of the data before and after filling, the model's predictive performance, and the consistency with biological indicators. The evaluation and validation module should also provide visualization tools to help users intuitively understand the filling effect, specifically: The evaluation and verification module uses statistical measures such as mean, variance, skewness, or kurtosis to compare the statistical characteristics of the data before and after imputation to verify the effectiveness of the imputation strategy: by comparing the overall characteristics of the dataset before and after imputation, if the imputed dataset is reasonably close to the complete dataset in these statistical measures, then the imputation strategy is considered to be effective in maintaining the original distribution characteristics of the data. The model performance is evaluated by comparing the prediction errors before and after imputation, thereby verifying the effectiveness of the imputation strategy; the evaluation metrics are mean squared error or score. The effectiveness of the imputation strategy is verified by calculating the degree of agreement between the imputation data and known biological facts or theoretical expectations.

6. A data imputation system for multiple missing mechanisms in biological processes based on incomplete information game theory according to claim 1, characterized in that, The display and human-computer interaction module provides data overview, visualization of missing mechanisms, comparison before and after filling, and parameter configuration and filling process management functions. It allows users to upload data, configure filling parameters, monitor the filling process, view filling reports and evaluation results, and retains a flexible customization interface to meet the professional needs and preferences of different users. Specifically: The display and human-computer interaction module provides a data overview function, which allows users to quickly view the basic statistical information, missing value distribution and data structure of the data after uploading it. The data overview is displayed intuitively in the form of charts and summary tables, which helps users to have a preliminary understanding of the data quality and missing value situation. The display and human-computer interaction module provides a missing mechanism visualization function. The missing mechanism visualization function is based on the results of the data missing mechanism identification module. The interface provides a graphical display of the proportion of different missing types and their distribution in the dataset, so as to help users intuitively understand the data missing pattern and provide a basis for selecting imputation strategies. The display and human-computer interaction module provides a before-and-after comparison function, which provides a series of visualization tools: box plots, density plots or time series plots, so that users can intuitively compare the distribution, statistical characteristics and patterns of data before and after filling, and confirm the effectiveness of the filling strategy. The visualization tools also include difference plots, which highlight the specific changes caused by filling, so that users can evaluate the potential impact of the filling effect on the analysis results. The display and human-computer interaction module provides parameter configuration and filling process management functions. These functions allow users to customize filling parameters, select filling methods, adjust model hyperparameters, and set missing mechanism assumptions through the interface. At the same time, it provides a process monitoring tool to track the status, progress, and resource consumption of the filling task in real time, ensuring that users can flexibly control and adjust the filling process.

Images