Method and system for processing laboratory test data based on machine learning technology

Abstract

The present invention discloses a method and system for processing laboratory test data based on machine learning technology, which relates to the field of data processing technology. The present invention decomposes signals through discrete wavelet transform, calculates the noise standard deviation based on the median of the highest frequency coefficient, reconstructs high signal-to-noise ratio data through dynamic threshold and soft threshold functions, and improves signal quality. The present invention uses a sliding window to extract time series signal statistics and FFT frequency domain features, combines the SIFT algorithm to extract spatial features, retains highly correlated features through mutual information screening, and reduces redundancy. A graph convolutional network anomaly detection model and an XGBoost-LightGBM weighted regression model are constructed, and contaminated data are eliminated through an abnormal probability threshold to obtain a normal regression prediction value. An LSTM network is used to predict a compensation amount according to environmental parameters, and calibrated laboratory data is obtained based on the normal regression prediction value combined with the predicted compensation amount. The present invention solves the problems of noise sensitivity, feature fragmentation, model isolation and environmental drift of multi-source data, and significantly improves the accuracy and robustness of laboratory analysis.

Description

Technical Field

[0001] The present invention belongs to the field of data processing technology, and more specifically, relates to a method and system for processing laboratory test data based on machine learning technology. Background Art

[0002] In high-end laboratory scenarios such as biomedicine, environmental monitoring, and materials science, instrument data is heterogeneous and multi-source (e.g., spectrometer waveforms, chromatograph values, microscopic images, etc.) and susceptible to interference from temperature, humidity, and voltage fluctuations. Traditional analysis methods rely on manual experience and segmented processing: first noise reduction, then feature extraction, and finally single-model prediction. This results in significant information loss, delayed process response, and severe cumulative errors due to environmental interference. The industry still needs an intelligent analysis system that can automatically integrate multimodal data, perform real-time noise reduction, and collaboratively optimize anomaly filtering and environmental compensation.

[0003] Existing laboratory data analysis systems have significant defects: noise processing of multi-source heterogeneous signals relies on fixed threshold noise reduction methods, which are difficult to adapt to dynamic noise environments; feature extraction is limited to a single modality and lacks multimodal fusion capabilities; anomaly detection and quantitative regression model training are separated, resulting in abnormal samples contaminating the regression results; environmental parameter interference is only handled through static compensation, which cannot respond to complex environmental changes in real time, ultimately resulting in reduced analysis accuracy and insufficient reliability. Summary of the Invention

[0004] (1) Technical problems solved

[0005] In response to the problems in the related art, the present invention provides a method for processing laboratory test data based on machine learning technology to overcome the above-mentioned technical problems existing in the existing related art.

[0006] (2) Technical solution

[0007] To solve the above technical problems, the present invention is achieved through the following technical solutions:

[0008] S1. Collect laboratory multi-source signal data and environmental parameter data; perform dynamic noise reduction on the laboratory multi-source signal data to obtain pre-processed laboratory multi-source data;

[0009] S2, extracting frequency domain features and spatial features from preprocessed laboratory multi-source data, and fusing them to obtain laboratory multi-source features;

[0010] Perform mutual information feature selection on multi-source laboratory features to obtain effective laboratory features;

[0011] S3. Construct an anomaly detection classification model and a quantitative regression model; train the anomaly detection classification model and the quantitative regression model respectively to obtain the final anomaly detection classification model and the final quantitative regression model;

[0012] S4. Use the final anomaly detection classification model and the final quantitative regression model to perform dual-model collaborative analysis on the effective laboratory characteristics to obtain the anomaly detection results and regression prediction values;

[0013] According to the anomaly detection results, the abnormal data in the regression prediction value is eliminated to obtain the normal regression prediction value;

[0014] S5. Based on the environmental parameter data, the predicted compensation amount is obtained in combination with the LSTM network; based on the normal regression prediction value and the predicted compensation amount, the calibration laboratory data is obtained;

[0015] This invention overcomes the technical problems of multi-source data noise sensitivity, insufficient feature utilization, isolated model prediction, and accumulated environmental interference through full-chain innovation of dynamic noise reduction, multimodal fusion, feature screening, dual-model collaboration, and environmental compensation. It provides laboratories with high-precision, robust, and adaptive intelligent analysis solutions, significantly improving the reliability and timeliness of detection results.

[0016] Preferably, the S1 comprises the following steps:

[0017] S11. Acquire laboratory multi-source signal data in real time through an industrial bus or API interface; the laboratory multi-source signal data includes structured signal data and unstructured signal data;

[0018] Synchronously collect environmental parameter data to obtain environmental parameter data;

[0019] S12, performing dynamic noise reduction processing on the laboratory multi-source signal data to obtain preprocessed laboratory multi-source data;

[0020] The present invention collects laboratory structured and unstructured data through the industrial bus / API dual track, synchronously obtains environmental parameters such as temperature and humidity, and realizes unified access of multi-source heterogeneous data; it adopts dynamic noise reduction processing to adaptively filter out noise and retain effective signal edge features, thereby improving the signal-to-noise ratio and providing a high-fidelity data foundation for subsequent analysis.

[0021] Preferably, the S12 includes the following steps:

[0022] S121, performing wavelet transform on the laboratory multi-source signal data to obtain a wavelet coefficient set; the wavelet coefficients include approximate coefficients and detail coefficients;

[0023] S122, calculating the noise standard deviation based on the wavelet coefficient set;

[0024] S123, calculating a real-time dynamic threshold according to the noise standard deviation;

[0025] S124, processing each wavelet coefficient in the wavelet coefficient set by a soft threshold function according to the real-time dynamic threshold to obtain a filtered wavelet coefficient set;

[0026] S125, reconstructing the wavelet signal in the filtered wavelet coefficient set by inverse wavelet transform to obtain preprocessed laboratory multi-source data;

[0027] The present invention decomposes the signal into approximate / detail coefficients through wavelet transform, calculates the noise standard deviation based on the median of the highest frequency detail coefficient, dynamically generates a threshold to adapt to the signal length, uses a soft threshold function to intelligently shrink the noise coefficient, and reconstructs a high-fidelity signal through wavelet inversion; adaptively filters out noise while retaining effective signal edge features, thereby improving the signal-to-noise ratio and avoiding the over-smoothing or under-denoising problems caused by fixed thresholds.

[0028] Preferably, said S2 comprises the following steps:

[0029] S21, calculating the local statistics of the time series signal data in the preprocessing laboratory multi-source data through a sliding window to obtain frequency domain features;

[0030] The SIFT algorithm is used to extract the key point descriptors of spatial data from the preprocessed laboratory multi-source data to obtain spatial features;

[0031] The frequency domain features and spatial features together constitute laboratory multi-source features;

[0032] S22, calculate the mutual information score set between the features and target variables of the preprocessed laboratory multi-source data;

[0033] Set the retention percentage; retain the valid features in the mutual information score set according to the retention percentage to obtain the valid laboratory features;

[0034] The present invention extracts local statistics and FFT frequency domain features of time series signals through a sliding window, and combines the SIFT algorithm to capture key point descriptors of spatial data to achieve time series-spatial multimodal feature fusion; calculates the mutual information score between features and target variables, retaining highly correlated features; and through deep mining of signal dynamic laws and spatial structure information, mutual information screening reduces feature redundancy, thereby improving model training efficiency and representation accuracy.

[0035] Preferably, the step S3 includes the following steps:

[0036] S31. Build an anomaly detection classification model based on graph convolutional networks and attention mechanisms; set the weight parameters of the anomaly detection classification model;

[0037] Based on XGBoost and LightGBM, a quantitative regression model is constructed; the quantitative regression model adopts the weighted fusion strategy of XGBoost and LightGBM dual models;

[0038] S32. Collect historical laboratory characteristic data;

[0039] S33, inputting historical laboratory feature data into an anomaly detection classification model for training;

[0040] During the training process, the random search and cross-validation algorithms are combined to find the optimal weight parameters of the anomaly detection classification model and obtain the optimal solution. The optimal solution is used as the weight parameter of the anomaly detection classification model to obtain the final anomaly detection classification model.

[0041] S34, inputting the historical laboratory characteristic data into the quantitative regression model for training, and optimizing the parameters of the quantitative regression model in combination with grid search to obtain the final quantitative regression model;

[0042] The present invention improves the analysis capability through a hybrid model architecture and intelligent optimization strategy; accurately captures data correlation features through graph convolutional network + attention mechanism to achieve anomaly detection, and enhances the robustness of regression prediction through weighted fusion of XGBoost and LightGBM; graph convolutional network processes topological data, attention mechanism focuses on key features, and XGBoost + LightGBM fusion takes into account both speed and accuracy; combines random search and cross-validation to optimize classification model weights, and grid search to tune regression model parameters; random search avoids grid calculation explosion, and cross-validation prevents overfitting, ensuring the reliability of optimal weights; directly improves the efficiency of automated analysis of laboratory data and reduces manual intervention errors; significantly improves the sensitivity of anomaly recognition and quantitative prediction accuracy, reduces the cost of manual parameter adjustment, and is particularly suitable for complex and high-dimensional laboratory data scenarios, improving the generalization ability of the model.

[0043] Preferably, the training process in S33 combines random search and cross-validation algorithms to find the optimal weight parameters of the anomaly detection classification model, and obtaining the optimal solution includes the following steps:

[0044] S331, set the weight parameter search space of the anomaly detection classification model to p, the maximum number of random searches to m, and the cross-validation fold to W;

[0045] S332. Randomly set the weight parameters of the anomaly detection classification model according to the weight parameter search space p to obtain an initial weight parameter set;

[0046] Initialize the optimal weight parameter of the anomaly detection classification model to u and the best performance to v, and randomly select the weight parameter of the anomaly detection classification model to be e from the initial weight parameter set;

[0047] S333. Divide the training data in the historical laboratory feature data into W folds. For each fold w, ; Use the training data from all historical laboratory feature data except the w-th fold to train the anomaly detection classification model, and use the training data from the w-th fold historical laboratory feature data to evaluate the anomaly detection classification model to obtain a set of performance indicators;

[0048] Calculate the mean of the performance indicators of all folds to get the average performance r,

[0049] If the average performance r> the best performance v, then update the best performance to r and the optimal weight parameter to e; otherwise, keep the original best performance and optimal weight parameter;

[0050] S334, repeat S333, and when the maximum number of random searches m is reached, stop the iteration and obtain the optimal solution;

[0051] The present invention optimizes the anomaly detection model by combining random search with cross-validation; efficiently samples weight combinations in the parameter space to avoid the computational overhead of grid search; divides the data into multiple training and evaluation steps to ensure the statistical stability of performance indicators and reduce the risk of overfitting; continuously compares the average performance with the best performance, and ultimately converges to the optimal weight; and balances efficiency and robustness within a limited number of searches, significantly improving the model's generalization ability. This method is particularly suitable for scenarios with limited laboratory data and the need to avoid evaluation bias.

[0052] Preferably, the S4 comprises the following steps:

[0053] S41. Input the effective laboratory characteristics into the final anomaly detection classification model and the final quantitative regression model respectively to obtain anomaly detection results and regression prediction values;

[0054] S42. Setting an abnormal probability threshold; marking valid laboratory features in the abnormal detection results whose abnormal probability is greater than or equal to the abnormal probability threshold as abnormal data;

[0055] Eliminate the regression prediction values corresponding to abnormal data from the regression prediction values to obtain normal regression prediction values;

[0056] The present invention improves the reliability of analysis through the collaboration of dual models; uses the classification model to identify abnormal data (probability ≥ threshold) and accurately mark abnormal samples; eliminates the regression prediction values corresponding to abnormal data and retains the normal regression prediction values; avoids outliers interfering with the regression results, and improves the accuracy and stability of quantitative predictions. It is particularly suitable for scenarios where laboratory data quality fluctuates.

[0057] Preferably, the S5 comprises the following steps:

[0058] S51. Build an LSTM network; set calibration rules; and update the LSTM network according to the calibration rules.

[0059] S52, inputting the environmental parameter data into the LSTM network to obtain the predicted compensation amount;

[0060] S53. Calculate calibration laboratory data based on the normal regression prediction value and the predicted compensation amount;

[0061] The present invention achieves dynamic calibration through LSTM time series modeling; uses LSTM to learn the impact of environmental parameters (such as temperature and humidity) on data and outputs predicted compensation values; combines normal regression predictions with compensation values to generate calibration laboratory data; significantly reduces systematic errors caused by environmental interference, improves data reliability and the accuracy of experimental conclusions, and is suitable for high-precision environmentally sensitive scenarios.

[0062] A machine learning-based laboratory test data processing system, used to implement the aforementioned machine learning-based laboratory test data processing method, includes a multi-source data acquisition and dynamic noise reduction module, a multimodal feature fusion and selection module, a dual-model collaborative training and optimization module, an anomaly perception regression prediction module, and an environmental compensation dynamic calibration module;

[0063] The multi-source data acquisition and dynamic noise reduction module is used for real-time acquisition and preprocessing of multi-source signals and environmental parameters in the laboratory. It acquires structured and unstructured data through an industrial bus or API interface, and simultaneously collects environmental parameters such as temperature, humidity, and voltage. It decomposes coefficients through discrete wavelet transform, estimates the noise standard deviation based on the median of the highest frequency coefficient, dynamically calculates the threshold, and then shrinks the noise coefficient and reconstructs the signal through a soft threshold function, ultimately outputting the denoised multi-source data.

[0064] The multimodal feature fusion and selection module is used to extract and fuse frequency domain and spatial features from preprocessed data; for time series signals, local statistics are calculated through a sliding window and FFT frequency domain features are extracted; for spatial data, the SIFT algorithm is used to extract key point descriptors; the fused multi-source features are optimized through mutual information feature selection, the mutual information score of the features and the target variable is calculated, high-scoring features are retained, redundant features are eliminated, and effective laboratory features are output;

[0065] The dual-model collaborative training and optimization module is used to build and train dual models for anomaly detection and quantitative regression. Based on the graph convolutional network and the attention mechanism, a graph structure is constructed, and the weight parameters are optimized using random search cross-validation. Random sampling is performed in the parameter space, and performance is evaluated through W-fold cross-validation. The optimal weights are iteratively updated to obtain the final anomaly detection classification model. By fusing XGBoost and LightGBM, training is carried out in two stages: first, XGBoost is used to predict the base value, and then LightGBM is used to learn the residual. Finally, the weighted fusion coefficient is optimized on the validation set through grid search, and the weight that minimizes the RMSE is selected to obtain the final quantitative regression model.

[0066] The anomaly perception regression prediction module is used to implement dual-model collaborative reasoning and anomaly filtering; input valid features into the final anomaly detection classification model and the final quantitative regression model respectively, output anomaly probability and regression prediction value; set an anomaly probability threshold, mark abnormal samples; eliminate the regression prediction value corresponding to the abnormal sample, and output a pure normal regression prediction result;

[0067] The environmental compensation dynamic calibration module eliminates environmental interference based on the LSTM network; inputs environmental parameters into the LSTM and calculates the predicted compensation amount through a gating mechanism; designs an incremental learning mechanism to optimize the LSTM; superimposes the normal regression prediction value and the compensation amount, and outputs the final laboratory data after environmental calibration.

[0068] (3) Beneficial effects

[0069] The present invention has the following beneficial effects:

[0070] This invention overcomes the technical problems of multi-source data noise sensitivity, insufficient feature utilization, isolated model prediction, and accumulated environmental interference through full-chain innovation of dynamic noise reduction, multimodal fusion, feature screening, dual-model collaboration, and environmental compensation. It provides laboratories with high-precision, robust, and adaptive intelligent analysis solutions, significantly improving the reliability and timeliness of detection results.

[0071] The present invention significantly improves signal quality through dynamic adaptive noise reduction; decomposes signal coefficients through discrete wavelet transform, accurately estimates noise standard deviation based on the median of the highest frequency detail coefficient, and dynamically calculates thresholds and combines them with soft threshold functions to intelligently shrink the noise coefficient, thoroughly filtering out noise while retaining effective signal edge features, thereby improving the signal-to-noise ratio of the reconstructed signal and providing a high-fidelity data foundation for subsequent analysis.

[0072] The present invention comprehensively enhances the information representation capability through multimodal feature fusion; a sliding window is used to extract local statistics and FFT frequency domain features for time series signals; key point descriptors are extracted for spatial data through the SIFT algorithm; the fused multi-source features are screened through mutual information scoring to retain highly correlated features, effectively reducing feature redundancy and significantly improving model training efficiency and representation accuracy.

[0073] The present invention achieves anomaly immunity through a dual-model collaborative mechanism; the anomaly detection model combines graph convolutional networks with the attention mechanism, optimizes parameters through random search and W-fold cross-validation, and improves the accuracy of anomaly classification; quantitative regression adopts a weighted fusion strategy of XGBoost and LightGBM, and reduces the prediction error after determining the optimal weight through grid search; contaminated samples are directly eliminated based on the anomaly detection results, eliminating the interference of outliers on the regression results from the source.

[0074] The present invention ensures long-term analysis accuracy through dynamic calibration of environmental compensation; the LSTM network dynamically learns the nonlinear relationship between environmental parameters and interference through a gating mechanism, and the cell state long-term memorizes the environmental change pattern and outputs an accurately predicted compensation amount; the design of an incremental learning mechanism optimizes LSTM to solve the problem of cumulative error caused by laboratory environmental drift.

[0075] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] In order to more clearly illustrate the technical solutions of the embodiments of the invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the invention. For ordinary technicians in this field, they can also obtain drawings based on these drawings without paying any creative work.

[0077] Figure 1 Schematic diagram of the process of the laboratory test data processing method based on machine learning technology of the present invention;

[0078] Figure 2 This is a flow chart of dynamic threshold denoising of laboratory multi-source signal data in the laboratory test data processing method based on machine learning technology of the present invention;

[0079] Figure 3 A schematic diagram of a process for obtaining a normal regression prediction value in a method for processing laboratory test data based on machine learning technology of the present invention;

[0080] Figure 4 Schematic diagram of the modules of the laboratory test data processing system based on machine learning technology of the present invention. DETAILED DESCRIPTION

[0081] The following will clearly and completely describe the technical solutions in the embodiments of the invention in conjunction with the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of the invention, not all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0082] In the description of the present invention, it should be understood that the terms "opening", "upper", "lower", "top", "middle", "inside" and the like indicating orientation or positional relationship are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the components or elements referred to must have a specific orientation, be constructed and operate in a specific orientation, and therefore cannot be understood as limiting the invention.

[0083] Example 1:

[0084] See also Figure 1 、 Figure 2 、 Figure 3 The present invention discloses a method for processing laboratory test data based on machine learning technology, comprising the following steps:

[0085] S1. Collect laboratory multi-source signal data and environmental parameter data; perform dynamic noise reduction on the laboratory multi-source signal data to obtain pre-processed laboratory multi-source data;

[0086] Said S1 comprises the following steps:

[0087] S11. Acquire laboratory multi-source signal data in real time via an industrial bus (Modbus / CAN) or API interface; the laboratory multi-source signal data includes structured signal data and unstructured signal data; the structured signal data includes detection values (concentration value, absorbance, etc.), and the unstructured signal data includes spectral waveforms, chromatograms, and mass spectra;

[0088] Synchronously collect environmental parameter signals to obtain environmental parameter signal data; the environmental parameter temperature data, humidity data, and device voltage value;

[0089] S12, performing dynamic noise reduction processing on the laboratory multi-source signal data to obtain preprocessed laboratory multi-source data;

[0090] The S12 includes the following steps:

[0091] S121, performing a wavelet transform (such as discrete wavelet transform (DWT)) on the laboratory multi-source signal data to obtain a wavelet coefficient set; the wavelet coefficients include approximate coefficients and detail coefficients; the highest frequency detail coefficients are mainly composed of noise;

[0092] S122. Calculate the noise standard deviation based on the wavelet coefficient set; the calculation formula is as follows:

[0093] ;

[0094] Where σ is the standard deviation of the noise; ck is the highest-frequency wavelet coefficient in the wavelet coefficient set; median(ck) is the median of the highest-frequency wavelet coefficients (used to eliminate the influence of signal components); the reason for dividing by 0.6745 is that for normally distributed noise, the relationship between MAD and standard deviation is MAD / 0.6745;

[0095] S123. Calculate the real-time dynamic threshold according to the noise standard deviation; the calculation formula is as follows:

[0096] ;

[0097] Where λ represents the real-time dynamic threshold (the wavelet coefficient used to distinguish noise from useful signals), and N represents the signal length of the laboratory multi-source signal data (the threshold increases with the signal length to accommodate longer signals);

[0098] S124, processing each wavelet coefficient in the wavelet coefficient set by a soft threshold function according to the real-time dynamic threshold to obtain a filtered wavelet coefficient set; the soft threshold function formula is as follows:

[0099] ;

[0100] Where y represents the wavelet coefficient in the wavelet coefficient set, and η(y,λ) represents the wavelet coefficient in the wavelet coefficient set after the soft threshold function processing; the wavelet coefficient greater than λ is subtracted from λ (shrinking to zero), the wavelet coefficient less than -λ is added to λ (shrinking to zero), and the coefficients with absolute values less than or equal to λ are set to zero (considered as noise);

[0101] S125, reconstructing the wavelet signal in the filtered wavelet coefficient set by inverse wavelet transform to obtain preprocessed laboratory multi-source data;

[0102] S2, extracting frequency domain features and spatial features from preprocessed laboratory multi-source data, and fusing them to obtain laboratory multi-source features;

[0103] Perform mutual information feature selection on multi-source laboratory features to obtain effective laboratory features;

[0104] The S2 comprises the following steps:

[0105] S21. Calculate the local statistics (mean, variance, skewness) of the time series signal data (such as spectral scan) in the preprocessed laboratory multi-source data through a sliding window (width = 50 sampling points) to obtain frequency domain features (such as the main frequency amplitude and power spectrum entropy after FFT transformation). The sliding window calculation formula is as follows:

[0106] ;

[0107] Where μt represents the mean of the t-th window, w represents the window length (i.e., the number of sample points of the time series signal data in the preprocessed laboratory multi-source data), and x represents the value of the time series signal at index i; σt2 represents the variance of the t-th window, 1 / (w-1) represents the denominator of the variance calculation of the unbiased estimate (using w-1 instead of w can provide a better estimate of the overall variance, especially when the sample size w is not large), and (xi-μt)2 represents the calculation of the square of the deviation of each sample point from the window mean;

[0108] The SIFT algorithm is used to extract key point descriptors of spatial data (such as cell microscopic images) from multi-source data in the preprocessed laboratory to obtain spatial features;

[0109] The frequency domain features and spatial features together constitute laboratory multi-source features;

[0110] S22. Calculate the mutual information score set between the features of the preprocessed laboratory multi-source data and the target variable; the calculation formula is as follows:

[0111] ;

[0112] Where X represents the feature variable in the features of the preprocessed laboratory multi-source data, and Y represents the target variable (such as category label, predicted value, etc.); I(X,Y) represents the mutual information score between the feature variable X and the target variable Y (i.e., it measures the degree to which the uncertainty of Y is reduced after knowing X, and vice versa, it measures the statistical dependence between X and Y. The larger the value, the stronger the correlation between X and Y); x and y represent all possible values of X and Y, respectively; p(x,y) represents the joint probability that X takes the value x and Y takes the value y; p(x) and p(y) represent the marginal probabilities that X takes the value x and Y takes the value y, respectively.

[0113] Set the retention percentage; retain the valid features in the mutual information score set according to the retention percentage to obtain the valid laboratory features; for example, retain the top 20% of the features with high mutual information scores and eliminate redundant features;

[0114] S3. Construct an anomaly detection classification model and a quantitative regression model; train the anomaly detection classification model and the quantitative regression model respectively to obtain the final anomaly detection classification model and the final quantitative regression model;

[0115] The S3 includes the following steps:

[0116] S31. Construct an anomaly detection classification model based on a graph convolutional network and an attention mechanism; set weight parameters of the anomaly detection classification model; constructing the anomaly detection classification model includes constructing a graph structure, designing a graph convolution layer, setting an attention mechanism, and designing an output layer; constructing the graph structure includes inputting training sample features, defining an adjacency matrix, and adding self-loops;

[0117] Based on XGBoost and LightGBM, a quantitative regression model is constructed; the quantitative regression model adopts the weighted fusion strategy of XGBoost and LightGBM dual models; the XGBoost model is based on the gradient boosting tree, and the complexity of the XGBoost model is controlled by regularization terms; based on the histogram algorithm acceleration, the leaf-wise growth strategy is adopted;

[0118] S32. Collect historical laboratory characteristic data;

[0119] S33, inputting historical laboratory feature data into an anomaly detection classification model for training;

[0120] During the training process, the random search and cross-validation algorithms are combined to find the optimal weight parameters of the anomaly detection classification model and obtain the optimal solution. The optimal solution is used as the weight parameter of the anomaly detection classification model to obtain the final anomaly detection classification model.

[0121] In the training process in S33, the optimal weight parameters of the anomaly detection classification model are found by combining random search and cross-validation algorithms to obtain the optimal solution, which includes the following steps:

[0122] S331, set the weight parameter search space of the anomaly detection classification model to p, the maximum number of random searches to m, and the cross-validation fold to W;

[0123] S332, according to the weight parameter search space p, randomly set the weight parameters of the anomaly detection classification model to obtain the initial weight parameter set , ji represents the i-th initial weight parameter, k represents the total number of initial weight parameters;

[0124] Initialize the optimal weight parameter of the anomaly detection classification model to u and the best performance to v, and start from the initial weight parameter set The weight parameter of the anomaly detection classification model is randomly selected as e;

[0125] The training data in the historical laboratory feature data is divided into W folds. For each fold w, ; Use the training data from all historical laboratory feature data except the w-th fold to train the anomaly detection classification model, and use the training data from the w-th fold historical laboratory feature data to evaluate the anomaly detection classification model to obtain the performance indicator set ; Where tw represents the wth performance indicator, and W represents the total number of performance indicators;

[0126] Calculate the mean of the performance indicators of all folds to obtain the average performance r. The calculation formula is as follows:

[0127] ;

[0128] If the average performance r> the best performance v, then update the best performance to r and the optimal weight parameter to e; otherwise, keep the original best performance and optimal weight parameter;

[0129] S334, repeat S333, and when the maximum number of random searches m is reached, stop the iteration and obtain the optimal solution;

[0130] S34, inputting the historical laboratory characteristic data into the quantitative regression model for training, and optimizing the parameters of the quantitative regression model in combination with grid search to obtain the final quantitative regression model;

[0131] The historical laboratory feature data was divided into a training set and a validation set. The quantitative regression model was trained in stages. In the first stage, the base model was trained independently using XGBoost, with 200 trees, a maximum depth of 7, a learning rate of 0.05, and a subsampling ratio of 0.8 to improve overfitting resistance. After the model training was completed, the prediction residuals of the training set were calculated as the second stage target. In the second stage, the XGBoost parameters were fixed, and LightGBM was used to specifically learn these residuals. The residual compensation model was configured with 150 leaf nodes, a learning rate of 0.1, and a feature sampling ratio of 0.7.

[0132] On the retained validation set, a grid search was performed with a step size of 0.01 for weight values in the range [0.5, 0.7]. At each weight value, the predictions of the two models were fused (α × XGB prediction + (1-α) × compensation prediction), and the validation set RMSE was calculated. The weight value corresponding to the minimum RMSE was selected as the final fusion coefficient. During prediction, XGBoost was first used to generate the basic prediction, and then LightGBM was used to generate the residual compensation. Finally, the fusion output was weighted according to the optimal weight to obtain the final quantitative regression model.

[0133] S4. Use the final anomaly detection classification model and the final quantitative regression model to perform dual-model collaborative analysis on the effective laboratory characteristics to obtain the anomaly detection results and regression prediction values;

[0134] According to the anomaly detection results, the abnormal data in the regression prediction value is eliminated to obtain the normal regression prediction value;

[0135] The S4 comprises the following steps:

[0136] S41. Input the effective laboratory characteristics into the final anomaly detection classification model and the final quantitative regression model respectively to obtain anomaly detection results and regression prediction values;

[0137] S42. Setting an abnormal probability threshold; marking valid laboratory features in the abnormal detection results whose abnormal probability is greater than or equal to the abnormal probability threshold as abnormal data;

[0138] Eliminate the regression prediction values corresponding to abnormal data from the regression prediction values to obtain normal regression prediction values;

[0139] S5. Predict the compensation amount based on the environmental parameter data in combination with the LSTM network; obtain the calibration laboratory data based on the normal regression prediction value combined with the predicted compensation amount;

[0140] The S5 comprises the following steps:

[0141] S51. Build an LSTM network; set calibration rules; and update the LSTM network according to the calibration rules.

[0142] For example, when the prediction error of 10 consecutive samples is greater than 5%, incremental learning is automatically triggered, and the parameters of the last two layers of LSTM are fine-tuned with the new data to obtain an updated LSTM network;

[0143] S52, input the environmental parameter data into the LSTM network to obtain the predicted compensation amount; the LSTM unit calculation process is as follows,

[0144] ;

[0145] Where it represents the input gate value at time step t (used to control what new information is written into the cell state), σ represents the Sigmoid activation function (used to map the input to the (0, 1) interval), Wi represents the weight matrix of the input gate, [ht-1,et] represents the concatenation of the hidden state of the previous time step and the environment parameters of the current time step into a vector as the input of the LSTM unit, and bi represents the bias vector of the input gate;

[0146] ft represents the forget gate value at time step t (controls how much past information needs to be discarded from the cell state), Wf represents the weight matrix of the forget gate, and bf represents the bias vector of the forget gate;

[0147] ot represents the output gate value at time step t (used to control which parts of the cell state will be output as hidden states), Wo represents the weight matrix of the output gate, and bo represents the bias vector of the output gate;

[0148] c1t represents the candidate cell state value at time step t (used to represent new information that may be added to the cell state at the current time step), tanh represents the hyperbolic tangent activation function (used to map the input to the [-1, 1] interval), Wc represents the weight matrix of the candidate cell state, and bc represents the bias vector of the candidate cell state;

[0149] ct represents the cell state value at time step t (used to store long-term memory), ct-1 represents the cell state at the previous time step, ft*ct-1 represents how much past cell state information is retained, and it*c1t represents how much new candidate information is added to the cell state;

[0150] ht represents the hidden state value at time step t (i.e., the predicted compensation amount), tanhct represents the application of the tanh function to the current cell state and maps it to the interval [-1, 1], and ot *tanhct represents the output gate that determines which information in the hidden state is important;

[0151] S53. Calculate the calibration laboratory data based on the normal regression prediction value combined with the predicted compensation amount; the calculation formula is as follows:

[0152] ;

[0153] Where f represents the calibration laboratory data, d represents the normal regression predicted value, and ht represents the predicted compensation amount.

[0154] Example 2:

[0155] See also Figure 4 , a laboratory test data processing system based on machine learning technology, used to implement the above-mentioned laboratory test data processing method based on machine learning technology, including a multi-source data acquisition and dynamic noise reduction module, a multimodal feature fusion and selection module, a dual-model collaborative training and optimization module, an anomaly perception regression prediction module and an environmental compensation dynamic calibration module;

[0156] The multi-source data acquisition and dynamic noise reduction module is used for real-time acquisition and preprocessing of multi-source signals and environmental parameters in the laboratory. It acquires structured and unstructured data through an industrial bus or API interface, and simultaneously collects environmental parameters such as temperature, humidity, and voltage. It decomposes coefficients through discrete wavelet transform, estimates the noise standard deviation based on the median of the highest frequency coefficient, dynamically calculates the threshold, and then shrinks the noise coefficient and reconstructs the signal through a soft threshold function, ultimately outputting the denoised multi-source data.

[0157] The multimodal feature fusion and selection module is used to extract and fuse frequency domain and spatial features from preprocessed data; for time series signals, local statistics are calculated through a sliding window and FFT frequency domain features are extracted; for spatial data, the SIFT algorithm is used to extract key point descriptors; the fused multi-source features are optimized through mutual information feature selection, the mutual information score of the features and the target variable is calculated, high-scoring features are retained, redundant features are eliminated, and effective laboratory features are output;

[0158] The dual-model collaborative training and optimization module is used to build and train dual models for anomaly detection and quantitative regression. Based on the graph convolutional network and the attention mechanism, a graph structure is constructed, and the weight parameters are optimized using random search cross-validation. Random sampling is performed in the parameter space, and performance is evaluated through W-fold cross-validation. The optimal weights are iteratively updated to obtain the final anomaly detection classification model. By fusing XGBoost and LightGBM, training is carried out in two stages: first, XGBoost is used to predict the base value, and then LightGBM is used to learn the residual. Finally, the weighted fusion coefficient is optimized on the validation set through grid search, and the weight that minimizes the RMSE is selected to obtain the final quantitative regression model.

[0159] The anomaly perception regression prediction module is used to implement dual-model collaborative reasoning and anomaly filtering; input valid features into the final anomaly detection classification model and the final quantitative regression model respectively, output anomaly probability and regression prediction value; set an anomaly probability threshold, mark abnormal samples; eliminate the regression prediction value corresponding to the abnormal sample, and output a pure normal regression prediction result;

[0160] The environmental compensation dynamic calibration module eliminates environmental interference based on the LSTM network; inputs environmental parameters into the LSTM and calculates the predicted compensation amount through a gating mechanism; designs an incremental learning mechanism to optimize the LSTM; superimposes the normal regression prediction value and the compensation amount, and outputs the final laboratory data after environmental calibration.

[0161] Throughout this specification, references to terms such as "one embodiment," "example," or "specific example" indicate that the specific features, structures, materials, or characteristics described in conjunction with that embodiment or example are included in at least one embodiment or example of the invention. In this specification, schematic representations of these terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0162] The preferred embodiments of the invention disclosed above are intended only to help illustrate the invention. These preferred embodiments do not exhaust all details, nor do they limit the invention to the specific embodiments described. Obviously, many modifications and variations are possible based on the content of this specification. These embodiments are selected and described in detail in this specification to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention.

Claims

1. A method for processing laboratory test data based on machine learning technology, characterized in that: The following steps are involved: S1. Collect laboratory multi-source signal data and environmental parameter data; Perform dynamic noise reduction processing on laboratory multi-source signal data to obtain pre-processed laboratory multi-source data; S2, extracting frequency domain features and spatial features from preprocessed laboratory multi-source data, and fusing them to obtain laboratory multi-source features; Perform mutual information feature selection on multi-source laboratory features to obtain effective laboratory features; S3, build anomaly detection classification model and quantitative regression model; The anomaly detection classification model and the quantitative regression model are trained separately to obtain the final anomaly detection classification model and the final quantitative regression model; The S3 includes the following steps: S31. Build an anomaly detection classification model based on graph convolutional networks and attention mechanisms; set the weight parameters of the anomaly detection classification model; Based on XGBoost and LightGBM, a quantitative regression model is constructed; the quantitative regression model adopts the weighted fusion strategy of XGBoost and LightGBM dual models; S32. Collect historical laboratory characteristic data; S33, inputting historical laboratory feature data into an anomaly detection classification model for training; During the training process, the random search and cross-validation algorithms are combined to find the optimal weight parameters of the anomaly detection classification model and obtain the optimal solution. The optimal solution is used as the weight parameter of the anomaly detection classification model to obtain the final anomaly detection classification model. S34, inputting the historical laboratory characteristic data into the quantitative regression model for training, and optimizing the parameters of the quantitative regression model in combination with grid search to obtain the final quantitative regression model; In the training process in S33, the optimal weight parameters of the anomaly detection classification model are found by combining random search and cross-validation algorithms to obtain the optimal solution, which includes the following steps: S331, set the weight parameter search space of the anomaly detection classification model to p, the maximum number of random searches to m, and the cross-validation fold to W; S332. Randomly set the weight parameters of the anomaly detection classification model according to the weight parameter search space p to obtain an initial weight parameter set; Initialize the optimal weight parameter of the anomaly detection classification model to u and the best performance to v, and randomly select the weight parameter of the anomaly detection classification model to be e from the initial weight parameter set; The training data in the historical laboratory feature data is divided into W folds. For each fold w, the training data in all the historical laboratory feature data except the w-th fold is used to train the anomaly detection classification model. The training data in the w-th fold is used to evaluate the anomaly detection classification model to obtain a set of performance indicators. Calculate the mean of the performance indicators of all folds to get the average performance r, If the average performance r> the best performance v, then update the best performance to r and the optimal weight parameter to e; otherwise, keep the original best performance and optimal weight parameter; S334, repeat S333, and when the maximum number of random searches m is reached, stop the iteration and obtain the optimal solution; S4. Use the final anomaly detection classification model and the final quantitative regression model to perform dual-model collaborative analysis on the effective laboratory characteristics to obtain the anomaly detection results and regression prediction values; According to the anomaly detection results, the abnormal data in the regression prediction value is eliminated to obtain the normal regression prediction value; S5. Based on the environmental parameter data, the predicted compensation amount is obtained in combination with the LSTM network; based on the normal regression prediction value and the predicted compensation amount, the calibration laboratory data is obtained.

2. The method for processing laboratory test data based on machine learning technology according to claim 1, characterized in that: Said S1 comprises the following steps: S11. Acquire laboratory multi-source signal data in real time through an industrial bus or API interface; the laboratory multi-source signal data includes structured signal data and unstructured signal data; Synchronously collect environmental parameter data to obtain environmental parameter data; S12. Perform dynamic noise reduction processing on the laboratory multi-source signal data to obtain preprocessed laboratory multi-source data.

3. The method for processing laboratory test data based on machine learning technology according to claim 2, characterized in that: The S12 includes the following steps: S121, performing wavelet transform on the laboratory multi-source signal data to obtain a wavelet coefficient set; the wavelet coefficients include approximate coefficients and detail coefficients; S122, calculating the noise standard deviation based on the wavelet coefficient set; S123, calculating a real-time dynamic threshold according to the noise standard deviation; S124, processing each wavelet coefficient in the wavelet coefficient set by a soft threshold function according to the real-time dynamic threshold to obtain a filtered wavelet coefficient set; S125. Reconstruct the wavelet signal in the filtered wavelet coefficient set through inverse wavelet transform to obtain preprocessed laboratory multi-source data.

4. The method for processing laboratory test data based on machine learning technology according to claim 1, characterized in that: The S2 comprises the following steps: S21, calculating the local statistics of the time series signal data in the preprocessing laboratory multi-source data through a sliding window to obtain frequency domain features; SIFT algorithm is used to extract key point descriptors of spatial data from preprocessed laboratory multi-source data to obtain spatial features; The frequency domain features and spatial features together constitute laboratory multi-source features; S22, calculate the mutual information score set between the features and target variables of the preprocessed laboratory multi-source data; Set the retention percentage; retain the valid features in the mutual information score set according to the retention percentage to obtain the valid laboratory features.

5. The method for processing laboratory test data based on machine learning technology according to claim 1, characterized in that: The S4 comprises the following steps: S41. Input the effective laboratory characteristics into the final anomaly detection classification model and the final quantitative regression model respectively to obtain anomaly detection results and regression prediction values; S42. Setting an abnormal probability threshold; marking valid laboratory features in the abnormal detection results whose abnormal probability is greater than or equal to the abnormal probability threshold as abnormal data; Eliminate the regression prediction values corresponding to abnormal data from the regression prediction values to obtain normal regression prediction values.

6. The method for processing laboratory test data based on machine learning technology according to claim 1, characterized in that The S5 comprises the following steps: S51. Build an LSTM network; set calibration rules; and update the LSTM network according to the calibration rules. S52, inputting the environmental parameter data into the LSTM network to obtain the predicted compensation amount; S53. Calculate the calibration laboratory data based on the normal regression prediction value and the predicted compensation amount.

7. A laboratory test data processing system based on machine learning technology, characterized in that: Implementing a method for processing laboratory test data based on machine learning technology as described in any one of claims 1 to 6, the system includes a multi-source data acquisition and dynamic noise reduction module, a multimodal feature fusion and selection module, a dual-model collaborative training and optimization module, an abnormal perception regression prediction module, and an environmental compensation dynamic calibration module; The multi-source data acquisition and dynamic noise reduction module is used for real-time acquisition and preprocessing of multi-source signals and environmental parameters in the laboratory; it acquires structured and unstructured data through the industrial bus or API interface and synchronously acquires environmental parameters; The coefficients are decomposed by discrete wavelet transform, the noise standard deviation is estimated based on the median of the highest frequency coefficient, the threshold is dynamically calculated, and the noise coefficient is shrunk and the signal is reconstructed through the soft threshold function, and finally the multi-source data after noise reduction is output; The multimodal feature fusion and selection module is used to extract and fuse frequency domain and spatial features from preprocessed data; for time series signals, local statistics are calculated through a sliding window and FFT frequency domain features are extracted; for spatial data, the SIFT algorithm is used to extract key point descriptors; the fused multi-source features are optimized through mutual information feature selection, the mutual information score of the features and the target variable is calculated, high-scoring features are retained, redundant features are eliminated, and effective laboratory features are output; The dual-model collaborative training and optimization module is used to build and train dual models for anomaly detection and quantitative regression. Based on the graph convolutional network and the attention mechanism, a graph structure is constructed, and the weight parameters are optimized using random search cross-validation. Random sampling is performed in the parameter space, and performance is evaluated through W-fold cross-validation. The optimal weights are iteratively updated to obtain the final anomaly detection classification model. By fusing XGBoost and LightGBM, training is carried out in two stages: first, XGBoost is used to predict the base value, and then LightGBM is used to learn the residual. Finally, the weighted fusion coefficient is optimized on the validation set through grid search, and the weight that minimizes the RMSE is selected to obtain the final quantitative regression model. The anomaly perception regression prediction module is used to realize dual-model collaborative reasoning and anomaly filtering; the effective features are input into the final anomaly detection classification model and the final quantitative regression model respectively, and the anomaly probability and regression prediction value are output; Set the abnormal probability threshold and mark abnormal samples; remove the regression prediction values corresponding to abnormal samples and output pure normal regression prediction results; The environmental compensation dynamic calibration module eliminates environmental interference based on the LSTM network; Environmental parameters are input into LSTM, and the predicted compensation amount is calculated through the gating mechanism; an incremental learning mechanism is designed to optimize LSTM; the normal regression prediction value and the compensation amount are superimposed to output the final laboratory data after environmental calibration.

8. A storage medium, characterized in that: A program is stored thereon, and when the program is executed by a processor, the method for processing laboratory test data based on machine learning technology as described in any one of claims 1 to 7 is implemented.

Images