Detection data processing method based on LIMS system
The proposed method automates data processing in LIMS systems using data cleaning, analysis, and machine learning to enhance accuracy and efficiency in handling multi-plate and multi-well test results, reducing manual errors and enabling real-time data visualization.
Patent Information
- Application Number
- CN202411318373.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-20
- Publication Date
- 2024-12-31
AI Technical Summary
Current LIMS systems face inefficiencies and deviations in processing multi-plate and multi-well test results due to manual test result processing, which increases workload and delays report issuance.
A detection data processing method for LIMS systems involving data acquisition, cleaning, analysis, model building, evaluation, and visualization, utilizing techniques like PCA, LDA, and machine learning models for automated data processing.
Improves data accuracy, reliability, and efficiency by automating data processing, reducing manual errors, and enabling real-time data analysis and visualization.
Abstract
Description
Technical Field
[0001] The present invention relates to the field of detection technology, and more specifically, to a detection data processing method based on a LIMS system. Background Art
[0002] In order to complete the collection, analysis, reporting and management of laboratory data and information, existing laboratories often use LIMS (Laboratory Information Management System) for intelligent management. LIMS is based on a computer local area network and is specially designed for the overall environment of a laboratory. It is an efficient integrated system that includes signal acquisition equipment, data communication software, and database management software. With the laboratory as the center, it organically combines the laboratory's business processes, environment, personnel, instruments and equipment, standard substances and solutions, chemical reagents, standard methods, books and materials, document records, scientific research management, project management, customer management and other factors.
[0003] In the current common LIMS system, when the test items use multi-plate and multi-well detection methods to generate multiple sets of test data, the test results need to be processed manually. Manual processing of test results increases the workload of inspectors and the probability of deviations. Failure to process test results in a timely manner may cause delays in the issuance of test reports. Therefore, the existing multi-plate and multi-well test result processing methods make the test efficiency low and the test is prone to deviations.
[0004] Therefore, a detection data processing method based on LIMS system is proposed. Summary of the invention
[0005] In order to overcome the above-mentioned defects of the prior art, the present invention provides a detection data processing method based on a LIMS system to solve the problems raised in the above-mentioned background technology.
[0006] To achieve the above object, the present invention provides the following technical solution: a detection data processing method based on a LIMS system, the data processing method comprising the following steps:
[0007] S1: Data acquisition, using the LIMS system to collect sensor data or other detection data and convert the data into a processable form;
[0008] S2: Data cleaning: clean the collected data and remove unreasonable data such as outliers and missing values;
[0009] S3: Data analysis, analyzing the cleaned data and extracting useful information;
[0010] S4: Model building: Based on the analysis results, build the corresponding model for prediction or classification;
[0011] S5: Model evaluation: Evaluate the established model to check its accuracy and reliability;
[0012] S6: Result display: The processed data and model results are displayed in a visual form for users to view and analyze.
[0013] Preferably, after cleaning the data in S2, the data needs to be converted into a format suitable for processing.
[0014] Preferably, in S3, useful features are extracted according to specific task requirements, such as extracting the location of the target object from the image and extracting keywords from the text.
[0015] Preferably, normalization processing is performed based on the extracted features so that data of different scales are comparable, for example, the temperature value is normalized to a range of 0 to 1.
[0016] Preferably, the high-dimensional data contained in the feature data extracted in S3 is processed using a dimensionality reduction method, such as principal component analysis (PCA) and linear discriminant analysis (LDA), to reduce the dimension of the data and improve data processing efficiency.
[0017] Preferably, the model building steps in S4 are: determining the prediction or classification target: according to the analysis results, determining the target to be predicted or classified, selecting the model type, dividing the training set and the test set, feature extraction and selection, model training and parameter adjustment, model evaluation and verification, and application of the model.
[0018] Preferably, when selecting the model type, a suitable model type is selected according to the prediction or classification target, and the model type can be a classification model of logistic regression, support vector machine, or decision tree, or a regression model of random forest or gradient boosting tree.
[0019] Preferably, when dividing the training set and the test set, the data set is divided into a training set and a test set for evaluating and verifying the established model, the training set is used to train the model, and the test set is used to evaluate the accuracy and generalization ability of the model.
[0020] Preferably, the model training and parameter adjustment are to train the established model through a training set and adjust the parameters to improve the accuracy and generalization ability of the model.
[0021] Preferably, in the model evaluation and verification, a test set is used to evaluate and verify the established model to check the prediction or classification accuracy and generalization ability of the model.
[0022] The present invention provides a method for processing detection data based on a LIMS system, which has the following beneficial effects:
[0023] Improve data accuracy and reliability: Through methods such as data preprocessing and data analysis, interference factors such as noise and outliers in the data can be cleaned up, thereby improving data accuracy and reliability.
[0024] Improve data processing efficiency: The LIMS system can distribute data processing tasks to multiple computers for parallel processing, thereby greatly improving data processing efficiency.
[0025] Easy data management and analysis: The LIMS system can store data in a standardized format in a cloud server, making it easier for users to manage and analyze data.
[0026] Support multi-device data collection: The LIMS system supports data collection from multiple types of sensors and devices to meet the needs of different application scenarios.
[0027] Facilitates data sharing and collaboration: The LIMS system can store data in the cloud, making it convenient for multiple users to share and collaborate on data. DETAILED DESCRIPTION
[0028] Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in the field without making any creative work shall fall within the scope of protection of the present invention.
[0029] Example 1
[0030] This embodiment 1 provides a method for processing detection data based on a LIMS system, and the data processing method comprises the following steps:
[0031] S1: Data acquisition, using the LIMS system to collect sensor data or other detection data and convert the data into a processable form;
[0032] S2: Data cleaning: clean the collected data to remove unreasonable data such as outliers and missing values. After cleaning the data, it is necessary to convert the data into a format suitable for processing;
[0033] S3: Data analysis: Analyze the cleaned data to extract useful information. According to specific task requirements, extract useful features, such as extracting the location of the target object from the image and extracting keywords from the text. Perform normalization processing based on the extracted features to make data of different scales comparable. For example, normalize the temperature value to the range of 0 to 1. Use dimensionality reduction methods such as principal component analysis (PCA) and linear discriminant analysis (LDA) to reduce the dimension of the high-dimensional data contained in the extracted feature data, thereby reducing the dimension of the data and improving data processing efficiency.
[0034] S4: Model building: Based on the analysis results, build the corresponding model for prediction or classification;
[0035] In specific implementation, the steps of model establishment are:
[0036] Determine the prediction or classification target: Based on the analysis results, determine the target that needs to be predicted or classified. For example, for blood glucose test data, the goal may be to predict whether the patient has diabetes.
[0037] Select model type: Select the appropriate model type based on the prediction or classification target. For example, you can select the logistic regression classification model. The algorithm formula of the logistic regression model is:
[0038] $$
[0039] y=\frac{1}{1+e^{-z}}
[0040] $$
[0041] Among them, $y$ represents the predicted probability value, $z$ represents the weighted sum of the input features, and $e$ represents a natural constant.
[0042] Specifically, $z$ can be expressed as:
[0043] $$
[0044] z=w_1x_1+w_2x_2+…+w_px_p
[0045] $$
[0046] Among them, $w_1,w_2,…,w_p$ represent the weights of the $1st, 2nd, …, pth $features, and $x_1,x_2,…,x_p$ represent the values of the $1st, 2nd, …, pth $features.
[0047] Substituting $z$ into $y$, we can get the predicted probability value:
[0048] $$
[0049] y=\frac{1}{1+e^{-z}}=\frac{1}{1+e^{w_1x_1+w_2x_2+…+w_px_p}}
[0050] $$
[0051] It can be seen that the prediction results of the logistic regression model are related to the weights and input features.
[0052] Divide the dataset into training and test sets: Divide the dataset into training and test sets to evaluate and validate the established model. The training set is used to train the model, and the test set is used to evaluate the accuracy and generalization ability of the model.
[0053] Feature extraction and selection: Based on the analysis results, extract and select features related to the prediction or classification target. For example, for blood sugar test data, you can extract features such as the patient's weight, age, and blood pressure.
[0054] Model training and parameter adjustment: Use the training set to train the established model and adjust the parameters to improve the accuracy and generalization ability of the model.
[0055] Model evaluation and validation: Use the test set to evaluate and validate the established model to check the model’s prediction or classification accuracy and generalization ability.
[0056] Apply the model: Apply the established model to actual scenarios for prediction or classification. For example, for diabetic patients, the established model can be used for real-time blood sugar prediction and classification;
[0057] S5: Model evaluation: Evaluate the established model to check its accuracy and reliability;
[0058] S6: Result display: The processed data and model results are displayed in a visual form for users to view and analyze.
[0059] Example 2
[0060] This embodiment 2 provides a detection data processing method based on a LIMS system, and the data processing method comprises the following steps:
[0061] S1: Data acquisition, using the LIMS system to collect sensor data or other detection data and convert the data into a processable form;
[0062] S2: Data cleaning: clean the collected data to remove unreasonable data such as outliers and missing values. After cleaning the data, it is necessary to convert the data into a format suitable for processing;
[0063] S3: Data analysis: Analyze the cleaned data to extract useful information. According to specific task requirements, extract useful features, such as extracting the location of the target object from the image and extracting keywords from the text. Perform normalization processing based on the extracted features to make data of different scales comparable. For example, normalize the temperature value to the range of 0 to 1. Use dimensionality reduction methods such as principal component analysis (PCA) and linear discriminant analysis (LDA) to reduce the dimension of the high-dimensional data contained in the extracted feature data, thereby reducing the dimension of the data and improving data processing efficiency.
[0064] S4: Model building: Based on the analysis results, build the corresponding model for prediction or classification;
[0065] In specific implementation, the steps of model establishment are:
[0066] Determine the prediction or classification target: Based on the analysis results, determine the target that needs to be predicted or classified. For example, for blood glucose test data, the goal may be to predict whether the patient has diabetes.
[0067] Select model type: Select the appropriate model type according to the prediction or classification target. You can select the support vector machine classification model, the classification model algorithm formula is:
[0068] $$
[0069] f(x)=\frac{1}{2}w^Tx+b
[0070] $$
[0071] Among them, $f(x)$ represents the prediction function, $w$ represents the weight vector, and $b$ represents the bias term. This formula shows how to map the input features to a high-dimensional space and find an optimal hyperplane in the space to maximize the interval between different categories.
[0072] In the support vector machine model, the optimal hyperplane can be expressed as:
[0073] $$
[0074] w^Tx+b=\max
[0075] $$
[0076] Among them, $x$ represents the input feature vector, $w$ represents the weight vector, and $b$ represents the bias term. This formula shows how to find a hyperplane in high-dimensional space so that sample points of different categories can be separated to the greatest extent.
[0077] The optimization goal of the support vector machine model is to minimize the classification error and maximize the margin to obtain better generalization ability and accuracy. During the training process, the support vector machine model obtains the optimal hyperplane by solving the Lagrange multiplier method.
[0078] Divide the dataset into training and test sets: Divide the dataset into training and test sets to evaluate and validate the established model. The training set is used to train the model, and the test set is used to evaluate the accuracy and generalization ability of the model.
[0079] Feature extraction and selection: Based on the analysis results, extract and select features related to the prediction or classification target. For example, for blood sugar test data, you can extract features such as the patient's weight, age, and blood pressure.
[0080] Model training and parameter adjustment: Use the training set to train the established model and adjust the parameters to improve the accuracy and generalization ability of the model.
[0081] Model evaluation and validation: Use the test set to evaluate and validate the established model to check the model’s prediction or classification accuracy and generalization ability.
[0082] Apply the model: Apply the established model to actual scenarios for prediction or classification. For example, for diabetic patients, the established model can be used for real-time blood sugar prediction and classification;
[0083] S5: Model evaluation: Evaluate the established model to check its accuracy and reliability;
[0084] S6: Result display: The processed data and model results are displayed in a visual form for users to view and analyze.
[0085] Example 3
[0086] This embodiment 3 provides a method for processing detection data based on a LIMS system, and the data processing method comprises the following steps:
[0087] S1: Data acquisition, using the LIMS system to collect sensor data or other detection data and convert the data into a processable form;
[0088] S2: Data cleaning: clean the collected data to remove unreasonable data such as outliers and missing values. After cleaning the data, it is necessary to convert the data into a format suitable for processing;
[0089] S3: Data analysis: Analyze the cleaned data to extract useful information. According to specific task requirements, extract useful features, such as extracting the location of the target object from the image and extracting keywords from the text. Perform normalization processing based on the extracted features to make data of different scales comparable. For example, normalize the temperature value to the range of 0 to 1. Use dimensionality reduction methods such as principal component analysis (PCA) and linear discriminant analysis (LDA) to reduce the dimension of the high-dimensional data contained in the extracted feature data, thereby reducing the dimension of the data and improving data processing efficiency.
[0090] S4: Model building: Based on the analysis results, build the corresponding model for prediction or classification;
[0091] In specific implementation, the steps of model establishment are:
[0092] Determine the prediction or classification target: Based on the analysis results, determine the target that needs to be predicted or classified. For example, for blood glucose test data, the goal may be to predict whether the patient has diabetes.
[0093] Select model type: Select the appropriate model type based on the prediction or classification target. You can select the random forest regression model, the algorithm formula of which is:
[0094] The random forest model is an integrated learning model for classification and regression analysis. Its algorithm formula is as follows:
[0095] $$
[0096] y=\frac{1}{1+e^{-z}}
[0097] $$
[0098] Among them, $y$ represents the predicted probability value, $z$ represents the weighted sum of the input features, and $e$ represents a natural constant. Specifically, $z$ can be expressed as:
[0099] $$
[0100] z=w_1x_1+w_2x_2+…+w_px_p
[0101] $$
[0102] Among them, $w_1,w_2,…,w_p$ represents the weights of the 1st, 2nd,…,pth features, and $x_1,x_2,…,x_p$ represents the values of the 1st, 2nd,…,pth features.
[0103] Substituting $z$ into $y$, we can get the predicted probability value:
[0104] $$
[0105] y=\frac{1}{1+e^{w_1x_1+w_2x_2+…+w_px_p}}
[0106] $$
[0107] It can be seen that the prediction results of the random forest model are related to the weights and input features. During the training process, the random forest model optimizes the weight parameters by minimizing the loss function to improve the accuracy and generalization ability of the model.
[0108] Divide the dataset into training and test sets: Divide the dataset into training and test sets to evaluate and validate the established model. The training set is used to train the model, and the test set is used to evaluate the accuracy and generalization ability of the model.
[0109] Feature extraction and selection: Based on the analysis results, extract and select features related to the prediction or classification target. For example, for blood sugar test data, you can extract features such as the patient's weight, age, and blood pressure.
[0110] Model training and parameter adjustment: Use the training set to train the established model and adjust the parameters to improve the accuracy and generalization ability of the model.
[0111] Model evaluation and validation: Use the test set to evaluate and validate the established model to check the model’s prediction or classification accuracy and generalization ability.
[0112] Apply the model: Apply the established model to actual scenarios for prediction or classification. For example, for diabetic patients, the established model can be used for real-time blood sugar prediction and classification;
[0113] S5: Model evaluation: Evaluate the established model to check its accuracy and reliability;
[0114] S6: Result display: The processed data and model results are displayed in a visual form for users to view and analyze.
[0115] Example 4
[0116] This embodiment 4 provides a method for processing detection data based on a LIMS system, and the data processing method comprises the following steps:
[0117] S1: Data acquisition, using the LIMS system to collect sensor data or other detection data and convert the data into a processable form;
[0118] S2: Data cleaning: clean the collected data to remove unreasonable data such as outliers and missing values. After cleaning the data, it is necessary to convert the data into a format suitable for processing;
[0119] S3: Data analysis: Analyze the cleaned data to extract useful information. According to specific task requirements, extract useful features, such as extracting the location of the target object from the image and extracting keywords from the text. Perform normalization processing based on the extracted features to make data of different scales comparable. For example, normalize the temperature value to the range of 0 to 1. Use dimensionality reduction methods such as principal component analysis (PCA) and linear discriminant analysis (LDA) to reduce the dimension of the high-dimensional data contained in the extracted feature data, thereby reducing the dimension of the data and improving data processing efficiency.
[0120] S4: Model building: Based on the analysis results, build the corresponding model for prediction or classification;
[0121] In specific implementation, the steps of model establishment are:
[0122] Determine the prediction or classification target: Based on the analysis results, determine the target that needs to be predicted or classified. For example, for blood glucose test data, the goal may be to predict whether the patient has diabetes.
[0123] Select model type: Select the appropriate model type based on the prediction or classification target. You can use the gradient boosting tree regression model, the regression model algorithm formula is:
[0124] $$
[0125] y=g(z)
[0126] $$
[0127] Among them, $y$ represents the predicted value, $z$ represents the weighted sum of the input features, and $g$ represents a nonlinear function. This formula shows how to map the input features to the predicted value.
[0128] In the gradient boosting tree model, each tree is a local model, and the prediction value of each tree is related to the weighted sum of the input features. During the training process, the model optimizes the weight parameters through continuous iteration to improve the accuracy and generalization ability of the model.
[0129] The optimization goal of the gradient boosting tree model is to minimize the loss function while maximizing the margin to obtain better generalization ability and accuracy. During the training process, the model obtains the optimal hyperplane by solving the Lagrange multiplier method.
[0130] Divide the dataset into training and test sets: Divide the dataset into training and test sets to evaluate and validate the established model. The training set is used to train the model, and the test set is used to evaluate the accuracy and generalization ability of the model.
[0131] Feature extraction and selection: Based on the analysis results, extract and select features related to the prediction or classification target. For example, for blood sugar test data, you can extract features such as the patient's weight, age, and blood pressure.
[0132] Model training and parameter adjustment: Use the training set to train the established model and adjust the parameters to improve the accuracy and generalization ability of the model.
[0133] Model evaluation and validation: Use the test set to evaluate and validate the established model to check the model’s prediction or classification accuracy and generalization ability.
[0134] Apply the model: Apply the established model to actual scenarios for prediction or classification. For example, for diabetic patients, the established model can be used for real-time blood sugar prediction and classification;
[0135] S5: Model evaluation: Evaluate the established model to check its accuracy and reliability;
[0136] S6: Result display: The processed data and model results are displayed in a visual form for users to view and analyze.
[0137] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A detection data processing method based on a LIMS system, characterized in that: The data processing method comprises the following steps: S1: Data acquisition, using the LIMS system to collect sensor data or other detection data and convert the data into a processable form; S2: Data cleaning: clean the collected data and remove unreasonable data such as outliers and missing values; S3: Data analysis, analyzing the cleaned data and extracting useful information; S4: Model building: Based on the analysis results, build the corresponding model for prediction or classification; S5: Model evaluation: Evaluate the established model to check its accuracy and reliability; S6: Result display: The processed data and model results are displayed in a visual form for users to view and analyze.
2. The method for processing detection data based on a LIMS system according to claim 1, characterized in that: After the data is cleaned in S2, it is necessary to convert the data into a format suitable for processing.
3. The method for processing detection data based on a LIMS system according to claim 1, characterized in that: In S3, useful features are extracted according to specific task requirements, such as extracting the location of the target object from the image and extracting keywords from the text.
4. The method for processing detection data based on a LIMS system according to claim 1, characterized in that: Normalization processing is performed based on the extracted features to make data of different scales comparable, for example, the temperature value is normalized to a range of 0 to 1.
5. The method for processing detection data based on a LIMS system according to claim 1, characterized in that: The high-dimensional data contained in the feature data extracted in S3 is processed using a dimensionality reduction method, such as principal component analysis (PCA) and linear discriminant analysis (LDA), to reduce the dimension of the data and improve data processing efficiency.
6. The method for processing detection data based on a LIMS system according to claim 1, characterized in that: The steps of model building in S4 are: determining the prediction or classification target: based on the analysis results, determining the target to be predicted or classified, selecting the model type, dividing the training set and the test set, feature extraction and selection, model training and parameter adjustment, model evaluation and verification, and application of the model.
7. The method for processing detection data based on a LIMS system according to claim 1, characterized in that: When selecting the model type, a suitable model type is selected according to the prediction or classification target. The model type can be a classification model of logistic regression, support vector machine, or decision tree, or a regression model of random forest or gradient boosting tree.
8. The method for processing detection data based on a LIMS system according to claim 1, characterized in that: When dividing the training set and the test set, the data set is divided into a training set and a test set for evaluating and verifying the established model, the training set is used to train the model, and the test set is used to evaluate the accuracy and generalization ability of the model.
9. According to claim 1, a detection data processing method based on the LIMS system is characterized by using: the model training and parameter adjustment, training the established model through the training set, and adjusting the parameters to improve the accuracy and generalization ability of the model.
10. The method for processing detection data based on a LIMS system according to claim 1, characterized in that: In the model evaluation and verification, the established model is evaluated and verified using a test set to check the prediction or classification accuracy and generalization ability of the model.