A method for evaluating the accuracy of measurement methods based on machine learning
By preprocessing and extracting features from spectral data using machine learning methods, a feature analysis model is constructed, and an efficient model is selected. This solves the problem of low efficiency in the accuracy assessment of spectral analysis methods and enables rapid and accurate spectral data assessment.
Patent Information
- Application Number
- CN202410609408.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-05-16
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2044-05-16
AI Technical Summary
The accuracy assessment efficiency of existing spectral analysis methods is low, mainly due to the presence of noise and redundant data in complex spectral data, which leads to low assessment efficiency.
By using machine learning-based methods, the initial spectral data of the spectral measuring instrument is obtained, preprocessed, and features are extracted to construct a feature analysis model. The model performance coefficients are then used to select efficient feature analysis models for evaluating the accuracy of the spectral analysis method.
This invention improves the efficiency of accuracy assessment for spectral analysis methods. Through feature extraction and model selection, it can quickly and accurately assess the accuracy of spectral data, thus solving the problem of low efficiency in existing technologies.
Smart Images

Figure CN118551209B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of data processing, and in particular to an accuracy evaluation method of a measurement method based on machine learning. Background Art
[0002] People often use a variety of test methods to achieve qualitative and quantitative requirements. Different requirements require different test methods. The field of chemistry involves a lot of analysis of properties and structures. Common chemical test methods include quantitative analysis, qualitative analysis, spectral analysis, chromatographic analysis, mass spectrometry, electrochemical analysis, etc. These methods analyze the properties, composition, structure, etc. of substances to achieve the purpose of detection, identification, and quantification of samples. It is very important to evaluate the accuracy of chemical test methods because accuracy directly affects the reliability and effectiveness of test results. Inaccurate test methods will affect the scientific nature and application value of experimental results. Machine learning is a branch of artificial intelligence that enables computers to learn from data and make predictions or decisions by building and training models. In the accuracy evaluation of chemical test methods, machine learning can be used to process large amounts of experimental data, establish prediction models for specific chemical test methods to predict new experimental results, and optimize the design and parameter selection of test methods to improve the accuracy and efficiency of test methods.
[0003] The accuracy of chemical test methods mainly includes applicability, accuracy, and repeatability. Specifically, the accuracy can be evaluated by combining control experiments, reference standards, and error analysis. For example, a control experiment can be conducted to compare the results obtained with known results to verify the accuracy of the test method. Error analysis can also be performed on experimental data to identify possible sources of error and take measures to correct and improve them.
[0004] For example, the process industry energy consumption optimization decision-making system and the method for improving the accuracy of evaluation results announced by the invention patent with announcement number: CN112734284B include: user parameter configuration module, modeling and analysis module, product and service release module and decision analysis visualization module; the user parameter configuration module can automatically obtain relevant parameters from the template analysis code and generate a visual configuration interface to improve the system's usability and generalization ability; the modeling and analysis module can use machine learning methods to extract normal operating data from historical data, build an energy consumption analysis model to evaluate whether the energy consumption at the current point is abnormal, and calculate the energy saving improvement space at the current point.
[0005] For example, the invention patent announcement with announcement number: CN116308762B discloses an artificial intelligence-based credibility assessment and credit processing method, including: determining the number of historical identity leakages based on the user's face recognition results, and taking the difference between the user's most recent identity leakage time and the current time as the leakage time difference, and when it is determined that there is no risk of identity misuse based on the leakage time difference and the number of historical identity leakages, the corrected similarity of face recognition is determined based on the user's face recognition similarity, the accuracy of the face recognition model, and the image clarity of face recognition, and in combination with the number of historical identity leakages and the type of mobile terminal, an evaluation model based on a machine learning algorithm is used to obtain the user's credibility, and based on the user's credibility and the basic credit limit, the user's credit limit is obtained, and remote credit processing is completed.
[0006] However, in the process of implementing the technical solution of the invention in the embodiments of the present application, the present application found that the above technology has at least the following technical problems:
[0007] In the prior art, the accuracy assessment of spectral analysis methods involves the analysis of a lot of complex spectral data, and there is noise and redundant data in the spectral data, which will significantly reduce the efficiency of accuracy assessment, resulting in the problem of low efficiency of accuracy assessment of spectral analysis methods. Summary of the invention
[0008] The embodiment of the present application solves the problem of low accuracy assessment efficiency of spectral analysis methods in the prior art by providing an accuracy assessment method for a measurement method based on machine learning, thereby improving the accuracy assessment efficiency of spectral analysis methods.
[0009] The embodiment of the present application provides an accuracy assessment method for a measurement method based on machine learning, comprising the following steps: preprocessing initial spectral data obtained by a spectral measuring instrument to obtain first spectral data, and performing feature extraction on the first spectral data to obtain first spectral feature data; comparing the obtained first spectral feature data with the first spectral data to obtain spectral feature coefficients, and judging whether the obtained spectral feature coefficients meet feature extraction conditions, if not, re-performing feature extraction, otherwise constructing a first spectral feature data set based on the first spectral feature data, the spectral feature coefficients being used to describe the accuracy of feature extraction of the first spectral data; training and verifying a candidate model through the first spectral feature data set to obtain a first feature analysis model, and screening the first feature analysis model through the model performance coefficient of the first feature analysis model to obtain a second feature analysis model, the model performance coefficient being used to describe the performance of the first feature analysis model; analyzing the spectral data input within a preset time period through the second feature analysis model, and feeding back the analysis results to a preset person for subsequent analysis.
[0010] Furthermore, the method for obtaining the spectral characteristic coefficient is as follows: the first spectral data is analyzed according to the principal component analysis method and the independent component analysis method to obtain the second spectral characteristic data; the first spectral characteristic data is compared with the second spectral characteristic data to obtain characteristic analysis data, and the characteristic analysis data includes a characteristic mean absolute error and a characteristic correlation coefficient; the feature extraction weight is obtained according to the subjective weighting method, and the feature extraction weight includes an accuracy weight and a consistency weight, and the spectral characteristic coefficient is obtained in combination with the obtained characteristic analysis data.
[0011] Furthermore, the specific calculation formula of the spectral characteristic coefficient is as follows:
[0012]
[0013] In the formula, m is the number of times the feature is extracted, m = 1, 2, ..., M, M is the total number of times the feature is extracted, GT m is the spectral characteristic coefficient of the mth feature extraction, GA m is the mean absolute error of the features extracted for the mth time, GX m is the feature correlation coefficient of the mth feature extraction, GA0 is the threshold of the feature mean absolute error, GX0 is the threshold of the feature correlation coefficient, α1 is the accuracy weight, and α2 is the consistency weight. Further,
[0014] Furthermore, the model performance coefficient is calculated using the following formula:
[0015]
[0016] Where q is the number of the first feature analysis model, q = 1, 2, ..., Q, Q is the total number of the first feature analysis models, MX q is the model performance coefficient of the qth first characteristic analysis model, MT q is the model time proportion of the qth first feature analysis model, MZ q is the model accuracy coefficient of the qth first feature analysis model, MZ0 is the reference model accuracy coefficient, δ is the model time proportion factor, and θ is the model accuracy factor.
[0017] Furthermore, the model accuracy coefficient is calculated using the following formula: determine the corresponding significant analysis deviation ratio based on the substance type and substance content obtained by inputting the verification set into the first feature analysis model; judge whether the significant analysis deviation ratio is greater than the threshold of the significant analysis deviation ratio, if it is, the corresponding model accuracy coefficient of the first feature analysis model is 0, otherwise the model verification data is obtained by combining the actual substance type and actual substance content corresponding to the verification set, and the average model verification data is obtained based on this, and the average model verification data includes the average substance type deviation and the average substance content deviation; obtain the corresponding F1 score through the confusion matrix of the first feature analysis model, and obtain the model accuracy coefficient of the first feature analysis model in combination with the obtained significant analysis deviation ratio and the average model verification data.
[0018] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:
[0019] 1. The initial spectral data of the designated experimental area is obtained by a spectral measurement instrument, and preprocessed and feature extracted to obtain the first spectral feature data. At the same time, the feature extraction is ensured to meet the feature extraction conditions according to the obtained spectral feature coefficients. Then, the first feature analysis model is constructed through the first spectral feature data, and the second feature analysis model is obtained by screening according to the model performance coefficient. Finally, the accuracy of the spectral analysis method is evaluated through the second feature analysis model, thereby realizing the rapid evaluation of the accuracy of the spectral data, and then realizing the improvement of the accuracy evaluation efficiency of the spectral analysis method, and solving the problem of low efficiency of the accuracy evaluation of the spectral analysis method in the prior art.
[0020] 2. The first spectral data are analyzed by principal component analysis and independent component analysis respectively to obtain the second spectral feature data, and then the first spectral feature data are compared with the second spectral feature data to obtain feature analysis data. Then, the feature extraction weight is obtained according to the subjective weighting method, and finally, the spectral feature coefficient is obtained by combining the obtained feature analysis data, thereby realizing the accuracy of numerical evaluation of feature extraction, and then realizing more accurate acquisition of the first spectral feature data.
[0021] 3. The model time ratio is obtained through the training process of the first feature analysis model, and then the model verification data is obtained according to the verification result of the first feature analysis model. Then, the model accuracy coefficient is obtained in combination with the F1 score of the first feature analysis model. Finally, the model performance coefficient of the first feature analysis model is obtained according to the obtained model time ratio and the model accuracy coefficient, thereby realizing the numerical description of the model performance of the first feature analysis model, and then realizing the acquisition of a more accurate second feature analysis model. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] Figure 1A flow chart of an accuracy assessment method of a measurement method based on machine learning provided in an embodiment of the present application;
[0023] Figure 2 A flowchart for obtaining spectral characteristic coefficients provided in an embodiment of the present application;
[0024] Figure 3 A schematic diagram of the change of spectral characteristic coefficients provided in an embodiment of the present application. DETAILED DESCRIPTION
[0025] The embodiment of the present application solves the problem of low accuracy evaluation efficiency of spectral analysis methods in the prior art by providing an accuracy evaluation method of a measurement method based on machine learning. A comparative experiment is set by obtaining spectral influence data of a specified experimental area and a corresponding reference range of the spectral influence data, and a spectral measuring instrument obtains first initial spectral data of a specified experimental area within a specified time period and second initial spectral data in the comparative experiment, and then the acquired data is preprocessed to obtain first spectral data, and the first spectral data is feature extracted in combination with a preset fitting function after initialization to obtain first spectral feature data, and the first spectral data is analyzed in combination with a principal component analysis method and an independent component analysis method to obtain second spectral feature data, and then spectral feature coefficients of feature extraction are obtained according to the first spectral feature data and the second spectral feature data to ensure that the accuracy of the extraction meets the feature extraction conditions, and then a first feature analysis model is constructed through the first spectral feature data, and the constructed model is screened in combination with the model performance coefficient of the first feature analysis model to obtain a second feature analysis model, and finally, the spectral data input within the preset time period is analyzed through the second feature analysis model to evaluate the accuracy of the spectral analysis method, thereby improving the accuracy evaluation efficiency of the spectral analysis method.
[0026] The technical solution in the embodiment of the present application is to solve the problem of low accuracy evaluation efficiency of the above-mentioned spectral analysis method, and the overall idea is as follows:
[0027] The initial spectral data of a specified experimental area within a specified time period is obtained by a spectral measuring instrument and the first spectral data is preprocessed. The first spectral data is then feature extracted to obtain first spectral feature data. The second spectral feature data obtained by the principal component analysis method and the independent component analysis method are combined to obtain the spectral feature coefficients of feature extraction to ensure that the extraction meets the feature extraction conditions. The first feature analysis model is then constructed through the first spectral feature data, and the second feature analysis model is obtained by screening through the corresponding model performance coefficient. Finally, the accuracy of the spectral analysis method is evaluated through the second feature analysis model, thereby achieving the effect of improving the accuracy evaluation efficiency of the spectral analysis method.
[0028] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.
[0029] like Figure 1 As shown, it is a flow chart of an accuracy assessment method of a measurement method based on machine learning provided in an embodiment of the present application. The accuracy assessment method of a measurement method based on machine learning provided in an embodiment of the present application comprises the following steps: preprocessing initial spectral data acquired by a spectral measuring instrument to obtain first spectral data, and extracting features from the first spectral data to obtain first spectral feature data, the first spectral data representing data after preprocessing the initial spectral data acquired by the spectral measuring instrument, and the first spectral feature data representing data corresponding to preset feature parameters in the first spectral data; comparing the obtained first spectral feature data with the first spectral data to obtain spectral feature coefficients, and judging whether the obtained spectral feature coefficients meet feature extraction conditions, if not, re-performing feature extraction, otherwise constructing a first spectral feature data set according to the first spectral feature data, the spectral feature coefficients being used to describe the first The accuracy of feature extraction from spectral data. The first spectral feature data set is a collection of first spectral feature data. The feature extraction conditions are set by the preset personnel according to the accuracy evaluation requirements of the spectral analysis method. The first feature analysis model is obtained by training and verifying the selected model through the first spectral feature data set, and the first feature analysis model is screened through the model performance coefficient of the first feature analysis model to obtain the second feature analysis model. The model performance coefficient is used to describe the performance of the first feature analysis model. The first feature analysis model is used to analyze the material characteristics corresponding to the spectral data of a specified time period. The second feature analysis model is the only model determined after the first feature analysis model is screened. The second feature analysis model meets the preset requirements, specifically the model performance requirements. The spectral data input within the preset time period is analyzed by the second feature analysis model, and the analysis results are fed back to the preset personnel for subsequent analysis.
[0030] In this embodiment, spectral analysis is used to analyze the structure and composition of a substance, such as infrared spectroscopy (IR), ultraviolet and visible spectrum (UV-Vis), etc.; the purpose of preprocessing the initial spectral data is to remove anomalies and noise in the initial spectral data, and the anomalies specifically include data anomalies, data missing, etc. The spectral data may also be affected by some interference factors, such as baseline drift, instrument noise, etc.; specifically, machine learning can be used to develop efficient data preprocessing algorithms, including noise filtering, baseline correction, signal enhancement, etc., to improve the quality and accuracy of spectral data; the selection of spectral measurement instruments is related to the spectral analysis method, generally including spectrophotometers, spectrometers, mass spectrometers, nuclear magnetic resonance spectrometers, etc., which can measure spectral data in different wavelength ranges, such as visible light, infrared light, ultraviolet light, etc.; the models to be selected include support vector machines, random forests, and deep learning models, and specific models can be selected from various types of The optimal model; before inputting the spectral data within a preset time period into the second feature analysis model, the data must also be preprocessed and feature extracted, wherein the method used for feature extraction is a method in which the spectral feature coefficients meet the feature extraction conditions; during the use of the second feature analysis model, the second feature analysis model can be further optimized according to actual needs and feedback information from the analysis results, so that the function of the second feature analysis model is more consistent with the needs; by constructing a second feature analysis model based on machine learning to analyze the material characteristics corresponding to the spectral data, the accuracy of the spectral analysis method can be evaluated more quickly and accurately according to the analysis results of the model, thereby improving the efficiency of the accuracy evaluation of the spectral analysis method, and effectively solving the problem of low efficiency of the accuracy evaluation of the spectral analysis method in the prior art.
[0031] Furthermore, a specific method for acquiring initial spectral data is as follows: acquiring spectral influence data of a specified experimental area and a corresponding reference range of spectral influence data, the spectral influence data including light intensity, temperature and humidity, the reference range of spectral influence data indicating a variation range of spectral influence data, specifically including a light intensity variation range, a temperature variation range and a humidity variation range, which can be obtained by statistically analyzing the light intensity, temperature and humidity of the specified experimental area in a historical time period to obtain the corresponding variation range; acquiring first initial spectral data of the specified experimental area in a specified time period through a spectral measuring instrument; setting up a comparative experiment in combination with the spectral influence data reference range and the spectral influence data, and acquiring second initial spectral data corresponding to the comparative experiment through a spectral measuring instrument, the purpose of the comparative experiment is to obtain more data for subsequent feature extraction, so as to further train a more accurate model, and except for differences in spectral influence data, other conditions should remain consistent; the initial spectral data include first initial spectral data and second initial spectral data.
[0032] In this embodiment, the light intensity has a direct effect on the absorption, emission or scattering characteristics of the sample. If the light conditions are unstable or uneven, the measurement results of the sample absorption spectrum may be inaccurate; temperature has a great influence on the optical properties of the sample, especially for liquid samples or solid samples; temperature changes may cause changes in the physical properties of the sample, thereby affecting its optical response. When performing spectral analysis, the temperature needs to be controlled to ensure the accuracy of the results; humidity also affects the physical properties of the sample, especially for samples with strong hygroscopicity, changes in humidity may cause changes in the sample structure, thereby affecting its optical properties. In a high humidity environment, the sample may absorb moisture, resulting in inaccurate spectral analysis results; the comparative experiment specifically refers to combining the reference range of the spectral influence data to adjust the spectral influence data. During the adjustment process, the adjustment range can be set each time, such as the temperature is increased by one degree each time in the control experiment, and the humidity and light intensity are set similarly. In the process of setting the control experiment, the rationality of the data should be ensured, that is, the mutual influence between light intensity, temperature and humidity needs to be considered; the increase in the sample size of the initial spectral data is achieved, thereby improving the model performance of the second characteristic analysis model.
[0033] Furthermore, the specific steps of feature extraction are as follows: fit the first spectral data through the preset fitting function after initialization, and obtain the peak position, peak height and peak width according to the fitting result, wherein the preset fitting function generally includes Gaussian function and Lorentz function, and the specific function form can be selected by observing and understanding the first spectral data. Initialization refers to the position, height and width of the initialization peak, and the initial values of these parameters can be estimated by heuristic methods, which can be determined according to actual conditions; the peak intensity is obtained in combination with the peak height, and the relative position and peak ratio of the peak are obtained in combination with the first spectral peak characteristic data. The relative position of the peak refers to the positional relationship between different peaks in the spectrum, and the peak ratio refers to the ratio of relative intensity or absorbance between different peaks, which reflects the relative content or abundance of different components in the sample.
[0034] In this embodiment, usually, the Gaussian function is used to fit the symmetric peak, and the Lorentz function is used to fit the asymmetric peak; common fitting algorithms in the process of fitting spectral data include the least squares method and the nonlinear least squares method, which can optimize the parameters to make the fitting function fit the actual data best; for the Gaussian function, the peak position is the mean obtained by fitting, and the peak width can be calculated by the standard deviation; the first spectral characteristic data includes the peak position, peak height and peak width, peak intensity, relative position of the peaks and peak ratio, the relative position of the peaks involves the distance between the peak top positions and the relative order of the peaks, because the peaks of specific components may appear in a certain relative position, this relative position relationship can provide important information about the properties or composition of the sample, for example, in the nuclear magnetic resonance spectrum, different hydrogen atoms may produce different peaks, and their relative positions can provide information about the molecular structure; the peak ratio can infer the relative amounts of different components in the sample by analyzing the relative ratio of the peaks, which is very important for quantitative analysis. For example, in infrared spectroscopy, the ratio of the absorption peak intensities of different functional groups can be used to estimate their relative contents in the sample; this enables more accurate acquisition of the first spectral characteristic data, thereby more accurately evaluating the accuracy of the spectral analysis method.
[0035] Further, such as Figure 2As shown, it is a flow chart for obtaining spectral characteristic coefficients provided in an embodiment of the present application, and the method for obtaining spectral characteristic coefficients is as follows: according to the principal component analysis method and the independent component analysis method, the first spectral data are analyzed respectively to obtain second spectral characteristic data, and the second spectral characteristic data represent the average values corresponding to the peak position, peak height, peak width, peak intensity, peak relative position and peak ratio extracted by the principal component analysis method and the independent component analysis method; the first spectral characteristic data are compared with the second spectral characteristic data to obtain characteristic analysis data, and the characteristic analysis data include characteristic mean absolute error and characteristic correlation coefficient, the characteristic mean absolute error represents the average absolute error between the first spectral characteristic data and the second spectral characteristic data, and the characteristic correlation coefficient represents the correlation coefficient between the first spectral characteristic data and the second spectral characteristic data; according to the subjective weighting method, the feature extraction weight is obtained, and the feature extraction weight includes accuracy weight and consistency weight, and the spectral characteristic coefficient is obtained in combination with the obtained characteristic analysis data, the accuracy weight represents the influence of the characteristic mean absolute error deviation on the spectral characteristic coefficient, the consistency weight represents the influence of the characteristic correlation coefficient deviation on the spectral characteristic coefficient, and the characteristic mean absolute error deviation and the characteristic correlation coefficient deviation refer to the ratios of the characteristic mean absolute error and the characteristic correlation coefficient to the corresponding average values, respectively.
[0036] In this embodiment, the process of principal component analysis on the first spectral data is as follows: first, the first spectral data is preprocessed to reduce the influence of noise and impurities in the data on the analysis results, and then the preprocessed data is constructed into a data matrix, in which each row represents a sample and each column represents a spectral data point. Finally, principal component analysis is performed on the data matrix to find the principal component (i.e., the main direction of change in the data), and the data is projected into the principal component space; the difference between the independent analysis method and the independent analysis method is that independent component analysis is performed on the data matrix to find independent components in the data, and the data is decomposed into these independent components, and then features are extracted to obtain corresponding data; mean absolute error (MAE) is similar to mean square error, but it is the average value of the calculated absolute value error, which can better reflect the size of the actual error; correlation coefficient (Correl at i on Coeff ici ent) calculates the correlation between the extracted features and the actual data, that is, the linear correlation between the extracted results and the actual results. The closer it is to 1, the more consistent the extracted features are with the actual data. Specifically, the feature mean absolute error is the average of the difference between the corresponding data, and the feature correlation coefficient is the average of the correlation coefficients of the corresponding data; the subjective weighting method refers to a method in which professionals use personal subjective experience to weight different indicators through scoring and grading in a qualitative way. Generally, the higher the score of the indicator data, the greater the corresponding weight. That is, this method uses the relative size information of the scoring numbers to calculate the weight. Commonly used methods in the subjective weighting method include hierarchical analysis method and priority diagram method, which can be determined according to needs, or the average of the two methods; the numeration of the feature extraction effect is realized, thereby improving the accuracy of feature extraction.
[0037] Further, such as Figure 3 As shown, it is a schematic diagram of the change of the spectral characteristic coefficient provided in the embodiment of the present application. In addition to being expressed by directly comparing the difference of the corresponding data, the spectral characteristic coefficient can also be calculated more accurately. The specific calculation formula of the spectral characteristic coefficient is as follows:
[0038]
[0039] In the formula, m is the number of times the feature is extracted, m = 1, 2, ..., M, M is the total number of times the feature is extracted, GT m is the spectral characteristic coefficient of the mth feature extraction, GA m is the mean absolute error of the features extracted for the mth time, GX mis the feature correlation coefficient of the mth feature extraction, GA0 is the threshold of the feature mean absolute error, and its value range is [0.05, 0.15]. GX0 is the threshold of the feature correlation coefficient, and its value range is [0.8, 1]. α1 is the accuracy weight, and its range is (0, 1). α2 is the consistency weight, and its range is also (0, 1), and α1+α2=1.
[0040] In this embodiment, specific values of the threshold of the feature mean absolute error and the threshold of the feature correlation coefficient are determined according to the actual application scenario; Figure 3 Taking α1=0.45、α2=0.55、GA0=0.1、GX0=0.9 as an example, Figure 3 It can be seen that with the increase of the feature mean absolute error, the spectral characteristic coefficient is constantly decreasing, but the decreasing trend is first slow and then fast. When the feature mean absolute error is not greater than the corresponding threshold, the spectral characteristic coefficient is 1; similarly, with the increase of the feature correlation coefficient, the spectral characteristic coefficient is constantly increasing, but the increasing trend is first fast and then slow. When the feature correlation coefficient is not less than the corresponding threshold, the spectral characteristic coefficient is 1; the calculation method of the spectral characteristic coefficient is helpful to process data with nonlinear relationships, which can eliminate data deviations, and at the same time more flexibly handle situations under different conditions, and can more intuitively and accurately describe the accuracy of feature extraction, so that the first spectral feature data can be obtained more accurately, and a more accurate second feature analysis model can be obtained to more accurately and quickly analyze the accuracy of spectral data, thereby achieving an improvement in the efficiency of accuracy assessment of spectral analysis methods.
[0041] Furthermore, a specific construction method of the first feature analysis model is as follows: after data cleaning and normalization of the first spectral feature data, a first spectral feature data set is constructed, and data division is performed to obtain a training set and a validation set; the training set is labeled according to the material feature information of the initial spectral data corresponding to the training set, and the labeled information may be the material composition and content, and the labeled training set is input into the candidate model for training until the model is in a convergence state to obtain the first feature analysis model. During the training process, the training process of the candidate model can also be optimized by the hyperparameter optimization method; the first feature analysis model is verified by the validation set, and the model is optimized according to the verification result, specifically including data enhancement, regularization and feature engineering.
[0042] In this embodiment, the data is cleaned and normalized to ensure data quality and comparability; the optimization in the training process can specifically use cross-validation techniques (such as grid search, random search) to adjust the hyperparameters of the model to improve model performance and generalization ability; taking random forest as an example, the number of trees, the depth of the trees, the minimum number of node samples in each tree, etc. can be adjusted; in the deep learning model, the learning rate, batch size, number of layers, the number of neurons in each layer, etc. can be adjusted; and a more accurate first feature analysis model is constructed, thereby obtaining a more accurate second feature analysis model to evaluate the accuracy of the spectral analysis method.
[0043] It should be understood that in model optimization, feature engineering refers to further processing and transformation of features to improve model performance, including feature selection, feature scaling, feature combination, feature dimensionality reduction, etc. For example, for deep learning models, it is particularly important to standardize or normalize the input data; regularization refers to the introduction of regularization terms during model training to prevent overfitting. For example, for linear models and deep learning models, L1 regularization or L2 regularization can be used. For tree models, the depth of the tree or the minimum number of samples of the node can be limited; data enhancement refers to the expansion of training data to increase data diversity and improve the generalization ability of the model.
[0044] Furthermore, a specific method for obtaining the model performance coefficient of the first feature analysis model is as follows: according to the training process of the first feature analysis model, model time data and average model time data are obtained, and the model time ratio is obtained accordingly, the model time data represents the training time of the first feature analysis model, and the model time ratio represents the deviation of the model time data of the first feature analysis model from the average model time data, specifically the ratio of the difference between the model time data and the average model time data and the average model time data, and the average model time data is the average value of the model time data of all first feature analysis models; according to the verification result of the first feature analysis model, model verification data is obtained, the model verification data includes material type deviation and material content deviation, and the model accuracy coefficient is obtained in combination with the F1 score of the first feature analysis model, the model accuracy coefficient is used to numerically describe the accuracy of the first feature analysis model, the material quantity deviation refers to the ratio of the material type obtained by the first feature analysis model to the actual material type, and the material content deviation refers to the ratio of the material content obtained by the first feature analysis model to the actual material content; the model performance coefficient of the first feature analysis model is obtained according to the obtained model time ratio and the model accuracy coefficient.
[0045] In this embodiment, for large data sets and complex models, the training time may be very long and the model time data is very large, which affects the actual usability of the model; the model situation of the first feature analysis model is uniformly evaluated through the model performance coefficient, thereby achieving a more intuitive reflection of the model performance differences of the first feature analysis model, and then achieving more accurate screening to obtain the second feature analysis model.
[0046] Furthermore, the model performance can be directly expressed by the model accuracy, but a more precise calculation can be performed. The model performance coefficient is calculated using the following formula:
[0047]
[0048] Where q is the number of the first feature analysis model, q = 1, 2, ..., Q, Q is the total number of the first feature analysis models, MX q is the model performance coefficient of the qth first characteristic analysis model, MT q is the model time proportion of the qth first feature analysis model, MZ q is the model accuracy coefficient of the qth first feature analysis model, MZ0 is the reference model accuracy coefficient, δ is the model time ratio factor, the value range is (0, 1), θ is the model accuracy factor, the value range is greater than 1; Taking θ=2 as an example, where e is a natural constant, the calculation formula of the model performance coefficient is:
[0049]
[0050] Where e is a natural constant.
[0051] In this embodiment, the statistical table of changes in the model performance coefficient is shown in Table 1:
[0052] Table 1 Statistics of model performance coefficient changes
[0053]
[0054] As can be seen from Table 1, the model performance coefficient of the first characteristic analysis model is jointly affected by the corresponding model time ratio and the model accuracy coefficient. The relationship between model performance cannot be inferred by comparing the size of a certain data alone. The model time ratio is negatively correlated with the model performance coefficient, but it does not mean that the smaller the model time ratio, the larger the corresponding model performance coefficient, such as the third and fourth groups of data, because the model performance coefficient is also related to the model accuracy coefficient; but when the model accuracy coefficient is greater than the reference model accuracy coefficient, the model performance coefficient increases with the increase of the model accuracy coefficient, such as the model performance coefficient in the third group of data is greater than that in other groups of data, because the corresponding model accuracy coefficient is the largest; the reference model accuracy coefficient is the average value of the accuracy coefficients of all first characteristic analysis models, which can be further determined in combination with actual application requirements; the numerical description of the model performance of the first characteristic analysis model is realized, thereby realizing more accurate screening of the second characteristic analysis model, so as to more quickly and accurately analyze the accuracy of the spectral analysis method.
[0055] Further, the model accuracy coefficient is calculated using the following formula: determine the corresponding significant analysis deviation ratio according to the substance types and substance contents obtained by inputting the validation set into the first feature analysis model, the significant analysis deviation ratio indicates the proportion of completely wrong situations according to the substance types and substance contents obtained in the validation set, or completely unusable situations, which requires relevant professionals to make judgments based on the obtained data; judge whether the significant analysis deviation ratio is greater than the threshold of the significant analysis deviation ratio, if so, the model accuracy coefficient of the corresponding first feature analysis model is 0, otherwise, the model verification data is obtained by combining the actual substance types and actual substance contents corresponding to the validation set, and the average model verification data is obtained accordingly, the average model verification data includes the average substance type deviation and the average substance content deviation, the average substance type deviation refers to the average value of the substance type deviations of all validation sets, and the average substance content deviation refers to the average value of the substance content deviations of all validation sets; obtain the corresponding F1 score through the confusion matrix of the first feature analysis model, and obtain the model accuracy coefficient of the first feature analysis model by combining the obtained significant analysis deviation ratio and the average model verification data; the model accuracy coefficient can be directly represented by the F1 score, but it can also be calculated more accurately, and the model accuracy coefficient is calculated using the following formula:
[0056]
[0057] Where q is the number of the first feature analysis model, q = 1, 2, ..., Q, Q is the total number of the first feature analysis models, MZ q is the model accuracy coefficient of the qth first feature analysis model, ZP q is the average substance type deviation of the qth first characteristic analysis model, HPq is the average material content deviation of the qth first characteristic analysis model, F1 q is the F1 score of the qth first feature analysis model, XP q is the significant analysis deviation ratio of the qth first feature analysis model, and XP0 is the threshold of the significant analysis deviation ratio.
[0058] In this embodiment, the threshold of the significant analysis deviation ratio is also determined by relevant professionals according to the type of substance. If the deviation is too obvious, the model accuracy coefficient can be directly considered to be 0, and the model is basically unusable. The general value range is 5% to 10%; the range of F1 score is [0, 1]. The main advantage is that it takes into account both false positives and false negatives. Therefore, when the number of labels 1 and 0 in the original data set is unbalanced, or the false positives and false negatives of the model are unbalanced and both are important, we can use F1 score to evaluate the model performance; first determine the range of the significant analysis deviation ratio of the first feature analysis model. When the significant analysis deviation ratio of the first feature analysis model is greater than the corresponding threshold, it means that the deviation is obvious, and the corresponding model accuracy coefficient is 0. On the contrary, the model accuracy coefficient increases with the increase of F1 score, and increases with the decrease of average substance type deviation and average substance content deviation. Such a calculation method can obtain the model accuracy coefficient more quickly and efficiently; specifically, when ZP q +HP q = 0, the corresponding model accuracy coefficient is only related to the F1 score, and the corresponding expression is when When , the model accuracy coefficient reaches the maximum value; it realizes a more intuitive and rapid description of the accuracy of the first feature analysis model, and combined with the model time proportion, it can numerically describe the corresponding model performance, thereby realizing a more accurate screening of the first feature analysis model to obtain the second feature analysis model to realize accurate and rapid evaluation of the accuracy of the spectral analysis method.
[0059] Furthermore, a specific method for screening the first feature analysis model to obtain the second feature analysis model is as follows: Step 1, compare the model performance coefficient of the first feature analysis model with the corresponding reference value. If it is not greater than the corresponding reference value, it indicates that the corresponding model does not meet the preset requirements, otherwise execute step 2; Step 2, determine whether the first feature analysis model that meets the preset requirements is unique. If it is unique, it is defined as the second feature analysis model. Otherwise, a first screening is performed according to the corresponding model time data to determine the model with the smallest model time data as the second feature analysis model to be selected; Step 3, determine whether the number of second feature analysis models to be selected is unique. If it is unique, it is defined as the second feature analysis model. Otherwise, a second screening is performed on the second feature analysis models to be selected according to the corresponding model screening parameters until the screened model is unique, so as to obtain the second feature analysis model. The model screening parameters mainly include the complexity, computing resources and stability of the model.
[0060] In this embodiment, in terms of the complexity of the model, if there is a simpler model among them, the simpler model will usually be selected, because the simpler model is easier to explain and understand, and is less prone to overfitting; considering the computational cost of the model, a model that is more efficient in using computing resources can be selected, which may mean selecting a model with fewer parameters or faster calculation speed; the stability of the model refers to the stability of the model output when the input data changes. When other aspects are similar, a more stable model can be selected; a more accurate screening is achieved to obtain the second characteristic analysis model, so that the material characteristics corresponding to the spectral data can be analyzed more quickly and accurately, and the accuracy of the spectral analysis method can be evaluated more quickly.
[0061] To summarize, the embodiment of the present application obtains the initial spectral data of the specified experimental area through a spectral measurement instrument and performs preprocessing and feature extraction to obtain the first spectral feature data. At the same time, the feature extraction is ensured to meet the feature extraction conditions based on the obtained spectral feature coefficients. Then, the first feature analysis model is constructed through the first spectral feature data, and the second feature analysis model is obtained by screening according to the model performance coefficient. Finally, the accuracy of the spectral analysis method is evaluated through the second feature analysis model, thereby realizing the rapid evaluation of the accuracy of the spectral data, and then realizing the improvement of the efficiency of the accuracy evaluation of the spectral analysis method, which solves the problem of low efficiency of the accuracy evaluation of the spectral analysis method in the prior art.
[0062] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0063] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0064] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0065] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0066] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.
[0067] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.
Claims
1. A method for evaluating the accuracy of a measurement method based on machine learning, characterized in that: The following steps are involved: Preprocessing initial spectrum data acquired by a spectrum measuring instrument to obtain first spectrum data, and extracting features from the first spectrum data to obtain first spectrum feature data; The obtained first spectral feature data is compared with the first spectral data to obtain a spectral feature coefficient, and it is determined whether the obtained spectral feature coefficient meets the feature extraction condition. If not, feature extraction is performed again. Otherwise, a first spectral feature data set is constructed according to the first spectral feature data, and the spectral feature coefficient is used to describe the accuracy of feature extraction of the first spectral data; The first feature analysis model is obtained by training and verifying the selected model through the first spectral feature data set, and the first feature analysis model is screened through the model performance coefficient of the first feature analysis model to obtain a second feature analysis model, wherein the model performance coefficient is used to describe the performance of the first feature analysis model; The specific calculation formula of the spectral characteristic coefficient is as follows: In the formula, m is the number of times the feature is extracted, m = 1, 2, ..., M, M is the total number of times the feature is extracted, GT m is the spectral characteristic coefficient of the mth feature extraction, GA m is the mean absolute error of the features extracted for the mth time, GX m is the feature correlation coefficient of the mth feature extraction, GA0 is the threshold of the feature mean absolute error, GX0 is the threshold of the feature correlation coefficient, α1 is the accuracy weight, and α2 is the consistency weight; Analyze the spectral data input within a preset time period through the second characteristic analysis model, and feed back the analysis results to the preset personnel for subsequent analysis; The model performance coefficient is calculated using the following formula: Where q is the number of the first feature analysis model, q = 1, 2, ..., Q, Q is the total number of the first feature analysis models, MX q is the model performance coefficient of the qth first characteristic analysis model, MT q is the model time proportion of the qth first feature analysis model, MZ q is the model accuracy coefficient of the qth first feature analysis model, MZ0 is the reference model accuracy coefficient, δ is the model time proportion factor, and θ is the model accuracy factor.
2. The accuracy evaluation method of a measurement method based on machine learning as claimed in claim 1, characterized in that: The specific method for obtaining the initial spectral data is as follows: Obtain the spectral impact data of the specified experimental area and the corresponding spectral impact data reference range; Acquiring first initial spectral data of a specified experimental area within a specified time period by a spectral measurement instrument; A comparative experiment is set up in combination with the spectral influence data reference range and the spectral influence data, and second initial spectral data corresponding to the comparative experiment is obtained through a spectral measuring instrument.
3. The accuracy evaluation method of a measurement method based on machine learning as claimed in claim 1, characterized in that: The specific steps of feature extraction are as follows: Fitting the first spectrum data by using the preset fitting function after initialization, and obtaining the spectrum peak position, spectrum peak height and spectrum peak width according to the fitting result; The peak intensity is obtained by combining the peak height, and the relative position and ratio of the peak are obtained by combining the first peak characteristic data.
4. The accuracy evaluation method of a measurement method based on machine learning as claimed in claim 1, characterized in that: The method for obtaining the spectral characteristic coefficient is as follows: Analyze the first spectrum data according to the principal component analysis method and the independent component analysis method to obtain the second spectrum characteristic data; Comparing the first spectrum feature data with the second spectrum feature data to obtain feature analysis data, wherein the feature analysis data includes a feature mean absolute error and a feature correlation coefficient; The feature extraction weights are obtained according to the subjective weighting method, wherein the feature extraction weights include accuracy weights and consistency weights, and the spectral feature coefficients are obtained in combination with the obtained feature analysis data.
5. The accuracy evaluation method of a measurement method based on machine learning as claimed in claim 1, characterized in that: The specific construction method of the first feature analysis model is as follows: After performing data cleaning and normalization processing on the first spectral feature data, a first spectral feature data set is constructed, and the data is divided into a training set and a validation set; The training set is labeled according to the material characteristic information of the initial spectral data corresponding to the training set, and the labeled training set is input into the candidate model for training until the model is in a convergent state to obtain a first characteristic analysis model; The first feature analysis model is verified through the verification set, and the model is optimized according to the verification result.
6. The accuracy evaluation method of a measurement method based on machine learning as claimed in claim 5, characterized in that: The specific method for obtaining the model performance coefficient of the first characteristic analysis model is as follows: The model time data and the average model time data are obtained according to the training process of the first feature analysis model, and the model time proportion is obtained accordingly; Obtaining model verification data according to the verification result of the first feature analysis model, the model verification data including substance type deviation and substance content deviation, and obtaining a model accuracy coefficient in combination with the F1 score of the first feature analysis model, the model accuracy coefficient being used to numerically describe the accuracy of the first feature analysis model; The model performance coefficient of the first feature analysis model is obtained according to the obtained model time proportion and model accuracy coefficient.
7. The accuracy evaluation method of a measurement method based on machine learning as claimed in claim 6, characterized in that: The model accuracy coefficient is calculated using the following formula: Determining the corresponding significant analysis deviation ratio according to the substance types and substance contents obtained by inputting the validation set into the first feature analysis model; Determine whether the significant analysis deviation ratio is greater than a threshold value of the significant analysis deviation ratio; if so, the model accuracy coefficient of the corresponding first feature analysis model is 0; otherwise, obtain model verification data by combining the actual substance type and the actual substance content corresponding to the verification set, and obtain average model verification data accordingly, wherein the average model verification data includes an average substance type deviation and an average substance content deviation; The corresponding F1 score is obtained through the confusion matrix of the first feature analysis model, and the model accuracy coefficient of the first feature analysis model is obtained by combining the obtained significant analysis deviation ratio and the average model verification data.
8. The accuracy evaluation method of a measurement method based on machine learning as claimed in claim 1, characterized in that: The specific method of screening the first feature analysis model to obtain the second feature analysis model is as follows: Step 1: compare the model performance coefficient of the first characteristic analysis model with the corresponding reference value. If it is not greater than the corresponding reference value, it indicates that the corresponding model does not meet the preset requirements, otherwise, execute step 2; Step 2: determine whether the first feature analysis model that meets the preset requirements is unique. If it is unique, define it as the second feature analysis model. Otherwise, perform a first screening based on the corresponding model time data to determine the model with the smallest model time data as the second feature analysis model to be selected. Step three, determine whether the number of second feature analysis models to be selected is unique. If so, define it as the second feature analysis model. Otherwise, perform a second screening on the second feature analysis models to be selected according to the corresponding model screening parameters until the screened model is unique, thereby obtaining the second feature analysis model.
