An artificial intelligence-based test data labeling method and system

Abstract

The present application relates to the technical field of data labeling, and more particularly to a test data labeling method and system based on artificial intelligence, which comprises a data processing module, a data labeling module and a labeling determination module; the data processing module collects multi-source test data, sequentially performs cleaning, denoising and timestamp alignment on the multi-source test data, and performs feature extraction on optimized test data; the data labeling module trains an initial model and labels the multi-source test data; the labeling determination module determines whether the reliability of the labeling result meets the requirements according to a labeling quality representation value; the screening adjustment module determines a screening constraint coefficient of multi-source test features according to a noise proportion in the optimized test data; and the verification adjustment module determines a labeling verification frequency coefficient according to a backlog rate of the optimized test data. The present application improves the labeling stability of test data.

Description

Technical Field

[0001] This invention relates to the field of data annotation technology, and in particular to an experimental data annotation method and system based on artificial intelligence. Background Technology

[0002] Existing technologies have developed preprocessing methods for multi-source heterogeneous data and employ artificial intelligence models to achieve automatic or assisted annotation. However, these methods generally suffer from a disconnect between multi-source data preprocessing and annotation processes. The annotation models lack generalization ability and robustness in scenarios with small sample sizes, strong noise, and multi-source heterogeneity. They also lack a closed-loop control mechanism for annotation quality judgment, parameter adjustment, and process optimization, making it difficult to meet the combined requirements of accuracy and timeliness in complex experimental scenarios such as high sampling rates, strong noise, and class imbalance. Furthermore, the absence of a closed-loop feedback mechanism fails to meet the stringent requirements for accuracy, processing efficiency, and result reliability in experimental data annotation, and the stability of data annotation is difficult to satisfy.

[0003] Chinese Patent Publication No. CN120296223A discloses a method and system for automatic data annotation based on model training. The method includes: S1, target content information collection: collecting target content information using a RAG large language model as the collection model based on user-input annotation tags; wherein the target content information includes at least one of text, images, and videos; S2, information filtering and preliminary evaluation: filtering the target content information according to a first evaluation condition to obtain first filtered data; wherein the first evaluation condition includes information timeliness and information relevance; S3, filtered data verification and feature feedback: verifying the first filtered data to obtain historical information, and feeding this information into subsequent target content information collection stages. The system provides feedback on the following steps: S4, Differential Feature Extraction and Labeling Preparation: Overflow filtering of the historical information is performed to obtain differential information, which is then labeled as data to be labeled; S5, Automatic Labeling and Error Rate Verification and Update: The data to be labeled is segmented using the data to be labeled to obtain labeled data. The labeled data is then verified and the error rate is updated based on the historical error rate; S6, Deviation Calculation and Monitoring Interval Generation: Deviation is calculated on the updated error rate, and a monitoring evaluation interval is generated based on the deviation. The error rate is updated during the next data labeling, and the output of the automatic labeling model is classified into frozen data and unfrozen data based on the real-time value of the updated error rate and the monitoring evaluation interval. Therefore, the automatic data labeling method and system based on model training suffer from problems such as reliance on uncontrollable open data sources and static filtering rules, an error-accumulating feedback mechanism, and delayed error rate verification, resulting in large fluctuations in labeling results and decreased stability of experimental data labeling. Summary of the Invention

[0004] To address this, the present invention provides an experimental data annotation method and system based on artificial intelligence, which overcomes the problems in the prior art where the annotation results fluctuate greatly due to the reliance on uncontrollable open data sources, static screening rules, feedback mechanisms that easily accumulate errors, and lagging error rate verification, leading to decreased stability of the annotation data.

[0005] To achieve the above objectives, the present invention provides an artificial intelligence-based experimental data annotation system, comprising: The data processing module includes an acquisition unit for collecting multi-source experimental data, a preprocessing unit connected to the acquisition unit for sequentially cleaning, denoising, and timestamp aligning the multi-source experimental data to obtain optimized experimental data, and a feature extraction unit connected to the preprocessing unit for extracting features from the optimized experimental data to obtain multi-source experimental features. The data annotation module, which is connected to the data processing module, includes a training unit for training an initial model based on the multi-source experimental features to obtain an artificial intelligence annotation model, and an annotation unit connected to the training unit for annotating multi-source experimental data based on the artificial intelligence annotation model to obtain an annotated dataset. The annotation judgment module, which is connected to the data processing module, is used to determine whether the reliability of the annotation results meets the requirements based on the annotation quality characterization value determined by the loss rate of multi-source experimental data and the annotation accuracy of the annotation dataset. The screening and adjustment module, which is connected to the annotation determination module, is used to determine the screening constraint coefficient of the multi-source test features based on the noise ratio in the optimized test data when the reliability of the annotation results does not meet the requirements. The verification and adjustment module, which is connected to the screening and adjustment module, is used to determine the labeling verification frequency coefficient based on the backlog rate of the optimized test data.

[0006] Furthermore, the annotation determination module determines the annotation quality characterization value based on the ratio of the loss rate of multi-source experimental data to the annotation accuracy of the annotation dataset in order to determine whether the reliability of the annotation results meets the requirements.

[0007] Furthermore, the annotation determination module determines that the reliability of the annotation result meets the requirements when the annotation quality characterization value is less than or equal to a preset characterization value. The annotation determination module determines that the reliability of the annotation result does not meet the requirements when the annotation quality characterization value is greater than the preset characterization value.

[0008] Furthermore, in response to the condition that the reliability of the annotation results does not meet the requirements, the screening and adjustment module determines whether the accuracy of the annotation results meets the requirements based on the noise ratio in the optimized test data.

[0009] Furthermore, the screening and adjustment module responds to the fact that the noise ratio in the optimized test data is less than or equal to a preset first ratio, and determines that the accuracy of the labeling results meets the requirements; The filtering and adjustment module responds to the fact that the noise ratio in the optimized test data is greater than the preset first ratio, and determines that the accuracy of the labeling results does not meet the requirements.

[0010] Furthermore, the screening and adjustment module increases the screening constraint coefficient of the multi-source test features in response to the noise ratio in the optimized test data being greater than a preset first ratio and less than or equal to a preset second ratio. The filtering and adjustment module responds to the fact that the noise ratio in the optimized test data is greater than the preset second ratio, initially determines that the timeliness of the annotation results does not meet the requirements, and determines whether the timeliness of the annotation results meets the requirements based on the backlog rate of the optimized test data.

[0011] Furthermore, the increase in the screening constraint coefficient of the multi-source test features is determined by optimizing the difference between the noise ratio in the test data and the preset first ratio.

[0012] Furthermore, in response to the condition that the noise ratio in the optimized test data is greater than the preset second ratio, the verification and adjustment module determines whether the timeliness of the annotation results meets the requirements based on the backlog rate of the optimized test data.

[0013] Furthermore, the verification and adjustment module determines that the timeliness of the annotation results meets the requirements when the backlog rate of the optimized test data is less than or equal to the preset backlog rate. The verification and adjustment module responds to the fact that the backlog rate of the optimized test data is greater than the preset backlog rate, determines that the timeliness of the annotation results does not meet the requirements, and reduces the annotation verification frequency coefficient.

[0014] Furthermore, the reduction in the labeled verification frequency coefficient is determined by the difference between the backlog rate of the optimized test data and the preset backlog rate.

[0015] This invention also provides an artificial intelligence-based experimental data annotation method, comprising: Multi-source experimental data is collected, and the multi-source experimental data is sequentially cleaned, denoised, and timestamp aligned to obtain optimized experimental data. Feature extraction is performed based on the optimized experimental data to obtain multi-source experimental features. The initial model is trained based on the multi-source experimental features to obtain an artificial intelligence annotation model, and the multi-source experimental data is annotated using the artificial intelligence annotation model to obtain an annotated dataset; The loss rate of multi-source experimental data and the annotation accuracy of the labeled dataset are obtained to determine the annotation quality characterization value, and the reliability of the annotation results is determined based on the annotation quality characterization value. If the reliability of the annotation results does not meet the requirements, the noise ratio in the optimized test data is obtained to determine whether the accuracy of the annotation results meets the requirements. If the accuracy of the annotation results does not meet the requirements, then determine whether it is necessary to increase the screening constraint coefficient of the multi-source experimental features; If it is not necessary to increase the screening constraint coefficient of multi-source test features, then the backlog rate of the optimized test data is used to determine the annotation verification frequency coefficient.

[0016] Compared with existing technologies, the beneficial effects of this invention are as follows: The system of this invention, through a data processing module, a data annotation module, an annotation judgment module, a screening and adjustment module, and a verification and adjustment module, determines the reliability of the annotation results based on the annotation quality characterization value determined by the loss rate of multi-source experimental data and the annotation accuracy of the annotation dataset. Since multi-source experimental data is susceptible to electromagnetic interference, and insufficient alignment accuracy between multiple sensors prevents precise association and collaborative matching of multi-source data, it causes distortion of the input signal and disorder of feature correspondence in the artificial intelligence annotation model, resulting in systematic deviations in key annotation information. By determining the reliability of the annotation results, the root cause of the problem can be accurately identified as insufficient front-end data fusion quality. This enables precise capture and measurement of feature-level confusion caused by electromagnetic interference and equipment delay, providing a basis for error correction for credible annotation of multi-source experimental data. The system adjusts the screening constraint coefficient of multi-source experimental features based on the noise ratio in the optimized experimental data. This addresses issues such as the scarcity of key state samples and severe class imbalance in the training data itself. This leads to the AI ​​model capturing noise during the learning process, rather than the essential mapping relationship between the actual state of the equipment and the observed signals. Consequently, it fails to accurately reflect the actual operating state of the equipment. Increasing the screening constraint coefficient of multi-source experimental features can improve the strictness of feature selection, strengthen the filtering of noisy features, and constrain the model to focus on features that have discriminative power in the annotation task, reducing interference from invalid features and improving the accuracy and reliability of the annotation results. Adjusting the annotation verification frequency coefficient based on the backlog rate of optimized experimental data is crucial. Due to the high data sampling rate and large amount of data collected during the experiment, the inference speed, data processing rate, and resource scheduling strategy of the AI ​​annotation model were not adapted to the high-speed acquisition rhythm, resulting in excessive time delay from data generation to annotation completion. This fails to meet the time constraints of real-time monitoring, online decision-making, and dynamic parameter adjustment in the experiment. Reducing the annotation verification frequency coefficient can lower the verification overhead and improve the data annotation processing speed, thereby reducing the time delay from data generation to annotation completion and improving the annotation stability of the experimental data.

[0017] Furthermore, the system described in this invention determines the reliability of the annotation results by setting preset characterization values. Since multi-source experimental data is susceptible to electromagnetic interference, and the alignment accuracy between multiple sensors is insufficient, multi-source data cannot achieve accurate association and collaborative matching, resulting in distortion of the input signal of the artificial intelligence annotation model and disorder of feature correspondence, leading to systematic deviations in key annotation information. By determining the reliability of the annotation results, the root cause of the problem can be accurately identified as insufficient front-end data fusion quality, achieving accurate capture and measurement of feature-level confusion caused by electromagnetic interference and equipment delay, providing a basis for error correction for credible annotation of multi-source experimental data, and further improving the annotation stability of experimental data.

[0018] Furthermore, the system described in this invention adjusts the screening constraint coefficient of multi-source experimental features by setting a preset first proportion and a preset second proportion. Due to the scarcity of key state samples and severe class imbalance in the training data, the artificial intelligence model captures noise during the learning process, rather than the essential mapping relationship between the actual state of the equipment and the observed signal, and cannot truly reflect the actual operating state of the equipment. By increasing the screening constraint coefficient of multi-source experimental features, the strictness of feature screening can be improved, the filtering of noisy features can be strengthened, the model can be constrained to focus on features that have a discriminative effect on the labeling task, the interference of invalid features can be reduced, the accuracy and reliability of the labeling results can be improved, and the labeling stability of the experimental data can be further improved.

[0019] Furthermore, the system described in this invention adjusts the annotation verification frequency coefficient by setting a preset backlog rate. Due to the high data sampling rate and large amount of data collected during the experiment, the inference speed, data processing rate, and resource scheduling strategy of the artificial intelligence annotation model are not adapted to the high-speed collection rhythm, resulting in an excessive time delay from data generation to annotation completion. This fails to meet the time constraints of real-time monitoring, online decision-making, and dynamic parameter adjustment in the experiment. By reducing the annotation verification frequency coefficient, the verification overhead can be reduced and the data annotation processing speed can be improved, thereby reducing the time delay from data generation to annotation completion and further improving the annotation stability of the experimental data. Attached Figure Description

[0020] Figure 1 This is a block diagram of the overall structure of the artificial intelligence-based experimental data annotation system according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating the process of determining the reliability of annotation results using an artificial intelligence-based experimental data annotation system according to an embodiment of the present invention. Figure 3 This is a flowchart illustrating the process of determining the screening constraint coefficients for multi-source experimental features in an artificial intelligence-based experimental data annotation system according to an embodiment of the present invention. Figure 4This is an overall flowchart of the experimental data annotation method based on artificial intelligence according to an embodiment of the present invention. Detailed Implementation

[0021] To make the objectives and advantages of the present invention clearer, the present invention will be further described below with reference to embodiments; it should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0022] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0023] Please see Figure 1 As shown, it is an overall structural block diagram of the experimental data annotation system based on artificial intelligence according to an embodiment of the present invention.

[0024] An embodiment of the present invention provides an artificial intelligence-based experimental data annotation system, comprising: The data processing module includes an acquisition unit for collecting multi-source experimental data, a preprocessing unit connected to the acquisition unit for sequentially cleaning, denoising, and timestamp aligning the multi-source experimental data to obtain optimized experimental data, and a feature extraction unit connected to the preprocessing unit for extracting features from the optimized experimental data to obtain multi-source experimental features. The data annotation module, which is connected to the data processing module, includes a training unit for training an initial model based on the multi-source experimental features to obtain an artificial intelligence annotation model, and an annotation unit connected to the training unit for annotating multi-source experimental data based on the artificial intelligence annotation model to obtain an annotated dataset. The annotation judgment module, which is connected to the data processing module, is used to determine whether the reliability of the annotation results meets the requirements based on the annotation quality characterization value determined by the loss rate of multi-source experimental data and the annotation accuracy of the annotation dataset. The screening and adjustment module, which is connected to the annotation determination module, is used to determine the screening constraint coefficient of the multi-source test features based on the noise ratio in the optimized test data when the reliability of the annotation results does not meet the requirements. The verification and adjustment module, which is connected to the screening and adjustment module, is used to determine the annotation verification frequency coefficient based on the backlog rate of the optimized test data.

[0025] Specifically, the multi-source test data includes the current of the servo drive, the rotational speed of the high-speed rotating shaft, and the temperature at the center of the reactor.

[0026] Specifically, the optimized test data includes the current of the servo drive after cleaning, the speed of the high-speed rotating shaft after noise reduction, and the temperature of the reactor center after timestamp alignment.

[0027] Specifically, the characteristics of multi-source testing include the current fluctuation amplitude of the servo driver per unit time, the rate of temperature drop at the center of the reactor, and the rate of change of the rotational speed of the high-speed rotating shaft.

[0028] Specifically, the process of training the initial model based on the multi-source experimental features to obtain the AI ​​labeled model involves constructing a time-series sample set from the multi-source experimental features and dividing it into a training set, a validation set, and a test set. The training set is then input into the initial model for forward training and parameter iteration. Model parameters are tuned based on the validation set, and model performance is verified based on the test set to obtain the AI ​​labeled model.

[0029] Specifically, the initial model is a basic model framework that already has the ability to analyze and label predictions of multi-source experimental features, but has not yet undergone parameter iteration optimization using data feature samples.

[0030] Specifically, the artificial intelligence annotation model can be a convolutional neural network model, a long short-term memory network model, or a graph neural network model, with the preferred embodiment being a convolutional neural network model.

[0031] Specifically, the process of annotating multi-source experimental data to obtain an labeled dataset based on an artificial intelligence annotation model involves inputting the multi-source experimental data into the artificial intelligence annotation model, performing fusion calculations on the multi-source experimental features, identifying the current operating status of the equipment, and outputting the labeled dataset.

[0032] Specifically, the labeled dataset includes a fault state set, a time node set, and an anomaly event set.

[0033] Specifically, the screening constraint coefficient of multi-source experimental features is a quantitative parameter that characterizes the strictness and filtering intensity of the artificial intelligence model in screening multi-source experimental features during the training process.

[0034] Specifically, the annotation verification frequency coefficient is a quantitative parameter that characterizes the execution frequency and verification density of the annotation system in verifying and checking the completed annotation results.

[0035] In implementation, the system of this invention, through a data processing module, a data annotation module, an annotation judgment module, a screening and adjustment module, and a verification and adjustment module, determines the reliability of the annotation results based on the annotation quality characterization value determined by the loss rate of multi-source experimental data and the annotation accuracy of the annotation dataset. Because multi-source experimental data is susceptible to electromagnetic interference, and insufficient alignment accuracy between multiple sensors prevents precise association and collaborative matching of multi-source data, it causes distortion of the input signal and disorder of feature correspondence in the artificial intelligence annotation model, leading to systematic deviations in key annotation information. By judging the reliability of the annotation results, the root cause of the problem can be accurately identified as insufficient front-end data fusion quality. This enables precise capture and measurement of feature-level confusion caused by electromagnetic interference and equipment delays, providing a basis for error correction for credible annotation of multi-source experimental data. The system adjusts the screening constraint coefficient of multi-source experimental features based on the noise ratio in the optimized experimental data. Due to problems such as scarce key state samples and severe class imbalance in the training data itself, the artificial intelligence... The model captures noise during the learning process, rather than the essential mapping relationship between the actual state of the equipment and the observed signal. Therefore, it cannot truly reflect the actual operating state of the equipment. By increasing the screening constraint coefficient of multi-source experimental features, the strictness of feature selection can be improved, strengthening the filtering of noisy features. This constrains the model to focus on features that have discriminative power in the annotation task, reducing interference from invalid features and improving the accuracy and reliability of the annotation results. The annotation verification frequency coefficient is adjusted based on the backlog rate of the optimized experimental data. Due to the high data sampling rate and large amount of data collected during the experiment, the inference speed, data processing rate, and resource scheduling strategy of the AI ​​annotation model were not adapted to the high-speed acquisition rhythm, resulting in excessive time delay from data generation to annotation completion. This fails to meet the time constraints of real-time monitoring, online decision-making, and dynamic parameter adjustment in the experiment. By reducing the annotation verification frequency coefficient, the verification overhead can be reduced, and the data annotation processing speed can be improved, thereby reducing the time delay from data generation to annotation completion and improving the annotation stability of the experimental data.

[0036] Please continue reading. Figure 2 As shown, it is a logical flowchart of the process of determining the reliability of the annotation results of the experimental data annotation system based on artificial intelligence in an embodiment of the present invention.

[0037] Specifically, the annotation determination module determines the annotation quality characterization value based on the ratio of the loss rate of multi-source experimental data to the annotation accuracy of the annotation dataset in order to determine whether the reliability of the annotation results meets the requirements.

[0038] Specifically, the annotation determination module determines that the reliability of the annotation result meets the requirements when the annotation quality characterization value is less than or equal to a preset characterization value. The annotation determination module determines that the reliability of the annotation result does not meet the requirements when the annotation quality characterization value is greater than the preset characterization value.

[0039] Understandably, in AI-based experimental data annotation systems, the core logic of using preset characterization values ​​to assess the reliability of annotation results is to transform the reliability of the annotation results into quantifiable characterization values ​​for judgment. The preset characterization value serves as the dividing line for determining whether the reliability of the annotation results meets the requirements. The preset characterization value can be set according to actual working conditions. The preset characterization value aims to ensure the stability and practicality of data annotation. Optionally, the preset characterization value is determined through a limited number of experiments by evaluating the effects of different characterization values ​​on data annotation. The determined preset characterization value should be neither too small nor cause excessive interference to the data annotation process. For example, the preset characterization value is generally selected within the range of [10%, 15%].

[0040] Preferably, the preset characterization value is 12% in a preferred embodiment.

[0041] Specifically, the loss rate of multi-source experimental data is the ratio of the amount of lost multi-source experimental data to the total amount of multi-source experimental data.

[0042] Specifically, the labeling accuracy rate of a labeled dataset is the ratio of the number of accurately labeled datasets to the total number of labeled datasets.

[0043] Specifically, accurate labeling means that the labeling result is consistent with the actual label.

[0044] In practice, the system described in this invention determines the reliability of the annotation results by setting preset characterization values. Since multi-source experimental data is susceptible to electromagnetic interference, and the alignment accuracy between multiple sensors is insufficient, multi-source data cannot achieve accurate association and collaborative matching, resulting in distortion of the input signal of the artificial intelligence annotation model and disorder of feature correspondence, leading to systematic deviations in key annotation information. By determining the reliability of the annotation results, the root cause of the problem can be accurately identified as insufficient front-end data fusion quality. This enables accurate capture and measurement of feature-level confusion caused by electromagnetic interference and equipment delay, providing a basis for error correction for credible annotation of multi-source experimental data and further improving the annotation stability of experimental data.

[0045] Please continue reading. Figure 3 The diagram shown is a logical flowchart of the process for determining the screening constraint coefficients of multi-source experimental features in the experimental data annotation system based on artificial intelligence according to an embodiment of the present invention.

[0046] Specifically, in response to the condition that the reliability of the annotation results does not meet the requirements, the screening and adjustment module determines whether the accuracy of the annotation results meets the requirements based on the noise ratio in the optimized test data.

[0047] Specifically, the screening and adjustment module responds to the fact that the noise ratio in the optimized test data is less than or equal to a preset first ratio, and determines that the accuracy of the labeling results meets the requirements; The filtering and adjustment module responds to the fact that the noise ratio in the optimized test data is greater than the preset first ratio, and determines that the accuracy of the labeling results does not meet the requirements.

[0048] Specifically, the screening and adjustment module increases the screening constraint coefficient of the multi-source test features in response to the noise ratio in the optimized test data being greater than a preset first ratio and less than or equal to a preset second ratio. The filtering and adjustment module responds to the fact that the noise ratio in the optimized test data is greater than the preset second ratio, initially determines that the timeliness of the annotation results does not meet the requirements, and determines whether the timeliness of the annotation results meets the requirements based on the backlog rate of the optimized test data.

[0049] Understandably, in AI-based experimental data annotation systems, the core logic of using preset first and second percentages to characterize whether the accuracy of the annotation results meets requirements is to convert the accuracy of the annotation results into quantifiable percentages for judgment. The preset first percentage serves as the dividing line for judging whether the accuracy of the annotation results meets requirements, and the preset second percentage serves as the dividing line for judging the two reasons that lead to the inaccuracy of the annotation results not meeting requirements. The preset first and second percentages can be set according to actual working conditions. The preset first and second percentages aim to ensure the stability and practicality of data annotation. Optionally, the preset first and second percentages are determined through a limited number of experiments by evaluating the effect of different percentages on data annotation. The determined preset first and second percentages should satisfy the condition that they are neither too small nor cause excessive interference to the data annotation process. For example, the preset first percentage is generally selected in the range of [5%, 15%], and the preset second percentage is generally selected in the range of [20%, 40%].

[0050] Preferably, the first percentage is 10% in the preferred embodiment and 30% in the preferred embodiment.

[0051] Specifically, the noise percentage in the optimized experimental data is the ratio of the number of noisy data points in the optimized experimental data to the total number of optimized experimental data points.

[0052] Specifically, the increase in the screening constraint coefficient of the multi-source test features is determined by optimizing the difference between the noise ratio in the test data and the preset first ratio.

[0053] Specifically, when the difference between the noise proportion in the optimized test data and the preset first proportion is within 2%, the screening constraint coefficient of the multi-source test feature is increased to 1.5 times the original value. When the difference between the noise proportion in the optimized test data and the preset first proportion exceeds 2%, on the basis of increasing to 1.5 times the original value, for every 1% exceeding the original value, the screening constraint coefficient of the multi-source test feature increases by 0.2%. For example, if the difference between the noise proportion in the optimized test data and the preset first proportion is 5%, and the current screening constraint coefficient of the multi-source test feature is 3%, the increased screening constraint coefficient of the multi-source test feature is 3×1.5+0.2×3=5.1%.

[0054] In implementation, the system of the present invention adjusts the screening constraint coefficient of multi-source experimental features by setting a preset first proportion and a preset second proportion. Due to the scarcity of key state samples and severe class imbalance in the training data, the artificial intelligence model captures noise during the learning process, rather than the essential mapping relationship between the actual state of the equipment and the observed signal, and cannot truly reflect the actual operating state of the equipment. By increasing the screening constraint coefficient of multi-source experimental features, the strictness of feature screening can be improved, the filtering of noisy features can be strengthened, the model is constrained to focus on features that have a discriminative effect on the labeling task, the interference of invalid features is reduced, the accuracy and reliability of the labeling results are improved, and the labeling stability of the experimental data is further improved.

[0055] Specifically, the verification and adjustment module, in response to the condition that the noise ratio in the optimized test data is greater than the preset second ratio, determines whether the timeliness of the annotation results meets the requirements based on the backlog rate of the optimized test data.

[0056] Specifically, the verification and adjustment module determines that the timeliness of the annotation results meets the requirements when the backlog rate of the optimized test data is less than or equal to the preset backlog rate. The verification and adjustment module responds to the fact that the backlog rate of the optimized test data is greater than the preset backlog rate, determines that the timeliness of the annotation results does not meet the requirements, and reduces the annotation verification frequency coefficient.

[0057] Understandably, in AI-based experimental data annotation systems, using a preset backlog rate to characterize whether the timeliness of the annotation results meets requirements involves converting the timeliness of the annotation results into a quantifiable backlog rate for judgment. The preset backlog rate serves as the dividing line for determining whether the timeliness of the annotation results meets requirements. The preset backlog rate can be set according to actual working conditions. The preset backlog rate aims to ensure the stability and practicality of data annotation. Optionally, the preset backlog rate is determined through a limited number of experiments by evaluating the effect of different backlog rates on data annotation. The determined preset backlog rate should be neither too small nor cause excessive interference to the data annotation process. For example, the preset backlog rate is generally selected in the range of [10%, 30%].

[0058] Preferably, the preset backlog rate is 20% in the preferred embodiment.

[0059] Specifically, the backlog rate of optimized experimental data is the ratio of the amount of optimized experimental data to be processed to the total amount of optimized experimental data.

[0060] Specifically, the reduction in the annotation verification frequency coefficient is determined by the difference between the backlog rate of the optimized test data and the preset backlog rate.

[0061] Specifically, when the difference between the backlog rate of the optimized test data and the preset backlog rate is within 2%, the annotation verification frequency coefficient is reduced to 0.9 times the original value. When the difference between the backlog rate of the optimized test data and the preset backlog rate exceeds 2%, the annotation verification frequency coefficient is reduced by 1% for every 1% exceeding the original value, in addition to being reduced to 0.9 times the original value. For example, if the difference between the backlog rate of the optimized test data and the preset backlog rate is 4%, and the current annotation verification frequency coefficient is 5%, the reduced annotation verification frequency coefficient will be 5 × 0.9 - 1 × 2 = 2.5%.

[0062] In practice, the system described in this invention adjusts the annotation verification frequency coefficient by setting a preset backlog rate. Due to the high data sampling rate and large amount of data collected during the experiment, the inference speed, data processing rate and resource scheduling strategy of the artificial intelligence annotation model are not adapted to the high-speed collection rhythm, resulting in an excessive time delay from data generation to annotation completion. This fails to meet the time constraints of real-time monitoring, online decision-making and dynamic parameter adjustment in the experiment. By reducing the annotation verification frequency coefficient, the verification overhead can be reduced and the data annotation processing speed can be improved, thereby reducing the time delay from data generation to annotation completion and further improving the annotation stability of the experimental data.

[0063] Please continue reading. Figure 4 The diagram shown is an overall flowchart of the experimental data annotation method based on artificial intelligence according to an embodiment of the present invention.

[0064] This invention provides an artificial intelligence-based experimental data annotation method, comprising: Step S1: Collect multi-source experimental data, and sequentially clean, denoise, and align the timestamps of the multi-source experimental data to obtain optimized experimental data. Based on the optimized experimental data, perform feature extraction to obtain multi-source experimental features. Step S2: Train the initial model based on the multi-source experimental features to obtain an artificial intelligence annotation model, and use the artificial intelligence annotation model to annotate the multi-source experimental data to obtain an annotated dataset; Step S3: Obtain the loss rate of multi-source experimental data and the annotation accuracy of the labeled dataset to determine the annotation quality characterization value, and determine whether the reliability of the annotation results meets the requirements based on the annotation quality characterization value; Step S4: If the reliability of the annotation result does not meet the requirements, the noise ratio in the optimized test data is obtained to determine whether the accuracy of the annotation result meets the requirements. Step S5: If the accuracy of the annotation results does not meet the requirements, determine whether it is necessary to increase the screening constraint coefficient of the multi-source experimental features. Step S6: If it is not necessary to increase the screening constraint coefficient of the multi-source test features, then obtain the backlog rate of the optimized test data to determine the annotation verification frequency coefficient.

[0065] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. An artificial intelligence-based experimental data annotation system, characterized in that, include: The data processing module includes an acquisition unit for collecting multi-source experimental data, a preprocessing unit connected to the acquisition unit for sequentially cleaning, denoising, and timestamp aligning the multi-source experimental data to obtain optimized experimental data, and a feature extraction unit connected to the preprocessing unit for extracting features from the optimized experimental data to obtain multi-source experimental features. The data annotation module, which is connected to the data processing module, includes a training unit for training an initial model based on the multi-source experimental features to obtain an artificial intelligence annotation model, and an annotation unit connected to the training unit for annotating multi-source experimental data based on the artificial intelligence annotation model to obtain an annotated dataset. The annotation judgment module, which is connected to the data processing module, is used to determine whether the reliability of the annotation results meets the requirements based on the annotation quality characterization value determined by the loss rate of multi-source experimental data and the annotation accuracy of the annotation dataset. The screening and adjustment module, which is connected to the annotation determination module, is used to determine the screening constraint coefficient of the multi-source test features based on the noise ratio in the optimized test data when the reliability of the annotation results does not meet the requirements. The verification and adjustment module, which is connected to the screening and adjustment module, is used to determine the annotation verification frequency coefficient based on the backlog rate of the optimized test data.

2. The artificial intelligence-based experimental data annotation system according to claim 1, characterized in that, The annotation determination module responds to the ratio of the loss rate of multi-source experimental data to the annotation accuracy of the annotation dataset to determine the annotation quality characterization value to determine whether the reliability of the annotation results meets the requirements.

3. The artificial intelligence-based experimental data annotation system according to claim 2, characterized in that, The annotation determination module determines that the reliability of the annotation result meets the requirements when the annotation quality characterization value is less than or equal to the preset characterization value. The annotation determination module determines that the reliability of the annotation result does not meet the requirements when the annotation quality characterization value is greater than the preset characterization value.

4. The artificial intelligence-based experimental data annotation system according to claim 3, characterized in that, The filtering and adjustment module, in response to the condition that the reliability of the annotation results does not meet the requirements, determines whether the accuracy of the annotation results meets the requirements based on the noise ratio in the optimized test data.

5. The artificial intelligence-based experimental data annotation system according to claim 4, characterized in that, The screening and adjustment module responds to the fact that the noise ratio in the optimized test data is less than or equal to a preset first ratio, and determines that the accuracy of the labeling results meets the requirements. The filtering and adjustment module responds to the fact that the noise ratio in the optimized test data is greater than the preset first ratio, and determines that the accuracy of the labeling results does not meet the requirements.

6. The artificial intelligence-based experimental data annotation system according to claim 5, characterized in that, The screening and adjustment module responds to the fact that the noise ratio in the optimized test data is greater than a preset first ratio and less than or equal to a preset second ratio by increasing the screening constraint coefficient of the multi-source test features. The filtering and adjustment module responds to the fact that the noise ratio in the optimized test data is greater than the preset second ratio, initially determines that the timeliness of the annotation results does not meet the requirements, and determines whether the timeliness of the annotation results meets the requirements based on the backlog rate of the optimized test data.

7. The artificial intelligence-based experimental data annotation system according to claim 6, characterized in that, The increase in the screening constraint coefficient of the multi-source test features is determined by the difference between the noise ratio in the optimized test data and the preset first ratio.

8. The artificial intelligence-based experimental data annotation system according to claim 7, characterized in that, The verification and adjustment module responds to the condition that the noise ratio in the optimized test data is greater than the preset second ratio, and determines whether the timeliness of the annotation results meets the requirements based on the backlog rate of the optimized test data.

9. The artificial intelligence-based experimental data annotation system according to claim 8, characterized in that, The verification and adjustment module determines that the timeliness of the annotation results meets the requirements when the backlog rate of the optimized test data is less than or equal to the preset backlog rate. The verification and adjustment module responds to the fact that the backlog rate of the optimized test data is greater than the preset backlog rate, determines that the timeliness of the annotation results does not meet the requirements, and reduces the annotation verification frequency coefficient. The reduction in the labeled verification frequency coefficient is determined by the difference between the backlog rate of the optimized test data and the preset backlog rate.

10. A labeling method applied to the artificial intelligence-based experimental data labeling system according to any one of claims 1-9, characterized in that, include: Multi-source experimental data is collected, and the multi-source experimental data is sequentially cleaned, denoised, and timestamp aligned to obtain optimized experimental data. Feature extraction is performed based on the optimized experimental data to obtain multi-source experimental features. The initial model is trained based on the multi-source experimental features to obtain an artificial intelligence annotation model, and the multi-source experimental data is annotated using the artificial intelligence annotation model to obtain an annotated dataset; The loss rate of multi-source experimental data and the annotation accuracy of the labeled dataset are obtained to determine the annotation quality characterization value, and the reliability of the annotation results is determined based on the annotation quality characterization value. If the reliability of the annotation results does not meet the requirements, the noise ratio in the optimized test data is obtained to determine whether the accuracy of the annotation results meets the requirements. If the accuracy of the annotation results does not meet the requirements, then determine whether it is necessary to increase the screening constraint coefficient of the multi-source experimental features; If it is not necessary to increase the screening constraint coefficient of multi-source test features, then the backlog rate of the optimized test data is used to determine the annotation verification frequency coefficient.

Images