High-precision time series labeling equipment
By introducing acquisition, preprocessing, comprehensive processing and optimization modules into the time series labeling equipment, using the methods of comparison and federated learning frameworks, the problem of time series labeling error handling in the prior art is solved, and high-precision, efficiency and consistency labeling results are achieved.
Patent Information
- Application Number
- CN202411844689.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-13
- Publication Date
- 2025-04-04
AI Technical Summary
Existing high-precision time series labeling equipment is prone to errors during the labeling process, which is difficult to effectively handle errors, resulting in large errors in the labeling result, increasing the labeling cost and cycle, and may mislead the direction of technology research and development.
Provide a high-precision time series labeling device, including an acquisition module, a preprocessing module, a comprehensive processing module and an optimization module. By comparing the benchmark dataset and feature dataset, a two-stage federated learning framework and nonlinear projection head are used to optimize errors that occur during the annotation process.
It significantly improves the accuracy, efficiency, consistency and comparability of labeling, reduces the degradation of model performance caused by labeling errors, enhances the effectiveness of data analysis, and supports a wider range of application scenarios.
Smart Images
Figure CN119760568A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of time series labeling, and in particular to a high-precision time series labeling device. Background Art
[0002] A data processing system refers to a system that uses computers to process information. It mainly processes, organizes, and calculates data information to obtain various analysis indicators, converts this information into a form that is easy for people to accept, and marks time series in the data processing system.
[0003] Time series annotation is based on an ordered observation data set associated with the time sequence. It uses random process theory and mathematical statistics methods to study the statistical laws followed by the data set, thereby inferring the data development trend and guiding the solution of practical problems. Scientific and correct prediction and analysis of various actual time series can produce huge economic and social benefits. Time series annotation has been widely used in industry, address, ecology, economy, meteorology, medicine and other fields.
[0004] The existing high-precision time series labeling equipment includes the following steps: data import, labeling rule setting, labeling process and result export. First, the time series data to be labeled needs to be imported into the sensor, and the labeling rules are set, including the labeling granularity, labeling category and labeling standard. According to the data provided by the sensor, the server labels the time series data according to the set rules; however, when labeling the data, a set of time series samples are collected through local sensors, and wrong labels may appear in the labeling process, which makes it difficult to effectively handle the wrong labels, resulting in large errors in the labeled results. In order to correct the wrong labels, additional manpower, material resources and time are required for review and modification. This will not only increase the labeling cost, but also prolong the labeling cycle. In the technology research and development based on time series data, wrong labeling data may mislead the research direction and hinder the progress and development of technology.
[0005] Therefore, it is necessary to provide a new high-precision time series labeling device to solve the above technical problems. Summary of the invention
[0006] In order to solve the above technical problems, the present invention provides a high-precision time series labeling device.
[0007] The high-precision time series labeling device provided by the present invention includes an acquisition module for acquiring a time series data set and a benchmark data set;
[0008] Preprocessing module, which performs data preprocessing on time series data sets;
[0009] Comprehensive processing module, used to extract features from the preprocessed time series dataset and benchmark dataset;
[0010] The optimization module compares the extracted features to obtain the mislabeled features, and distinguishes the contrast loss. The nonlinear projection head is used to perform nonlinear transformation on the original features and enhanced features, and the pre-supervision model θ is added. pre In order to improve the distinguishability of the dataset, a two-stage federated learning framework is used to optimize the errors that occur during the labeling process.
[0011] Preferably, the labeling method of the time series labeling device specifically includes the following steps:
[0012] S1. Collect a benchmark data set through the acquisition module: The time series data is established according to the i samples collected by the sensor in the acquisition module, and the time series Ti is equally divided into K small sequences, each of which is defined as an event, i.e., a data set;
[0013] S2: Preprocess the collected data set and establish a pre-supervision model θ pre ;
[0014] S3. Establish an external supervision model based on the preprocessed data set Perform feature extraction;
[0015] S4. Obtain feature set Z = [Z1, Z2, ..., Z N ], where Z is the feature set for feature extraction;
[0016] S5. Compare the benchmark dataset with the obtained feature dataset to find out the possible errors in the annotation process. The nonlinear transformation uses a function F to represent the transformation process, mapping the original data x to the point y in the new space. ′ :(x,y ′ ) = F(x), where F is a polynomial, and for a polynomial transformation, a quadratic polynomial is used: y ′ =(x 2 ,x), and map it into a two-dimensional space, where the original feature x is a one-dimensional time series;
[0017] S6. Optimize the feature data set, including the following operations:
[0018] First, a two-stage federated learning framework is used to establish a mapping function f k (θ; x): x
[0019] →[0,1] cOptimize the error of timestamp mismatch between the timestamp and the benchmark dataset in the annotation process, where θ is the federated learning framework, x is the input space, c is the number of categories, and y = [0,1] c For the time series space, the mapping function satisfies in is the real label;
[0020] S7. Labeling based on the optimized time series data.
[0021] Preferably, the sensor data collection in S1 mainly includes the following steps:
[0022] Step 1: Observe the i-th sample and define a time series function;
[0023] Step 2: When there are N observation samples, a data set containing N time series is obtained;
[0024] Step 3: Convert the time series T i Divide into K small sequences, each of which is called a subsequence of time;
[0025] Step 4: Define the generated subsequence as a data set.
[0026] Preferably, the federated learning framework consists of a server and multiple devices, and each round is implemented by cyclically repeating the local learning and model aggregation process until convergence, at which time the server activates some edge devices and sends the latest model θ G Sent to the device, the activated device learns and trains on the local data. Each device simultaneously performs stochastic gradient descent parameter optimization on its local model parameters and trains on its own private dataset Dk to minimize the local loss.
[0027] Preferably, data preprocessing mainly involves standardizing and normalizing the data.
[0028] Preferably, the federated learning framework further comprises the following steps:
[0029] Step 1. The server uses limited D B The benchmark data is obtained by minimizing the pre-supervised loss Get the pre-supervised model θ pre ;
[0030] Step 2: Server uses D B Benchmark data and pre-supervised model θ pre , based on fine-tuning model loss Generate an externally supervised model for device clean dataset sampling Then the external supervision model And initialize the training model Sending equipment;
[0031] Step 3: Edge device k receives the training task and pre-supervision model at the same time, then collects time series data X according to the task and annotates the label Y to the expert to generate a private local dataset D. k ;
[0032] Step 4: Edge device k uses external supervision model For local private data D k Noise filtering to filter out clean labels Datasets;
[0033] Step 5: Edge device k based on local clean data The global model θ will be obtained G , where θ G Using the formula Local model parameters in
[0034] Step 6: Server usage formula Upload the model to the aggregate edge device during the tth round of training And generate the current global model It is then sent to the edge device for the next iteration, and Step 5 and Step 6 are repeated until most of the noise is filtered out.
[0035] Preferably, the foreign supervision model in Step 2 The training specifically includes the following steps:
[0036] (1) Data enhancement: transform the time series into an enhanced time series through δ;
[0037] (2) Feature extraction: extract the features of the time series and the enhanced time series respectively;
[0038] (3) Self-supervision of the model: predict the contrastive loss and transform the pre-trained model into an external training model based on the benchmark data.
[0039] Preferably, the self-supervised model is used for pre-training, and the external supervised model Used for fine-tuning and global training.
[0040] Compared with the related art, the high-precision time series labeling device provided by the present invention has the following beneficial effects:
[0041] 1. Comparing the benchmark dataset with the feature dataset can timely discover possible errors or inconsistencies in the annotation process, provide clear guidance for subsequent optimization, and use the comparison method to help reduce the degradation of model performance caused by annotation errors and improve the accuracy and reliability of the overall analysis;
[0042] 2. The risk of error propagation is reduced by using a labeling result that gradually approaches the truth. Hierarchical optimization makes the labeling process more transparent and controllable. According to the different characteristics of the data, targeted optimization processing can be performed at the corresponding level. When the data changes, only adjustments and optimizations need to be made at the affected level without the need to comprehensively modify the entire labeling system.
[0043] 3. Labeling based on optimized time series data can significantly improve the accuracy, efficiency, consistency and comparability of labeling, improve the effect of subsequent data analysis, and support a wider range of application scenarios;
[0044] 4. The use of gradually approaching the real annotation results not only improves the accuracy and efficiency of annotation, but also better captures the key features and information in the time series data, providing strong support for subsequent data analysis and application;
[0045] 5. A two-stage federated learning framework is used to identify errors in labeling and filter noise in the time series data collected by the sensor, making the time series labeling more accurate and close to the true value.
[0046] 6. By introducing a nonlinear projection head to perform nonlinear transformation on the original features and enhanced features, the discriminability of the pre-supervised model is significantly increased, which enables the model to more clearly distinguish the differences between different features in the representation space X, thereby improving the accuracy and robustness of the annotation. BRIEF DESCRIPTION OF THE DRAWINGS
[0047] Figure 1 A flowchart of a high-precision time series labeling device module provided by the present invention;
[0048] Figure 2 A flowchart of a high-precision time series labeling device provided by the present invention;
[0049] Figure 3 A flowchart of the time series analysis provided by the present invention;
[0050] Figure 4 A flowchart of a two-stage federated learning framework provided by the present invention;
[0051] Figure 5 The external supervision model provided by the present invention Training flow chart;
[0052] Figure 6 This is a flowchart of the time series comparison processing provided by the present invention. DETAILED DESCRIPTION
[0053] The present invention will be further described below in conjunction with the accompanying drawings and implementation modes.
[0054] Please refer to Figures 1 to 6 ,in, Figure 1 A flowchart of a high-precision time series labeling device module provided by the present invention; Figure 2 A flowchart of a high-precision time series labeling device provided by the present invention; Figure 3 A flowchart of the time series analysis provided by the present invention; Figure 4 A flowchart of a two-stage federated learning framework provided by the present invention; Figure 5 The external supervision model provided by the present invention Training flow chart; Figure 6 This is a flowchart of the time series comparison processing provided by the present invention.
[0055] In the specific implementation process, Figures 1 to 6 As shown, it includes an acquisition module for acquiring time series data sets and benchmark data sets;
[0056] Preprocessing module, which performs data preprocessing on time series data sets;
[0057] Comprehensive processing module, used to extract features from the preprocessed time series dataset and benchmark dataset;
[0058] The optimization module compares the extracted features to obtain the mislabeled features, and distinguishes the contrast loss. The nonlinear projection head is used to perform nonlinear transformation on the original features and enhanced features, and the pre-supervision model θ is added. pre To improve the distinguishability of the labels, a two-stage federated learning framework is used to optimize the errors that occur during the labeling process, making them easier to distinguish in the representation space X.
[0059] It should be further explained that in the discriminative contrast loss, a nonlinear projection head is used to perform nonlinear transformation on the features extracted from the benchmark dataset and the features extracted from the data collected by the sensor to increase the pre-supervised model θ pre The discriminability of the representation is to increase the similarity between the maximized sample and its positive sample, and at the same time increase the similarity between the minimized sample and its negative sample, so that the final representation is more discriminative.
[0060] refer to Figure 1 , Figure 2 , Figure 3 , Figure 6 As shown, the following steps are included:
[0061] S1. Collect a benchmark data set through the acquisition module: The time series data is established according to the i samples collected by the sensor in the acquisition module, and the time series Ti is equally divided into K small sequences, each of which is defined as an event, i.e., a data set;
[0062] S2, preprocessing the collected data set;
[0063] S3. Establish an external supervision model based on the preprocessed data set Perform feature extraction;
[0064] S4. Obtain feature set Z = [Z1, Z2, ..., Z N ], where Z is the feature set for feature extraction;
[0065] S5, comparing the benchmark dataset with the obtained feature dataset to find out the possible errors in the annotation process;
[0066] S6. Optimize the feature data set, including the following operations:
[0067] First, a two-stage federated learning framework is used to establish a mapping function f k (θ; x): x
[0068] →[0,1] c Optimize the errors that occur during the labeling process, where θ is the hierarchical optimization, x is the input space, c is the number of categories, and y = [0,1] c For the time series space, the mapping function satisfies in is the real label;
[0069] S7, labeling according to the optimized time series data;
[0070] Data preprocessing mainly standardizes and normalizes the data;
[0071] It should be noted that the data preprocessing of the data set collected by the sensor includes data cleaning, data arrangement and data conversion. First, data cleaning is performed to check and delete duplicate records in the data set. For missing data points, interpolation methods are used to compensate, maintaining the integrity of the data and the accuracy of the analysis results. After data cleaning, the time series data is sorted according to subsequences, and the data is standardized or normalized after sorting;
[0072] Secondly, an external supervision model is established based on the preprocessed time series Compare the benchmark dataset with the preprocessed feature dataset to find out the possible errors in the annotation process, and use the discriminative contrast loss in the external supervision model to increase the discriminability;
[0073] It should be further explained that the mapping function satisfies The optimization formula is:
[0074]
[0075] in, is the cross entropy loss function of the kth device.
[0076] The sensor data collection in S1 includes the following steps:
[0077] Step 1: Observe the i-th sample and define the time series function as:
[0078] T i ={x1, x2, ..., x n}x j ∈R D ,j=1,2,3,...,n
[0079] Where n is the length of the time series, and D is the latitude of the observation indicator;
[0080] Step 2: When there are N observation samples, a data set containing N time series is obtained:
[0081] D = {T1, T2, ..., T N}
[0082] Step 3: Convert the time series T i Divide it into K small sequences, then each small sequence is called a subsequence of time, and the definition of a subsequence is:
[0083] X k ={x (k-1)*L+1 , x (k-1)*L+2 , ..., x (k*L)}
[0084] Among them, X k ∈R L*D , (K=1, 2, ..., K0), L is a hyperparameter with different values in different datasets;
[0085] Step 4: Define the generated subsequence as a data set.
[0086] It should be noted that for the time series, a temperature sensor was selected, which recorded temperature data once an hour for 72 consecutive hours (i.e., n = 72), for one of the observation indicators (i.e., latitude D = 1), namely, temperature;
[0087] Divide this 72-hour time series into K small sequences. Assuming K = 6, each subsequence will contain data from L = 72 / 6 = 12 time points. These six subsequences are defined as:
[0088] Subsequence 1: {T1, T2, ..., T 12}
[0089] Subsequence 2:{T 13 , T 14 , ..., T 24} ......
[0091] Subsequence 6:{T 61 , T 62 , ..., T 72}
[0092] Define these subsequences as a new dataset;
[0093] It should be further explained that the federated learning framework consists of a server and multiple devices. Each round is achieved by cyclically repeating the local learning and model aggregation process until convergence. At this time, the server activates some edge devices and sends the latest model θ G Sent to the device, the activated device learns and trains on the local data. Each device also performs stochastic gradient descent parameter optimization on its local model parameters and trains on its own private data set Dk to minimize the local loss. The calculation formula is:
[0094]
[0095] Where η is the local learning rate, is the local cross entropy loss;
[0096] The models aggregated by the server are:
[0097]
[0098] Where |D| is the total data volume of all devices, |D k | is the data volume of device k;
[0099] It should be noted that when training on the data set Dk, the above process needs to be continued until convergence.
[0100] refer to Figure 4 , Figure 5 , Figure 6 As shown in the figure, the federated learning framework also includes the following steps:
[0101] Step 1. The server uses limited D B The benchmark data is obtained by minimizing the pre-supervised loss Get the pre-supervised model θ pre ;
[0102] Step 2: Server uses D B Benchmark data and pre-supervised model θ pre , based on fine-tuning model loss Generate an externally supervised model for device clean dataset sampling Then the external supervision model And initialize the training model Sending equipment;
[0103] Step 3: Edge device k receives the training task and pre-supervision model at the same time, then collects time series data X according to the task and annotates the label Y to the expert to generate a private local dataset D. k ;
[0104] Step 4: Edge device k uses external supervision model For local private data D k Noise filtering to filter out clean labels Datasets;
[0105] Step 5: Edge device k based on local clean data The global model θ will be obtained G , where θ G Using the formula Local model parameters in
[0106] Step 6: Server usage formula Model uploaded during round t training on the aggregate edge device And generate the current global model It is then sent to the edge device for the next iteration, and Step 5 and Step 6 are repeated until most of the noise is filtered out;
[0107] It should be noted that through the pre-supervised model θ pre and external supervision models The federated learning framework can effectively identify and filter noise in local data on the device side, reducing the impact of data quality issues on model training and improving the accuracy and reliability of the model.
[0108] It should be further explained that the noise filtering in Step 4 ensures that the edge device uses clean, high-quality data during the training process, and in Step 5, the edge device can adaptively adjust the learning rate based on the characteristics of the local data and the performance during the training process, which helps to accelerate the convergence process and avoid overfitting problems;
[0109] It needs to be further explained that the pre-supervised model θ pre In the absence of any manual annotation, the model is designed to automatically learn from the data and generate useful feature representations by designing preset tasks, and the pre-supervised model θ pre Self-supervised learning is needed, which can extract supervisory signals from unlabeled data and establish a pre-supervised model θ pre It can reduce the reliance on manual annotation and reduce the cost of annotation;
[0110] Step 2: Chinese and foreign supervision model The training specifically includes the following steps:
[0111] (1) Data enhancement: transform the time series into an enhanced time series through δ;
[0112] (2) Feature extraction: extract the features of the time series and the enhanced time series respectively;
[0113] (3) Self-supervision of the model: predicting contrast loss and transforming the pre-trained model into an external training model based on benchmark data;
[0114] And the self-supervised model is used for pre-training, the external supervision model For fine-tuning and global training
[0115] The discriminative contrast loss uses a nonlinear projection head to perform nonlinear transformation on the original features and enhanced features, and adds a pre-supervisory model θ pre The distinctiveness of , making it easier to distinguish in the representation space X;
[0116] It should be noted that by enhancing the time series data, the external supervision model Ability to learn more diverse data representations, thereby improving resistance to noise and outliers, making external supervision models More stable and reliable in practical applications;
[0117] It should be further explained that through self-supervised learning tasks, the external supervision model Able to perform training without manual annotation, making full use of a large amount of unlabeled data resources.
[0118] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments, and the present invention can be implemented in other specific forms without departing from the spirit or basic features of the present invention. Therefore, no matter from which point of view, the embodiments should be regarded as exemplary and non-restrictive. The scope of the present invention is defined by the attached claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention, and any figure mark in the claims should not be regarded as limiting the claims involved.
[0119] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.
Claims
1. High-precision time series labeling equipment, characterized by: It includes a collection module for collecting time series data sets and benchmark data sets; Preprocessing module, which performs data preprocessing on time series data sets; Comprehensive processing module, used to extract features from the preprocessed time series dataset and benchmark dataset; The optimization module compares the extracted features to obtain the mislabeled features, and distinguishes the contrast loss. The nonlinear projection head is used to perform nonlinear transformation on the original features and enhanced features, and the pre-supervision model θ is added. pre In order to improve the distinguishability of the dataset, a two-stage federated learning framework is used to optimize the errors that occur during the labeling process.
2. The high-precision time series labeling device according to claim 1, characterized in that: The labeling method of the time series labeling device specifically includes the following steps: S1. Collect a benchmark data set through the acquisition module: The time series data is established according to the i samples collected by the sensor in the acquisition module, and the time series Ti is equally divided into K small sequences, each of which is defined as an event, i.e., a data set; S2: Preprocess the collected data set and establish a pre-supervision model θ pre ; S3. Establish an external supervision model based on the preprocessed data set Perform feature extraction; S4. Obtain feature set Z = [Z1, Z2, ..., Z N ], where Z is the feature set for feature extraction; S5. Compare the benchmark dataset with the obtained feature dataset to find out the possible errors in the annotation process. The nonlinear transformation uses a function F to represent the transformation process, mapping the original data x to the point y in the new space. ′ :(x,y ′ ) = F(x), where F is a polynomial, and for a polynomial transformation, a quadratic polynomial is used: y ′ =(x 2 ,x), and map it into a two-dimensional space, where the original feature x is a one-dimensional time series; S6. Optimize the feature data set: Use a two-stage federated learning framework to establish a mapping function f k (θ; x) : x → [0, 1] c Optimize the error of timestamp mismatch between the timestamp and the benchmark dataset in the annotation process, where θ is the federated learning framework, x is the input space, c is the number of categories, and y = [0,1] c For the time series space, the mapping function satisfies in is the real label; S7. Labeling based on the optimized time series data.
3. The high-precision time series labeling device according to claim 2, characterized in that: The sensor data collection in S1 includes the following steps: Step 1: Observe the i-th sample and define the time series function as: T i {x1,x2,...,x n }x j ∈R D ,j=1,2,3,...,n Where n is the length of the time series, and D is the latitude of the observation indicator; Step 2: When there are N observation samples, a data set containing N time series is obtained: D={T1,T2,...,T N } Step 3: Convert the time series T i Divide it into K small sequences, then each small sequence is called a subsequence of time, and the definition of a subsequence is: X k ={x (k-1)*L+1 ,x (k-1)*L+2 ,...,x (k*L) } Among them, X k ∈R L*D , (K=1, 2, ..., K0), L is a hyperparameter with different values in different datasets; Step 4: Define the generated subsequence as a data set.
4. The high-precision time series labeling device according to claim 2, characterized in that: The federated learning framework consists of a server and multiple devices. Each round is implemented by cyclically repeating the local learning and model aggregation process until convergence. At this time, the server activates some edge devices and sends the latest model θ G Sent to the device, the activated device learns and trains on the local data. Each device also performs stochastic gradient descent parameter optimization on its local model parameters and trains on its own private data set Dk to minimize the local loss. The calculation formula is: Where η is the local learning rate, is the local cross entropy loss.
5. The high-precision time series labeling device according to claim 4, characterized in that: The models aggregated by the server are: Where |D| is the total data volume of all devices, |D k | is the data volume of device k.
6. The high-precision time series labeling device according to claim 1, characterized in that: Data preprocessing mainly involves standardizing and normalizing the data.
7. The high-precision time series labeling device according to claim 2, characterized in that: The federated learning framework further includes the following steps: Step 1. The server uses limited D B The benchmark data is obtained by minimizing the pre-supervised loss Get the pre-supervised model θ pre ; Step 2: Server uses D B Benchmark data and pre-supervised model θ pre , based on the fine-tuning model loss Generate an externally supervised model for device clean dataset sampling Then the external supervision model And initialize the training model Sending equipment; Step 3: Edge device k receives the training task and pre-supervision model at the same time, then collects time series data X according to the task and annotates the label Y to the expert to generate a private local dataset D. k ; Step 4: Edge device k uses external supervision model For local private data D k Noise filtering to filter out clean labels Datasets; Step 5: Edge device k based on local clean data The global model θ will be obtained G , where θ G Using the formula Local model parameters in Step 6: Server usage formula Model uploaded during round t training on the aggregate edge device And generate the current global model It is then sent to the edge device for the next iteration, and Step 5 and Step 6 are repeated until most of the noise is filtered out.
8. The high-precision time series labeling device according to claim 7, characterized in that: The foreign supervision model in Step 2 The training specifically includes the following steps: (1) Data enhancement: transform the time series into an enhanced time series through δ; (2) Feature extraction: extract the features of the time series and the enhanced time series respectively; (3) Self-supervision of the model: predict the contrastive loss and transform the pre-trained model into an external training model based on the benchmark data.
9. The high-precision time series labeling device according to claim 8, characterized in that: The self-supervision of the model is used for pre-training, and the external supervision model Used for fine-tuning and global training.