Error tag data identification method, device, electronic device and readable storage medium

By introducing the concept of intra-class spatial angle and additive angular margin penalty into the power text classification model, and combining it with a small sample sampling method, the problems of low accuracy and low efficiency in identifying error labels in power field operation texts are solved, and efficient and accurate error label data identification and dataset construction are achieved.

CN117349771BActive Publication Date: 2026-11-17EAST CHINA BRANCH OF STATE GRID CORP
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202311303298.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-10-09
Publication Date
2026-11-17
Estimated Expiration
2043-10-09

AI Technical Summary

Technical Problem

In existing technologies, text classification models in the power industry suffer from low accuracy and inefficiency due to the limited number of training sample sets and the presence of erroneous labels. This is especially true in power field operation texts that are highly specialized and have high text similarity, where it is difficult to effectively identify erroneous labels.

Method used

By normalizing the weight vector and distributing it on a hypersphere, the concept of intra-class spatial angle is proposed, and an additive angular margin penalty is introduced. The ArcFace method is used to separate erroneous label data from correct label data. Combined with a small sample sampling method, the label correction results fed back by the verifiers are obtained for data adjustment, thereby improving the recognition effect and efficiency.

Benefits of technology

It achieves efficient and accurate construction of high-quality datasets, reduces the amount of data to be manually verified, and improves the accuracy and efficiency of identifying erroneous labeled data, making it suitable for accurate risk level rating of power field operation texts.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117349771B_ABST
    Figure CN117349771B_ABST
Patent Text Reader

Abstract

This application discloses a method, apparatus, electronic device, and readable storage medium for identifying mislabeled data, relating to the field of power system technology. It normalizes the weight vector and distributes it on a hypersphere, thereby proposing the concept of intra-class spatial angle. This angle is used to replace the predicted probability as the confidence standard for the prediction result. Based on the analysis of the impact of mislabeled data on the intra-class spatial angle distribution, the ArcFace method is used to introduce an additive angle margin penalty, achieving separation between mislabeled and correct labeled data, facilitating the selection of the spatial angle threshold. An automatic spatial angle threshold selection method and a small sample sampling method are proposed to further improve the mislabeled data identification effect, reduce the amount of data manually checked, and enable the construction of high-quality sample sets for text classification tasks in professional fields.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of power system technology, and in particular to a method, apparatus, electronic device and readable storage medium for identifying error-labeled data. Background Technology

[0002] With the development and maturation of natural language processing technology, power industry texts, due to their inherent data value, have gradually become a research hotspot in power industry data mining. However, current datasets of various business texts in the power industry lag behind general domain datasets in both quality and quantity. Specifically, in the power industry text classification problem, the limited number of training samples and the unpredictable proportion of mislabeled entries in the classification labels often make it difficult to break through the ceiling of classification accuracy simply by selecting and improving the classification model. Text classification often employs neural network models, and the quality of the dataset, as the knowledge acquisition source for the neural network model, directly determines the application effect of the model. Therefore, research on mislabeled data identification in datasets used for text classification training is of great significance.

[0003] In related technologies, the maximum softmax probability (MSP) is typically selected as the basis for identifying mislabeled data. This means the maximum predicted probability is used as the confidence level of the prediction result, and a threshold is set. Suspicious data includes data with a confidence level below the threshold, and data with a high confidence level but whose prediction result is inconsistent with the label value. However, the applicant recognizes that this method is difficult to select the confidence threshold for and is not suitable for highly specialized power field operation texts with high text similarity, resulting in low recognition accuracy and inefficiency. Summary of the Invention

[0004] In view of this, this application provides a method, apparatus, electronic device and readable storage medium for identifying mislabeled data. The main purpose is to solve the problem that the method of selecting the maximum predicted probability value for identifying mislabeled data is difficult to select the confidence threshold and is not suitable for power field operation texts with strong professionalism and high text similarity, resulting in low recognition accuracy and low efficiency. The application aims to achieve efficient and accurate construction of high-quality datasets.

[0005] According to a first aspect of this application, a method for identifying error-labeled data is provided, the method comprising: Acquire sample set data and convert the sample set data into target sample data according to a preset conversion format; The target sample data is input into a pre-trained sample set prediction model to obtain intra-class spatial angle distribution information, and suspicious data is extracted from the target sample data according to the preset bimodal distribution rule and the intra-class spatial angle distribution information. The suspicious data is input into a small sample set prediction model for category prediction to obtain highly suspicious data, and the highly suspicious data is sent to the terminal held by the verification personnel. When the verification personnel receive the label correction results based on their terminal, they use the label correction results to adjust the target sample data to obtain the specified sample set data.

[0006] According to a second aspect of this application, an error label data identification device is provided, the device comprising: The conversion module is used to acquire sample set data and convert the sample set data into target sample data according to a preset conversion format. The extraction module is used to input the target sample data into a pre-trained sample set prediction model to obtain intra-class spatial angle distribution information, and to extract suspicious data from the target sample data according to a preset bimodal distribution rule and the intra-class spatial angle distribution information. The prediction module is used to input the suspicious data into a small sample set prediction model for category prediction to obtain highly suspicious data, and to send the highly suspicious data to the terminal held by the verification personnel. The adjustment module is used to adjust the target sample data using the label correction results fed back by the inspector based on the terminal they are holding, so as to obtain the specified sample set data.

[0007] According to a third aspect of this application, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described in any of the first aspects above.

[0008] According to a fourth aspect of this application, a readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any one of the first aspects above.

[0009] Using the above technical solution, this application provides a method, apparatus, electronic device, and readable storage medium for identifying mislabeled data. This application acquires sample set data, converts the sample set data into target sample data according to a preset conversion format, inputs the target sample data into a pre-trained sample set prediction model, normalizes the weight vectors, distributes them on a hypersphere, and proposes the concept of intra-class spatial angle. This angle replaces the prediction probability as the confidence standard of the prediction result, obtaining the intra-class spatial angle distribution information. Based on the analysis of the influence of mislabeled data on the intra-class spatial angle distribution, the ArcFace method is used to introduce an additive angle margin penalty. The separation between incorrect and correct label data facilitates the selection of spatial angle thresholds and the extraction of suspicious data from the target sample data according to preset bimodal distribution rules and intra-class spatial angle distribution information. Suspicious data is then input into a small sample set prediction model for category prediction to obtain highly suspicious data. This highly suspicious data is then sent to the terminal held by the verification personnel. When the label correction results are received from the verification personnel based on their terminals, the target sample data is adjusted using the label correction results to obtain high-quality sample set data. Then, a small sample sampling method is proposed to further improve the identification effect and efficiency of incorrect label data and reduce the amount of data to be manually verified.

[0010] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0011] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 This illustration shows a flowchart of a method for identifying error tag data according to an embodiment of this application; Figure 2A This illustration shows a flowchart of a method for identifying error tag data according to an embodiment of this application; Figure 2B This illustration shows a schematic diagram of the intra-class spatial angle distribution of feature vectors containing error labels provided in an embodiment of this application. Figure 2C This illustration shows a schematic diagram of the spatial angle distribution after introducing additive angular margin constraints according to an embodiment of this application. Figure 2DA flowchart of an error label data identification method provided in an embodiment of this application is shown; Figure 2E The illustration shows the application effects of different methods provided in the embodiments of this application on different datasets; Figure 2F This illustration shows a schematic diagram of the intra-class spatial angle distribution of the general dataset provided in an embodiment of this application; Figure 2G This illustration shows a schematic diagram of the intraclass spatial angle distribution of power field operation text provided in an embodiment of this application; Figure 3A This illustration shows a structural diagram of an error tag data identification method provided in an embodiment of this application; Figure 3B This illustration shows a structural diagram of an error tag data identification method provided in an embodiment of this application; Figure 4 A schematic diagram of the device structure of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0012] Exemplary embodiments of the present application will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this application will be thorough and complete, and will fully convey the scope of the present application to those skilled in the art.

[0013] This application provides a method for identifying error label data, such as... Figure 1 As shown, the method includes: 101. Obtain the sample set data and convert it into the target sample data according to the preset conversion format.

[0014] The power field operation record document documents the work plan information for power production sites, including work time, work content, voltage level, professional type, and work risk level. Work risk levels are subdivided from high to low into Level I to Level V. The classification of work risk levels is based on relevant guidelines and standards, such as the "Regulations on Work Safety Risk Management of State Grid Corporation of China." However, these guidelines and standards are revised as the power system develops, and previously correct levels may no longer be correct under new evaluation criteria. When new guidelines are issued recently and business personnel are not yet familiar with the new content, incorrect ratings are difficult to avoid. Furthermore, guidelines cannot cover all situations. When situations not covered in the guidelines occur, business personnel need to make subjective ratings based on specific circumstances and personal experience, which also makes it difficult to avoid incorrect ratings. For power field operations, risk level ratings cannot be wrong; that is, the classification labels cannot be incorrect. Incorrect risk level ratings can lead to the development of more stringent work plans on-site, wasting social resources, or even cause work safety accidents, endangering the safe operation of the power grid and the personal safety of workers. When using power field operation texts containing mislabeled data to train a classification model, there is an upper limit to the accuracy of the model's predictions.

[0015] To address this issue, this application proposes a method for identifying mislabeled data. By normalizing the weight vector, the class center vectors of each category are distributed on a hypersphere, thus introducing the concept of intra-class spatial angle and analyzing the impact of mislabeled data on its distribution. Based on the characteristics of the intra-class spatial angle distribution, an additive angular margin penalty is added to separate mislabeled data from correct label data. Then, a small-sample sampling method is combined to achieve efficient and accurate identification of mislabeled data. The implementing entity of this application can be a label data identification system. This system relies on the computing power of a server to provide services to users. The server can be a standalone server or a server providing basic cloud computing services such as cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms, in order to improve the identification effect and efficiency of mislabeled data and reduce the amount of data requiring manual verification.

[0016] In this embodiment, the label data recognition system acquires sample set data and obtains a preset conversion format, wherein the preset conversion format can be a "text label" format. Then, the label data recognition system converts the sample set data into target sample data according to the preset conversion format, so that the subsequent label data recognition system can identify erroneous label data in the target sample data.

[0017] 102. Input the target sample data into the pre-trained sample set prediction model to obtain the intra-class spatial angle distribution information, and extract suspicious data from the target sample data according to the preset bimodal distribution rule and the intra-class spatial angle distribution information.

[0018] Since existing methods for identifying mislabeled data are difficult to improve through algorithm or model replacement, this application utilizes the ArcFace (additive penalty angle algorithm) method to introduce additive angle margin penalty to train a BERT (Bidirectional Encoder Representations from Transformers, a pre-trained language representation model) single-text classification model, resulting in a sample set prediction model. The BERT model is a neural network model proposed by Google AI (Artificial Intelligence) team, which has achieved excellent results in various natural language processing tasks. In this embodiment, the label data identification system inputs the target sample data into the pre-trained sample set prediction model to obtain the intra-class spatial angle distribution information, i.e., the intra-class spatial angle distribution of the target sample data. Then, the label data identification system obtains a preset bimodal distribution rule and extracts suspicious data from the target sample data according to the preset bimodal distribution rule and the intra-class spatial angle distribution information. The preset bimodal distribution rule detects whether the obtained intra-class spatial angle distribution is a clearly separated bimodal distribution, enabling more accurate extraction of suspicious data from the target sample data.

[0019] 103. Input the suspicious data into the small sample set prediction model to perform category prediction, obtain highly suspicious data, and send the highly suspicious data to the terminal held by the verification personnel.

[0020] In this embodiment, if a small sample set prediction model is detected, the label data identification system inputs suspicious data into the small sample set prediction model for category prediction, resulting in highly suspicious data. By using a small sample sampling method to filter suspicious data, highly suspicious data can be obtained from the sample set, thereby achieving efficient and accurate identification of erroneous label data. Then, the label data identification system sends the highly suspicious data to the terminal held by the verification personnel for manual verification.

[0021] 104. When the label correction results are received from the verification personnel based on their terminals, the target sample data is adjusted using the label correction results to obtain the specified sample set data.

[0022] In this embodiment, when the label data recognition system receives the label correction results from the inspector based on their terminal, it adjusts the target sample data using these results to obtain a specified sample set, which is a complete sample set with high label accuracy. Therefore, the label data recognition system employs an automatic spatial angle threshold selection method, which achieves excellent error label data recognition. Then, a small sample sampling method further enhances the error label data recognition effect, ultimately outputting a complete sample set of data for label correction.

[0023] The method provided in this application involves acquiring sample set data, converting the sample set data into target sample data according to a preset conversion format, inputting the target sample data into a pre-trained sample set prediction model, normalizing the weight vectors to distribute them on a hypersphere, and proposing the concept of intra-class spatial angle. This angle is used to replace the prediction probability as the confidence standard of the prediction result, thus obtaining the intra-class spatial angle distribution information. Based on the analysis of the influence of mislabeled data on the intra-class spatial angle distribution, the ArcFace method is used to introduce an additive angle margin penalty to achieve a balance between mislabeled data and correct labeled data. The separation of these components facilitates the selection of spatial angle thresholds and the extraction of suspicious data from the target sample data according to the preset bimodal distribution rules and intra-class spatial angle distribution information. The suspicious data is then input into a small sample set prediction model for category prediction to obtain highly suspicious data. This highly suspicious data is then sent to the terminal held by the verification personnel. When the label correction results are received from the verification personnel based on their terminals, the label correction results are used to adjust the target sample data to obtain high-quality sample set data. Then, a small sample sampling method is proposed to further improve the identification effect and efficiency of erroneous label data and reduce the amount of data to be manually verified.

[0024] Furthermore, as a refinement and extension of the specific implementation methods of the above embodiments, and in order to fully illustrate the specific implementation process of this embodiment, this application provides another method for identifying error tag data, such as... Figure 2A As shown, the method includes: 201. Obtain the sample set data and convert it into the target sample data according to the preset conversion format.

[0025] In this embodiment of the application, the label data recognition system acquires sample set data, acquires a preset conversion format, and converts the sample set data into target sample data according to the preset conversion format, so that the label data recognition system can identify erroneous label data in the sample set data.

[0026] 202. Input the target sample data into the pre-trained sample set prediction model to obtain the intra-class spatial angle distribution information.

[0027] In this embodiment, by normalizing the weight vector, the category center vectors of each category are distributed on a hypersphere. This leads to the concept of intra-class spatial angle, and the impact of erroneous label data on its distribution is analyzed. Thus, based on the characteristics of the intra-class spatial angle distribution, an additive angular margin penalty is applied to separate erroneous and correct label data.

[0028] In text classification tasks, the probability of a neural network predicting the category of an input text is calculated using the following formula: Formula 1: Formula 1: ( ) in, This is the predicted probability matrix for the category to which the input text x belongs. Let x be the text feature vector of the input text, with dimension 1. d is the dimension of the character vector, and W is the weight coefficient matrix of the fully connected layer with dimension d. C represents the number of categories in the classification task.

[0029] The loss function is a function that calculates the difference between the model's predicted values ​​and the actual values ​​of the input data. It is often used to guide the training direction of neural network models; the smaller the loss value, the better the model training effect. In text classification, the Softmax-based cross-entropy loss function is one of the commonly used loss functions. The cross-entropy loss function is given by the following formula: Formula 2:

[0030] in, Let x be the cross-entropy loss function for the input text. The value y represents whether the input text x belongs to the i-th class; it is 1 if it does, and 0 otherwise. The matrix y formed by these values ​​is the one-hot encoding of the true distribution of the input text x. Given input text x, predict the probability that it belongs to the i-th class. Let be the row vector of the i-th row in W, which is the weight vector.

[0031] Therefore, this application is in In the resulting feature vector distribution space, the space formed by the feature vectors of input text belonging to the same category is called the category subspace, and the center vector of the category subspace is called the category center vector. Due to the adjustment of the weight vector... Implement the feature vector of input text of the same category Since they are distributed in similar distribution spaces, the weight vectors... It is equivalent to the class center vector of class i.

[0032] Furthermore, this application defines the intra-class spatial angle as follows: the input text feature vector with class label i in the sample set... , with the class center vector of the i-th class The spatial angle between This represents the intra-class spatial angle. Similarly, the inter-class spatial angle is defined as follows: the input text feature vector with class label i in the sample set... , and the Class center vector The spatial angle between The angle between the spaces of the classes.

[0033] In summary, this application adjusts Formula 1 and Formula 2 based on the intra-class spatial angle to obtain the following Formula 3 for calculating the prediction probability and the following Formula 4 for the adjusted loss function: Formula 3:

[0034] Formula 4:

[0035] in, This represents the probability that the input text is predicted to be of class i. Let the feature vector of the input text be the spatial angle between the center vector of class i. This is the loss value after normalizing the weight vector. From this, we can see that the output of the neural network is... A negative correlation, that is The smaller the value, the greater the probability that the input text is predicted to belong to class i.

[0036] In order to accurately identify mislabeled data, this application has adjusted the loss function again. The adjusted loss function is as follows: Formula 5: Formula 5:

[0037] in, The overall loss value for the sample set containing mislabeled data. Let p be the p-th data point with true class j and sample set label i. Therefore, the overall loss value of the sample set containing mislabeled data consists of two parts: the loss value of the correctly labeled data and the loss value of the mislabeled data. This means that the intra-class spatial angle distribution of the complete sample set will consist of two sub-distributions: the intra-class spatial angle distribution of the correctly labeled data and the intra-class spatial angle distribution of the mislabeled data.

[0038] Because there are differences between category i and category j—category j lacks the textual features of category i or their textual features are opposite—when data from category j is forcibly classified as category i, its convergence difficulty will be greater than that of data from category i. In other words, the convergence difficulty for mislabeled data is greater than that for correctly labeled data. This means that the loss value for mislabeled data is relatively large. As shown in the readjusted loss function, the intra-class space angle of mislabeled data is larger than that of correctly labeled data. Figure 2B As shown, this is reflected in the intra-class spatial angle distribution. The angular margin distribution curve of the entire sample set data containing mislabeled data will exhibit a bimodal distribution. The two peaks are located at the distribution centers of the intra-class spatial angle distribution of correct label data and mislabeled data, respectively, with the correct label data located on the left. This application defines region I to the left of the distribution center of the correct label data as the high-confidence interval, region II between the distribution centers of the correct and mislabeled data as the medium-confidence interval, and region III to the right of the distribution center of the mislabeled data as the low-confidence interval. Thus, the data characteristics of each region are derived: Region I has a small intra-class spatial angle, high confidence, very few mislabeled data, and the prediction results are basically consistent with the labels; Region II has a moderate intra-class spatial angle, moderate confidence, only a portion of the mislabeled data, and the prediction results are partially consistent with the labels; Region III has a large intra-class spatial angle, low confidence, a large number of mislabeled data, and the prediction results are only slightly consistent with the labels.

[0039] Therefore, in order to further enhance the text feature extraction capability of the model, this application uses the ArcFace method in the field of face recognition to add an additive angular margin penalty m to the intra-class spatial angle, and the target loss function is as follows: Formula 6: Formula 6:

[0040] It should be noted that a penalty term is introduced for the intra-class spatial angle, artificially increasing this angle value, while keeping the inter-class spatial angle unchanged. Furthermore, to ensure convergence during model training, after introducing the additive angular margin penalty, the intra-class spatial angle is controlled to not exceed... / 2.

[0041] Finally, this application proposes the concept of intra-class spatial angle and, based on the distribution characteristics of intra-class spatial angle, applies an additive angular margin penalty to obtain the prediction probability calculation formula and the target loss function.

[0042] In one optional implementation, the label data recognition system acquires training samples, reads text feature vectors from these samples, then acquires the weight coefficient matrix of the fully connected layer, reads multiple weight vectors from this matrix, and calculates the spatial angles between the text feature vectors and these weight vectors to obtain multiple intra-class spatial angles. Next, the system acquires a prediction probability calculation formula, uses this formula to calculate the multiple intra-class spatial angles to obtain multiple probability values, then acquires an additive penalty angle algorithm and a loss function. The system obtains the loss function value from the additive penalty angle algorithm and adjusts the loss function to obtain the target loss function. Then, the system acquires the additive penalty angle, uses the target loss function and the additive penalty angle to normalize the multiple intra-class spatial angles to obtain the target loss value, and corrects the spatial angles based on the additive penalty angle to obtain corrected intra-class spatial angles. Finally, the system uses the additive penalty angle algorithm to construct a model using the multiple probability values, the target loss value, and the corrected intra-class spatial angles to obtain a sample set prediction model.

[0043] In this way, the label data recognition system inputs the target sample data into a pre-trained sample set prediction model to obtain the intra-class spatial angle distribution information. For example... Figure 2C As shown, after introducing an additive angular margin penalty in text classification, the overall intra-class spatial angle increases, and its distribution shifts to the right. To counteract the penalty, the model learns to extract more discriminative text features to achieve better classification results, thus reducing the intra-class spatial angle and shifting its distribution to the left. Since the convergence difficulty of correct and incorrect label data differs, the leftward shift of their intra-class spatial angle distributions is inconsistent, thereby separating correct and incorrect label data and facilitating the identification of incorrect label data. Therefore, under this distribution condition, the selection of the spatial angle threshold is relatively simple, and the proposed automatic spatial angle threshold selection method exhibits excellent error label data identification performance.

[0044] 203. Use the preset bimodal distribution rule to detect the spatial angle distribution information within the class; if the spatial angle distribution information within the class does not conform to the preset bimodal distribution rule, then execute the following step 204; if the spatial angle distribution information within the class conforms to the preset bimodal distribution rule, then execute the following step 205.

[0045] In this embodiment, the label data recognition system uses a preset bimodal distribution rule to detect the spatial angle distribution information within a class. If the detected spatial angle distribution information within a class does not conform to the preset bimodal distribution rule, it indicates that the spatial angle distribution within the class is a clearly separated bimodal distribution, and step 204 is executed. If the detected spatial angle distribution information within a class conforms to the preset bimodal distribution rule, it indicates that the spatial angle distribution within the class is not a clearly separated bimodal distribution, and step 205 is executed. This allows for more accurate extraction of suspicious data from the target sample data, thereby improving the accuracy of identifying erroneous label data.

[0046] 204. If the intra-class spatial angle distribution information does not conform to the preset bimodal distribution rule, the intra-class spatial angle distribution statistics are performed in the target sample data using a preset range and preset unit interval to obtain the intra-class spatial angle distribution map. Suspicious data are extracted from the target sample data based on the intra-class spatial angle distribution map, and the following step 206 is executed.

[0047] In this embodiment, the label data recognition system obtains a preset range and a preset unit interval, and uses the preset range and preset unit interval to perform intra-class spatial angle distribution statistics on the target sample data to obtain an intra-class spatial angle distribution map, that is, in The distribution of intra-class spatial angles of the sample set data within a range is statistically analyzed at intervals of 0.002 radians. The intra-class spatial angle distribution map includes a first peak interval and a second peak interval. The first peak interval is... Distributed The text is then processed. Next, the tag data recognition system determines the target spatial angle threshold within the second peak interval according to preset unit intervals. The number of spatial angle threshold values ​​distributed within the preset unit interval is less than the preset angle value, meaning the number of values ​​distributed is less than 1. Angle starting position The system uses this as a threshold for the spatial angle. Then, the label data recognition system determines the position of the target spatial angle threshold within the first and second peak intervals, and designates the interval to the right of this position as the target data distribution interval. Finally, the label data recognition system acquires sample data within the target data distribution interval from the target sample data, treating this sample data as suspicious data. This improves the identification effect and efficiency of erroneous label data and reduces the amount of data requiring manual verification.

[0048] It should be noted that, in order to reduce the influence of random factors, this application performs smoothing processing on the spatial angle distribution within the class, that is, setting the left and right sides of the spatial angle. The sliding window is used, and the average number of texts within the window is taken as the text distribution quantity. The leftmost or rightmost edges of the spatial distribution within the class have extremely high or low confidence levels, and therefore have virtually no impact on the filtering results for erroneous label data; therefore, no smoothing is applied.

[0049] 205. If the intra-class spatial angle distribution information is detected to conform to the preset bimodal distribution rule, the first peak position and the second peak position are determined according to the intra-class spatial angle distribution information. The specified spatial angle threshold is determined according to the first peak position and the second peak position. The target sample data is filtered using the specified spatial angle threshold to obtain suspicious data. Then, the following step 206 is executed.

[0050] In this embodiment, if the intra-class spatial angle distribution information is detected to conform to a preset bimodal distribution rule, the label data recognition system selects the lowest valley value between the two peaks as the threshold for the spatial angle. Specifically, the label data recognition system determines the positions of the first peak and the second peak based on the intra-class spatial angle distribution information, and determines the region between the first peak position and the second peak position based on the intra-class spatial angle distribution information, designating this region as the valley region. Then, the label data recognition system uses the lowest value in the valley region as the specified spatial angle threshold and uses the specified spatial angle threshold to filter the target sample data, obtaining suspicious data. This reduces the amount of data that needs to be manually checked, enabling the rapid and accurate identification of suspicious data and improving the efficiency of identifying erroneous label data.

[0051] 206. Detect whether a small sample set prediction model exists; if a small sample set prediction model is detected, proceed to step 208 below; if no small sample set prediction model is detected, proceed to step 207 below.

[0052] In this embodiment, the label data recognition system utilizes a few-sample sampling method to achieve efficient and accurate identification of erroneous label data. First, the label data recognition system detects whether a few-sample set prediction model exists; if no few-sample set prediction model is detected, it means that the model needs to be trained first, and then step 207 is executed; if a few-sample set prediction model is detected, it means that category prediction can be performed directly, and then step 208 is executed.

[0053] 207. If no small sample set prediction model is detected, random uniform sampling is performed on the target sample data to obtain a small sample set. The small sample set is then used to construct a small sample set prediction model. The following step 208 is then executed.

[0054] In this embodiment, if no small sample set prediction model is detected, the label data recognition system performs random and uniform sampling on the target sample data to obtain a small sample set, where the size of the small sample set is 10% of the complete sample set. Next, the label data recognition system acquires multiple labels from the suspicious data and extracts multiple labels to be verified from the small sample set that match these labels. Subsequently, the label data recognition system sends these multiple labels to the terminal held by the verification personnel. When multiple verified labels are received from the verification personnel based on their terminal, the label data recognition system adjusts the small sample set using these verified labels to obtain a target small sample set. Then, the label data recognition system uses the target small sample set to build a model, obtaining a small sample set prediction model. Training the small sample set prediction model using a small sample set with high label accuracy can further improve the identification effect of erroneous label data.

[0055] 208. If a small sample set prediction model is detected, the suspicious data is input into the small sample set prediction model for category prediction to obtain highly suspicious data, and the highly suspicious data is sent to the terminal held by the inspector, and the following step 209 is executed.

[0056] In this embodiment, if a small sample set prediction model is detected, the label data recognition system inputs suspicious data into the small sample set prediction model for category prediction, obtaining the intra-class spatial angle distribution information of the small samples. Then, the label data recognition system extracts distribution information that is inconsistent with the intra-class spatial angle distribution information from the small sample intra-class spatial angle distribution information, and designates the data corresponding to the distribution information in the suspicious data as highly suspicious data. Alternatively, the label data recognition system reads multiple suspicious label values ​​from the suspicious data and multiple sample label values ​​from the target sample data, designating the suspicious label values ​​that are inconsistent with the multiple sample label values ​​as highly suspicious data, which can further improve the recognition effect and efficiency of erroneous label data.

[0057] 209. When the label correction results are received from the verification personnel based on their terminals, the target sample data is adjusted using the label correction results to obtain the specified sample set data.

[0058] In this embodiment, when the label correction results are obtained from the verification personnel based on their terminals, the label data recognition system adjusts the target sample data using the label correction results to obtain a specified sample set of data. This allows for the manual verification of highly suspicious data labels to obtain a complete sample set with high label accuracy, reducing the amount of data manually verified and improving the efficiency and accuracy of erroneous label data identification. In an optional implementation, the label data recognition system also uses the specified sample set data to train and update a small sample set prediction model, ensuring the accuracy of the small sample set prediction model and thus improving the accuracy of erroneous label data identification.

[0059] Based on the above process, the flowchart of the error label data identification method proposed in this application embodiment is as follows: like Figure 2D As shown, a new sample set of data is input and suspicious data is filtered. Specifically, the new sample set data is input into the training to obtain a complete sample set prediction model, and it is determined whether there is a clear bimodal distribution. If so, the lowest value between the two peaks is used as the spatial angle threshold, and suspicious data in the new sample set is filtered out. If not, the spatial angle of the largest distribution number located to the left of the first peak is used as the spatial angle threshold, and suspicious data in the new sample set is filtered out. In this way, the ArcFace method can be used to train the BERT single-text classification model to obtain the intra-class spatial angle distribution of the sample set, and the angle threshold is automatically selected based on its distribution characteristics to filter out suspicious data in the sample set. Next, it is determined whether a small sample prediction model exists. If not, a small sample set prediction model is trained, that is, a small sample is sampled, and the small sample set data labels are manually checked. The small sample set prediction model is then trained and highly suspicious data is filtered out. If so, the small sample set prediction model is directly used to predict the category of suspicious data, highly suspicious data is filtered out from the suspicious data, and then the labels of highly suspicious data are manually checked to correct the erroneous labels. The small sample set prediction model is then trained and updated. Finally, the complete sample set data for label correction is output. This application proposes the concept of intra-class spatial angle and uses it to replace the predicted probability value as the confidence standard of the prediction result. Then, by introducing an additive angle margin penalty, it achieves the separation of mislabeled data and correct labeled data, and proposes an automatic selection method for spatial angle threshold. Combined with a small sample sampling method, it can achieve efficient and accurate identification of mislabeled data. Moreover, the mislabeled data identification method of this application has versatility and can be applied to the construction of datasets in other fields.

[0060] To verify the feasibility and effectiveness of the proposed error-labeled data identification method based on intra-class spatial angle constraints and small-sample sampling, the following analyses were conducted: 180,000 data entries across 10 classes were selected from the general dataset THUCNews; and 36,146 power field operation text samples from a provincial company within one month were also analyzed. Specifically, 15% of the data from each category in THUCNews were randomly selected, and their category labels were randomly modified, resulting in a sample set with an error rate of 15%. The risk levels of the power field operation text sample set were manually checked, correcting 380 erroneous risk level data entries, resulting in an error rate of 1.05%.

[0061] This application uses the MSP method, which uses the maximum predicted probability value as the basis for identifying mislabeled data, as a comparison method. This method is a commonly used method for identifying mislabeled data. The method uses the maximum predicted probability value as the confidence level of the prediction result and sets a threshold. Suspicious data are those with a confidence level lower than the set threshold, and those with high confidence levels but whose prediction results are inconsistent with the label values.

[0062] To facilitate comparison between the MSP method and the error label data identification method of this application, the predicted probability value and the angle between the intra-class space are normalized to a threshold value ratio. For example... Figure 2E The results show the application effects of the two methods on different datasets. Compared with the MSP method, the error label data identification method of this application has better performance in different datasets. At the same time, the performance of the method is better than that of the MSP method in most threshold value ratios, indicating that the overall performance of the error label data identification method of this application is better than that of the MSP method.

[0063] When the error label data identification method of this application is applied to a general dataset with an error rate of 15%, the intra-class spatial angle distribution is as follows: Figure 2FAs shown, it exhibits a clear bimodal distribution. The spatial angle corresponding to the left peak is 0.324°, with 3229 data points. When the spatial angle is 0.76°, the number of data points is 33, which is 1% of the maximum value. Based on the automatic selection method for spatial angle threshold, 0.76° is used as the threshold, thus identifying a total of 32879 suspicious data points. Further filtering using a small sample method yields 31585 highly suspicious data points, reducing the number of data points by 1294. Therefore, a comparison of the application effect and optimal performance of the error label data identification method of this application was obtained. At optimal performance, the spatial angle threshold was 1.349°, the F1 value was 0.9474, the recall was 94.64%, and the precision was 94.84%. When the spatial angle threshold was automatically selected and small sample sampling was not used, the spatial angle threshold was 0.760°, the F1 value was 0.8871, the recall was 98.37%, and the precision was 80.78%. When the spatial angle threshold was automatically selected and small sample sampling was used, the spatial angle threshold was 0.760°, the F1 value was 0.9067, the recall was 98.37%, and the precision was 84.09%. The error label data identification method of this application is quite close to the optimal performance, demonstrating the effectiveness of the error label data identification method of this application. In practical applications, the proportion of error label data in the sample set is unknown; therefore, a conservative identification strategy is generally adopted, that is, to identify as many error label data as possible. The error label data identification method of this application has a recall rate higher than the optimal performance, but a precision rate lower than the optimal performance, which meets the requirements of practical applications.

[0064] When the error label data identification method of this application is applied to a power field operation text sample set, its intra-class spatial angle distribution is as follows: Figure 2G As shown, the spatial angle distribution within classes of power field operation texts exhibits a subtle bimodal distribution, but it still falls within... A second peak appears between the two peaks. Based on the automatic selection method for the spatial angle threshold, the spatial angle corresponding to the lowest value between the two peaks, 0.836, is selected as the spatial angle threshold, thus filtering out 3030 suspicious data points. Using a small sample sampling method, 2198 highly suspicious data points are obtained, reducing the number of data points by 832. Therefore, a comparison of the error label data identification effect and optimal performance of this application is obtained. At optimal performance, the spatial angle threshold is 1.015, the F1 value is 0.2350, the recall is 55.53%, and the precision is 14.90%. When the spatial angle threshold selection method is automatic and small sample sampling is not used, the spatial angle threshold is 0.836, the F1 value is 0.1690, the recall is 76.05%, and the precision is 9.51%. When the spatial angle threshold selection method is automatic and small sample sampling is used, the spatial angle threshold is 0.836, the F1 value is 0.2172, the recall is 73.68%, and the precision is 12.74%. Although the intra-class spatial angle distribution of power field operation texts differs significantly from that of general datasets, the error label data identification method of this application achieves essentially the same recognition effect on error label data, with a small gap from the optimal performance and a higher recall rate than the optimal performance. It can solve the problem of error label data identification in power field operation texts, that is, detect and identify power field operation texts with incorrect on-site operation risk level ratings in actual power field operation texts. This not only improves the quality of the sample set and thus enhances the performance of the power field operation risk rating model, but also allows the error-prone sample set to serve as a basis for business training and standard supplementation.

[0065] In summary, the application examples in both the general dataset and the power field operation text sample set demonstrate the feasibility and effectiveness of the error label data identification method proposed in this application, proving it to be an efficient and accurate method for error label data identification.

[0066] The method provided in this application involves acquiring sample set data, converting the sample set data into target sample data according to a preset conversion format, inputting the target sample data into a pre-trained sample set prediction model, normalizing the weight vectors to distribute them on a hypersphere, and proposing the concept of intra-class spatial angle. This angle is used to replace the prediction probability as the confidence standard of the prediction result, thus obtaining the intra-class spatial angle distribution information. Based on the analysis of the influence of mislabeled data on the intra-class spatial angle distribution, the ArcFace method is used to introduce an additive angle margin penalty to achieve a balance between mislabeled data and correct labeled data. The separation of these components facilitates the selection of spatial angle thresholds and the extraction of suspicious data from the target sample data according to the preset bimodal distribution rules and intra-class spatial angle distribution information. The suspicious data is then input into a small sample set prediction model for category prediction to obtain highly suspicious data. This highly suspicious data is then sent to the terminal held by the verification personnel. When the label correction results are received from the verification personnel based on their terminals, the label correction results are used to adjust the target sample data to obtain high-quality sample set data. Then, a small sample sampling method is proposed to further improve the identification effect and efficiency of erroneous label data and reduce the amount of data to be manually verified.

[0067] Furthermore, as Figure 1 To specifically implement the method, this application provides an error tag data identification device, such as... Figure 3A As shown, the device includes: a conversion module 301, an extraction module 302, a prediction module 303, and an adjustment module 304.

[0068] The conversion module 301 is used to acquire sample set data and convert the sample set data into target sample data according to a preset conversion format. Extraction module 302 is used to input the target sample data into a pre-trained sample set prediction model to obtain intra-class spatial angle distribution information, and to extract suspicious data from the target sample data according to a preset bimodal distribution rule and the intra-class spatial angle distribution information; The prediction module 303 is used to input the suspicious data into a small sample set prediction model for category prediction to obtain highly suspicious data, and to send the highly suspicious data to the terminal held by the verification personnel. The adjustment module 304 is used to adjust the target sample data using the label correction result when it receives the label correction result fed back by the inspector based on the terminal he / she holds, so as to obtain the specified sample set data.

[0069] In a specific application scenario, the extraction module 302 is used to detect the intra-class spatial angle distribution information using the preset bimodal distribution rule; if the intra-class spatial angle distribution information is detected to conform to the preset bimodal distribution rule, a preset range and a preset unit interval are obtained, and intra-class spatial angle distribution statistics are performed in the target sample data using the preset range and the preset unit interval to obtain an intra-class spatial angle distribution map, which includes a first peak interval and a second peak interval; a target spatial angle threshold is determined in the second peak interval according to the preset unit interval, wherein the number of distributions of the target spatial angle threshold in the preset unit interval is less than a preset angle value; the position of the target spatial angle threshold is determined in the first peak interval and the second peak interval, and the interval to the right of the position is taken as the target data distribution interval; sample data located in the target data distribution interval is obtained in the target sample data, and the sample data is taken as the suspicious data.

[0070] In a specific application scenario, the extraction module 302 is used to determine the first peak position and the second peak position based on the intra-class spatial angle distribution information if the intra-class spatial angle distribution information does not conform to the preset bimodal distribution rule; determine the region between the first peak position and the second peak position based on the intra-class spatial angle distribution information, and designate the region as a valley region; use the lowest value in the valley region as a specified spatial angle threshold, and use the specified spatial angle threshold to filter the target sample data to obtain the suspicious data.

[0071] In specific application scenarios, the prediction module 303 is used to input the suspicious data into the small sample set prediction model for category prediction to obtain the intra-class spatial angle distribution information of the small sample; extract the distribution information that is inconsistent with the intra-class spatial angle distribution information from the intra-class spatial angle distribution information, and take the data corresponding to the distribution information in the suspicious data as the highly suspicious data; or, read multiple suspicious label values ​​from the suspicious data, read multiple sample label values ​​from the target sample data, and take the suspicious label values ​​that are inconsistent with the multiple sample label values ​​as the highly suspicious data.

[0072] In specific application scenarios, such as Figure 3B As shown, the device also includes: a first training module 305, a second training module 306, and an update module 307.

[0073] The first training module 305 is used to acquire training samples, read text feature vectors from the training samples; acquire a fully connected layer weight coefficient matrix, read multiple weight vectors from the fully connected layer weight coefficient matrix, calculate the spatial angles between the text feature vectors and the multiple weight vectors respectively, and obtain multiple intra-class spatial angles; acquire a prediction probability calculation formula, use the prediction probability calculation formula to calculate the multiple intra-class spatial angles, and obtain multiple probability values; acquire an additive penalty angle algorithm and a loss function, acquire a loss function value in the additive penalty angle algorithm, use the loss function value to adjust the loss function, and obtain a target loss function; acquire an additive penalty angle, use the target loss function and the additive penalty angle to normalize the multiple intra-class spatial angles, and obtain a target loss value; correct the multiple spatial angles according to the additive penalty angle, and obtain corrected multiple intra-class spatial angles; use the additive penalty angle algorithm to construct a model using the multiple probability values, the target loss value, and the corrected multiple intra-class spatial angles, and obtain the sample set prediction model.

[0074] The second training module 306 is used to detect whether the small sample set prediction model exists; if the existence of the small sample set prediction model is detected, the step of inputting the suspicious data into the small sample set prediction model is executed; if the absence of the small sample set prediction model is detected, random uniform sampling is performed on the target sample data to obtain a small sample set; multiple labels are obtained from the suspicious data, multiple labels to be verified that match the multiple labels are extracted from the small sample set, and the multiple labels to be verified are sent to the terminal held by the verification personnel; when multiple verified labels are obtained from the verification personnel based on the terminal they hold, the small sample set is adjusted using the multiple verified labels to obtain a target small sample set; the target small sample set is used to construct the model to obtain the small sample set prediction model.

[0075] The update module 307 is used to train and update the small sample set prediction model using the specified sample set data.

[0076] The apparatus provided in this application acquires sample set data, converts the sample set data into target sample data according to a preset conversion format, inputs the target sample data into a pre-trained sample set prediction model, normalizes the weight vectors and distributes them on a hypersphere, thereby proposing the concept of intra-class spatial angle. This angle is used to replace the prediction probability as the confidence standard of the prediction result, obtaining the intra-class spatial angle distribution information. Based on the analysis of the influence of incorrectly labeled data on the intra-class spatial angle distribution, the ArcFace method is used to introduce an additive angle margin penalty to achieve a balance between incorrectly labeled data and correctly labeled data. The separation of these components facilitates the selection of spatial angle thresholds and the extraction of suspicious data from the target sample data according to the preset bimodal distribution rules and intra-class spatial angle distribution information. The suspicious data is then input into a small sample set prediction model for category prediction to obtain highly suspicious data. This highly suspicious data is then sent to the terminal held by the verification personnel. When the label correction results are received from the verification personnel based on their terminals, the label correction results are used to adjust the target sample data to obtain high-quality sample set data. Then, a small sample sampling method is proposed to further improve the identification effect and efficiency of erroneous label data and reduce the amount of data to be manually verified.

[0077] It should be noted that other corresponding descriptions of the functional units involved in the error tag data identification device provided in this application embodiment can be found by referring to... Figure 1 and Figures 2A to 2G The corresponding description in [the document] will not be repeated here.

[0078] In an exemplary embodiment, see Figure 4 Furthermore, an electronic device is provided, comprising a bus, a processor, a memory, and a communication interface. It may also include an input / output interface and a display device, wherein the various functional units can communicate with each other via the bus. The memory stores a computer program, and the processor executes the program stored in the memory to perform the error tag data identification method described in the above embodiments.

[0079] A readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the error tag data identification method.

[0080] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented in hardware or by using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) and includes several instructions to cause a computer device (such as a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0081] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of a preferred embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing this application.

[0082] Those skilled in the art will understand that the modules in the apparatus of the implementation scenario can be distributed within the apparatus of the implementation scenario as described, or they can be located in one or more apparatuses different from this implementation scenario, with corresponding changes. The modules of the above-described implementation scenario can be combined into one module, or they can be further divided into multiple sub-modules.

[0083] The serial numbers in this application are for descriptive purposes only and do not represent the superiority or inferiority of the implementation scenario.

[0084] The above disclosures are only a few specific implementation scenarios of this application. However, this application is not limited to these. Any variations that can be conceived by those skilled in the art should fall within the protection scope of this application.

Claims

1. A method for identifying error label data, characterized in that, include: Acquire sample set data, and convert the sample set data into target sample data according to a preset conversion format. The sample set data is power field operation text data carrying category labels. The sample set data includes multiple sample text data. The preset conversion format is used to combine each sample text data and the category label of each sample text data in the form of text-label. The target sample data is input into a pre-trained sample set prediction model to obtain intra-class spatial angle distribution information. Suspicious data is extracted from the target sample data according to a preset bimodal distribution rule and the intra-class spatial angle distribution information. This includes: detecting the distribution pattern indicated by the intra-class spatial angle distribution information according to the preset bimodal distribution rule; determining a spatial angle threshold based on the detected distribution pattern; and identifying sample data in the target sample data whose intra-class spatial angle is greater than the spatial angle threshold as suspicious data. The sample set prediction model employs an additive angle margin penalty algorithm. The model obtained by training the BERT single-text classification model, wherein the intra-class spatial angle distribution information is the distribution of the intra-class spatial angle of each sample data in the target sample data output by the sample set prediction model, wherein the intra-class spatial angle is the spatial angle between the text feature vector of the sample data with the class label of class i and the class center vector of class i, wherein the class center vector of class i is a weight vector distributed on the hypersphere after normalization, and the preset bimodal distribution rule is used to detect whether the distribution indicated by the intra-class spatial angle distribution information is a clearly separated bimodal distribution; The suspicious data is input into a small sample set prediction model for category prediction to obtain highly suspicious data, and the highly suspicious data is sent to the terminal held by the verification personnel. The highly suspicious data is the sample data in the suspicious data whose predicted category obtained by the small sample set prediction model is inconsistent with the label value of the sample data in the suspicious data. When the verification personnel receive the label correction results based on their terminal, they use the label correction results to adjust the target sample data to obtain the specified sample set data.

2. The method according to claim 1, characterized in that, Before inputting the target sample data into a pre-trained sample set prediction model to obtain the intra-class spatial angle distribution information, the method further includes: Obtain training samples and read text feature vectors from the training samples; Obtain the weight coefficient matrix of the fully connected layer, read multiple weight vectors from the weight coefficient matrix of the fully connected layer, calculate the spatial angle between the text feature vector and the multiple weight vectors respectively, and obtain multiple intra-class spatial angles. The multiple weight vectors are normalized weight vectors distributed on a hypersphere, and each weight vector is equivalent to the class center vector of a class. Obtain the prediction probability calculation formula, and use the prediction probability calculation formula to calculate the included angle of the multiple intra-class spaces to obtain multiple probability values; The process of obtaining an additive penalty angle algorithm and a loss function, obtaining a loss function value in the additive penalty angle algorithm, and adjusting the loss function using the loss function value to obtain a target loss function includes: obtaining the loss function value in the additive penalty angle algorithm, and introducing an additive penalty angle into the loss function according to the additive penalty angle algorithm to obtain the target loss function, wherein the target loss function is used to calculate the loss value corresponding to the intra-class space angle after adding the additive penalty angle; Obtaining the additive penalty angle, and normalizing the multiple intra-class spatial angles using the target loss function and the additive penalty angle to obtain the target loss value, includes: determining the true intra-class spatial angle corresponding to the true class of the training sample among the multiple intra-class spatial angles; adding the additive penalty angle to the true intra-class spatial angle to obtain the penalized intra-class spatial angle; normalizing the penalized intra-class spatial angle with the remaining angles among the multiple intra-class spatial angles other than the true intra-class spatial angle using the target loss function to obtain the normalized probability value of the training sample corresponding to the true class; and calculating the target loss value based on the normalized probability value using the target loss function. Correcting the plurality of spatial angles according to the additive penalty angle to obtain the corrected plurality of intra-class spatial angles includes: correcting the real intra-class spatial angles according to the additive penalty angle, while keeping the other angles in the plurality of intra-class spatial angles other than the real intra-class spatial angles unchanged, to obtain the corrected plurality of intra-class spatial angles; The additive penalty angle algorithm is used to construct a model using the multiple probability values, the target loss value, and the corrected multiple intra-class space angles to obtain the sample set prediction model. This includes: using the multiple probability values ​​as the prediction output of the BERT single-text classification model, using the target loss value as the training direction of the BERT single-text classification model, and using the corrected multiple intra-class space angles as the confidence metric for the prediction result to train the BERT single-text classification model to obtain the sample set prediction model.

3. The method according to claim 1, characterized in that, The step of extracting suspicious data from the target sample data according to the preset bimodal distribution rule and the intra-class spatial angle distribution information includes: The pre-defined bimodal distribution rule is used to detect the spatial angle distribution information within the class. If the intra-class spatial angle distribution information is detected to be inconsistent with the preset bimodal distribution rule, a preset range and a preset unit interval are obtained, and the intra-class spatial angle distribution is statistically analyzed in the target sample data using the preset range and the preset unit interval to obtain an intra-class spatial angle distribution map, wherein the intra-class spatial angle distribution map includes a first peak interval and a second peak interval. Determining a target spatial angle threshold in the first peak interval and the second peak interval according to the preset unit interval includes: traversing the angle interval covered by the first peak interval and the second peak interval from the second peak interval according to the preset unit interval, and determining the starting angle of the unit interval with a distribution number less than a preset angle value as the target spatial angle threshold, wherein the preset angle value is a distribution number threshold, and the distribution number of the target spatial angle threshold in the preset unit interval is less than the preset angle value; The position of the target spatial angle threshold is determined in the first peak interval and the second peak interval, and the interval to the right of the position is taken as the target data distribution interval, wherein the target spatial angle threshold is located within the angle interval covered by the first peak interval and the second peak interval; Sample data that falls within the target data distribution range is obtained from the target sample data, and this sample data is used as the suspicious data.

4. The method according to claim 3, characterized in that, After detecting the intra-class spatial angle distribution information using the preset bimodal distribution rule, the method further includes: If the intra-class spatial angle distribution information is detected to conform to the preset bimodal distribution rule, then the positions of the first peak and the second peak are determined based on the intra-class spatial angle distribution information. Based on the intra-class spatial angle distribution information, the region between the first peak position and the second peak position is determined, and the region is taken as the valley region; The lowest value in the valley region is used as a specified spatial angle threshold. The specified spatial angle threshold is used to filter the target sample data to obtain the suspicious data.

5. The method according to claim 1, characterized in that, Before inputting the suspicious data into a small sample set prediction model for category prediction to obtain highly suspicious data, the method further includes: Detect whether the small sample set prediction model exists; If the existence of the small sample set prediction model is detected, then the step of inputting the suspicious data into the small sample set prediction model is executed; If the small sample set prediction model is not detected, random uniform sampling is performed on the target sample data to obtain a small sample set; Multiple tags are obtained from the suspicious data; multiple tags to be verified that match the multiple tags are extracted from the small sample set; and the multiple tags to be verified are sent to the terminal held by the verification personnel. When multiple verified tags are obtained from the verification personnel based on their terminals, the small sample set is adjusted using the multiple verified tags to obtain the target small sample set; The target small sample set is used to construct the model, resulting in the small sample set prediction model.

6. The method according to claim 1, characterized in that, The step of inputting the suspicious data into a small sample set prediction model for category prediction to obtain highly suspicious data includes: The suspicious data is input into the small sample set prediction model to predict the category, and the spatial angle distribution information within the small sample class is obtained. Extract distribution information that is inconsistent with the intra-class spatial angle distribution information of the small sample, and designate the data corresponding to the distribution information in the suspicious data as the highly suspicious data; or, Multiple suspicious label values ​​are read from the suspicious data, and multiple sample label values ​​are read from the target sample data. Suspicious label values ​​that are inconsistent with the multiple sample label values ​​are taken as highly suspicious data.

7. The method according to claim 1, characterized in that, The method further includes: The small sample set prediction model is trained and updated using the specified sample set data.

8. A device for identifying error label data, characterized in that, include: The conversion module is used to acquire sample set data and convert the sample set data into target sample data according to a preset conversion format. The sample set data is power field operation text data carrying category labels. The sample set data includes multiple sample text data. The preset conversion format is used to combine each sample text data and the category label of each sample text data in the form of text-label. The extraction module is used to input the target sample data into a pre-trained sample set prediction model to obtain intra-class spatial angle distribution information, and to extract suspicious data from the target sample data according to a preset bimodal distribution rule and the intra-class spatial angle distribution information. This includes: detecting the distribution pattern indicated by the intra-class spatial angle distribution information according to the preset bimodal distribution rule; determining a spatial angle threshold based on the detected distribution pattern; and identifying sample data in the target sample data whose intra-class spatial angle is greater than the spatial angle threshold as suspicious data. The sample set prediction model uses additive angular margins. The penalty algorithm is used to train the BERT single-text classification model. The intra-class spatial angle distribution information is the distribution of the intra-class spatial angle of each sample data in the target sample data output by the sample set prediction model. The intra-class spatial angle is the spatial angle between the text feature vector of the sample data with the class label of class i and the class center vector of class i. The class center vector of class i is a weight vector distributed on the hypersphere after normalization. The preset bimodal distribution rule is used to detect whether the distribution indicated by the intra-class spatial angle distribution information is a clearly separated bimodal distribution. The prediction module is used to input the suspicious data into a small sample set prediction model for category prediction to obtain highly suspicious data, and to send the highly suspicious data to the terminal held by the verification personnel. The highly suspicious data is the sample data in the suspicious data whose predicted category obtained by the small sample set prediction model is inconsistent with the label value of the sample data in the suspicious data. The adjustment module is used to adjust the target sample data using the label correction results fed back by the inspector based on the terminal they are holding, so as to obtain the specified sample set data.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.