Data labeling method and device, electronic equipment and storage medium

Abstract

The invention discloses a data annotation method and device, electronic equipment and a storage medium, and belongs to the technical field of computers. According to the data annotation method, annotation related information is obtained, a sample data set and an anomaly detection algorithm are determined based on the annotation related information, the sample data set is annotated through the anomaly detection algorithm, and initial annotation data are obtained. In this way, the sample data set can be labeled.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data labeling method, device, electronic device and storage medium. Background Art

[0002] Data labeling is the process of labeling personnel using labeling tools to process data. In some fields, such as the battery field, it is necessary to rely on the labeled data obtained after labeling to perform a series of operations. Therefore, how to label sample data sets has become a problem that needs to be solved. Summary of the invention

[0003] In view of the above problems, the embodiments of the present application provide a data labeling method, device, electronic device and storage medium, aiming to solve the technical problem of how to label a sample data set.

[0004] In a first aspect, an embodiment of the present application provides a data labeling method, including: obtaining labeling-related information involved in data labeling, determining a sample data set and an anomaly detection algorithm based on the labeling-related information, labeling the sample data set through the anomaly detection algorithm, and obtaining initial labeling data.

[0005] The embodiment of the present application obtains labeling related information to determine a sample data set and an anomaly detection algorithm, and then labels the sample data set through the anomaly detection algorithm. In this way, the sample data set can be labeled.

[0006] In some embodiments, obtaining annotation-related information involved in data annotation includes: displaying a first page; and receiving annotation-related information input through the first page.

[0007] The embodiment of the present application can provide a first page for data annotation, and the annotator can enter the annotation related information in the first page. This annotation method combined with artificial intelligence makes it easier for the annotator to perform data annotation.

[0008] In some embodiments, the labeling related information includes a sample data set identifier and an anomaly detection algorithm identifier. Determining the sample data set and the anomaly detection algorithm based on the labeling related information includes: determining the sample data set based on the sample data set identifier; determining the anomaly detection algorithm based on the anomaly detection algorithm identifier.

[0009] The embodiment of the present application can determine the sample data set according to the sample data set identifier, and determine the anomaly detection algorithm according to the anomaly detection algorithm identifier, so that when there are multiple sample data sets and multiple anomaly detection algorithms, the sample data set and anomaly detection algorithm required for subsequent labeling can be accurately determined.

[0010] In some embodiments, the labeling related information includes a labeling type and attribute information associated with an anomaly detection algorithm. Labeling the sample data set through the anomaly detection algorithm to obtain initial labeling data includes: labeling the sample data set through the anomaly detection algorithm, the labeling type and the attribute information to obtain initial labeling data.

[0011] The embodiments of the present application can combine existing anomaly detection algorithms, annotation types, and attribute information to annotate sample data sets to accurately determine the category of the initial annotated data obtained by the annotation.

[0012] In some embodiments, the annotation type includes a characteristic value and / or an original value. When the annotation type includes a characteristic value, the initial annotation data includes a first type of data and a first annotation result; the first type of data includes data in the sample data set that matches the anomaly detection algorithm, and the first annotation result is used to annotate the data in the first type of data that has anomalies. When the annotation type includes an original value, the initial annotation data includes a second type of data and a second annotation result; the second type of data includes data corresponding to the attribute information in the sample data set, and the second annotation result is used to annotate the data in the second type of data that is related to the cause of the anomaly; and the second annotation result is the same as the first annotation result.

[0013] The embodiment of the present application can facilitate understanding of abnormal data based on the first category of data and the first annotation result in the initial annotation data, and can facilitate analysis of possible causes of the abnormality based on the second category of data and the second annotation result in the initial annotation data.

[0014] In some embodiments, the labeling related information includes a corresponding intelligent labeling condition when the intelligent labeling is enabled. Labeling the sample data set by an anomaly detection algorithm to obtain initial labeling data includes: labeling the sample data set by an anomaly detection algorithm and the intelligent labeling condition to obtain initial labeling data.

[0015] The embodiments of the present application can combine intelligent labeling conditions and existing anomaly detection algorithms to label sample data sets, thereby further accurately determining the labeling results in the initial labeled data.

[0016] In some embodiments, the sample data set includes battery data.

[0017] The embodiments of the present application can realize the labeling of battery data involved in the battery field.

[0018] In some embodiments, after the sample data set is labeled by an anomaly detection algorithm to obtain initial labeled data, the method further includes: displaying the initial labeled data.

[0019] The embodiment of the present application can visually display the initial annotation data so that the annotator can intuitively understand the initial annotation data obtained by annotation.

[0020] In some embodiments, before determining the sample data set and the anomaly detection algorithm based on the annotation related information, it also includes: sending the annotation related information to the server. After annotating the sample data set through the anomaly detection algorithm to obtain the initial annotation data, it also includes: receiving the initial annotation data returned by the server.

[0021] In the embodiment of the present application, the server may use an anomaly detection algorithm to annotate the sample data set, thereby increasing the speed of data annotation.

[0022] In some embodiments, labeling a sample data set by using an anomaly detection algorithm to obtain initial labeling data includes: labeling a locally stored sample data set by using a locally stored anomaly detection algorithm to obtain initial labeling data.

[0023] The embodiment of the present application can directly use the locally stored anomaly detection algorithm to annotate the locally stored sample data set without sending the annotation related information to the server for annotation through the server, thereby reducing data transmission.

[0024] In some embodiments, after the initial annotation data is displayed, the method further includes: receiving a correction operation on the initial annotation data; and in response to the correction operation, correcting the initial annotation data to obtain first target annotation data.

[0025] The embodiment of the present application can perform secondary manual annotation on the basis of the initial annotation data to correct the initial annotation data, thereby improving the annotation efficiency and accuracy of the sample data set.

[0026] In some embodiments, after the initial annotation data is modified in response to the modification operation to obtain the first target annotation data, the method further includes: displaying the first target annotation data.

[0027] The embodiment of the present application can visually display the first target annotation data so that the annotator can intuitively understand the first target annotation data obtained after correction.

[0028] In some embodiments, in response to a correction operation, the initial annotation data is corrected to obtain first target annotation data, including: in response to the correction operation, obtaining the annotation start time and the annotation end time; and correcting the initial annotation data according to the annotation start time and the annotation end time to obtain the first target annotation data.

[0029] In the embodiment of the present application, the initial marking data can be corrected according to the marking start time and the marking end time, and the correction method is relatively simple.

[0030] In some embodiments, before the initial annotation data is corrected according to the annotation start time and the annotation end time to obtain the first target annotation data, the method further includes: obtaining the annotation description and configuration information in response to the correction operation. The annotation description is used to indicate the judgment basis for correcting the initial annotation data; the configuration information is used to configure the display effect of the corrected annotation result in the first target annotation data.

[0031] The embodiment of the present application can provide annotation instructions so that when viewing the first target annotation data obtained after the current correction, the judgment basis of the current correction operation can be directly understood. In addition, the display effect of the correction annotation result can be configured through configuration information, so that the annotator can distinguish the annotation results in the initial annotation data and the correction annotation results after the correction operation, thereby improving the annotation experience of the annotator.

[0032] In some embodiments, in response to the correction operation, the initial annotation data is corrected to obtain the first target annotation data, including: in response to the correction operation, each initial annotation sub-data in a plurality of initial annotation sub-data is corrected to obtain the first target annotation data.

[0033] The embodiment of the present application uses a linkage annotation method to correct each of the multiple initially annotated sub-data, thereby improving the correction efficiency of the corresponding correction operation during the secondary manual annotation.

[0034] In a second aspect, an embodiment of the present application provides a data labeling device, including: a first acquisition module, a determination module, and a labeling module. The first acquisition module is used to acquire labeling related information involved in data labeling; the determination module is used to determine a sample data set and anomaly detection algorithm based on the labeling related information; the labeling module is used to label the sample data set through the anomaly detection algorithm to obtain initial labeling data.

[0035] In a third aspect, an embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to call the computer program to execute the above-mentioned data labeling method.

[0036] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program or instruction is stored. When the computer program or instruction is executed, the above-mentioned data labeling method is implemented.

[0037] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0039] Figure 1 A schematic diagram of a data annotation method according to some embodiments of the present application;

[0040] Figure 2 A schematic diagram of a flow chart of a data annotation method according to some other embodiments of the present application;

[0041] Figure 3 A schematic diagram of the first page in the data annotation process of some embodiments of the present application;

[0042] Figure 4 A schematic diagram of displaying a first annotation pop-up window on a first page in some embodiments of the present application;

[0043] Figure 5 A flowchart of a data annotation method according to some embodiments of the present application;

[0044] Figure 6 A schematic diagram of initial annotation data displayed on a second page in some embodiments of the present application;

[0045] Figure 7 A schematic diagram of another type of initial annotation data displayed on the second page in some embodiments of the present application;

[0046] Figure 8 A flowchart of a data annotation method according to some further embodiments of the present application;

[0047] Fig. 9 A schematic diagram of displaying a second annotation pop-up window on a second page in some embodiments of the present application;

[0048] Fig.10 For some embodiments of this application Figure 6 Schematic diagram of the initial annotation data after correction;

[0049] Fig.11 For some embodiments of this application Figure 7 Schematic diagram of the initial annotation data after correction;

[0050] Fig.12 A schematic diagram of the structure of a data annotation device in some embodiments of the present application;

[0051] Fig.13 This is a schematic diagram of the structure of an electronic device according to some embodiments of the present application. DETAILED DESCRIPTION

[0052] The following embodiments of the technical solution of the present application will be described in detail in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present application, and are therefore only used as examples, and cannot be used to limit the scope of protection of the present application.

[0053] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by technicians in the technical field to which this application belongs; the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit this application; the terms "including" and "having" in the specification and claims of this application and the above-mentioned figure descriptions and any variations thereof are intended to cover non-exclusive inclusions.

[0054] In the description of the embodiments of the present application, the technical terms "first", "second", etc. are only used to distinguish different objects, and cannot be understood as indicating or implying relative importance or implicitly indicating the number, specific order or primary and secondary relationship of the indicated technical features. In the description of the embodiments of the present application, the meaning of "multiple" is more than two, unless otherwise clearly and specifically defined.

[0055] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0056] In the description of the embodiments of the present application, the term "and / or" is only a description of the association relationship of the associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0057] In the description of the embodiments of the present application, the term "multiple" refers to more than two (including two). Similarly, "multiple groups" refers to more than two groups (including two groups), and "multiple pieces" refers to more than two pieces (including two pieces).

[0058] In the description of the embodiments of the present application, the technical terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, which are only for the convenience of describing the embodiments of the present application and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the embodiments of the present application.

[0059] In the description of the embodiments of the present application, unless otherwise clearly specified and limited, technical terms such as "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, and it can be the internal connection of two elements or the interaction relationship between two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the embodiments of the present application can be understood according to the specific circumstances.

[0060] In some fields, such as the battery field, it is necessary to rely on the labeled data obtained after labeling to perform a series of operations. For example, the labeled data obtained after labeling can be compared with the actual abnormal time period in the sample data to verify the detection accuracy of the anomaly detection algorithm and evaluate the quality of the anomaly detection algorithm.

[0061] When it is verified that the quality of the anomaly detection algorithm does not meet the requirements, the anomaly detection algorithm can be further optimized. Specifically, the threshold selected by the anomaly detection algorithm can be optimized, such as the preset voltage threshold, preset temperature threshold, and preset resistance threshold mentioned later, and the algorithm logic of the anomaly detection algorithm can also be optimized. In addition, a new anomaly detection algorithm can be generated based on the labeled data obtained after labeling.

[0062] Therefore, how to label the sample data set becomes a problem that needs to be solved.

[0063] Based on the above considerations, in order to realize the labeling of sample data sets, the embodiments of the present application propose a data labeling method, device, electronic device and storage medium, which obtains labeling-related information to determine the sample data set and the anomaly detection algorithm, and then labels the sample data set through the anomaly detection algorithm. In this way, the sample data set can be labeled.

[0064] In order to better understand the embodiments of the present application, the data labeling method, device, electronic device and storage medium provided according to the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0065] The execution subject of the data annotation method provided in the embodiment of the present application may be an electronic device, which includes but is not limited to a personal computer, a laptop computer, a smart phone, a tablet computer, a wearable device (such as a smart watch, a smart bracelet), etc. It should be noted that the above execution subjects do not constitute a limitation on the embodiments of the present application.

[0066] In some embodiments of the present application, reference Figure 1 As shown, Figure 1 The following is a flow chart of a data annotation method according to some embodiments of the present application. The data annotation method may include:

[0067] S10: Obtaining annotation-related information of participants in data annotation.

[0068] In some examples, annotation-related information input by annotators may be obtained, and the annotation-related information is used to determine a sample data set and anomaly detection algorithm involved in data annotation.

[0069] S20: Determine a sample data set and anomaly detection algorithm based on the labeled related information.

[0070] The sample data set may include pre-collected sample data, and the anomaly detection algorithm is an algorithm for detecting whether there are anomalies in the sample data.

[0071] It should be noted that the anomaly detection algorithm is a piece of code written by relevant personnel to detect whether there are anomalies in sample data. It is not a neural network model obtained through training.

[0072] S30: Annotate the sample data set using an anomaly detection algorithm to obtain initial annotated data.

[0073] The sample data set may be annotated by an anomaly detection algorithm to obtain initial annotated data. The initial annotated data may include the corresponding sample data in the sample data set and the annotated result of the anomaly detection algorithm on the corresponding sample data in the sample data set, and the annotated result may be the time period when the sample data has anomalies.

[0074] In this way, after the initial annotation data is obtained by annotating in the above manner, the annotator can compare the initial annotation data with the actual abnormal time period in the sample data to verify the detection accuracy of the anomaly detection algorithm, so as to evaluate the quality of the anomaly detection algorithm, so that the initial annotation data can provide data support for the research and development of the anomaly detection algorithm and the verification of its effect. After optimizing the anomaly detection algorithm based on the initial annotation data and generating an anomaly detection algorithm with higher detection accuracy, the optimized anomaly detection algorithm can be used to efficiently improve the accuracy of the detected sample data anomalies.

[0075] The data labeling method proposed in the embodiment of the present application obtains labeling related information to determine a sample data set and an anomaly detection algorithm, and then labels the sample data set through the anomaly detection algorithm. In this way, the sample data set can be labeled.

[0076] In some embodiments, the sample data set includes battery data.

[0077] In this way, the sample data included in the sample data set may be battery data, and the battery data may be battery data collected from a battery in the vehicle under any operating condition. For example, the battery data may be battery data collected from a battery in the vehicle during charging or driving.

[0078] The battery in the vehicle may include one or more battery packs, each of which may include multiple cells. Accordingly, the battery data may include multiple data, such as the voltage value of each cell in the battery pack, the current value of each cell in the battery pack, multiple temperature values ​​of the battery pack, multiple insulation resistance values ​​of the battery pack, and the state of charge (SOC) value of the battery pack.

[0079] It can be understood that the battery data in the embodiments of the present application is not limited to the battery data collected from the batteries in the vehicle, and it can also be the battery data collected from the energy storage battery or other types of batteries. The embodiments of the present application are not limited to this.

[0080] Taking the sample data in the sample data set as battery data as an example, the anomaly detection algorithm may be an analysis algorithm for the battery field, which may be an algorithm for detecting whether a battery has an abnormality.

[0081] Since the battery data includes multiple types, the abnormality detection algorithms for detecting the battery data may also include multiple types, and each abnormality detection algorithm corresponds to at least one type of battery data. In some embodiments, the abnormality detection algorithms include but are not limited to: an excessive pressure difference detection algorithm, an excessive temperature difference detection algorithm, and an insulation resistance value too low detection algorithm.

[0082] Among them, the voltage difference detection algorithm is used to detect whether the battery has an abnormality of excessive voltage difference. Specifically, it can be determined whether the battery has an abnormality of excessive voltage difference based on the size relationship between the voltage difference value of the same battery pack in the sample data set and the preset voltage threshold. The voltage difference value can be the difference between the highest voltage value and the lowest voltage value among the voltage values ​​of each battery cell in the same battery pack at the same time; the preset voltage threshold can be set according to the actual situation.

[0083] When the voltage difference is greater than the preset voltage threshold, that is, there is a large difference in voltage values ​​between different cells in the battery pack, it can be considered that the battery has an abnormality of excessive voltage difference; when the voltage difference is less than or equal to the preset voltage threshold, that is, the difference in voltage values ​​between different cells in the battery pack is small, it can be considered that the battery does not have an abnormality of excessive voltage difference.

[0084] The excessive temperature difference detection algorithm is used to detect whether the battery has an abnormality of excessive temperature difference. Specifically, it can be determined whether the battery has an abnormality of excessive temperature difference based on the relationship between the temperature difference value of the same battery pack in the sample data set and the preset temperature threshold. The temperature difference value can be the difference between the highest temperature value and the lowest temperature value among the temperature values ​​collected by each temperature sensor in the same battery pack at the same time; the preset temperature threshold can be set according to the actual situation.

[0085] When the temperature difference is greater than the preset temperature threshold, that is, there is a large difference in temperature values ​​at different locations in the battery pack, it can be considered that the battery has an abnormality of excessive temperature difference; when the temperature difference is less than or equal to the preset temperature threshold, that is, the difference between the temperature values ​​at different locations in the battery pack is small, it can be considered that the battery does not have an abnormality of excessive temperature difference.

[0086] The low insulation resistance detection algorithm is used to detect whether the battery has an abnormality of low insulation resistance. Specifically, it can be determined whether the battery has an abnormality of low insulation resistance based on the size relationship between the lowest insulation resistance of the same battery pack in the sample data set and the preset resistance threshold. The lowest insulation resistance can be the minimum value among multiple insulation resistance values ​​of the same battery pack at the same time; the preset resistance threshold can be set according to actual conditions.

[0087] When the minimum insulation resistance is less than the preset resistance threshold, it can be considered that the battery has an abnormality of too low insulation resistance; when the minimum insulation resistance is greater than or equal to the preset resistance threshold, it can be considered that the battery does not have an abnormality of too low insulation resistance.

[0088] The data labeling method proposed in the embodiment of the present application can realize the labeling of battery data involved in the battery field.

[0089] In some embodiments, the annotation personnel can input the annotation related information in the first page to obtain the annotation related information. Figure 2 The above step S10 may specifically include:

[0090] S101, displaying the first page.

[0091] It is understandable that the first page may also be referred to as a data annotation page, which is used to receive annotation-related information input by an annotator. Figure 3 A first page 400 is shown.

[0092] S102, receiving annotation related information input through the first page.

[0093] The annotator can operate on the first page 400 to input the annotation related information, so that the annotation related information input by the annotator through the first page 400 is received.

[0094] The data labeling method proposed in the embodiment of the present application can provide a first page used for data labeling, and the labeling personnel can enter the labeling related information in the first page. This labeling method combined with artificial intelligence facilitates the labeling personnel to perform data labeling.

[0095] In some embodiments, the labeling related information includes a sample data set identifier and an anomaly detection algorithm identifier. Therefore, the above step S20 may specifically include: determining the sample data set based on the sample data set identifier; and determining the anomaly detection algorithm based on the anomaly detection algorithm identifier.

[0096] The sample data set identifier is used to determine the sample data set involved in data annotation, and the sample data set identifier can be defined by the file name of the sample data set, etc. The anomaly detection algorithm identifier is used to determine the anomaly detection algorithm involved in data annotation, and the anomaly detection algorithm identifier can be defined by the algorithm name of the anomaly detection algorithm, etc.

[0097] Take the example of the annotator entering the annotation related information on the first page. Figure 3 As shown, the first page 400 includes a device identification selection control 401 , a sample data set identification selection control 402 , and an anomaly detection algorithm identification selection control 403 .

[0098] The annotator can select any device identification (ie, device ID) in the drop-down list of the device identification selection control 401. The device identification may refer to the identification of a vehicle, and each vehicle has a unique identification.

[0099] There is a corresponding relationship between the device identifier and the sample data set identifier, and each device identifier corresponds to one or more sample data set identifiers. By selecting the corresponding device identifier from the drop-down list of the device identifier selection control 401, one or more sample data set identifiers corresponding to the selected device identifier are further displayed in the drop-down list of the sample data set identifier selection control 402. The annotator can continue to select any sample data set identifier in the drop-down list of the sample data set identifier selection control 402.

[0100] For example, the device identifier selected by the annotator in the drop-down list of the device identifier selection control 401 is: 04570204e762a6d2c, and the sample data set identifier selected by the annotator in the drop-down list of the sample data set identifier selection control 402 is: 04570204e762a6d2c_2023-01-29.csv.

[0101] The annotator may also select any anomaly detection algorithm identifier in the drop-down list of the anomaly detection algorithm identifier selection control 403. For example, the anomaly detection algorithm identifier selected by the annotator in the drop-down list of the anomaly detection algorithm identifier selection control 403 is: excessive pressure difference detection algorithm.

[0102] And, if Figure 3 As shown, the first page 400 also includes a query control 407. After receiving the sample data set identifier input through the device identifier selection control 401 and the sample data set identifier selection control 402, and the anomaly detection algorithm identifier input through the anomaly detection algorithm identifier selection control 403, the annotator can operate the query control 407, such as clicking. In response to the operation input to the query control 407, the electronic device or server can determine the sample data set based on the sample data set identifier, and determine the anomaly detection algorithm based on the anomaly detection algorithm identifier, so that the electronic device or server annotates the sample data set through the anomaly detection algorithm to obtain initial annotated data.

[0103] In addition, if Figure 3 As shown, the first page 400 also includes a reset control 408. The reset control 408 is used to delete the annotation related information input on the first page 400 after being triggered.

[0104] The data labeling method proposed in the embodiment of the present application can determine the sample data set according to the sample data set identifier, and determine the anomaly detection algorithm according to the anomaly detection algorithm identifier, so that when there are multiple sample data sets and multiple anomaly detection algorithms, the sample data sets and anomaly detection algorithms required for subsequent labeling can be accurately determined.

[0105] In some embodiments, the labeling related information may include, in addition to the sample data set identifier and the anomaly detection algorithm identifier, the labeling type and attribute information associated with the anomaly detection algorithm. Therefore, the above step S30 may specifically include: labeling the sample data set using the anomaly detection algorithm, the labeling type and the attribute information to obtain initial labeling data.

[0106] Take the example of the annotator entering the annotation related information on the first page. Figure 3 As shown, the first page 400 may include a label type selection control 405 and a property selection control 404 in addition to the above-mentioned device identification selection control 401 , sample data set identification selection control 402 , and anomaly detection algorithm identification selection control 403 .

[0107] The annotator can also select the characteristic value and the original value in the annotation type selection control 405 to determine whether the annotation type used for data annotation is one of the characteristic value and the original value, or both.

[0108] The annotator can also select various attribute information corresponding to the attribute selection control 404 to determine the attribute information used for data annotation. The attribute information selected by the annotator can include one or more attribute information.

[0109] For example, if the anomaly detection algorithm selected by the labeler is identified as the excessive pressure difference detection algorithm, the first page 400 displays various attribute information including current, maximum temperature, minimum temperature and average temperature. Among them, the current in the attribute information can be used for data labeling by default, while the maximum temperature, minimum temperature and average temperature in the attribute information can be selected by the labeler according to the actual situation.

[0110] After receiving the sample data set identifier input through the device identifier selection control 401 and the sample data set identifier selection control 402, the anomaly detection algorithm identifier input through the anomaly detection algorithm identifier selection control 403, the annotation type input through the annotation type selection control 405, and the attribute information input through the attribute selection control 404, the annotator can operate the query control 407. In response to the operation input to the query control 407, the electronic device or server can determine the sample data set based on the sample data set identifier and determine the anomaly detection algorithm based on the anomaly detection algorithm identifier, thereby allowing the electronic device or server to annotate the sample data set through the anomaly detection algorithm, the annotation type and the attribute information to obtain initial annotated data.

[0111] The data labeling method proposed in the embodiment of the present application can be combined with existing anomaly detection algorithms, labeling types and attribute information to label sample data sets to accurately determine the category of the initial labeled data obtained by labeling.

[0112] In some embodiments, the annotation type includes a characteristic value and / or an original value. When the annotation type includes a characteristic value, the initial annotation data includes a first type of data and a first annotation result; the first type of data includes data in the sample data set that matches the anomaly detection algorithm, and the first annotation result is used to annotate the data in the first type of data that has anomalies. When the annotation type includes an original value, the initial annotation data includes a second type of data and a second annotation result; the second type of data includes data corresponding to the attribute information in the sample data set, and the second annotation result is used to annotate the data in the second type of data that is related to the cause of the anomaly; and the second annotation result is the same as the first annotation result.

[0113] Each anomaly detection algorithm has one or more fixed matching data, and different anomaly detection algorithms may match different data. The data matched with the anomaly detection algorithm refers to the data used to input into the anomaly detection algorithm to detect whether there is an anomaly. The first annotation result can be the time period in which the anomaly exists in the first type of data.

[0114] For example, the abnormality detection algorithm may be an excessive voltage difference detection algorithm, and the data matching the excessive voltage difference detection algorithm in the sample data set includes the average voltage value, the highest voltage value, and the lowest voltage value. The average voltage value represents the average value of the voltage values ​​of each battery cell in the same battery pack at the same time; the highest voltage value represents the maximum value of the voltage values ​​of each battery cell in the same battery pack at the same time; and the lowest voltage value represents the minimum value of the voltage values ​​of each battery cell in the same battery pack at the same time.

[0115] In this way, when the anomaly detection algorithm is an excessive voltage difference detection algorithm, the first type of data includes the average voltage value, the highest voltage value and the lowest voltage value in the sample data set, and the first annotation result can be the time period in which there are anomalies in the average voltage value, the highest voltage value and the lowest voltage value in the sample data set.

[0116] Each anomaly detection algorithm corresponds to one or more pre-set attribute information. The attribute information corresponding to different anomaly detection algorithms may be the same or different. The attribute information corresponding to the anomaly detection algorithm indicates the attribute information that may be associated with the anomaly detected by the anomaly detection algorithm. The second annotation result is the direct use of the first annotation result as the annotation result of the second type of data.

[0117] For example, the abnormality detection algorithm may be an excessive voltage difference detection algorithm, and the attribute information that may be associated with the voltage difference of the battery pack may include current, maximum temperature, minimum temperature, and average temperature, etc., that is, it is possible that the voltage difference of the battery pack increases due to excessive current, or it is possible that the voltage difference of the battery pack increases due to excessive temperature. Therefore, the attribute information corresponding to the excessive voltage difference detection algorithm includes current, maximum temperature, minimum temperature, and average temperature; accordingly, the data corresponding to the attribute information in the sample data set includes current value, maximum temperature value, minimum temperature value, and average temperature value. The maximum temperature value represents the maximum value of the temperature values ​​collected by each temperature sensor in the same battery pack at the same time; the minimum temperature value represents the minimum value of the temperature values ​​collected by each temperature sensor in the same battery pack at the same time; and the average temperature value represents the average value of the temperature values ​​collected by each temperature sensor in the same battery pack at the same time.

[0118] In this way, when the anomaly detection algorithm is an excessive pressure difference detection algorithm, the second type of data includes the current value, maximum temperature value, minimum temperature value and average temperature value in the sample data set, and the second annotation result directly adopts the first annotation result as the annotation result of the current value, maximum temperature value, minimum temperature value and average temperature value in the sample data set.

[0119] Therefore, the initial annotated data may include the corresponding sample data in the sample data set, and the annotated results of the corresponding sample data in the sample data set by the anomaly detection algorithm. The sample data in the initial annotated data includes the first category of data and / or the second category of data mentioned above. Furthermore, when the initial annotated data includes the first category of data, the annotated results in the initial annotated data include the first annotated results mentioned above; when the initial annotated data includes the second category of data, the annotated results in the initial annotated data include the second annotated results mentioned above.

[0120] The data labeling method proposed in the embodiment of the present application can facilitate understanding of abnormal data based on the first category of data in the initial labeled data and the first labeling result, and can facilitate analysis of possible causes of the abnormality based on the second category of data in the initial labeled data and the second labeling result.

[0121] In some embodiments, the labeling related information may include not only the sample data set identifier and the anomaly detection algorithm identifier, but also the corresponding intelligent labeling conditions when the intelligent labeling is started. Therefore, the above step S30 may specifically include: labeling the sample data set through the anomaly detection algorithm and the intelligent labeling conditions to obtain initial labeling data.

[0122] When smart labeling is enabled, if the anomaly detection algorithm itself supports algorithm labeling, the initial labeling data may include the initial labeling data corresponding to the anomaly detection algorithm and the initial labeling data corresponding to the smart labeling conditions. Of course, smart labeling may not be enabled, in which case the initial labeling data may include the initial labeling data corresponding to the anomaly detection algorithm.

[0123] It should be noted that some anomaly detection algorithms may support algorithm labeling, and the initial labeling data may include the labeling results of the corresponding sample data in the sample data set; some anomaly detection algorithms may not support algorithm labeling, and the initial labeling data may not include the labeling results of the corresponding sample data in the sample data set.

[0124] Take the example of the annotator entering the annotation related information on the first page. Figure 3 As shown, the first page 400 may include an intelligent labeling switch control 406 in addition to the above-mentioned device identification selection control 401 , sample data set identification selection control 402 , and anomaly detection algorithm identification selection control 403 .

[0125] The annotator can also determine whether to start the smart annotation according to the actual situation. If it is determined that the smart annotation needs to be started, the annotator can turn on the smart annotation switch control 406. In response to the turning on operation of the smart annotation switch control 406, Figure 4 As shown, a first annotation pop-up window 500 may be displayed on the first page 400. The first annotation pop-up window 500 includes a smart annotation condition input control 501, a first save control 502, and a first cancel control 503. The first cancel control 503 is used to cancel the currently started smart annotation after being triggered.

[0126] The labeler can input the smart labeling condition in the smart labeling condition input control 501. In some implementations, the smart labeling condition can be a threshold range input by the labeler. The anomaly detection algorithm can be an excessive voltage difference detection algorithm. In this case, the smart labeling condition can be a voltage threshold input by the labeler. For example, the voltage threshold input by the labeler can be 50mV.

[0127] After receiving the smart annotation conditions input into the smart annotation condition input control 501, the annotator can operate the first save control 502, such as clicking, etc., in response to the operation input into the first save control 502, so that the electronic device obtains the corresponding smart annotation conditions when the smart annotation is started.

[0128] After receiving the sample data set identifier input through the device identifier selection control 401 and the sample data set identifier selection control 402, the anomaly detection algorithm identifier input through the anomaly detection algorithm identifier selection control 403, and the intelligent labeling condition input through the intelligent labeling switch control 406 and the first labeling pop-up window 500, the labeling personnel can operate the query control 407. In response to the operation input to the query control 407, the electronic device or server can determine the sample data set based on the sample data set identifier, and determine the anomaly detection algorithm based on the anomaly detection algorithm identifier, so that the electronic device or server can label the sample data set through the anomaly detection algorithm and the intelligent labeling condition to obtain initial labeling data.

[0129] The data labeling method proposed in the embodiment of the present application can combine intelligent labeling conditions and existing anomaly detection algorithms to label sample data sets, thereby further accurately determining the labeling results in the initial labeled data.

[0130] It is understandable that the annotator can also input the sample data set identifier through the device identifier selection control 401 and the sample data set identifier selection control 402, input the anomaly detection algorithm identifier through the anomaly detection algorithm identifier selection control 403, input the annotation type through the annotation type selection control 405, input the attribute information through the attribute selection control 404, and input the intelligent annotation condition after turning on the intelligent annotation switch control 406. Afterwards, the annotator can operate the query control 407, and in response to the operation input to the query control 407, the electronic device or server can determine the sample data set based on the sample data set identifier, and determine the anomaly detection algorithm based on the anomaly detection algorithm identifier, so that the electronic device or server can annotate the sample data set through the anomaly detection algorithm, annotation type, attribute information and intelligent annotation conditions to obtain initial annotation data.

[0131] In some embodiments, after the sample data set is annotated by the anomaly detection algorithm to obtain the initial annotated data, the initial annotated data can also be displayed. Figure 5 After the above step S30, the method may further include:

[0132] S40: Displaying initial annotation data.

[0133] In one implementation, the initial annotation data is displayed in a chart form on the second page. Where the initial annotation data includes the first category of data and the first annotation result corresponding to the first category of data, the first annotation result selects the data with abnormalities in the first category of data in the form of an annotation box. Where the initial annotation data includes the second category of data and the second annotation result corresponding to the second category of data, the second annotation result selects the data related to the cause of the abnormality in the second category of data in the form of an annotation box.

[0134] In one case, if the annotator selects a feature value in the annotation type selection control 405 displayed on the first page 400 , the initial annotation data obtained by annotation includes a first category of data and a first annotation result corresponding to the first category of data.

[0135] For example, when the anomaly detection algorithm selected by the annotator on the first page 400 is identified as the excessive pressure difference detection algorithm, and the annotation type selected on the first page 400 is the characteristic value, after receiving the operation input to the query control 407, the first page 400 may be displayed as follows: Figure 6 The second page 600 is shown.

[0136] like Figure 6 As shown, the second page 600 displays initial annotation data, and the initial annotation data includes first-category data and a first annotation result corresponding to the first-category data.

[0137] The first type of data includes the average voltage value, the highest voltage value, and the lowest voltage value in the sample data set, and the average voltage value, the highest voltage value, and the lowest voltage value are displayed in a chart in accordance with time changes. Figure 6 As shown, the horizontal axis represents the time when the data such as the average voltage value, the highest voltage value and the lowest voltage value in the sample data set are collected; the vertical axis represents the voltage values ​​of the data such as the average voltage value, the highest voltage value and the lowest voltage value in the sample data set; and each curve segment represents the change trend of the average voltage value, the highest voltage value and the lowest voltage value collected according to the collection time.

[0138] The first annotation result may be a time period in which anomalies exist in the average voltage value, the highest voltage value, and the lowest voltage value in the sample data set. Figure 6 As shown, the first annotation result 61 is in the form of an annotation box, which selects the battery data with abnormalities in the average voltage value, the highest voltage value and the lowest voltage value.

[0139] In another case, if the annotator selects the original value in the annotation type selection control 405 displayed on the first page 400, and selects the corresponding attribute information in the attribute selection control 404 displayed on the first page 400, the initial annotation data obtained includes the second category of data and the second annotation result corresponding to the second category of data.

[0140] For example, if the anomaly detection algorithm selected by the annotator on the first page 400 is an excessive pressure difference detection algorithm, the annotation type selected on the first page 400 is an original value, and the attribute information selected on the first page includes current, maximum temperature, minimum temperature, and average temperature, after receiving an operation input to the query control 407, the first page 400 may be displayed as follows: Figure 7 The second page 600 is shown.

[0141] like Figure 7 As shown, the second page 600 displays initial annotation data, and the initial annotation data includes second category data and second annotation results corresponding to the second category data.

[0142] Among them, the second category of data includes the current value, maximum temperature value, minimum temperature value and average temperature value in the sample data set, and the current value, maximum temperature value, minimum temperature value and average temperature value are displayed separately in the form of charts according to time changes, that is, the current value, maximum temperature value, minimum temperature value and average temperature value are not displayed in the same chart.

[0143] like Figure 7 As shown, the horizontal axis in the chart corresponding to the current value represents the time when the current value in the sample data set is collected, the vertical axis in the chart corresponding to the current value represents the specific numerical value of the current value in the sample data set, and the curve segment in the chart corresponding to the current value represents the change trend of the current value collected according to the collection time.

[0144] like Figure 7 As shown, the horizontal axis in the chart corresponding to the maximum temperature value represents the time when the maximum temperature value in the sample data set is collected, the vertical axis in the chart corresponding to the maximum temperature value represents the specific numerical value of the maximum temperature value in the sample data set, and the curve segment in the chart corresponding to the maximum temperature value represents the changing trend of the maximum temperature value collected according to the collection time.

[0145] like Figure 7 As shown, the horizontal axis in the chart corresponding to the lowest temperature value represents the time when the lowest temperature value in the sample data set is collected, the vertical axis in the chart corresponding to the lowest temperature value represents the specific numerical value of the lowest temperature value in the sample data set, and the curve segment in the chart corresponding to the lowest temperature value represents the changing trend of the lowest temperature value collected according to the collection time.

[0146] like Figure 7 As shown, the horizontal axis in the chart corresponding to the average temperature value represents the time when the average temperature value in the sample data set is collected, the vertical axis in the chart corresponding to the average temperature value represents the specific numerical value of the average temperature value in the sample data set, and the curve segment in the chart corresponding to the average temperature value represents the changing trend of the average temperature value collected according to the collection time.

[0147] like Figure 7 As shown, the second annotation result 62 is in the form of an annotation box, which selects data related to the cause of abnormality in the current value, the highest temperature value, the lowest temperature value and the average temperature value, which may cause the average voltage value, the highest voltage value and the lowest voltage value to be abnormal. Figure 7 The time period of the data related to the cause of the abnormality selected by the second annotation result 62 is Figure 6 The time periods in which the anomalies are framed by the first annotation result 61 are the same.

[0148] In this way, the labeling personnel can analyze the second labeling results 62 corresponding to the current value, the maximum temperature value, the minimum temperature value and the average temperature value to determine the possible reasons for the abnormal time period in the average voltage value, the maximum voltage value and the minimum voltage value.

[0149] It can be understood that if the annotator selects characteristic value and original value in the annotation type selection control 405 displayed on the first page 400, the first category of data and its first annotation result, and the second category of data and its second annotation result will be displayed in the form of charts on the same second page 600.

[0150] It should be noted that Figure 7 The scenario shown is for the initial annotation data after multiple different attribute information annotations are compared and displayed under the same anomaly detection algorithm. Of course, the embodiment of the present application can also compare and display the initial annotation data after multiple different anomaly detection algorithms are annotated. In this way, after selecting the device identifier and the sample data set identifier in the first page 400, at least two anomaly detection algorithm identifiers can be selected, and each anomaly detection algorithm identifier selects its corresponding annotation type, attribute information, and whether to start intelligent annotation, etc., so that the initial annotation data is obtained by annotating with at least two anomaly detection algorithms, and the initial annotation data corresponding to different anomaly detection algorithms are displayed separately in the form of charts.

[0151] The embodiment of the present application can visually display the initial annotation data so that the annotator can intuitively understand the initial annotation data obtained by annotation.

[0152] In some embodiments, the server can determine the sample data set and the anomaly detection algorithm based on the annotation related information, and annotate the sample data set by the anomaly detection algorithm to obtain the initial annotation data. Therefore, before the above step S20, it also includes: sending the annotation related information to the server; after the above step S30, it also includes: receiving the initial annotation data returned by the server.

[0153] The sample data sets and the anomaly detection algorithm may be stored in the server. Furthermore, the server may provide the electronic device with sample data set identifiers corresponding to the multiple sample data sets and anomaly detection algorithm identifiers corresponding to the multiple anomaly detection algorithms.

[0154] After obtaining the annotation-related information involved in data annotation, the electronic device may send the annotation-related information to the server, wherein the annotation-related information includes the sample data set identifier and the anomaly detection algorithm identifier.

[0155] The server may call a stored sample data set according to the sample data set identifier, and call a stored anomaly detection algorithm according to the anomaly detection algorithm identifier. Then, the server may annotate the sample data set using the anomaly detection algorithm to obtain initial annotated data. The server may return the annotated initial annotated data to the electronic device, so that the electronic device may receive the initial annotated data returned by the server.

[0156] The data labeling method proposed in the embodiment of the present application can use an anomaly detection algorithm to label the sample data set through the server, which can improve the speed of data labeling.

[0157] In some embodiments, the electronic device can determine the sample data set and the anomaly detection algorithm based on the annotation related information, and annotate the sample data set by the anomaly detection algorithm to obtain the initial annotated data. Therefore, the above step S30 can specifically include: annotating the locally stored sample data set by the locally stored anomaly detection algorithm to obtain the initial annotated data.

[0158] The sample data set and the anomaly detection algorithm may be stored in the electronic device. Thus, after obtaining the sample data set identifier and the anomaly detection algorithm identifier, the electronic device may call the locally stored sample data set according to the sample data set identifier, and call the locally stored anomaly detection algorithm according to the anomaly detection algorithm identifier.

[0159] The electronic device may annotate the sample data set by using an anomaly detection algorithm to obtain initial annotated data, wherein the initial annotated data may include the corresponding sample data in the sample data set and the annotated result of annotating the corresponding sample data in the sample data set by using the anomaly detection algorithm.

[0160] The embodiment of the present application can directly use the locally stored anomaly detection algorithm to annotate the locally stored sample data set without sending the annotation related information to the server for annotation through the server, thereby reducing data transmission.

[0161] In some embodiments, the displayed initial annotation data can also be modified, so refer to Figure 8 , after the above S40, further comprising:

[0162] S50: receiving a correction operation on the initial annotation data.

[0163] After the initial annotation data is displayed, the annotator can also perform correction operations on the initial annotation data according to the actual situation, that is, perform secondary manual annotation on the initial annotation data.

[0164] The correction operation includes at least one of an add operation, a modify operation, and a delete operation. The add operation is used to add corresponding annotation results to the sample data that is not annotated by the anomaly detection algorithm; the modify operation is used to modify the annotation results of the sample data annotated by the anomaly detection algorithm; and the delete operation is used to delete the annotation results of the sample data annotated by the anomaly detection algorithm.

[0165] S60: In response to the correction operation, the initial labeled data is corrected to obtain first target labeled data.

[0166] After receiving the correction operation on the initial annotation data, the initial annotation data may be corrected, such as adding annotation results to the initial annotation data, modifying the annotation results in the initial annotation data, and deleting the annotation results in the initial annotation data.

[0167] It is understandable that the initial labeled data may include the corresponding sample data in the sample data set and the labeling result of labeling the corresponding sample data in the sample data set using the anomaly detection algorithm. The first target labeled data includes the sample data in the initial labeled data and the revised labeling result after the revised labeling result in the initial labeled data.

[0168] The data labeling method proposed in the embodiment of the present application can perform secondary manual labeling on the basis of the initial labeled data to correct the initial labeled data, thereby improving the labeling efficiency and accuracy of the sample data set.

[0169] In some embodiments, after the above step S60, the method further includes: displaying the first target annotation data.

[0170] After the first target annotation data is obtained after correction, the first target annotation data may be displayed in a chart form on the second page.

[0171] The data labeling method proposed in the embodiment of the present application can visually display the first target labeling data so that the labeling personnel can intuitively understand the first target labeling data obtained after correction.

[0172] In some embodiments, the above step S60 may specifically include: in response to the correction operation, obtaining the labeling start time and the labeling end time; and correcting the initial labeling data according to the labeling start time and the labeling end time to obtain the first target labeling data.

[0173] by Figure 6 As an example, the initial annotation data shown is used as an example. Figure 6 When the first annotation result 61 displayed on the far right is unreasonable, the annotator can trigger a correction operation by sliding his finger to the left or right at the position of the first annotation result 61 displayed on the far right, or by dragging the mouse to slide to the left or right at the position of the first annotation result 61 displayed on the far right.

[0174] In response to a corrective action, such as Fig. 9 As shown, a second annotation pop-up window 700 can be displayed on the second page 600, and the second annotation pop-up window 700 is used for related settings of secondary manual annotation. The second annotation pop-up window 700 includes an annotation start time selection control 701 and an annotation end time selection control 702.

[0175] Taking the modification operation as a modification operation or a deletion operation as an example, after the second annotation pop-up window 700 is displayed on the second page 600, the annotation start time selection control 701 is automatically filled with Figure 6 The marking start time of the first marking result 61 displayed on the far right and the marking end time selection control 702 will be automatically filled in. Figure 6 The marking end time of the first marking result 61 displayed on the far right.

[0176] If the correction operation is a modification operation, the annotator can modify the currently entered annotation start time in the annotation start time selection control 701 , and / or modify the currently entered annotation end time in the annotation end time selection control 702 .

[0177] For example, after the second annotation pop-up window 700 is displayed on the second page 600, the annotation start time automatically filled in the annotation start time selection control 701 can be 2023-01-31 21:28, and the annotation end time automatically filled in the annotation end time selection control 702 can be 2023-01-31 22:14. The annotator can then modify the annotation start time to 2023-01-31 21:06 in the annotation start time selection control 701, and modify the annotation end time to 2023-01-31 22:30 in the annotation end time selection control 702.

[0178] If the correction operation is a deletion operation, the annotator can delete the annotation start time automatically filled in the annotation start time selection control 701 , and delete the annotation end time automatically filled in the annotation end time selection control 702 .

[0179] In addition, the second annotation pop-up window 700 further includes a second save control 705 and a second cancel control 706. The second save control 705 is used to save the information filled in the second annotation pop-up window 700 after being triggered, and the second cancel control 706 is used to cancel the corresponding correction operation of the current manual annotation after being triggered.

[0180] After receiving the marking start time input through the marking start time selection control 701 and the marking end time input through the marking end time selection control 702, the marking staff can operate the second saving control 705, such as clicking. In response to the operation input to the second saving control 705, the initial marking data is modified or deleted according to the marking start time and marking end time filled in the second marking pop-up window 700, and the first target marking data is obtained.

[0181] If the marking personnel are Fig. 9 In the second annotation pop-up window 700 shown, when modifying the annotation start time and the annotation end time, as shown in FIG. Fig.10 As shown, the initial annotation data displayed on the second page 600 can be replaced with the first target annotation data.

[0182] from Fig.10 It can be seen from the figure that the sample data in the first target labeled data is the same as the sample data in the initial labeled data, and the correction operation does not correct the sample data in the initial labeled data; and the first target labeled data includes the corrected labeling result after this correction, such as Figure 6 A modified annotation result 63 is obtained by modifying the first annotation result 61 displayed on the far right.

[0183] Understandably, it is also possible to Figure 6In the initial annotation data shown in FIG. 1 , the first annotation result 61 is not displayed. By sliding with a finger or dragging a mouse, a second page 600 is displayed as shown in FIG. Fig. 9 In the second annotation pop-up window 700 shown, at this time, the annotation start time automatically filled in the annotation start time selection control 701, and the annotation end time automatically filled in the annotation end time selection control 702 can be determined according to the time corresponding to the start position and the end position of the sliding operation. The annotator can also fine-tune the annotation start time automatically filled in the annotation start time selection control 701, and / or fine-tune the annotation end time automatically filled in the annotation end time selection control 702. The annotator can operate the second save control 705, and in response to the operation input to the second save control 705, the annotation result can be added to the initial annotation data according to the annotation start time and the annotation end time filled in the second annotation pop-up window 700.

[0184] The data labeling method proposed in the embodiment of the present application can correct the initial labeling data according to the labeling start time and the labeling end time, and the correction method is relatively simple.

[0185] In some embodiments, Fig. 9 As shown, the second annotation pop-up window 700 may include an annotation description input control 703 and an annotation box configuration selection control 704 in addition to the above-mentioned annotation start time selection control 701 and annotation end time selection control 702. Therefore, before the initial annotation data is corrected according to the annotation start time and the annotation end time to obtain the first target annotation data, it also includes: in response to the correction operation, obtaining the annotation description and configuration information.

[0186] The annotation description is used to indicate the basis for determining whether to modify the initial annotation data; and the configuration information is used to configure the display effect of the modified annotation result in the first target annotation data.

[0187] The annotator can also input the annotation description of this revision in the annotation description input control 703. The annotator can also select configuration information from the drop-down list of the annotation box configuration selection control 704. Fig. 9 As shown, the annotation box configuration selection control 704 can be used to configure the color of the annotation box, for example, the color of the selected annotation box can be red.

[0188] Of course, the annotation box configuration selection control 704 may also include a control for configuring the line type of the annotation box, such as the annotation box may be a solid line or a dashed line, etc. Alternatively, the annotation box configuration selection control 704 may also include a control for configuring the line width of the annotation box.

[0189] The data annotation method proposed in the embodiment of the present application can be annotated so that when the first target annotation data obtained after the current correction is viewed later, the judgment basis of the current correction operation can be directly understood. In addition, the display effect of the correction annotation result can also be configured through configuration information, so that the annotator can distinguish the annotation results in the initial annotation data and the correction annotation results after the correction operation, thereby improving the annotation experience of the annotator.

[0190] In some embodiments, the above step S60 may specifically include: in response to the correction operation, correcting each of the multiple initial annotation sub-data to obtain the first target annotation data.

[0191] by Figure 7 Taking the displayed initial annotation data as an example, the initial annotation data may include multiple initial annotation sub-data, such as the multiple initial annotation sub-data are: current value and its second annotation result, maximum temperature value and its second annotation result, minimum temperature value and its second annotation result, and average temperature value and its second annotation result.

[0192] Determined by the marking staff Figure 7 If the second annotation result 62 shown is unreasonable, the annotator can slide left or right at the position corresponding to any initial annotation sub-data by finger or dragging the mouse, etc., to trigger a correction operation. For example, a correction operation is performed on the initial annotation sub-data corresponding to the current value and its second annotation result.

[0193] In response to the corrective action, Figure 7 The second page 600 is shown to display a second annotation pop-up window 700 , wherein the second annotation pop-up window 700 includes an annotation start time selection control 701 , an annotation end time selection control 702 , an annotation description input control 703 , and an annotation frame configuration selection control 704 .

[0194] Taking the correction operation as an add operation as an example, you can fine-tune the annotation start time automatically filled in the annotation start time selection control 701 in the second annotation pop-up window 700, and / or the annotation end time automatically filled in the annotation end time selection control 702; in addition, you can also enter the annotation description of this correction in the annotation description input control 703, and select configuration information in the drop-down list of the annotation box configuration selection control 704.

[0195] Next, the annotator can operate the second save control 705. In response to the operation input to the second save control 705, each of the multiple initial annotation sub-data can be modified according to the annotation start time, annotation end time, annotation description and configuration information filled in the second annotation pop-up window 700, so as to obtain the following: Fig.11The first target labeling data is shown.

[0196] from Fig.11 It can be seen that the sample data in the first target annotation data is the same as the sample data in the initial annotation data, and the correction operation does not correct the sample data in the initial annotation data; and the first target annotation data includes the correction result after this correction, such as the correction result 64 obtained by performing the addition operation on the basis of the second annotation result 62 in the initial annotation data. In addition, the same correction result is added to the current value and its second annotation result, the highest temperature value and its second annotation result, the lowest temperature value and its second annotation result, and the average temperature value and its second annotation result.

[0197] It can be understood that, when the second category data and its second annotation results corresponding to multiple attribute information are displayed simultaneously on the second page 600, or when the first category data and its first annotation results, as well as the second category data and its second annotation results are displayed simultaneously on the second page 600, a linkage annotation method can be adopted. When a correction operation is triggered on any initial annotation sub-data, each of the multiple initial annotation sub-data can be corrected.

[0198] The data labeling method proposed in the embodiment of the present application uses a linkage labeling method to correct each of the multiple initially labeled sub-data, thereby improving the correction efficiency of the corresponding correction operation during the secondary manual labeling.

[0199] In some embodiments, after the above step S60, the method further includes: temporarily storing the corrected annotation result in the first target annotation data to the currently displayed second page.

[0200] After the annotator fills in the annotation start time in the annotation start time selection control 701 in the second annotation pop-up window 700, fills in the annotation end time in the annotation end time selection control 702, enters the revised annotation description in the annotation description input control 703, and selects configuration information in the drop-down list of the annotation box configuration selection control 704, the annotator can operate the second save control 705. In response to the operation input to the second save control 705, the revised annotation result in the first target annotation data can also be temporarily saved to the currently displayed second page 600.

[0201] The revised annotation result includes the annotation start time, the annotation end time, the annotation description and the configuration information.

[0202] The data labeling method proposed in the embodiment of the present application can temporarily store the corrected labeling results in the first target labeling data to the currently displayed second page after correcting the initial labeling data to obtain the first target labeling data, so as to facilitate the subsequent storage of the corrected labeling results.

[0203] In some embodiments, after the step of temporarily storing the revised annotation results in the first target annotation data to the currently displayed second page, it also includes: receiving a first operation on a third save control; in response to the first operation, saving the revised annotation results temporarily stored to the second page to the server.

[0204] like Fig.10 and Fig.11 As shown, the second page 600 also includes a third save control 65. After the annotator determines that all the initial annotated data have been corrected, the annotator can operate the third save control 65, such as clicking. In response to the operation input to the third save control 65, the corrected annotation result temporarily stored in the second page can be sent to the server, so that the server saves the corrected annotation result.

[0205] The data labeling method proposed in the embodiment of the present application can send the corrected labeling result corrected by the correction operation to the server through the third save control in the second page, without sending the sample data in the initial labeling data to the server, thereby reducing the amount of data transmitted.

[0206] In some embodiments, after the step of temporarily storing the revised annotation results in the first target annotation data to the currently displayed second page, it also includes: saving the revised annotation results temporarily stored to the second page to the server at every preset time interval.

[0207] After temporarily storing the revised annotation results in the first target annotation data to the currently displayed second page, the revised annotation results temporarily stored to the second page can be sent to the server at every preset time interval, so that the server can save the revised annotation results. The preset time can be set according to actual conditions. For example, the preset time can be 1 minute.

[0208] The data annotation method proposed in the embodiment of the present application can send the revised annotation results corrected by the correction operation to the server at a fixed time, without sending the sample data in the initial annotation data to the server, thereby reducing the amount of data transmitted. In addition, the revised annotation results can be automatically saved to the server without triggering the control, thereby reducing the operation of the annotation personnel.

[0209] In some embodiments, after the step of temporarily storing the revised annotation results in the first target annotation data to the currently displayed second page, it also includes: receiving a second operation on the restore control; in response to the second operation, restoring the first target annotation data displayed on the second page to the initial annotation data.

[0210] like Fig.10 and Fig.11 As shown, the second page 600 also includes a restore control 66. When the annotator is not satisfied with the revised annotation result or needs to revise the initial annotation data again, as long as the revised annotation result is not saved to the server, the annotator can operate the restore control 66, such as clicking, etc. In response to the operation input to the restore control 66, the first target annotation data displayed on the second page can be restored to the initial annotation data.

[0211] The data labeling method proposed in the embodiment of the present application can restore the first target labeling data displayed on the second page to the initial labeling data through the restore control 66 in the second page, so that the labeling personnel can correct the initial labeling data again.

[0212] In some embodiments, after the step of saving the revised annotation results temporarily stored in the second page to the server, it also includes: receiving an opening operation for the second page; in response to the opening operation, obtaining second target annotation data from the server, the second target annotation data being data generated by the server based on the initial annotation data and the revised annotation results, and the second target annotation data is the same as the first target annotation data; and displaying the second target annotation data on the second page.

[0213] After the revised annotation results are saved to the server, when the annotator wants to re-enter the previously annotated annotation data, the annotator can trigger an opening operation on the second page 600. In response to the opening operation on the second page 600, the second target annotation data can be obtained from the server.

[0214] Since the revised annotation result corrected by the correction operation was sent to the server before, and the sample data in the initial annotation data was not sent to the server, the server will generate second target annotation data based on the revised annotation result and the initial annotation data, and the server also returns the second target annotation data.

[0215] The data annotation method proposed in the embodiment of the present application can obtain the second target annotation data from the server when the opening operation of the second page is triggered, so that the annotator can view the previously annotated annotation data again.

[0216] Based on the same inventive concept, the embodiment of the present application also provides a data labeling device for implementing the data labeling method involved above. The implementation scheme for solving the problem provided by the data labeling device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more data labeling device embodiments provided below can refer to the limitations on the data labeling method above, and will not be repeated here.

[0217] Fig.12 The present invention is a schematic diagram of the structure of the data annotation device of some embodiments of the present application. Specifically, the data annotation device 120 includes: a first acquisition module 1201, a determination module 1202 and an annotation module 1203. The first acquisition module 1201 is used to obtain the annotation related information involved in data annotation; the determination module 1202 is used to determine the sample data set and the anomaly detection algorithm based on the annotation related information; the annotation module 1203 is used to annotate the sample data set through the anomaly detection algorithm to obtain the initial annotation data.

[0218] In some embodiments, the first acquisition module 1201 is specifically used to display the first page; and receive annotation related information input through the first page.

[0219] In some embodiments, the labeling related information includes a sample data set identifier and an anomaly detection algorithm identifier. The determination module 1202 is specifically configured to determine the sample data set based on the sample data set identifier; and determine the anomaly detection algorithm based on the anomaly detection algorithm identifier.

[0220] In some embodiments, the annotation related information includes the annotation type and attribute information associated with the anomaly detection algorithm. The annotation module 1203 is specifically configured to annotate the sample data set using the anomaly detection algorithm, the annotation type and the attribute information to obtain initial annotation data.

[0221] In some embodiments, the annotation type includes a characteristic value and / or an original value. When the annotation type includes a characteristic value, the initial annotation data includes a first type of data and a first annotation result; the first type of data includes data in the sample data set that matches the anomaly detection algorithm, and the first annotation result is used to annotate the data in the first type of data that has anomalies. When the annotation type includes an original value, the initial annotation data includes a second type of data and a second annotation result; the second type of data includes data corresponding to the attribute information in the sample data set, and the second annotation result is used to annotate the data in the second type of data that is related to the cause of the anomaly; and the second annotation result is the same as the first annotation result.

[0222] In some embodiments, the labeling related information includes corresponding intelligent labeling conditions when intelligent labeling is enabled. The labeling module 1203 is specifically configured to label the sample data set using an anomaly detection algorithm and intelligent labeling conditions to obtain initial labeling data.

[0223] In some embodiments, the sample data set includes battery data.

[0224] In some embodiments, the data annotation device 120 further includes a first display module. The first display module is used to display the initial annotation data.

[0225] In some embodiments, the data annotation device 120 further includes a sending module and a receiving module. The sending module is used to send the annotation related information to the server; the receiving module is used to receive the initial annotation data returned by the server.

[0226] In some embodiments, the labeling module 1203 is specifically used to label the locally stored sample data set through the locally stored anomaly detection algorithm to obtain initial labeling data.

[0227] In some embodiments, the data labeling device 120 further includes a correction operation receiving module and a correction module. The correction operation receiving module is used to receive a correction operation on the initial labeling data; the correction module is used to correct the initial labeling data in response to the correction operation to obtain the first target labeling data.

[0228] In some embodiments, the data labeling device 120 further includes a second display module. The second display module is used to display the first target labeling data.

[0229] In some embodiments, the correction module is specifically used to obtain the marking start time and the marking end time in response to the correction operation; and correct the initial marking data according to the marking start time and the marking end time to obtain the first target marking data.

[0230] In some embodiments, the data annotation device 120 further includes a second acquisition module. The second acquisition module is used to obtain the annotation description and configuration information in response to the correction operation; wherein the annotation description is used to indicate the judgment basis for correcting the initial annotation data; and the configuration information is used to configure the display effect of the corrected annotation result in the first target annotation data.

[0231] In some embodiments, the correction module is specifically configured to correct each of the plurality of initial labeled sub-data in response to the correction operation to obtain the first target labeled data.

[0232] Fig.13The structure diagram of the electronic device of some embodiments of the present application. Specifically, the electronic device 130 may include: a processor 1301, a memory 1302, a bus 1303 and a communication interface 1304, and the processor 1301, the communication interface 1304 and the memory 1302 are connected via the bus 1303. The memory 1302 stores a computer program that can be run on the processor 1301. When the processor 1301 runs the computer program, the data labeling method provided in any of the above embodiments of the present application is executed.

[0233] The implementation principle and technical effect of the electronic device provided in the embodiment of the present application are similar to those of the above-mentioned method embodiment, and will not be repeated here.

[0234] An embodiment of the present application also provides a computer-readable storage medium, in which a computer program or instruction is stored. When the computer program or instruction is executed, the data labeling method provided in any of the above-mentioned embodiments is executed.

[0235] Among them, the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable programmable read only memory (EPROM), programmable read only memory (PROM), read only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0236] The computer-readable storage medium provided in the embodiment of the present application has similar implementation principles and technical effects to those of the above-mentioned method embodiment, and will not be described in detail here.

[0237] The embodiment of the present application also provides a computer program product, which, when running on a computer, enables the computer to execute the data labeling method provided by any of the above embodiments. Its implementation principle and technical effect are similar to those of the above method embodiments, and will not be repeated here.

[0238] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application, and they should all be included in the scope of the claims and specification of the present application. In particular, as long as there is no structural conflict, the various technical features mentioned in the various embodiments can be combined in any way. The present application is not limited to the specific embodiments disclosed herein, but includes all technical solutions that fall within the scope of the claims.

Claims

1. A data labeling method, characterized in that: include: Obtain information related to annotations of participants in data annotation; Determine a sample data set and anomaly detection algorithm based on the labeled related information; The sample data set is annotated using the anomaly detection algorithm to obtain initial annotated data.

2. The method according to claim 1, characterized in that The obtaining of the annotation-related information of the participants in the data annotation includes: Display the first page; The annotation related information input through the first page is received.

3. The method according to claim 1 or 2, characterized in that: The annotation related information includes a sample data set identifier and an anomaly detection algorithm identifier; The determining of the sample data set and the anomaly detection algorithm based on the labeling related information includes: Determining the sample data set based on the sample data set identifier; The anomaly detection algorithm is determined based on the anomaly detection algorithm identifier.

4. The method according to any one of claims 1 to 3, characterized in that The annotation related information includes the annotation type and attribute information associated with the anomaly detection algorithm; The step of labeling the sample data set by using the anomaly detection algorithm to obtain initial labeling data includes: The sample data set is labeled using the anomaly detection algorithm, the labeling type and the attribute information to obtain the initial labeled data.

5. The method according to claim 4, characterized in that The annotation type includes characteristic value and / or original value; In the case where the annotation type includes the feature value, the initial annotation data includes a first category of data and a first annotation result; the first category of data includes data in the sample data set that matches the anomaly detection algorithm, and the first annotation result is used to annotate data with anomalies in the first category of data; In the case where the annotation type includes the original value, the initial annotation data includes a second category of data and a second annotation result; the second category of data includes data corresponding to the attribute information in the sample data set, and the second annotation result is used to annotate the data in the second category of data that is related to the cause of the anomaly; and the second annotation result is the same as the first annotation result.

6. The method according to any one of claims 1 to 5, characterized in that The annotation related information includes corresponding intelligent annotation conditions when intelligent annotation is enabled; The step of labeling the sample data set by using the anomaly detection algorithm to obtain initial labeling data includes: The sample data set is labeled using the anomaly detection algorithm and the intelligent labeling condition to obtain the initial labeled data.

7. The method according to claim 1, characterized in that The sample data set includes battery data.

8. The method according to claim 1, characterized in that After the sample data set is labeled by the anomaly detection algorithm to obtain initial labeled data, the method further includes: The initial annotation data is displayed.

9. The method according to any one of claims 1 to 8, characterized in that Before determining the sample data set and the anomaly detection algorithm based on the labeled relevant information, the method further includes: Sending the annotation related information to a server; After the sample data set is labeled by the anomaly detection algorithm to obtain initial labeled data, the method further includes: Receive the initial annotation data returned by the server.

10. The method according to any one of claims 1 to 8, characterized in that The step of labeling the sample data set by using the anomaly detection algorithm to obtain initial labeling data includes: The locally stored sample data set is labeled using the locally stored anomaly detection algorithm to obtain the initial labeled data.

11. The method according to claim 8, characterized in that After displaying the initial annotation data, the method further includes: receiving a correction operation on the initial annotation data; In response to the correction operation, the initial labeled data is corrected to obtain first target labeled data.

12. The method according to claim 11, characterized in that After the initial annotation data is modified in response to the modification operation to obtain the first target annotation data, the method further includes: The first target annotation data is displayed.

13. The method according to claim 11, characterized in that The step of correcting the initial annotation data in response to the correction operation to obtain first target annotation data includes: In response to the correction operation, obtaining a marking start time and a marking end time; The initial marking data is corrected according to the marking start time and the marking end time to obtain the first target marking data.

14. The method according to claim 13, characterized in that Before the initial annotation data is corrected according to the annotation start time and the annotation end time to obtain the first target annotation data, the method further includes: In response to the correction operation, obtaining annotation instructions and configuration information; The annotation description is used to indicate the basis for determining whether to modify the initial annotation data; and the configuration information is used to configure the display effect of the modified annotation result in the first target annotation data.

15. The method according to claim 11, characterized in that The step of correcting the initial annotation data in response to the correction operation to obtain first target annotation data includes: In response to the correction operation, each of the plurality of initial labeled sub-data is corrected to obtain the first target labeled data.

16. A data labeling device, characterized in that: include: A first acquisition module is used to acquire annotation related information involved in data annotation; A determination module, used to determine a sample data set and an anomaly detection algorithm based on the annotation related information; The labeling module is used to label the sample data set through the anomaly detection algorithm to obtain initial labeling data.

17. An electronic device, characterized in that: It comprises a memory and a processor, the memory is used to store a computer program, and the processor is used to call the computer program to execute the data labeling method as described in any one of claims 1 to 15.

18. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program or instruction, and when the computer program or instruction is executed, the data labeling method according to any one of claims 1 to 15 is implemented.

Images