Data processing method and device, equipment and storage medium
By obtaining the query time and target creation time, and using the record information table and query list to analyze the information in the data labeling stage, the problem of the inability to quickly locate the data labeling problem is solved, and the data labeling efficiency and process traceability are improved.
Patent Information
- Application Number
- CN202310558212.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-17
- Publication Date
- 2024-11-22
AI Technical Summary
During the automatic driving data labeling process, the inability to quickly locate the problem, resulting in low data labeling efficiency. The existing technology cannot monitor the processing of each stage in the data labeling process in real time, resulting in data loss or slow progress.
By obtaining the query time and the target creation time of the data annotation event, the target data annotation event is determined using the record information table, and the target query data annotation stage is determined based on the query list, and the target information in the target query data annotation stage is analyzed to quickly locate abnormalities, so as to improve Data labeling efficiency.
It realizes rapid positioning of abnormalities during the data labeling process, improves data labeling efficiency, and ensures traceability and real-time monitoring capabilities of the data labeling process.
Smart Images

Figure CN119004298A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of electronic technology, and specifically to a data processing method, device, equipment and storage medium. Background Art
[0002] Deep learning methods are increasingly recognized and valued by the academic community, and are also widely used in the field of autonomous driving. The training of neural network models in the field of deep learning requires a large amount of labeled data for learning, and the quality of the labeled data often directly determines the accuracy of model training.
[0003] Data labeling in the autonomous driving industry refers to the process of labeling and annotating various objects in original materials such as images, text, voice, video, 3D point clouds, etc. after collecting data from various autonomous driving scenarios through classification, segmentation, key point and line labeling, framing, tracking and other processing methods, and converting multi-source heterogeneous data into recognizable model training data, thereby becoming valuable data for machine learning.
[0004] The data annotation process of scene data annotation events generally includes data creation, data recovery, automatic annotation, manual annotation and other stages. Due to network delays and the different processing time of data in each stage, data loss or slow data annotation progress may occur during the data annotation process. In related technologies, only the final annotation data can be obtained for the data corresponding to the scene annotation, and the processing status of each stage in the data annotation process cannot be known. If the final annotation data is not obtained during the data annotation process, the problem cannot be quickly located, resulting in low efficiency of data annotation. Summary of the invention
[0005] In view of this, the present application provides a process data processing method, device, equipment and storage medium, so as to solve the problem in the prior art that if the final annotation data cannot be obtained within the expected time, the problem cannot be located quickly, resulting in low data annotation efficiency.
[0006] In a first aspect, an embodiment of the present application provides a data processing method, including:
[0007] Get the query time and the target creation time of the data annotation event;
[0008] Obtaining a record information table of data annotation events, and determining a target data annotation event in the record information table of data annotation events based on a target creation time of the data annotation event; the record information table of data annotation events is used to record identification information and time information of a completed data annotation stage of a created data annotation event; the target data annotation event is a data annotation event whose creation time recorded in the record information table of data annotation events is consistent with the target creation time;
[0009] Obtaining a query list, and determining a target query data annotation stage based on the query list; the query list records identification information of at least one data annotation stage of target information to be counted;
[0010] The query target time period is determined based on the query time and the target creation time, and in the record information table of the data annotation event, the target information of the target data annotation event included in the query target time period during the target query data annotation phase is determined.
[0011] In a possible implementation manner of the first aspect, the target information includes quantity information;
[0012] In the record information table of the data annotation event, determining the target information of the target data annotation event contained in the target query data annotation phase within the query target time period includes:
[0013] In the record information table of the data annotation events, the quantity information of the target data annotation events included in the query target time period during the target query data annotation phase is determined.
[0014] In a possible implementation manner of the first aspect, the target information includes time difference information;
[0015] In the record information table of the data annotation event, determining the target information of the target data annotation event contained in the target query data annotation phase within the query target time period includes:
[0016] In the record information table of the data annotation event, the time difference information corresponding to the target query data annotation phase of the target data annotation event within the query target time period is determined.
[0017] In a possible implementation manner of the first aspect,
[0018] In the record information table of the data annotation event, determining the time difference information corresponding to the target query data annotation phase of the target data annotation event within the query target time period includes:
[0019] Acquire data annotation process information; the data annotation process information includes identification information of each data annotation stage included in the data annotation process, and the identification information of each data annotation stage is arranged according to the execution order of each data annotation stage in the data annotation process;
[0020] For the identification information of each target query data marking stage in the query list, based on the data marking process information, determine whether there is identification information of the first target data marking stage corresponding to the identification information of the target query data marking stage in the query list; if not, determine the identification information of the second target data marking stage corresponding to the identification information of the target query data marking stage according to the query list and the data marking process information, and determine the identification information of the fourth data marking stage corresponding to the identification information of the target query data marking stage according to the data marking process information, and store the identification information of the target query data marking stage, the identification information of the fourth target data marking stage and the identification information of the second target data marking stage as a first query subset in the first set; if it exists, store the identification information of the target query data marking stage and the identification information of the first target data marking stage as a second query subset in the second set; wherein, the identification information of the first target data marking stage is the identification information of the target query data marking stage in the data marking process information. the identification information of the previous data annotation stage of the identification information of the query list; the identification information of the second target data annotation stage is the identification information of the third target data annotation stage of the query list when the identification information of the third target data annotation stage exists in the identification information of at least one data annotation stage contained in the query list; the identification information of the third target data annotation stage is the identification information of the previous data annotation stage of the identification information of the target query data annotation stage when the identification information of at least one data annotation stage contained in the query list is arranged in the execution order of the data annotation process; or, the identification information of the second target data annotation stage is the identification information of the first data annotation stage in the data annotation process information when the identification information of the third target data annotation stage does not exist in the identification information of at least one data annotation stage contained in the query list; the identification information of the fourth target data annotation stage is the identification information of the data annotation stage between the identification information of the target query data annotation stage and the identification information of the second target data annotation stage in the data annotation process information;
[0021] According to the record information table of the data annotation event, the first set and the second set, the time difference information corresponding to the query target time period in each target query data annotation stage is determined.
[0022] In a possible implementation of the first aspect, for the identification information of each target query data annotation stage in the query list, based on the data annotation process information, determining whether there is identification information of a first target data annotation stage corresponding to the identification information of the target query data annotation stage in the query list; if not, determining the identification information of a second target data annotation stage corresponding to the identification information of the target query data annotation stage according to the query list and the data annotation process information, and determining the identification information of a fourth data annotation stage corresponding to the identification information of the target query data annotation stage according to the data annotation process information, storing the identification information of the target query data annotation stage, the identification information of the fourth target data annotation stage, and the identification information of the second target data annotation stage as a first query subset in a first set; if present, storing the identification information of the target query data annotation stage and the identification information of the first target data annotation stage as a second query subset in a second set includes:
[0023] Determine identification information of a first data annotation stage and identification information of a second data annotation stage in the query list; the identification information of the first data annotation stage is identification information of the first data annotation stage when identification information of at least one data annotation stage included in the query list is arranged in the order of execution of the data annotation process; the identification information of the second data annotation stage is identification information of the last data annotation stage when identification information of at least one data annotation stage included in the query list is arranged in the order of execution of the data annotation process;
[0024] Detecting whether the identification information of the first data annotation stage is the identification information of the first data annotation stage in the data annotation process information;
[0025] If not, then determine the identification information of each data annotation stage before the first data annotation stage in the data annotation process information, store the identification information of each data annotation stage before the first data annotation stage in the data annotation process and the identification information of the first data annotation stage as a first query subset in the first set, and perform the time difference information determination step; wherein the time difference information determination step includes:
[0026] Determining the first data annotation stage as the current stage;
[0027] Detecting whether the current stage is the second data labeling stage;
[0028] If not, determining the identification information of the next data annotation stage of the current stage as the identification information of the target stage in the data annotation process information, and updating the value of the jump parameter to a first preset value;
[0029] Detecting whether the query list contains identification information of the target stage;
[0030] If the query list does not contain the identification information of the target stage, the identification information of the current stage is recorded in the first information, the identification information of the current stage is updated to the identification information of the target stage, and the step of determining the identification information of the next data annotation stage of the current stage as the identification information of the target stage in the data annotation process information is re-executed, and the value of the jump parameter is updated to the first preset value, until the step of detecting whether the query list contains the identification information of the target stage is detected, until the query list contains the identification information of the target stage; or,
[0031] If the query list contains the identification information of the target stage, detecting whether the value of the jump parameter is a second preset value;
[0032] If it is the second preset value, the identification information of the current stage and the identification information of the target stage are stored in the second set as a second query subset, the identification information of the current stage is updated to the identification information of the target stage, the value of the jump parameter is updated to the preset initial value, and the step of detecting whether the current stage is the second data labeling stage is re-executed until it is detected that the current stage is the second data labeling stage; or,
[0033] If it is not the second preset value, the identification information of the current stage, the identification information of the target stage and the identification information of the data annotation stage recorded in the first information are stored in the first set as a first query subset, the identification information of the current stage is updated to the identification information of the target stage, the value of the jump parameter is updated to the preset initial value, and the step of detecting whether the current stage is the second data annotation stage is re-executed until it is detected that the current stage is the second data annotation stage;
[0034] The method of determining the time difference information corresponding to the query target time period in each target query data annotation phase according to the record information table of the data annotation event, the first set and the second set includes:
[0035] If the current stage is the second data labeling stage, the time difference information corresponding to the query target time period of each target query data labeling stage is determined according to the record information table of the data labeling event, the first set and the second set.
[0036] In a possible implementation manner of the first aspect, the method further includes:
[0037] If the identification information of the first data labeling stage is the identification information of the first data labeling stage in the data labeling process information, the identification information of the first data labeling stage is stored in the first set as a first query subset, and the time difference information determination step is performed.
[0038] In a possible implementation manner of the first aspect, determining, according to the record information table of the data annotation event, the first set, and the second set, the time difference information corresponding to each data annotation stage in the query list includes:
[0039] When the first set is a non-empty set, for each first query subset included in the first set, determining, in the record information table of the data annotation event, a time difference between every two adjacent data annotation stages in all data annotation stages included in the first query subset within the query target time period;
[0040] Based on the time difference between every two adjacent data annotation stages in all data annotation stages included in the first query subset within the query target time period, at least one of the maximum time difference, the minimum time difference and the average time difference corresponding to the target query data annotation stage in the first query subset is determined.
[0041] In a possible implementation manner of the first aspect, determining, according to the record information table of the data annotation event, the first set, and the second set, the time difference information corresponding to each data annotation stage in the query list includes:
[0042] When the second set is a non-empty set, for each second query subset included in the second set, determining, in the record information table of the data annotation event, a time difference between two data annotation stages included in the second query subset within the query target time period;
[0043] Based on the time difference between two data annotation stages included in the second query subset within the query target time period, at least one of the maximum time difference, the minimum time difference and the average time difference corresponding to the target query data annotation stage in the second query subset is determined.
[0044] In a possible implementation manner of the first aspect, the time information includes completion time information, and the method further includes:
[0045] Obtaining completed data annotation stage information to be stored for at least one data annotation event; wherein the completed data annotation stage information includes identification information and completion time of the completed data annotation stage;
[0046] Based on the completed data annotation stage information to be stored for the at least one data annotation event, the record information table of the data annotation event is updated.
[0047] In a possible implementation manner of the first aspect, the time information further includes time difference information between two adjacent completed data annotation stages;
[0048] The updating of the record information table of the data labeling event based on the completed data labeling stage information to be stored of the at least one data labeling event comprises:
[0049] For each of the completed data annotation stage information to be stored for the at least one data annotation event, determining in the record information table of the data annotation event whether the data annotation event already exists;
[0050] If so, then the completion time of the fifth target data annotation stage of the data annotation event is determined in the record information table of the data annotation event; the fifth target data annotation stage of the data annotation event is the previous data annotation stage of the completed data annotation stage to be stored of the data annotation event;
[0051] Based on the completion time in the completed data labeling stage information to be stored of the data labeling event and the completion time information of the fifth target data labeling stage of the data labeling event, determine the time difference information between the completed data labeling stage to be stored of the data labeling event and the fifth target data labeling stage;
[0052] The time difference information between the completed data marking stage to be stored of the data marking event and the fifth target data marking stage and the completed data marking stage information to be stored of the at least one data marking event are stored in the record information table of the data marking event.
[0053] In a possible implementation manner of the first aspect, the method further includes:
[0054] If not, the completed data marking stage information to be stored of the at least one data marking event is stored in the record information table of the data marking event.
[0055] In a possible implementation manner of the first aspect, the method further includes:
[0056] Get attribute condition information;
[0057] In the record information table of the data annotation event, determining the target information of the target data annotation event contained in the query target time period in each target query data annotation stage includes:
[0058] In the record information table of the data annotation events, target information of the target data annotation events satisfying the attribute condition information contained in the query target time period in each target query data annotation phase is determined.
[0059] In a second aspect, an embodiment of the present application provides a data processing device, including:
[0060] An acquisition unit, used to acquire the query time and the target creation time of the data annotation event;
[0061] A processing unit, used to obtain a record information table of data annotation events, and determine a target data annotation event in the record information table of data annotation events based on a target creation time of the data annotation event; the record information table of data annotation events is used to record identification information and time information of a completed data annotation stage of a created data annotation event; the target data annotation event is a data annotation event whose creation time recorded in the record information table of data annotation events is consistent with the target creation time;
[0062] The processing unit is further used to obtain a query list and determine the target query data annotation stage based on the query list; the query list records identification information of at least one data annotation stage of the target information to be counted;
[0063] The processing unit is also used to determine the query target time period based on the query time and the target creation time, and in the record information table of the data annotation event, determine the target information of the target data annotation event contained in the query target time period in each target query data annotation stage.
[0064] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory for storing computer program instructions and a processor for executing the program instructions, wherein when the computer program instructions are executed by the processor, the electronic device is triggered to execute any one of the methods described in the first aspect above.
[0065] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute any method described in the first aspect.
[0066] Adopt the scheme provided by the embodiment of the present application, obtain the query time and the target creation time of the data annotation event; obtain the record information table of the data annotation event, and determine the target data annotation event in the record information table of the data annotation event based on the target creation time of the data annotation event; the record information table of the data annotation event is used to record the identification information and time information of the completed data annotation stage of the created data annotation event; the target data annotation event is a data annotation event whose creation time recorded in the record information table of the data annotation event is consistent with the target creation time; obtain the query list, and determine the target query data annotation stage based on the query list; the query list records the identification information of at least one data annotation stage of the target information to be counted; determine the query target time period based on the query time and the target creation time, and determine the target information of the target data annotation event contained in each target query data annotation stage within the query target time period in the record information table of the data annotation event. Among them, the target information can reflect the processing of the target data annotation event in the data annotation process. In this way, if the final labeled data is not obtained during the data labeling process, the target information of the target data labeling event contained in the target query data labeling stage within the query target time period can be obtained. In this way, by analyzing the target information contained in the target query data labeling stage, the abnormal data labeling stage can be quickly located to improve the data labeling efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0067] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0068] Figure 1 A flowchart of a data processing method provided in an embodiment of the present application;
[0069] Figure 2 A flowchart of another data processing method provided in an embodiment of the present application;
[0070] Figure 3 A schematic diagram of a scenario of a data processing method provided in an embodiment of the present application;
[0071] Figure 4 A flowchart of another data processing method provided in an embodiment of the present application;
[0072] Figure 5 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application;
[0073] Figure 6A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0074] In order to better understand the technical solution of the present application, the embodiments of the present application are described in detail below with reference to the accompanying drawings.
[0075] It should be clear that the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.
[0076] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. The singular forms "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.
[0077] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.
[0078] Deep learning methods are increasingly recognized and valued by the academic community, and are also widely used in the field of autonomous driving. The training of neural network models in the field of deep learning requires a large amount of labeled data for learning, and the quality of the labeled data often directly determines the accuracy of model training.
[0079] Data labeling in the autonomous driving industry refers to the process of labeling and annotating various objects in original materials such as images, text, voice, video, 3D point clouds, etc. after collecting data from various autonomous driving scenarios through classification, segmentation, key point and line labeling, framing, tracking and other processing methods, and converting multi-source heterogeneous data into recognizable model training data, thereby becoming valuable data for machine learning.
[0080] The annotation process of scene data annotation events generally includes data creation, data recovery, automatic annotation, manual annotation and other stages. Due to network delays and the different processing time of data in each stage, data loss or slow data annotation progress may occur during the data annotation process. In related technologies, the data corresponding to the scene annotation can only obtain the final annotation data, and the processing status of each stage in the data annotation process cannot be known. If the final annotation data cannot be obtained within the expected time, the problem cannot be quickly located, resulting in low data annotation efficiency.
[0081] In view of the above problems, the embodiments of the present application provide a data processing method, device, equipment and storage medium, which obtains the query time and the target creation time of the data annotation event; obtains the record information table of the data annotation event, and determines the target data annotation event in the record information table of the data annotation event based on the target creation time of the data annotation event; the record information table of the data annotation event is used to record the identification information and time information of the completed data annotation stage of the created data annotation event; the target data annotation event is a data annotation event whose creation time recorded in the record information table of the data annotation event meets the target creation time; obtains a query list, and determines the target query data annotation stage based on the query list; the query list records the identification information of at least one data annotation stage of the target information to be counted; determines the query target time period based on the query time and the target creation time, and determines the target information of the target data annotation event contained in each target query data annotation stage within the query target time period in the record information table of the data annotation event. The target information of the target data annotation event contained in the query target time period in each target query data annotation stage of the data annotation process can reflect the processing of the target data annotation event in the target query data annotation stage of the data annotation process. In this way, if the final labeled data is not obtained during the data labeling process, the target information of the target data labeling event contained in the target query data labeling stage within the query target time period can be obtained, so that the abnormal data labeling stage can be quickly located by analyzing the target information contained in the target query data labeling stage, thereby improving the data labeling efficiency. The following is a detailed explanation.
[0082] See also Figure 1 , is a flow chart of a data processing method provided in an embodiment of the present application, the method comprising:
[0083] Step S101, obtaining the query time and the target creation time of the data annotation event.
[0084] In an embodiment of the present application, when it is necessary to confirm the completion of the data annotation stage in the data annotation process within a certain time period, the target information of the data annotation events in the data annotation stage within the time period can be collected. Therefore, it is necessary to determine the data annotation events involved in the time period. The first data annotation stage of the data annotation process is the creation stage of the data annotation, that is, each data identification event must be created first, and the data annotation event to be queried can be determined based on the creation time. Based on this, the start time of the time period to be queried can be determined as the creation time of the data annotation event, and the query time can be determined as the end time of the time period, so as to determine the time period to be queried.
[0085] In some embodiments, the user may set the query time and the target creation time of the data annotation event according to actual needs. At this time, the data processing device may obtain the query time and the target creation time of the data annotation event set by the user.
[0086] Step S102: Obtain a record information table of data annotation events, and determine a target data annotation event in the record information table of data annotation events based on a target creation time of the data annotation event.
[0087] The data annotation event record information table is used to record the identification information and time information of the completed data annotation stage of the created data annotation event. The target data annotation event is a data annotation event whose creation time recorded in the data annotation event record information table matches the target creation time.
[0088] In an embodiment of the present application, during the data annotation process of a data annotation event, relevant information of the completed data annotation stage can be stored in the record information table of the data annotation event. For example, after a data annotation event is created, it is the creation stage in the data annotation process that has been completed. Then, the record information table of the data annotation event records the time information of the creation stage of each created data annotation event, such as the creation completion time, etc. After obtaining the target creation time of the data annotation event, the data processing device can obtain the record information table of the data annotation event, and based on the identification information and time information of the completed data annotation stage of each data annotation event recorded in the record information table of the data annotation event, determine the target data identification event whose creation time matches the target creation time. At this time, the number of target data identification events is at least one.
[0089] In some embodiments, the data processing device may obtain a record information table of data annotation events in its own storage device, or may obtain a record information table of data annotation events from other devices such as a server, and this application does not impose any restrictions on this.
[0090] In an embodiment of the present application, the target data annotation event is a data annotation event whose creation time recorded in the record information table of the data annotation event is consistent with the target creation time. That is, after obtaining the target creation time of the data annotation event, the data processing device can detect whether the data annotation event recorded in the record information table of the data annotation event has a data annotation event whose creation time is consistent with the target creation time in the acquired record information table of the data annotation event. For example, when the target creation time is a time period, it is detected whether the data annotation event recorded in the record information table of the data annotation event has a data annotation event whose creation time is within the target creation time. If so, the data annotation event whose creation time is within the target creation time is determined as the target data annotation event. Or, when the target creation time is a time point, it is detected whether the data annotation event recorded in the record information table of the data annotation event has a data annotation event whose creation time is the same as the target creation time. If so, the data annotation event whose creation time is the same as the target creation time is determined as the target data annotation event. In this way, by detecting the creation time of each data annotation event recorded in the record information table of the data annotation event, it is possible to determine which data annotation events were created at the target creation time and determine them as the target data annotation events.
[0091] In some embodiments, if the target creation time is a time point, the creation time being the same as the target creation time means that the time difference between the creation time and the target creation time is not greater than a preset time threshold. That is, the data processing device needs to detect the time difference between the creation time of each data annotation event recorded in the record information table of the data annotation event and the target creation time, and determine the data annotation event whose time difference is not greater than the preset time threshold as the target data annotation event.
[0092] Step S103: Obtain a query list, and determine the target query data labeling stage based on the query list.
[0093] The query list records identification information of at least one data annotation stage of target information to be counted.
[0094] In an embodiment of the present application, when it is necessary to count the target information of the target data annotation event contained in the data annotation stage within a certain time period, it is necessary to determine the data annotation stage of the target information to be counted. At this time, a query list can be obtained, and the query list records the identification information of the data annotation stage of the target information that needs to be counted, which is pre-set by the user. The data annotation stage corresponds to the identification information of the data annotation stage one by one, and the data annotation stage can be determined based on the identification information of the data annotation stage. After obtaining the query list, the data processing device can determine the data annotation stage of the target information to be counted based on the query list, that is, the target query data annotation stage.
[0095] In some embodiments, the data processing device may directly determine the data marking stage corresponding to the identification information of all data marking stages recorded in the query list as the target query data marking stage.
[0096] Step S104: determine the query target time period based on the query time and the target creation time, and determine the target information of the target data annotation event included in the query target time period during the target query data annotation phase in the record information table of the data annotation event.
[0097] In an embodiment of the present application, when it is necessary to confirm the completion of the target query data annotation stage within a certain time period, the target information of the target data annotation event of the target query data annotation stage within the time period can be counted. At this time, it is necessary to first determine the time period. The data processing device can determine the start time of the time period as the target creation time of the data annotation event, and determine the query time as the end time of the time period, and calculate the time period, that is, determine the query target time period. The identification information and time information of the completed data annotation stage of the created data annotation event are recorded in the record information table of the data annotation event. Therefore, the identification information and time information of the completed data annotation stage corresponding to the target data annotation event can be determined in the record information table of the data annotation event, and then the target information of the target data annotation event contained in the query target time period of the target query data annotation stage is determined according to the identification information and time information of the completed data annotation stage corresponding to the target data annotation event.
[0098] As a possible implementation method, the target information includes quantity information, that is, the quantity information of target data annotation events included in the query time period in each target query phase needs to be counted.
[0099] At this time, in the record information table of data annotation events, it is determined that the target information of the target data annotation events included in the target query data annotation phase within the query target time period includes:
[0100] In the record information table of data annotation events, the quantity information of target data annotation events included in the target query data annotation phase within the query target time period is determined.
[0101] In an embodiment of the present application, after determining the target data labeling event, the data processing device can query the identification information and time information of the completed data labeling stage of the target data labeling event in the record information table of the data labeling event, and determine the identification information and time information of the completed target query data labeling stage of the target data labeling event from the identification information and time information of the completed data labeling stage of the target data labeling event according to the identification information of the target query data labeling stage. According to the identification information and time information of the completed target query data labeling stage of the target data labeling event, the number of target data labeling events included in the target query data labeling stage within the query target time period is counted.
[0102] The quantity information of the data annotation events in the query target time period during the data annotation stage can reflect the processing of the target data annotation events in the data annotation stage in the data annotation process, so the target information can include quantity information. If the final annotation data is not obtained during the data annotation process, the quantity information of the data annotation events in the query target time period during the target query data annotation stage can be analyzed to locate the data annotation stage where the abnormality occurs. For example, during the query target time period, all target data annotation events complete the entire data annotation process. At this time, the quantity information of the target data annotation events contained in each data annotation stage of the data annotation process during the query target time period is the same, which is equal to the number of target data annotation events. If the difference between the quantity information of the data annotation events in the query target time period during the target query data annotation stage and the quantity information of the data annotation events in the query target time period during the previous target query data annotation stage is greater than the preset difference threshold, it means that the target query data annotation stage has an abnormality, which may be that the processing speed of the target query data annotation stage is slow, or that the data annotation event has lost data when it is transmitted to the data annotation device of the target query data annotation stage.
[0103] In some embodiments, a visualization graph may be generated based on the number of target data annotation events included in the target query data annotation stage within the query target time period. In the visualization graph, each target query data annotation stage corresponds to a graphic element. The length of the graphic element is proportional to the number of target data annotation events included in the target query data annotation stage within the query target time period. For example, the more target data annotation events included in the target query data annotation stage within the query target time period, the longer the graph corresponding to the target query data annotation stage. The arrangement order between the image elements corresponding to the target query data annotation stage can be arranged according to the execution order of the target query stage in the data annotation process. For example, the top-down order of the image elements corresponding to the target query data annotation stage corresponds to the execution order of the target query data annotation stage in the data annotation process. In this way, the visualization graph generated based on the number of target data annotation events included in the target query data annotation stage within the query target time period can more intuitively reflect the processing of the target query data annotation event in the target query data annotation stage.
[0104] As a possible implementation method, the target information includes time difference information, that is, the time difference information of target data annotation events included in the query time period in each target query phase needs to be counted.
[0105] At this time, in the record information table of data annotation events, it is determined that the target information of the target data annotation events included in the query target time period in each target query data annotation stage includes:
[0106] In the record information table of the data marking event, the time difference information corresponding to the target query data marking phase of the target data marking event within the query target time period is determined.
[0107] In an embodiment of the present application, after determining the target data labeling event, the data processing device can query the identification information and time information of the completed data labeling stage of the target data labeling event in the record information table of the data labeling event. According to the identification information of the target query data labeling stage, the identification information and time information of the completed data labeling stage of the target data labeling event are used to determine the identification information and time information of the completed target query data labeling stage of the target data labeling event and the data labeling stage before the target query data labeling stage. According to the identification information and time information of the completed target query data labeling stage of the target data labeling event and the data labeling stage before the target query data labeling stage, the time difference information corresponding to the completed target query data labeling stage of the target data labeling event within the query target time period is statistically calculated.
[0108] In some embodiments, the time difference information corresponding to the query target time period in each target query data annotation stage can be represented in a visual graph. For example, the time difference information of the data annotation stage can be represented by color depth, and the larger the time difference, the darker the color. Of course, it can also be represented in other ways, such as by the width of the graphic element, and this application does not limit this. In this way, the processing time consumption of the target query data annotation stage of the target data annotation event can be more intuitively reflected.
[0109] As a possible implementation, in the record information table of the data annotation event, the time difference information corresponding to the target query data annotation phase of the target data annotation event within the query target time period includes:
[0110] Obtain data labeling process information; the data labeling process information includes identification information of each data labeling stage included in the data labeling process, and the identification information of each data labeling stage is arranged according to the execution order of each data labeling stage in the data labeling process. For the identification information of each target query data labeling stage in the query list, based on the data labeling process information, determine whether there is identification information of the first target data labeling stage corresponding to the identification information of the target query data labeling stage in the query list; if not, determine the identification information of the second target data labeling stage corresponding to the identification information of the target query data labeling stage according to the query list and the data labeling process information, and determine the identification information of the fourth data labeling stage corresponding to the identification information of the target query data labeling stage according to the data labeling process information, and store the identification information of the target query data labeling stage, the identification information of the fourth target data labeling stage and the identification information of the second target data labeling stage as a first query subset in the first set; if it exists, store the identification information of the target query data labeling stage and the identification information of the first target data labeling stage as a second query subset in the second set; wherein, the identification information of the first target data labeling stage is the identification information of the target query data labeling stage in the data labeling process information. The identification information of the second target data annotation stage is the identification information of the third target data annotation stage in the query list when the identification information of the third target data annotation stage exists in the identification information of at least one data annotation stage included in the query list, and the identification information of the third target data annotation stage is the identification information of the previous data annotation stage of the identification information of the target query data annotation stage when the identification information of at least one data annotation stage included in the query list is arranged in the execution order of the data annotation process; or, the identification information of the second target data annotation stage is the identification information of the first data annotation stage in the data annotation process information when the identification information of the third target data annotation stage does not exist in the identification information of at least one data annotation stage included in the query list; the identification information of the fourth target data annotation stage is the identification information of the data annotation stage between the identification information of the target query data annotation stage and the identification information of the second target data annotation stage in the data annotation process information. According to the record information table of the data annotation event, the first set and the second set, the time difference information corresponding to each target query data annotation stage in the query target time period is determined.
[0111] In an embodiment of the present application, it is necessary to refer to the execution order of each data annotation stage in the data annotation process. Therefore, it is necessary to arrange the identification information of each data annotation stage according to the execution order of each data annotation stage in the data annotation process and store it as data annotation process information.
[0112] For the identification information of each target query data marking stage in the query list, the identification information of at least one data marking stage contained in the query list is arranged according to the execution order of the data marking process. When the query list contains the identification information of the previous data marking stage of the target query data marking stage, the time difference information corresponding to the target query data marking stage in the query target time period is related to the time difference between the target query data marking stage and the previous data marking stage of the target query data marking stage within the query target time period; or when there is a data marking stage between the target query data marking stage and the previous data marking stage in the data marking process, the time difference information corresponding to the target query data marking stage in the query target time period is related to the time difference between the target query data marking stage, the previous data marking stage of the target query data marking stage and the interval data marking stage within the query target time period. Therefore, when determining the time difference corresponding to the target query data marking stage of the target data marking event within the query target time period, it is necessary to store the identification information of the target query data marking stage and the corresponding identification information of the above-mentioned data marking stage as a query subset.
[0113] It should be understood that in the embodiment of the present application, two adjacent data labeling stages refer to being adjacent when sorted in the execution order in the data labeling process.
[0114] In the above process, the identification information of the previous data annotation stage of the target query data annotation stage determined in the query list is the identification information arranged before the identification information of the target query data annotation stage when the identification information of each target query data annotation stage in the query list is arranged according to the execution order of the data annotation process. Since the target query data annotation stage in the query list does not necessarily include all the data annotation stages in the data annotation process, the identification information of the previous data annotation stage of the target query data annotation stage may be the identification information of the previous data annotation stage adjacent to the target query data annotation stage. It may also be the identification information of a data annotation stage that is not adjacent to the target query data annotation stage and is executed before the target query data annotation stage.
[0115] Based on whether the previous data annotation stage of the target query data annotation stage is adjacent to the target query data annotation stage, different steps of determining the time difference information corresponding to the target query data annotation stage in the query target time period are performed. That is, according to whether the previous data annotation stage of the target query data annotation stage is adjacent to the target query data annotation stage, the identification information of the target query data annotation stage and the identification information of the previous data annotation stage are respectively stored in different query subsets. For example, if the target query data annotation stage is not adjacent to its previous data annotation stage, the identification information of the target query data annotation stage, the identification information of the previous data annotation stage of the target query data annotation stage, and the identification information of the data annotation stage separated by the two in the data annotation process information are made into a first query subset, and the first query subset can be stored in the first set. If the target query data annotation stage is adjacent to its previous data annotation stage, the identification information of the target query data annotation stage and the identification information of the previous data annotation stage of the target query data annotation stage are made into a second query subset, and the second query subset can be stored in the second set.
[0116] When the query list does not contain identification information of the previous data annotation stage of the target query data annotation stage, that is, when the identification information of at least one data annotation stage contained in the query list is arranged in the execution order of the data annotation process, the identification information of the target data query data annotation stage is the identification information of the first target query data annotation stage in the arranged query list. At this time, the time difference information corresponding to the target query data annotation stage in the query target time period is related to the time difference of the target query data annotation stage and the data annotation process information before the target query data annotation stage in the query target time period. For ease of implementation, the identification information of the target query data annotation stage and the identification information of the data annotation process information before the target query data annotation stage can be stored in the first set as a first query subset.
[0117] That is to say, for the identification information of each target query data annotation stage in the query list, the identification information of the previous adjacent data annotation stage of the identification information of the target query data annotation stage is determined according to the data annotation process information, that is, the identification information of the first target data annotation stage is determined. After the identification information of the first target data annotation stage is determined, it is detected whether there is the identification information of the first target data annotation stage in the query list. For example, the identification information of each target data annotation stage recorded in the query list can be compared with the identification information of the first target data annotation stage to determine whether there is the identification information of the first target data annotation stage in the query list. If so, the identification information of the target data annotation stage and the identification information of the first target data annotation stage are taken as a second query subset and stored in the second set.
[0118] If it does not exist, it means that there is no identification information of the previous data annotation stage adjacent to the target query data annotation stage in the query list. At this time, it can be detected whether there is identification information of the third target data annotation stage in the query list. Among them, the identification information of the third target data annotation stage is the identification information of the data annotation stage that is ranked before the identification information of the target query data annotation stage when the identification information of at least one data annotation stage contained in the query list is sorted according to the execution order of the data annotation process. That is, when the identification information of at least one data annotation stage contained in the query list is sorted according to the execution order of the data annotation process, if the identification information of the target query data annotation stage is ranked first, then there is no identification information of the third target data annotation stage for the identification information of the target query data annotation stage. If the identification information of the target query data annotation stage is not ranked first, there is identification information of the third target data annotation stage for the identification information of the target query data annotation stage. Therefore, the identification information of at least one data annotation stage contained in the query list can be sorted according to the execution order of the data annotation process according to the data annotation process information, so as to determine whether there is identification information of the third target data annotation stage in the query list. If the query list contains identification information of the third target data labeling stage, the identification information of the third target data labeling stage is determined as the identification information of the second target data labeling stage. Since the second target data labeling stage and the target query data labeling stage are not two adjacent data labeling stages, there are other data labeling stages between the second target data labeling stage and the target query data labeling stage in the data labeling process. At this time, the identification information of other data labeling stages can be determined based on the data labeling process information. That is, the identification information of the fourth target data labeling stage is determined. At this time, the identification information of the target query data labeling stage, the identification information of the second target data labeling stage, and the identification information of the fourth target data labeling stage can be stored in the first set as a first query subset.
[0119] If the identification information of the third target data annotation stage does not exist in the query list, at this time, the identification information of the first data annotation process in the data annotation process information can be determined as the identification information of the second target data annotation stage according to the data annotation process information. At this time, the second target data annotation stage and the target query data annotation stage may be two adjacent data annotation stages. That is, the second target data annotation stage is the previous data annotation stage adjacent to the target query data annotation stage, and there is no data annotation stage between the second target data annotation stage and the target query data annotation stage. The identification information of the second target data annotation stage and the identification information of the target query data annotation stage can be directly stored in the first set as a first query subset. Alternatively, when the second target data annotation stage and the target query data annotation stage are not two adjacent data annotation stages, it can be determined according to the data annotation process information that there is another data annotation stage between the second target data annotation stage and the target query data annotation stage. That is, the identification information of the fourth target data annotation stage is determined. The identification information of the target query data marking stage, the identification information of the second target data marking stage, and the identification information of the fourth target data marking stage are taken as a first query subset and stored in the first set.
[0120] It should be noted that when the identification information of the target query data annotation stage is the identification information of the first data annotation stage in the data annotation process information, the identification information of the target query data annotation stage can be directly stored in the first set as a first query subset. The identification information of the target query data annotation stage can also be used as the identification information of its second target data annotation stage, so that the identification information of the second target data annotation stage and the identification information of the target query data annotation stage can be directly stored in the first set as a first query subset. This application does not impose any restrictions on this.
[0121] The record information table of the data annotation event records the identification information and time information of the completed data annotation stage of the created data annotation event, the first set and the second set record the query subsets corresponding to the target query data annotation stage, and the query subset corresponding to each target query data annotation stage records the identification information of the data annotation stage related to the time difference information of the target query data annotation stage. In this way, the time difference information corresponding to the target query data annotation stage in the query target time period can be determined based on the record information table of the data annotation event, the first set and the second set.
[0122] As a possible implementation method, for the convenience of implementation, when determining the query subset corresponding to each target query data annotation in the query list, the identification information of each data annotation stage contained in the data annotation process information can be sequentially detected according to the data annotation process information to determine whether it exists in the query list, as described in detail below. Figure 2 As shown, for the identification information of each target query data marking stage in the query list, based on the data marking process information, determine whether there is identification information of the first target data marking stage corresponding to the identification information of the target query data marking stage in the query list; if not, determine the identification information of the second target data marking stage corresponding to the identification information of the target query data marking stage according to the query list and the data marking process information, and determine the identification information of the fourth data marking stage corresponding to the identification information of the target query data marking stage according to the data marking process information, and store the identification information of the target query data marking stage, the identification information of the fourth target data marking stage and the identification information of the second target data marking stage as a first query subset in the first set; if it exists, store the identification information of the target query data marking stage and the identification information of the first target data marking stage as a second query subset in the second set; according to the record information table of the data marking event, the first set and the second set, determine that the time difference information corresponding to each target query data marking stage in the query target time period includes:
[0123] Step S201: determine identification information of the first data marking stage and identification information of the second data marking stage in the query list.
[0124] Among them, the identification information of the first data marking stage is the identification information of the first data marking stage when the identification information of at least one data marking stage contained in the query list is arranged in the order of the data marking process information; the identification information of the second data marking stage is the identification information of the last data marking stage when the identification information of at least one data marking stage contained in the query list is arranged in the order of the data marking process information.
[0125] In an embodiment of the present application, after obtaining the data annotation process information, the data processing device can learn the execution order of each data annotation stage according to the data annotation process information. After obtaining the query list, the identification information of the first data annotation stage when arranged in the order of the data annotation process information can be determined in the query list according to the execution order of each data annotation stage, that is, the identification information of the first data annotation stage is determined. And the identification information of the last data annotation stage when arranged in the order of the data annotation process information in the query list can be determined, that is, the identification information of the second data annotation stage is determined.
[0126] Among them, since the identification information of the second target data annotation stage is not included in the query list, the identification information of the first data annotation stage in the data annotation process information can be confirmed as the identification information of the second target data annotation stage. Therefore, it is necessary to first determine the identification information of the first data annotation stage in the query list in order to calculate the time difference information between the identification information of the first data annotation stage and the identification information of the second target data annotation stage.
[0127] As a possible implementation method, the identification information of at least one data labeling stage contained in the query list is arranged in the order of the data labeling process information, so that the identification information of the first data labeling stage and the identification information of the second data labeling stage in the query list can be determined.
[0128] As a possible implementation method, check whether the query list contains the process information of the first data annotation stage in the data annotation process information. If so, determine the identification information of the first data annotation stage as the process information of the first data annotation stage in the data annotation process information. If not, check whether the query list contains the process information of the second data annotation stage in the data annotation process information. And so on, until the first data annotation stage is determined. Similarly, check whether the query list contains the process information of the last data annotation stage in the data annotation process information. If so, determine the identification information of the last data annotation stage as the process information of the second data annotation stage in the data annotation process information. If not, check whether the query list contains the process information of the second to last data annotation stage in the data annotation process information. And so on, until the second data annotation stage is determined.
[0129] Step S202: Detect whether the identification information of the first data labeling stage is the identification information of the first data labeling stage in the data labeling process information.
[0130] In an embodiment of the present application, after determining the identification information of the first data annotation stage in the query list, the data processing device can determine the identification information of the first data annotation stage in the data annotation process information, and compare the identification information of the first data annotation stage with the identification information of the first data annotation stage to determine whether the identification information of the first data annotation stage is the identification information of the first data annotation stage in the data annotation process information.
[0131] It should be noted that the data processing device performs the following steps differently according to different detection results. When it is detected that the identification information of the first data annotation stage is not the identification information of the first data annotation stage in the data annotation process information, step S203 and steps S205-S211 are executed. When it is detected that the identification information of the first data annotation stage is the identification information of the first data annotation stage in the data annotation process information, steps S204-S211 are executed.
[0132] Step S203: If not, determine the identification information of each data annotation stage before the first data annotation stage in the data annotation process information, and store the identification information of each data annotation stage before the first data annotation stage and the identification information of the first data annotation stage as a first query subset in the first set.
[0133] In an embodiment of the present application, when the data processing device determines the time difference information of the target data marking stage, it determines the time difference information between the previous data marking stage of the target data marking stage and the target data marking stage in the query list. However, the previous data marking stage of the first data marking stage does not exist in the query list. Therefore, it can be considered that the time difference information of the first data marking stage is related to all the data marking stages before it. At this time, the identification information of each data marking stage before the identification information of the first data marking stage can be determined according to the data marking process information, that is, the identification information of the second target data marking stage of the first data marking stage and the identification information of the fourth target data marking stage are determined. And the identification information of each data marking stage before the first data marking stage and the identification information of the first data marking stage are stored in the first set as a first query subset.
[0134] Step S204: If yes, store the identification information of the first data labeling stage as a first query subset in the first set.
[0135] In an embodiment of the present application, if the identification information of the first data labeling stage is the identification information of the first data labeling stage in the data labeling process information, it can be determined that there is no time difference information in the first data labeling stage, or its time information is only related to itself. At this time, the identification information of the first data labeling stage can be stored in the first set as a first query subset.
[0136] Step S205: determine the first data labeling stage as the current stage.
[0137] Step S206: Detect whether the current stage is the second data labeling stage.
[0138] In an embodiment of the present application, after determining the current stage, the data processing device needs to detect whether the current stage is the last data annotation stage of the query list. At this time, it can be detected whether the current stage is the second data annotation stage. For example, the identification information of the current stage can be compared with the identification information of the second data annotation stage. If the identification information of the current stage is the same as the identification information of the second data annotation stage, it can be determined that the current stage is the second data annotation stage. If the identification information of the current stage is different from the identification information of the second data annotation stage, it can be determined that the current stage is not the second data annotation stage.
[0139] When it is determined that the current stage is not the second data labeling stage, the following step S207 is continued to be executed. When it is determined that the current stage is the second data labeling stage, the process jumps to execute step S211.
[0140] Step S207: If not, determine the identification information of the next data annotation stage of the current stage as the identification information of the target stage in the data annotation process information, and update the value of the jump parameter to the first preset value.
[0141] In the embodiment of the present application, after determining the current stage, the data processing device can determine the identification information of the data annotation stage adjacent to the current stage and whose execution order is after the current stage in the data annotation process information, that is, determine the identification information of the next data annotation stage of the current stage in the data annotation process information. The next data annotation stage of the current stage determined is used as the target stage, and the identification information of the next data annotation stage of the current stage is used as the identification information of the target stage.
[0142] After determining the identification information of the target stage, the data processing device may increase the value of the jump parameter by a first preset value.
[0143] The first preset value is preset by the user according to actual needs. For example, the first preset value is 1. At this time, each time the identification information of the target stage is determined, the value of the jump parameter can be increased by 1. Of course, the first preset value can also be other values, which is not limited by the present application.
[0144] Step S208: Check whether the query list contains identification information of the target stage.
[0145] In the embodiment of the present application, after determining the identification information of the target stage, the data processing device can detect whether the identification information of the target stage exists in the query list. For example, the identification information of the target stage can be compared with the identification information of each target query data annotation stage recorded in the query list. If the identification information of the target stage is recorded in the query list, it can be determined that the query list contains the identification information of the target stage. If the identification information of the target stage is not recorded in the query list, it can be determined that the query list does not contain the identification information of the target stage.
[0146] When it is determined that the query list does not contain the identification information of the target stage, the following step S209a is continued to be executed. When the query list contains the identification information of the target stage, the process jumps to execute step S209b.
[0147] Step S209a: If the query list does not contain the identification information of the target stage, the identification information of the current stage is recorded in the first information, the identification information of the current stage is updated to the identification information of the target stage, and the step is re-executed to determine the identification information of the next data annotation stage of the current stage as the identification information of the target stage in the data annotation process information, and the value of the jump parameter is updated to the first preset value, and it is detected that the query list contains the identification information of the target stage.
[0148] In an embodiment of the present application, when the data processing device determines that the query list does not contain the identification information of the target stage, it means that the determined target stage is a fourth target data annotation stage in the above process, and the identification information of the target stage is recorded in the first information. Since in this embodiment, it is detected whether the query list contains the identification information of the data annotation stage recorded in the data annotation process information, therefore, after determining the identification information of the first data annotation stage, it is necessary to determine whether the query list contains the identification information of the data annotation stage located after the identification information of the first data annotation stage in the data annotation process information, so as to determine the query subset of each target query data annotation stage in the query list. Therefore, after recording the identification information of the target stage in the first information, the identification information of the current stage can be updated to the identification information of the target stage. That is, the value of the identification information of the current stage is updated to the value of the identification information of the target stage, and it is necessary to re-determine whether the identification information of the target stage of the updated current stage is in the query list. At this time, after updating the identification information of the current stage to the identification information of the target stage, the above steps S207 to S208 can be re-executed.
[0149] Step S209b: If the query list contains identification information of the target stage, detect whether the value of the jump parameter is a second preset value.
[0150] In the embodiment of the present application, when the data processing device determines that the identification information of the target phase is included in the query list, the data processing device can determine the data annotation phase related to the time difference of the target phase. At this time, it can be detected whether the value of the jump parameter is the second preset value.
[0151] The second preset value is preset by the user according to actual needs. In some embodiments, the second preset value is the sum of the preset initial value of the jump parameter and the first preset value.
[0152] The data processing device performs different steps according to different detection results: when it is detected that the value of the jump parameter is not the second preset value, the following step S210a is performed; when it is detected that the value of the jump parameter is the second preset value, the following step S210b is performed.
[0153] Step S210a: If it is not the second preset value, the identification information of the current stage, the identification information of the target stage and the identification information of the data labeling stage recorded in the first information are stored in the first set as a first query subset, the identification information of the current stage is updated to the identification information of the target stage, the value of the jump parameter is updated to the preset initial value, and the step of detecting whether the current stage is the second data labeling stage is re-executed until it is detected that the current stage is the second data labeling stage.
[0154] In an embodiment of the present application, when the data processing device determines that the query list contains the identification information of the target stage, and the value of the jump parameter is not the second preset value, it means that the identification information of the current stage does not exist in the query list, and the first information records the identification information of the second target data annotation stage in the above process, or records the identification information of the second target data annotation stage and the identification information of the fourth target data annotation stage, and the identification information of the current stage is also the identification information of the fourth target data annotation stage in the above process, and the identification information of the target stage at this time is the identification information of the target query data annotation stage in the above process. At this time, the identification information of the current stage, the identification information of the target stage, and the identification information of the data annotation stage recorded in the first information can be stored in the first set as a first query subset.
[0155] The data processing device also needs to continue to detect whether the query list contains the identification information of the data annotation stage after the identification information of the target stage in the data annotation process information, so as to determine the query subset of other target query data annotation stages in the query list. At this time, it is necessary to update the identification information of the current stage to the identification information of the target stage, update the value of the jump parameter to the preset initial value, and re-execute the above steps S206 to S209b.
[0156] Step S210b, if it is the second preset value, the identification information of the current stage and the identification information of the target stage are stored in the second set as a second query subset, the identification information of the current stage is updated to the identification information of the target stage, the value of the jump parameter is updated to the preset initial value, and the step of detecting whether the current stage is the second data labeling stage is re-executed until it is detected that the current stage is the second data labeling stage.
[0157] That is, when the data processing device determines that the query list contains the identification information of the target stage and the value of the jump parameter is the second preset value, it means that the query list contains the identification information of the current stage and the identification information of the target stage, and the current stage and the target stage are two adjacent data labeling stages. At this time, the identification information of the current stage and the identification information of the target stage can be directly taken as a second query subset and stored in the second set.
[0158] The data processing device also needs to continue to detect whether the query list contains the identification information of the data annotation stage after the identification information of the target stage in the data annotation process information, so as to determine the query subset of other target query data annotation stages in the query list. At this time, it is necessary to update the identification information of the current stage to the identification information of the target stage, update the value of the jump parameter to the preset initial value, and re-execute the above steps S206 to S209b.
[0159] Step S211: If the current stage is the second data labeling stage, determine the time difference information corresponding to the query target time period of each target query data labeling stage according to the record information table of the data labeling event, the first set and the second set.
[0160] In an embodiment of the present application, if the current stage is the second data annotation stage, it means that the query subset corresponding to each target query data annotation stage in the query list has been determined. At this time, when the first set and the second set are both non-empty sets, the data processing device can determine the time difference information corresponding to each target query data annotation stage in the query target time period based on the record information table of the data annotation event, the first set, and the second set. When the first set is a non-empty set and the second set is an empty set, the data processing device can determine the time difference information corresponding to each target query data annotation stage in the query target time period based on the record information table of the data annotation event and the first set. When the first set is an empty set and the second set is a non-empty set, the data processing device can determine the time difference information corresponding to each target query data annotation stage in the query target time period based on the record information table of the data annotation event and the second set.
[0161] As a possible implementation manner, determining the time difference information corresponding to each data annotation stage in the query list according to the record information table of the data annotation event, the first set, and the second set includes:
[0162] When the first set is a non-empty set, for each first query subset contained in the first set, in the record information table of the data labeling event, the time difference between every two adjacent data labeling stages in all the data labeling stages contained in the first query subset within the query target time period is determined; based on the time difference between every two adjacent data labeling stages in all the data labeling stages contained in the first query subset within the query target time period, at least one of the maximum time difference, the minimum time difference and the average time difference corresponding to the target query data labeling stage in the first query subset is determined.
[0163] In an embodiment of the present application, the identification information and time information of the completed data annotation stage of the data annotation event are recorded in the record information table of the data annotation event. When it is determined that the first set is a non-empty set, for each first query subset in the first set, in the record information table of the data annotation event, the time information of the data annotation stage contained in the first query subset is determined, and then the time difference between every two adjacent data annotation stages in all data annotation stages contained in the first query subset within the query target time period can be determined. In some embodiments, when the time information includes the completion time, the difference between the completion time of every two adjacent data annotation stages in all data annotation stages contained in the first query subset can be calculated in the record information table of the data annotation event to determine the time difference between every two adjacent data annotation stages in all data annotation stages contained in the first query subset within the query target time period. In some embodiments, when the time information includes the time difference information between two adjacent completed data annotation stages, the time difference between every two adjacent data annotation stages in all data annotation stages contained in the first query subset within the query target time period can be directly determined in the record information table of the data annotation event.
[0164] When the first query subset only includes identification information of one data labeling stage, it can be considered that there is no time difference in the target query data labeling stage, and the maximum time difference, minimum time difference and average time difference corresponding to the target query data labeling stage are all determined to be 0.
[0165] When the first query subset includes identification information of two data annotation stages, it means that the two data annotation stages are two adjacent data annotation stages, and the maximum time difference, the minimum time difference and the average time difference among the time differences between the two adjacent data annotation stages within the query target time period are determined. Among them, the average time difference is the sum of the time differences between the two adjacent data annotation stages within the query target time period, and the ratio of the number of target data annotation events contained in the target query data annotation stage in the first query subset. The above maximum time difference, minimum time difference and average time difference are determined as the maximum time difference, minimum time difference and average time difference corresponding to the target query data annotation stage.
[0166] When the first query subset may include identification information of at least three data annotation stages, the time difference between all adjacent data annotation stages in the first query subset within the query target time period is determined, so that among the time differences determined above, the maximum value is determined as the maximum time difference, and the minimum value is determined as the minimum time difference. And the average time difference is calculated based on all the determined time differences. For example, the sum of the time differences between all adjacent data annotation stages in the first query subset within the query target time period is calculated, and the ratio of the sum of the number of target data annotation events contained in all fourth target data annotation stages and target query data annotation stages in the first query subset within the query target time period is determined, and the ratio is determined as the average time difference. The above maximum time difference, minimum time difference and average time difference are determined as the maximum time difference, minimum time difference and average time difference corresponding to the target query data annotation stage.
[0167] As a possible implementation manner, determining the time difference information corresponding to each data annotation stage in the query list according to the record information table of the data annotation event, the first set, and the second set includes:
[0168] When the second set is a non-empty set, for each second query subset included in the second set, in the record information table of the data labeling event, the time difference between two data labeling stages included in the second query subset within the query target time period is determined; based on the time difference between the two data labeling stages included in the second query subset within the query target time period, at least one of the maximum time difference, the minimum time difference and the average time difference corresponding to the target stage in the second query subset is determined.
[0169] In an embodiment of the present application, the two data annotation stages included in the second query subset are two adjacent data annotation stages. The identification information and time information of the completed data annotation stage of the data annotation event recorded in the record information table of the data annotation event. In some embodiments, when the time information includes the completion time, the time difference between the two data annotation stages included in the second query subset within the query target time period can be determined in the record information table of the data annotation event by calculating the difference between the completion times of the two adjacent data annotation stages included in the second query subset. In some embodiments, when the time information includes the time difference information between two adjacent completed data annotation stages, the time difference between the two data annotation stages included in the second query subset within the query target time period can be directly determined in the record information table of the data annotation event.
[0170] After determining the time difference between the two data annotation stages included in the second query subset within the query target time period, the maximum time difference, minimum time difference and average time difference between the two data annotation stages included in the second query subset within the query target time period can be determined. Among them, the average time difference is the sum of the time differences between the two data annotation stages included in the second query subset within the query target time period, and the ratio of the number of target data annotation events included in the target query data annotation stage in the second query subset. The above maximum time difference, minimum time difference and average time difference are determined as the maximum time difference, minimum time difference and average time difference corresponding to the target query data annotation stage.
[0171] As a possible implementation method, for the identification information of each data annotation stage in the query list, based on the data annotation process information, it is determined whether there is identification information of the first target data annotation stage corresponding to the identification information of the data annotation stage in the query list; if not, the identification information of the second target data annotation stage corresponding to the identification information of the data annotation stage is determined according to the query list and the data annotation process information, and the identification information of the fourth data annotation stage corresponding to the identification information of the data annotation stage is determined according to the data annotation process information, and the identification information of the data annotation stage, the identification information of the fourth target data annotation stage and the identification information of the second target data annotation stage are stored as a first query subset in the first set; if it exists, the identification information of the data annotation stage and the identification information of the first target data annotation stage are stored as a second query subset in the second set, including:
[0172] Determine a first query list, the first query list contains identification information of at least one data annotation stage for which time difference information is to be counted within the query target time period, and the identification information of at least one data annotation stage is sorted in the order of the data annotation process. Obtain the identification information of the first data annotation stage contained in the first query list, and detect whether the identification information of the first data annotation stage is the identification information of the first data annotation stage of the data annotation process. If it is not the identification information of the first data annotation stage of the data annotation process, determine the identification information of each data annotation stage before the first data annotation stage in the data annotation process, and store the identification information of each data annotation stage before the first data annotation stage and the identification information of the first data annotation stage in the data annotation process as a first query subset in the first set. If it is the identification information of the first data annotation stage of the data annotation process, store the identification information of the first data annotation stage as a first query subset in the first set. Determine the identification information of the first data annotation stage contained in the first query list as the first target query data annotation stage, and detect whether the identification information of the first target query data annotation stage is the identification information of the last data annotation stage of the first query list. If not, the identification information of the second data annotation stage included in the first query list is determined as the identification information of the second target query data annotation stage. It is detected whether the identification information of the second target query data annotation stage is the identification information of the next data annotation stage of the first target query data annotation stage in the data annotation process.
[0173] If the identification information of the second target query data marking stage is the identification information of the next data marking stage of the first target query data marking stage in the data marking process, then the identification information of the first target query data marking stage and the identification information of the second target query data marking stage are stored in the second set as a second query subset, the identification information of the first target query data marking stage in the first query list is deleted, the identification information of at least one data marking stage contained in the first query list is updated, and the step of determining the identification information of the first data marking stage contained in the first query list as the identification information of the first target query data marking stage is re-executed, and detecting whether the identification information of the first target query data marking stage is the identification information of the last data marking stage in the first query list, and then the step of updating the identification information of at least one data marking stage contained in the first query list is performed until it is detected that the identification information of the first target data marking stage is the identification information of the last data marking stage in the first query list.
[0174] Alternatively, if the identification information of the second target query data marking stage is not the identification information of the next data marking stage of the first target query data marking stage in the data marking process, then determine the identification information of at least one data marking process between the first target query data marking stage and the second target query data marking stage; and store the identification information of the first target query data marking stage, the identification information of at least one data marking process between the first target query data marking stage and the second target query data marking stage, and the identification information of the second target query data marking stage as a first query subset in the first set, delete the first target query data marking stage in the first query list, update the identification information of at least one data marking stage contained in the first query list, and re-execute the step of determining the first data marking stage contained in the first query list as the first target query data marking stage, and detect whether the first target data marking stage is the last data marking stage of the first query list, to the step of updating the identification information of at least one data marking stage contained in the first query list, until it is detected that the first target query data marking stage is the last data marking stage of the first query list.
[0175] If it is detected that the first target query data marking stage is the last data marking stage of the first query list, the time difference information corresponding to each data marking stage in the query list in the query target time period is determined according to the record information table of the data marking event, the first set and the second set.
[0176] In an embodiment of the present application, when determining the query subset corresponding to the target query data annotation stage, it is necessary to determine the execution order of the target query data annotation stage in the data annotation process, that is, to determine the first query list. The identification information of the first data annotation stage contained in the first query list is obtained. At this time, the identification information of the first data annotation stage contained in the first query list is the first data annotation stage in the above process. Therefore, if the identification information of the first data annotation stage is not the identification information of the first data annotation stage of the data annotation process, the identification information of each data annotation stage before the first data annotation stage in the data annotation process is determined, and the identification information of each data annotation stage before the first data annotation stage in the data annotation process and the identification information of the first data annotation stage are stored in the first set as a first query subset. If the identification information of the first data annotation stage is the identification information of the first data annotation stage of the data annotation process, the identification information of the first data annotation stage is stored in the first set as a first query subset.
[0177] The identification information of the first data annotation stage included in the first query list is determined as the first target query data annotation stage, and it is detected whether the identification information of the first target query data annotation stage is the identification information of the last data annotation stage of the first query list. That is, it is detected whether the identification information of the first target query data annotation stage is the identification information of the second data annotation stage in the above process.
[0178] If the identification information of the first target query data annotation stage is not the identification information of the last data annotation stage in the first query list, the identification information of the second data annotation stage contained in the first query list is determined as the identification information of the second target query data annotation stage. Detect whether the identification information of the second target query data annotation stage is the identification information of the next data annotation stage of the first target query data annotation stage in the data annotation process. If the identification information of the second target query data annotation stage is the identification information of the next data annotation stage of the first target query data annotation stage in the data annotation process, it means that the identification information of the first target query data annotation stage and the second target query data annotation stage are two adjacent data annotation stages in the data annotation process. Therefore, the identification information of the first target query data annotation stage and the identification information of the second target query data annotation stage are stored in the second set as a second query subset. That is, determine the query subset corresponding to the second target query data annotation stage. After determining the query subset of the second target query data annotation stage, continue to determine the query subsets corresponding to other target query data annotation stages. The query subsets corresponding to the other target query data annotation stages are irrelevant to the first target query data annotation stage, so the identification information of the first target query data annotation stage in the first query list is deleted, and the identification information of at least one data annotation stage contained in the first query list is updated. Re-execute the step of determining the identification information of the first data annotation stage contained in the first query list as the identification information of the first target query data annotation stage, and detect whether the identification information of the first target query data annotation stage is the identification information of the last data annotation stage in the first query list, and then step of updating the identification information of at least one data annotation stage contained in the first query list until it is detected that the identification information of the first target data annotation stage is the identification information of the last data annotation stage in the first query list.
[0179] If the identification information of the second target query data annotation stage is not the identification information of the next data annotation stage of the first target query data annotation stage in the data annotation process, it means that the identification information of the first target query data annotation stage and the second target query data annotation stage are not adjacent in the data annotation process. Determine the identification information of at least one data annotation process between the first target query data annotation stage and the second target query data annotation stage, and store the identification information of the first target query data annotation stage, the identification information of at least one data annotation process between the first target query data annotation stage and the second target query data annotation stage, and the identification information of the second target query data annotation stage as a first query subset in the first set. That is, determine the query subset corresponding to the second target query data annotation stage. After determining the query subset of the second target query data annotation stage, continue to determine the query subsets corresponding to other target query data annotation stages. The query subsets corresponding to other target query data annotation stages are irrelevant to the first target query data annotation stage, so the identification information of the first target query data annotation stage in the first query list is deleted, and the identification information of at least one data annotation stage contained in the first query list is updated. Re-execute step to determine the identification information of the first data labeling stage contained in the first query list as the identification information of the first target query data labeling stage, and detect whether the identification information of the first target query data labeling stage is the identification information of the last data labeling stage of the first query list, to step to update the identification information of at least one data labeling stage contained in the first query list until it is detected that the identification information of the first target data labeling stage is the identification information of the last data labeling stage of the first query list.
[0180] If it is detected that the first target query data annotation stage is the last data annotation stage of the first query list, it means that the query subset corresponding to each target query data annotation stage has been determined. According to the record information table of the data annotation event, the first set and the second set, the time difference information corresponding to each data annotation stage in the query list in the query target time period can be determined.
[0181] Exemplarily, assume that the various data annotation stages in the data annotation process are arranged in the order of execution in the data annotation process: creation stage, stage A, stage B, stage C, stage D, stage E, stage F. The preset initial value of the jump parameter is 0, the first preset value is 1, and the first preset value is updated to increase the value of the jump parameter by the first preset value, that is, the value of the jump parameter is increased by 1, and the second preset value is the sum of the initial preset value and the first preset value, which is 1.
[0182] Assume that the target creation time set by the user is T1, the query time is T2, and the data annotation stages of the target information to be counted are stage B, stage C, and stage E. The target information includes quantity information and time difference information, and the time difference information is set to the average time difference.
[0183] The data processing device obtains the query time and the target creation time, where the target creation time is T1 and the query time is T2. T1 is a time range. The data processing device obtains the record information table of the data annotation event, and determines the target data annotation event in the record information table of the data annotation event based on the target creation time of the data annotation event. The target data annotation event is a data annotation event whose creation time is within the target creation time T1.
[0184] The data processing device obtains a query list, which records the identification information of the data annotation stage of the target information to be counted, that is, the identification information of stage B, stage C, and stage E. Based on the query list, the data processing device determines that the target query data annotation stage is stage B, stage C, and stage E.
[0185] The data processing device determines the query target time period as T1-T2 based on the query time and the target creation time. The record information table of the data annotation event records the identification information and time information of the completed data annotation stage of the target data annotation event. Among them, the time information includes the completion time and the time difference information of the two adjacent completed data annotation stages. The data processing device determines the record information table of the target data annotation event in the record information table of the data annotation event. In the record information table of the target data annotation event, the identification information and time information of the completed target query data annotation stage of the target data annotation event are determined. Detect whether the completion time of the target query data annotation stage meets the query target time period. Assume that there are 100 target data annotation events that complete the B stage within the query target time period, 80 target data annotation events that complete the C stage within the query target time period, and 30 target data annotation events that complete the E stage within the query target time period. The data processing device determines that the number information of the target data annotation events contained in the target query data annotation stages B, C, and E in the query target time period T1-T2 is 100, 80, and 30.
[0186] The data processing device obtains data flow information. The identification information of the data annotation stages contained in the data flow information is arranged according to the execution order in the data annotation process, namely, the identification information of the creation stage, the identification information of stage A, the identification information of stage B, the identification information of stage C, the identification information of stage D, the identification information of stage E, and the identification information of stage F.
[0187] The data processing device sorts the identification information of the target query data labeling stage contained in the query list according to the execution order in the data labeling process, and determines that the identification information of the first data labeling stage is the identification information of stage B, and the identification information of the second data labeling stage is the identification information of stage E.
[0188] The data processing device determines that the identification information of the first data annotation stage is not the identification information of the creation stage. The identification information of each data annotation stage before the first data annotation stage, i.e., stage B, in the data annotation process is determined to be the identification information of the creation stage and the identification information of stage A, and the identification information of the creation stage, the identification information of stage A, and the identification information of stage B are stored in the first set as a first query subset.
[0189] The first data annotation stage, i.e., stage B, is determined as the current stage. It is determined that the current stage is not the second data annotation stage, i.e., stage E. The identification information of the next data annotation stage of the identification information of the current stage, i.e., the identification information of stage B, in the data annotation process information is determined as the identification information of the target stage. That is, the identification information of stage C is determined as the identification information of the target stage. The value of the jump parameter is updated to the first preset value, i.e., the value of the jump parameter is increased by 1, and the value of the jump parameter is updated from the preset initial value 0 to 1. It is determined that the query list contains the identification information of the target stage, i.e., stage C. It is determined that the value 1 of the jump parameter is the second preset value 1. The identification information of the current stage and the identification information of the target stage are stored in the second set as a second query subset, i.e., the identification information of stage B and the identification information of stage C are stored in the second set as a second query subset. The identification information of the current stage is updated to the identification information of the target stage, i.e., the identification information of the current stage is updated to the identification information of the stage C. The value of the jump parameter is updated to the preset initial value 0.
[0190] It is determined that the current stage C is not the second data annotation stage E. In the data annotation process information, the identification information of the next data annotation stage of the current stage C is determined as the identification information of the target stage, that is, the identification information of the D stage is determined as the identification information of the target stage. The value of the jump parameter is updated to the first preset value, that is, the value of the jump parameter is updated to 1. It is determined that the query list does not contain the identification information of the target stage, that is, the D stage. The identification information of the current stage, that is, the C stage, is recorded in the first information. The identification information of the current stage is updated to the identification information of the target stage, that is, the D stage. In the data annotation process information, the identification information of the next data annotation stage of the current stage D is determined as the identification information of the target stage, that is, the identification information of the E stage is determined as the identification information of the target stage. The value of the jump parameter 1 is updated to the first preset value, that is, the value of the jump parameter is updated to 2. It is determined that the query list contains the identification information of the target stage, that is, the E stage. It is determined that the value 2 of the jump parameter is not the second preset value 1. The identification information of the current stage, the identification information of the target stage, and the identification information of the data annotation stage recorded in the first information are stored in the first set as a first query subset, that is, the identification information of the D stage, the identification information of the E stage, and the identification information of the C stage are stored in the first set as a first query subset. The identification information of the current stage is updated to the identification information of the target stage, i.e., the E stage, and the value of the jump parameter is updated to the preset initial value 0.
[0191] It is determined that the current stage, namely stage E, is the second data annotation stage E. According to the record information table of the data annotation event, the data annotation stages included in the first set and the data annotation stages included in the second set, the time difference information corresponding to each data annotation stage in the query list is determined.
[0192] The first set includes two first query subsets. For the first query subset including the identification information of the creation stage, the identification information of stage A, and the identification information of stage B, in the record information table of data annotation events, the time difference between the creation stage and stage A within the query target time period, and the time difference between stage A and stage B within the query target time period are determined. The number of target data annotation events included in stage B within the query target time period is 100, assuming that the number of target data annotation events included in stage A within the query target time period is 200. The average time difference avgB corresponding to stage B within the query target time period is determined as the sum of the time difference between the creation stage and stage A within the query target time period, and the time difference between stage A and stage B within the query target time period, and the ratio of the sum of the number of target data annotation events included in stage A and stage B within the query target time period, 300. Similarly, for the first query subset including the identification information of stage C, the identification information of stage D, and the identification information of stage E, the time difference between stage C and stage D within the query target time period, and the time difference between stage D and stage E within the query target time period are determined in the record information table of data annotation events. The number of target data annotation events included in stage E within the query target time period is 30, assuming that the number of target data annotation events included in stage D within the query target time period is 40. The average time difference avgE corresponding to stage E within the query target time period is determined as the ratio of the sum of the time difference between stage C and stage D within the query target time period, and the time difference between stage D and stage E within the query target time period, to the sum of the number of target data annotation events included in stage A and stage B within the query target time period, 70.
[0193] In this example, the second set includes a second query subset. For the first query subset including the identification information of stage B and the identification information of stage C, the time difference between stage B and stage C within the query target time period is determined in the record information table of the data annotation event. The average time difference avgC corresponding to stage C is determined as the sum of the time differences between stage B and stage C within the query target time period and the ratio of the number of target data annotation events 80 included in stage C within the query target time period.
[0194] Generate a visualization graph based on the quantity information and time difference information of the target data annotation events included in the target query data annotation phase within the query target time period. Each target query data annotation phase corresponds to a rectangular element. The length of the rectangle is proportional to the amount of the quantity information, and the more the quantity, the longer the corresponding rectangle. The filling point density of the rectangle can represent the time difference information, and the larger the time difference, the greater the density. For the target query data annotation phases B, C, and E, the corresponding quantity information is 100, 80, and 30 respectively, and the corresponding time difference information is avgB, avgC, and avgE respectively. Assume avgB < avgC < avgE. Refer to Figure 3 , which is a schematic diagram of the visualization graph generated based on the quantity information and time difference information of the target data annotation events included in phases B, C, and E within the query target time period.
[0195] See Figure 4 , which is a schematic flowchart of a data processing method provided by an embodiment of the present application. The method includes:
[0196] Step 401: Obtain the information of the completed data annotation phases to be stored for at least one data annotation event.
[0197] Among them, the information of the completed data annotation phases includes the identification information and completion time of the completed data annotation phases.
[0198] In an embodiment of the present application, after each data annotation phase of a data annotation event is completed, the relevant information of the completed data annotation phase needs to be stored in a record information table. At this time, after each data annotation phase of a data annotation event is completed, the information of its completed data annotation phase can be stored as the information of the completed data annotation phases to be stored in a storage device, such as stored in a cache. At this time, the data processing device can obtain the information of the completed data annotation phases of the data annotation event in the storage device so as to store it in the record information table of the corresponding data annotation event.
[0199] Step 402: Update the record information table of the data annotation event based on the information of the completed data annotation phases to be stored for at least one data annotation event.
[0200] In an embodiment of the present application, after obtaining the information of the completed data annotation phases to be stored for at least one data annotation event, the data processing device can store the information of the completed data annotation phases to be stored for at least one data annotation event in the record information table of the corresponding data annotation event to update the record information table, so that the record information table of the data annotation event records the identification information and time information of the completed data annotation phases of the created data annotation event.
[0201] As a possible implementation method, since the information of the completed data labeling stage includes the identification information and completion time of the completed data labeling stage, when storing the information of the completed data labeling stage in the record information table, the completion time of the completed data labeling stage can be directly used as the time information of the data labeling stage and directly stored in the record information table.
[0202] As a possible implementation method, each data labeling event has identification information, and the identification information and time information of the completed data labeling stage of each data labeling event are respectively stored in the record information table of the data labeling event that records the identification information of the data labeling event. The completed data labeling stage information to be stored of the data labeling event also includes the identification information of the data labeling event. The data processing device can determine whether there is a record information table of the data labeling event that records the identification information of the data labeling event in the database through the identification information of the data labeling event. If it exists, the completed data labeling stage information to be stored of the data labeling event is directly stored in the record information table of the data labeling event that records the identification information; if it does not exist, a record information table of the data labeling event that records the identification information is constructed, and then the completed data labeling stage information to be stored of the data labeling event is stored in the record information table of the data labeling event that records the identification information.
[0203] As a possible implementation method, the completion time of the completed data annotation stage of the data annotation event is stored in the record information table of the data annotation event in the form of a doubly linked list. A doubly linked list is a type of linked list, and each of its data nodes has two pointers, pointing to the direct successor and direct predecessor respectively. Therefore, starting from any node in the doubly linked list, its predecessor node and successor node can be easily accessed. Each data annotation event corresponds to a doubly linked list. The doubly linked list includes data nodes for recording the identification information and time information of each completed data annotation stage, and the data nodes are arranged according to the execution order of the data annotation stage in the data annotation process. In this way, it is very convenient to find the completion time of the completed data annotation stage of the data annotation event.
[0204] As a possible implementation manner, the time information further includes time difference information between two adjacent completed data annotation stages. Based on the completed data annotation stage information to be stored for at least one data annotation event, updating the record information table of the data annotation event includes:
[0205] For each of the completed data annotation stage information to be stored for at least one data annotation event, determining in the record information table of the data annotation event whether the data annotation event already exists;
[0206] If so, the completion time of the fifth target data annotation stage of the data annotation event is determined in the record information table of the data annotation event; wherein the fifth target data annotation stage of the data annotation event is the previous data annotation stage of the completed data annotation stage to be stored of the data annotation event;
[0207] Based on the completion time in the completed data labeling stage information to be stored of the data labeling event and the completion time information of the fifth target data labeling stage of the data labeling event, determine the time difference information between the completed data labeling stage to be stored of the data labeling event and the fifth target data labeling stage;
[0208] The time difference information between the completed data marking stage to be stored in the data marking event and the fifth target data marking stage, and the completed data marking stage information to be stored in at least one data marking event are stored in the record information table of the data marking event.
[0209] If not, the completed data annotation stage information to be stored of at least one data annotation event is stored in the record information table of the data annotation event.
[0210] In the embodiment, for the completed data annotation stage information to be stored for a data annotation event, it is determined whether the data annotation event exists in the record information table of the data annotation event. For example, it can be determined whether the data annotation event exists in the record information table of the data annotation event based on the identification information of the data annotation event.
[0211] If it exists, it means that the data labeling event has the identification information and time information of the completed data labeling stage in the record information table. During the data labeling process, each time a data labeling stage is completed, a completed data labeling stage information is generated and stored in the storage device. Therefore, the completed data labeling stage information is stored according to the execution order in the data labeling process. The record information table of the data labeling event records the identification information and time information of all completed data labeling stages before the completed data labeling stage to be stored. Therefore, the completion time of the previous data labeling stage of the completed data labeling stage to be stored of the data labeling event can be determined in the record information table, that is, the completion time of the fifth target data labeling stage can be determined in the record information table.
[0212] The difference between the completion time in the completed data annotation stage information to be stored in the data annotation event and the completion time of the fifth target data annotation stage of the data annotation event can be calculated to determine the time difference information between the completed data annotation stage to be stored in the data annotation event and the fifth target data annotation stage. The completion time in the completed data annotation stage information to be stored in the data annotation event and the time difference information between the completed data annotation stage to be stored in the data annotation event and the fifth target data annotation stage are stored in the record information table of the data annotation event to update the record information table.
[0213] Step 403: Obtain attribute condition information;
[0214] In the embodiment of the present application, data annotation events can be screened based on attribute condition information, wherein the attribute condition information includes data annotation event production line, data annotation event type, data annotation event label, etc.
[0215] In some embodiments, the user may set the attribute condition information according to actual needs. In this case, the data processing device may obtain the attribute condition information set by the user.
[0216] Step 404: Obtain the query time and the target creation time of the data annotation event.
[0217] For details, please refer to step S101 and no further details will be given here.
[0218] Step 405: Obtain a record information table of data annotation events, and determine a target data annotation event in the record information table of data annotation events based on the target creation time of the data annotation event.
[0219] For details, please refer to step S102 and no further details will be given here.
[0220] Step 406: Obtain a query list, and determine the target query data labeling stage based on the query list.
[0221] For details, please refer to step S103 and no further details will be given here.
[0222] Step 407: In the record information table of data annotation events, target information of target data annotation events satisfying attribute condition information contained in the target query data annotation phase within the query target time period is determined.
[0223] In an embodiment of the present application, data annotation events can be screened based on attribute condition information, and accordingly, target information of target data annotation events that meet the attribute condition information contained in each target query data annotation stage within the query target time period can be determined.
[0224] As a possible implementation method, the target data annotation events that satisfy the attribute condition information may be determined first, and then the target information of the target data annotation events that satisfy the attribute condition information contained in each data annotation stage of the data annotation process within the target query time period may be determined. Alternatively, the target information of the target data annotation events contained in each data annotation stage of the data annotation process within the target query time period may be determined first, and then the target information of the target data annotation events that satisfy the attribute condition information contained in each data annotation stage of the data annotation process within the target query time period may be determined. Among them, in the record information table of the data annotation events, the number of target data annotation events contained in each data annotation stage of the data annotation process within the target query time period may be determined with reference to step S104, which will not be described in detail here.
[0225] As a possible implementation manner, the attribute condition information of the data annotation event is stored in the record information table of the data annotation event.
[0226] refer to Figure 5 , is a schematic diagram of the structure of a vehicle data acquisition device provided in an embodiment of the present application. Figure 5 As shown, the device comprises:
[0227] The acquisition unit 501 is used to acquire the query time and the target creation time of the data annotation event.
[0228] Processing unit 502 is used to obtain the record information table of data annotation events, and determine the target data annotation event in the record information table of data annotation events based on the target creation time of the data annotation events; the record information table of data annotation events is used to record the identification information and time information of the completed data annotation stage of the created data annotation event; the target data annotation event is a data annotation event whose creation time recorded in the record information table of the data annotation event is consistent with the target creation time.
[0229] The processing unit 502 is further configured to obtain a query list and determine a target query data annotation stage based on the query list; the query list records identification information of at least one data annotation stage of target information to be counted.
[0230] The processing unit 502 is also used to determine the query target time period based on the query time and the target creation time, and to determine the target information of the target data annotation event contained in the query target time period during the target query data annotation phase in the record information table of the data annotation event.
[0231] As a possible implementation manner, the target information includes quantity information.
[0232] The processing unit 502 is specifically configured to determine, in the record information table of data annotation events, quantity information of target data annotation events included in the target query data annotation phase within the query target time period.
[0233] As a possible implementation manner, the target information includes time difference information.
[0234] The processing unit 502 is specifically configured to determine, in the record information table of the data annotation event, the time difference information corresponding to the query target time period in the target query data annotation phase.
[0235] As a possible implementation method, processing unit 502 is specifically used to obtain data labeling process information; the data labeling process information includes identification information of each data labeling stage included in the data labeling process, and the identification information of each data labeling stage is arranged according to the execution order of each data labeling stage in the data labeling process. For the identification information of each target query data marking stage in the query list, based on the data marking process information, determine whether there is identification information of the first target data marking stage corresponding to the identification information of the target query data marking stage in the query list; if not, determine the identification information of the second target data marking stage corresponding to the identification information of the target query data marking stage according to the query list and the data marking process information, and determine the identification information of the fourth data marking stage corresponding to the identification information of the target query data marking stage according to the data marking process information, and store the identification information of the target query data marking stage, the identification information of the fourth target data marking stage and the identification information of the second target data marking stage as a first query subset in the first set; if it exists, store the identification information of the target query data marking stage and the identification information of the first target data marking stage as a second query subset in the second set; wherein, the identification information of the first target data marking stage is the identification information of the target query data marking stage in the data marking process information. The identification information of the second target data annotation stage is the identification information of the third target data annotation stage in the query list when the identification information of the third target data annotation stage exists in the identification information of at least one data annotation stage included in the query list; the identification information of the third target data annotation stage is the identification information of the previous data annotation stage of the identification information of the target query data annotation stage when the identification information of at least one data annotation stage included in the query list is arranged in the execution order of the data annotation process; or, the identification information of the second target data annotation stage is the identification information of the first data annotation stage in the data annotation process information when the identification information of the third target data annotation stage does not exist in the identification information of at least one data annotation stage included in the query list; the identification information of the fourth target data annotation stage is the identification information of the data annotation stage in the data annotation process information between the identification information of the target query data annotation stage and the identification information of the second target data annotation stage. According to the record information table of the data annotation event, the first set and the second set, the time difference information corresponding to each target query data annotation stage in the query target time period is determined.
[0236] As a possible implementation, the processing unit 502 is specifically used to determine the identification information of the first data annotation stage and the identification information of the second data annotation stage in the query list; the identification information of the first data annotation stage is the identification information of the first data annotation stage when the identification information of at least one data annotation stage contained in the query list is arranged in the execution order of the data annotation process; the identification information of the second data annotation stage is the identification information of the last data annotation stage when the identification information of at least one data annotation stage contained in the query list is arranged in the execution order of the data annotation process; detect whether the identification information of the first data annotation stage is the identification information of the first data annotation stage in the data annotation process information; if not, determine the identification information of each data annotation stage before the first data annotation stage in the data annotation process information, store the identification information of each data annotation stage before the first data annotation stage in the data annotation process and the identification information of the first data annotation stage as a first query subset in the first set, and execute the time difference information determination step. Among them, the time difference information determination step includes:
[0237] The first data annotation stage is determined as the current stage.
[0238] Check whether the current stage is the second data labeling stage.
[0239] If not, the identification information of the next data annotation stage of the current stage is determined as the identification information of the target stage in the data annotation process information, and the value of the jump parameter is updated to the first preset value.
[0240] Check whether the query list contains the identification information of the target stage.
[0241] If the query list does not contain the identification information of the target stage, the identification information of the current stage is recorded in the first information, the identification information of the current stage is updated to the identification information of the target stage, and the step is re-executed to determine the identification information of the next data annotation stage of the current stage as the identification information of the target stage in the data annotation process information, and the value of the jump parameter is updated to the first preset value, and then the step of detecting whether the query list contains the identification information of the target stage is performed, until it is detected that the query list contains the identification information of the target stage.
[0242] Alternatively, if the query list contains identification information of the target stage, it is detected whether the value of the jump parameter is a second preset value.
[0243] If it is the second preset value, the identification information of the current stage and the identification information of the target stage are stored in the second set as a second query subset, the identification information of the current stage is updated to the identification information of the target stage, the value of the jump parameter is updated to the preset initial value, and the steps are re-executed to detect whether the current stage is the second data labeling stage until it is detected that the current stage is the second data labeling stage.
[0244] Alternatively, if it is not the second preset value, the identification information of the current stage, the identification information of the target stage and the identification information of the data labeling stage recorded in the first information are stored in the first set as a first query subset, the identification information of the current stage is updated to the identification information of the target stage, the value of the jump parameter is updated to the preset initial value, and the steps are re-executed to detect whether the current stage is the second data labeling stage until it is detected that the current stage is the second data labeling stage.
[0245] If the current stage is the second data marking stage, the time difference information corresponding to the query target time period of each target query data marking stage is determined according to the record information table of the data marking event, the first set and the second set.
[0246] As a possible implementation method, the processing unit 502 is specifically used to store the identification information of the first data labeling stage as a first query subset in the first set if the identification information of the first data labeling stage is the identification information of the first data labeling stage in the data labeling process information, and execute the time difference information determination step.
[0247] As a possible implementation method, the processing unit 502 is specifically used to determine, for each first query subset included in the first set, in the record information table of the data annotation event, the time difference between each two adjacent data annotation stages in all the data annotation stages included in the first query subset within the query target time period when the first set is a non-empty set; based on the time difference between each two adjacent data annotation stages in all the data annotation stages included in the first query subset within the query target time period, determine at least one of the maximum time difference, the minimum time difference and the average time difference corresponding to the target query data annotation stage in the first query subset.
[0248] As a possible implementation method, the processing unit 502 is specifically used to determine, for each second query subset included in the second set, in the record information table of the data labeling event, the time difference between two data labeling stages included in the second query subset within the query target time period when the second set is a non-empty set; based on the time difference between the two data labeling stages included in the second query subset within the query target time period, determine at least one of the maximum time difference, the minimum time difference and the average time difference corresponding to the target query data labeling stage in the second query subset.
[0249] As a possible implementation manner, the time information includes completion time information.
[0250] Processing unit 502 is specifically used to obtain completed data annotation stage information to be stored for at least one data annotation event; wherein the completed data annotation stage information includes identification information and completion time of the completed data annotation stage; based on the completed data annotation stage information to be stored for at least one data annotation event, update the record information table of the data annotation event.
[0251] As a possible implementation manner, the time information also includes time difference information between two adjacent completed data annotation stages.
[0252] Processing unit 502 is specifically used to determine whether a data marking event already exists in the record information table of the data marking event for each of the completed data marking stage information to be stored of at least one data marking event; if so, determine the completion time of the fifth target data marking stage of the data marking event in the record information table of the data marking event; the fifth target data marking stage of the data marking event is the previous data marking stage of the completed data marking stage to be stored of the data marking event; based on the completion time in the completed data marking stage information to be stored of the data marking event and the completion time information of the fifth target data marking stage of the data marking event, determine the time difference information between the completed data marking stage to be stored of the data marking event and the fifth target data marking stage; store the time difference information between the completed data marking stage to be stored of the data marking event and the fifth target data marking stage, and the completed data marking stage information of at least one data marking event to be stored in the record information table of the data marking event.
[0253] As a possible implementation manner, the processing unit 502 is specifically configured to store the completed data annotation stage information of at least one data annotation event to be stored in the record information table of the data annotation event if no.
[0254] As a possible implementation manner, the acquisition unit 501 is specifically configured to acquire attribute condition information.
[0255] The processing unit 502 is specifically configured to determine, in the record information table of data annotation events, target information of target data annotation events that meet the attribute condition information and are contained in the query target time period in each target query data annotation phase.
[0256] Corresponding to the above embodiments, the present application also provides an electronic device. Figure 6A schematic diagram of the structure of an electronic device provided in an embodiment of the present invention, the electronic device 600 may include: a processor 601, a memory 602 and a communication unit 603. These components communicate through one or more buses, and those skilled in the art will understand that the structure of the electronic device shown in the figure does not constitute a limitation on the embodiment of the present invention, and it may be a bus structure or a star structure, and may also include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.
[0257] The communication unit 603 is used to establish a communication channel so that the electronic device can communicate with other devices, receive user data sent by other devices or send user data to other devices.
[0258] The processor 601 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire electronic device. It runs or executes software programs and / or modules stored in the memory 602, and calls data stored in the memory to perform various functions of the electronic device and / or process data. The processor can be composed of an integrated circuit (IC), for example, it can be composed of a single packaged IC, or it can be composed of multiple packaged ICs with the same or different functions. For example, the processor 601 can include only a central processing unit (CPU). In an embodiment of the present invention, the CPU can be a single computing core or multiple computing cores.
[0259] The memory 602 is used to store the execution instructions of the processor 601. The memory 602 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.
[0260] When the execution instructions in the memory 602 are executed by the processor 601, the electronic device 600 can execute Figure 1 or Figure 4 Some or all of the steps in the illustrated embodiments.
[0261] In a specific implementation, the present invention further provides a computer storage medium, wherein the computer storage medium may store a program, and when the program is executed, the program may include some or all of the steps in each embodiment of the data processing method provided by the present invention. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.
[0262] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution in the embodiments of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention or some parts of the embodiments.
[0263] In this specification, the same or similar parts between the various embodiments can be referred to each other. In particular, for the device embodiment and the terminal embodiment, since they are basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description in the method embodiment.
Claims
1. A data processing method, characterized in that: include: Get the query time and the target creation time of the data annotation event; Obtaining a record information table of data annotation events, and determining a target data annotation event in the record information table of data annotation events based on a target creation time of the data annotation event; the record information table of data annotation events is used to record identification information and time information of a completed data annotation stage of a created data annotation event; the target data annotation event is a data annotation event whose creation time recorded in the record information table of data annotation events is consistent with the target creation time; Obtaining a query list, and determining a target query data annotation stage based on the query list; The query list records identification information of at least one data annotation stage of target information to be counted; The query target time period is determined based on the query time and the target creation time, and in the record information table of the data annotation event, the target information of the target data annotation event included in the query target time period during the target query data annotation phase is determined.
2. The method according to claim 1, characterized in that: The target information includes quantity information; In the record information table of the data annotation event, determining the target information of the target data annotation event contained in the target query data annotation phase within the query target time period includes: In the record information table of the data annotation events, the quantity information of the target data annotation events included in the query target time period during the target query data annotation phase is determined.
3. The method according to claim 1, characterized in that: The target information includes time difference information; In the record information table of the data annotation event, determining the target information of the target data annotation event contained in the target query data annotation phase within the query target time period includes: In the record information table of the data annotation event, the time difference information corresponding to the target query data annotation phase of the target data annotation event within the query target time period is determined.
4. The method according to claim 3, characterized in that In the record information table of the data annotation event, determining the time difference information corresponding to the target query data annotation phase of the target data annotation event within the query target time period includes: Acquire data annotation process information; the data annotation process information includes identification information of each data annotation stage included in the data annotation process, and the identification information of each data annotation stage is arranged according to the execution order of each data annotation stage in the data annotation process; For the identification information of each target query data marking stage in the query list, based on the data marking process information, determine whether there is identification information of the first target data marking stage corresponding to the identification information of the target query data marking stage in the query list; if not, determine the identification information of the second target data marking stage corresponding to the identification information of the target query data marking stage according to the query list and the data marking process information, and determine the identification information of the fourth data marking stage corresponding to the identification information of the target query data marking stage according to the data marking process information, and store the identification information of the target query data marking stage, the identification information of the fourth target data marking stage and the identification information of the second target data marking stage as a first query subset in the first set; if it exists, store the identification information of the target query data marking stage and the identification information of the first target data marking stage as a second query subset in the second set; wherein, the identification information of the first target data marking stage is the identification information of the target query data marking stage in the data marking process information. the identification information of the previous data annotation stage of the identification information of the query list; the identification information of the second target data annotation stage is the identification information of the third target data annotation stage of the query list when the identification information of the third target data annotation stage exists in the identification information of at least one data annotation stage contained in the query list; the identification information of the third target data annotation stage is the identification information of the previous data annotation stage of the identification information of the target query data annotation stage when the identification information of at least one data annotation stage contained in the query list is arranged in the execution order of the data annotation process; or, the identification information of the second target data annotation stage is the identification information of the first data annotation stage in the data annotation process information when the identification information of the third target data annotation stage does not exist in the identification information of at least one data annotation stage contained in the query list; the identification information of the fourth target data annotation stage is the identification information of the data annotation stage between the identification information of the target query data annotation stage and the identification information of the second target data annotation stage in the data annotation process information; According to the record information table of the data annotation event, the first set and the second set, the time difference information corresponding to the query target time period in each target query data annotation stage is determined.
5. The method according to claim 4, characterized in that For the identification information of each target query data marking stage in the query list, based on the data marking process information, determine whether there is identification information of the first target data marking stage corresponding to the identification information of the target query data marking stage in the query list; if not, determine the identification information of the second target data marking stage corresponding to the identification information of the target query data marking stage according to the query list and the data marking process information, and determine the identification information of the fourth data marking stage corresponding to the identification information of the target query data marking stage according to the data marking process information, and store the identification information of the target query data marking stage, the identification information of the fourth target data marking stage and the identification information of the second target data marking stage as a first query subset in the first set; If so, storing the identification information of the target query data annotation stage and the identification information of the first target data annotation stage as a second query subset in the second set includes: Determine identification information of a first data annotation stage and identification information of a second data annotation stage in the query list; the identification information of the first data annotation stage is identification information of the first data annotation stage when identification information of at least one data annotation stage included in the query list is arranged in the order of execution of the data annotation process; the identification information of the second data annotation stage is identification information of the last data annotation stage when identification information of at least one data annotation stage included in the query list is arranged in the order of execution of the data annotation process; Detecting whether the identification information of the first data annotation stage is the identification information of the first data annotation stage in the data annotation process information; If not, then determine the identification information of each data annotation stage before the first data annotation stage in the data annotation process information, store the identification information of each data annotation stage before the first data annotation stage in the data annotation process and the identification information of the first data annotation stage as a first query subset in the first set, and perform the time difference information determination step; wherein the time difference information determination step includes: Determining the first data annotation stage as the current stage; Detecting whether the current stage is the second data labeling stage; If not, determining the identification information of the next data annotation stage of the current stage as the identification information of the target stage in the data annotation process information, and updating the value of the jump parameter to a first preset value; Detecting whether the query list contains identification information of the target stage; If the query list does not contain the identification information of the target stage, the identification information of the current stage is recorded in the first information, the identification information of the current stage is updated to the identification information of the target stage, and the step of determining the identification information of the next data annotation stage of the current stage as the identification information of the target stage in the data annotation process information is re-executed, and the value of the jump parameter is updated to the first preset value, until the step of detecting whether the query list contains the identification information of the target stage is detected, until the query list contains the identification information of the target stage; or, If the query list contains the identification information of the target stage, detecting whether the value of the jump parameter is a second preset value; If it is the second preset value, the identification information of the current stage and the identification information of the target stage are stored in the second set as a second query subset, the identification information of the current stage is updated to the identification information of the target stage, the value of the jump parameter is updated to the preset initial value, and the step of detecting whether the current stage is the second data labeling stage is re-executed until it is detected that the current stage is the second data labeling stage; or, If it is not the second preset value, the identification information of the current stage, the identification information of the target stage and the identification information of the data annotation stage recorded in the first information are stored in the first set as a first query subset, the identification information of the current stage is updated to the identification information of the target stage, the value of the jump parameter is updated to the preset initial value, and the step of detecting whether the current stage is the second data annotation stage is re-executed until it is detected that the current stage is the second data annotation stage; The method of determining the time difference information corresponding to the query target time period in each target query data annotation phase according to the record information table of the data annotation event, the first set and the second set includes: If the current stage is the second data labeling stage, the time difference information corresponding to the query target time period of each target query data labeling stage is determined according to the record information table of the data labeling event, the first set and the second set.
6. The method according to claim 5, characterized in that Also includes: If the identification information of the first data labeling stage is the identification information of the first data labeling stage in the data labeling process information, the identification information of the first data labeling stage is stored in the first set as a first query subset, and the time difference information determination step is performed.
7. The method according to any one of claims 4 to 6, characterized in that: The step of determining the time difference information corresponding to each data annotation stage in the query list according to the record information table of the data annotation event, the first set, and the second set includes: When the first set is a non-empty set, for each first query subset included in the first set, determining, in the record information table of the data annotation event, a time difference between every two adjacent data annotation stages in all data annotation stages included in the first query subset within the query target time period; Based on the time difference between every two adjacent data annotation stages in all data annotation stages included in the first query subset within the query target time period, at least one of the maximum time difference, the minimum time difference and the average time difference corresponding to the target query data annotation stage in the first query subset is determined.
8. The method according to any one of claims 4 to 6, characterized in that: The step of determining the time difference information corresponding to each data annotation stage in the query list according to the record information table of the data annotation event, the first set, and the second set includes: When the second set is a non-empty set, for each second query subset included in the second set, determining, in the record information table of the data annotation event, a time difference between two data annotation stages included in the second query subset within the query target time period; Based on the time difference between two data annotation stages included in the second query subset within the query target time period, at least one of the maximum time difference, the minimum time difference and the average time difference corresponding to the target query data annotation stage in the second query subset is determined.
9. The method according to claim 1, characterized in that: The time information includes completion time information, and the method further includes: Obtaining completed data annotation stage information to be stored for at least one data annotation event; wherein the completed data annotation stage information includes identification information and completion time of the completed data annotation stage; Based on the completed data annotation stage information to be stored for the at least one data annotation event, the record information table of the data annotation event is updated.
10. The method according to claim 9, characterized in that The time information also includes time difference information between two adjacent completed data annotation stages; The updating of the record information table of the data labeling event based on the completed data labeling stage information to be stored of the at least one data labeling event comprises: For each of the completed data annotation stage information to be stored for the at least one data annotation event, determining in the record information table of the data annotation event whether the data annotation event already exists; If so, then the completion time of the fifth target data annotation stage of the data annotation event is determined in the record information table of the data annotation event; the fifth target data annotation stage of the data annotation event is the previous data annotation stage of the completed data annotation stage to be stored of the data annotation event; Based on the completion time in the completed data labeling stage information to be stored of the data labeling event and the completion time information of the fifth target data labeling stage of the data labeling event, determine the time difference information between the completed data labeling stage to be stored of the data labeling event and the fifth target data labeling stage; The time difference information between the completed data marking stage to be stored of the data marking event and the fifth target data marking stage and the completed data marking stage information to be stored of the at least one data marking event are stored in the record information table of the data marking event.
11. The method according to claim 10, characterized in that Also includes: If not, the completed data marking stage information to be stored of the at least one data marking event is stored in the record information table of the data marking event.
12. The method according to claim 1, characterized in that Also includes: Get attribute condition information; In the record information table of the data annotation event, determining the target information of the target data annotation event contained in the query target time period in each target query data annotation stage includes: In the record information table of the data annotation events, target information of the target data annotation events satisfying the attribute condition information contained in the query target time period in each target query data annotation phase is determined.
13. A data processing device, characterized in that: include: An acquisition unit, used to acquire the query time and the target creation time of the data annotation event; A processing unit, used to obtain a record information table of data annotation events, and determine a target data annotation event in the record information table of data annotation events based on a target creation time of the data annotation event; the record information table of data annotation events is used to record identification information and time information of a completed data annotation stage of a created data annotation event; the target data annotation event is a data annotation event whose creation time recorded in the record information table of data annotation events is consistent with the target creation time; The processing unit is further used to obtain a query list and determine the target query data annotation stage based on the query list; the query list records identification information of at least one data annotation stage of the target information to be counted; The processing unit is also used to determine the query target time period based on the query time and the target creation time, and to determine, in the record information table of the data annotation event, the target information of the target data annotation event contained in the query target time period during the target query data annotation phase.
14. An electronic device, characterized in that: The device comprises a memory for storing computer program instructions and a processor for executing the program instructions, wherein when the computer program instructions are executed by the processor, the electronic device is triggered to execute the method according to any one of claims 1 to 12.
15. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 12.
