Sample data processing method, device, electronic device and medium

Abstract

The embodiments of the present disclosure provide a sample data processing method, a sample data processing device, an electronic device, and a computer-readable medium; and relate to the field of big data processing technology. The sample data processing method comprises: step S1: obtaining a screening model by training a training data set containing annotations; step S2: obtaining sample data to be processed and performing initial annotations on the sample data to be processed; step S3: outputting a processing result of the sample data to be processed through the screening model, wherein the processing result includes a confidence level, and extracting usable samples from the samples to be processed based on the processing result, wherein the confidence level output by the screening model for the usable samples is less than a first threshold; step S4: performing a secondary annotation on the usable samples using the initial annotations, and expanding the training data set with the usable samples containing the secondary annotations. The technical solution of the embodiments of the present disclosure can improve the efficiency and accuracy of sample annotation.

Description

Technical Field

[0001] The present disclosure relates to the technical field of big data processing, and in particular to a sample data processing method, a sample data processing device, an electronic device, and a computer-readable medium. Background Art

[0002] In recent years, machine learning has rapidly developed and has become a core technology in various fields, such as image recognition, natural language processing, and autonomous driving. Machine learning requires sample data, which must then be labeled to establish a mapping between input and output and generate a model. Therefore, labeling sample data is crucial.

[0003] The labeling of sample data currently relies mainly on manual work. Although manual labeling has high accuracy, its efficiency is very low and the cost is high. Therefore, improving the efficiency of sample labeling has become a research focus.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute prior art known to ordinary technicians in the field. Summary of the Invention

[0005] The purpose of the embodiments of the present disclosure is to provide a sample data processing method, a sample data processing device, an electronic device and a computer-readable medium, which can automatically perform initial labeling on sample data, screen out high-quality usable samples through initial labeling, and then perform secondary labeling to obtain labeled samples with higher accuracy, thereby improving the labeling efficiency of sample data and the accuracy of labeling.

[0006] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by practice of the present disclosure.

[0007] According to a first aspect of an embodiment of the present disclosure, a sample data processing method is provided, comprising: step S1: obtaining a screening model by training a training data set containing annotations; step S2: obtaining sample data to be processed, and performing an initial annotation on the sample data to be processed; step S3: outputting a processing result of the sample data to be processed through the screening model, wherein the processing result includes a confidence level, and extracting available samples from the samples to be processed based on the processing result, wherein the confidence level output by the screening model for the available samples is less than a first threshold value; step S4: performing a secondary annotation on the available samples using the initial annotation, and expanding the training data set with the available samples containing the secondary annotations.

[0008] In an exemplary embodiment of the present disclosure, the method further includes: step S5: updating the screening model through the expanded training data set and updating the sample data to be processed; step S6: looping through the updated screening model and the updated sample data to be processed.

[0009] In an exemplary embodiment of the present disclosure, when the evaluation index of the screening model meets a preset condition, the execution of the loop is stopped.

[0010] In an exemplary embodiment of the present disclosure, the initial labeling includes labeling the position of the target object in the sample data to be processed. When the processing result includes the position of the target object in the sample data to be processed, extracting the available samples from the samples to be processed based on the processing result includes: extracting the sample data to be processed as the available samples whose deviation between the position of the processing result output by the screening model and the position of the initial labeling is greater than a second threshold.

[0011] In an exemplary embodiment of the present disclosure, the initial labeling includes labeling the outline of the target object in the sample data to be processed. When the processing result includes the outline of the target object in the sample data to be processed, extracting the available samples from the samples to be processed based on the processing result includes: extracting the sample data to be processed whose deviation between the outline of the processing result output by the screening model and the initially labeled outline is greater than a third threshold as the available sample.

[0012] In an exemplary embodiment of the present disclosure, the processing result includes the category of the target object in the sample data to be processed; and the secondary labeling of the available samples and the expansion of the training data set with the available samples containing the secondary labeling include:

[0013] A template corresponding to the target object is obtained according to the category of the target object; the available samples are secondary labeled to obtain a secondary labeling result; a similarity evaluation index between the template and the secondary labeling result is calculated, and the available samples whose similarity evaluation index is greater than a fourth threshold are used to expand the training data set.

[0014] In an exemplary embodiment of the present disclosure, the initial labeling includes labeling the category of the target object in the sample data to be processed, and extracting the available samples from the samples to be processed based on the processing results includes: when the category of the target object in the processing results is inconsistent with the category of the initially labeled target object, using the corresponding sample to be processed as the available sample.

[0015] According to a second aspect of an embodiment of the present disclosure, a sample data processing device is provided, which may include a model training module, a sample labeling module, a data screening module, and a sample data determination module.

[0016] Among them, the model training module is used to train a screening model using a training data set containing annotations; the sample annotation module is used to obtain the sample data to be processed and perform initial annotation on the sample data to be processed; the data screening module is used to output the processing results of the sample data to be processed through the screening model, wherein the processing results include confidence, and the available samples in the samples to be processed are extracted according to the processing results, wherein the confidence output of the screening model for the available samples is less than a first threshold; the sample data determination module is used to perform secondary annotation on the available samples using the initial annotation, and expand the training data set with the available samples containing the secondary annotations.

[0017] In an exemplary embodiment of the present disclosure, the sample data processing apparatus further includes a data updating module and a circulation module.

[0018] Among them, the data updating module is used to update the screening model through the expanded training data set and update the sample data to be processed.

[0019] A loop module is used to loop the model training module, sample labeling module, data screening module, sample data determination module and data update module through the updated screening model and the updated sample data to be processed.

[0020] In an exemplary embodiment of the present disclosure, the loop module may be configured to stop executing the loop when the evaluation index of the screening model meets a preset condition.

[0021] In an exemplary embodiment of the present disclosure, the processing result further includes the position or outline of the target object in the sample data to be processed.

[0022] In an exemplary embodiment of the present disclosure, the initial labeling includes labeling the position of the target object in the sample data to be processed. When the processing result includes the position of the target object in the sample data to be processed, the data screening module is used to: extract the sample data to be processed whose deviation between the position of the processing result output by the screening model and the position of the initial labeling is greater than a second threshold as the available sample.

[0023] In an exemplary embodiment of the present disclosure, the initial labeling includes labeling the outline of the target object in the sample data to be processed. When the processing result includes the outline of the target object in the sample data to be processed, the data screening module is used to: extract the sample data to be processed whose deviation between the outline of the processing result output by the screening model and the outline of the initial labeling is greater than a third threshold as the available sample.

[0024] In an exemplary embodiment of the present disclosure, the processing result includes the category of the target object in the sample data to be processed; the sample data determination module can be used to: obtain a template corresponding to the target object according to the category of the target object; perform secondary annotation on the available samples to obtain secondary annotation results; calculate a similarity evaluation index between the template and the secondary annotation results, and expand the training data set with available samples whose similarity evaluation index is greater than a fourth threshold.

[0025] In an exemplary embodiment of the present disclosure, the initial labeling includes labeling the category of the target object in the sample data to be processed, and the data screening module is used to: when the category of the target object in the processing result is inconsistent with the category of the initially labeled target object, the corresponding sample to be processed is used as the available sample.

[0026] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: one or more processors; a storage device for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the sample data processing method described in the first aspect of the above embodiment.

[0027] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable medium is provided, on which a computer program is stored. When the program is executed by a processor, the sample data processing method as described in the first aspect of the above embodiment is implemented.

[0028] The sample data processing method, sample data processing device, electronic device and computer-readable medium provided in the embodiments of the present disclosure initially mark the sample data to be processed, use a screening model to select available samples that meet the requirements from the sample data to be processed, and then perform a secondary marking on the available samples to expand the training data set, thereby obtaining samples containing labels. This not only eliminates the need for manual labeling of samples, thereby improving the sample labeling efficiency, but also allows samples to be screened to obtain high-quality sample data that meets the requirements, thereby improving the training efficiency and accuracy of the machine learning model.

[0029] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The accompanying drawings are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification, are used to explain the principles of the present disclosure. Obviously, the drawings described below are only some embodiments of the present disclosure, and those skilled in the art can derive other drawings based on these drawings without inventive effort. In the drawings:

[0031] Figure 1 A schematic diagram schematically illustrates an exemplary system architecture of a sample data processing method or a sample data processing device applied to an embodiment of the present disclosure;

[0032] Figure 2 The flowchart of the sample data processing method according to an embodiment of the present disclosure is schematically shown;

[0033] Figure 3 Schematically shows a flow chart of a sample data processing method according to another embodiment of the present disclosure;

[0034] Figure 4 A block diagram schematically shows a sample data processing apparatus according to an embodiment of the present disclosure;

[0035] Figure 5 A schematic diagram of the structure of a computer system suitable for implementing an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0036] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete and will fully convey the concepts of the example embodiments to those skilled in the art.

[0037] In addition, the described features, structures or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, well-known methods, devices, implementations or operations are not shown or described in detail to avoid blurring various aspects of the present disclosure.

[0038] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically separate entities. That is, these functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0039] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, while others may be combined or partially combined. Therefore, the actual execution order may vary depending on the actual situation.

[0040] In this specification, the terms "a", "an", "the", "said" and "at least one" are used to indicate the presence of one or more elements / components / etc.; the terms "comprising", "including" and "having" are used to express open-ended inclusion, and mean that additional elements / components / etc. may exist in addition to the listed elements / components / etc.; the terms "first", "second", "third", etc. are used only as labels and are not intended to limit the quantity of their objects.

[0041] Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.

[0042] Figure 1 A schematic diagram showing a system architecture of an exemplary application environment that can be applied to the sample data processing method or sample data processing apparatus according to the embodiments of the present disclosure.

[0043] like Figure 1 As shown, system architecture 100 may include one or more terminal devices 101, 102, 103, a network 104, and a server 105. Network 104 is a medium for providing a communication link between terminal devices 101, 102, 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0044] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Terminal devices 101, 102, and 103 can be various electronic devices with display screens and support web browsing, including but not limited to desktop computers, portable computers, smartphones and tablet computers, wearable devices, virtual reality devices, smart homes, etc.

[0045] The server 105 may be a server that provides various services, such as a background management server that provides support for devices operated by users using the terminal devices 101, 102, and 103. The background management server may analyze and process received requests and other data, and feed back the processing results to the terminal device.

[0046] For example, the server 105 can use a training data set containing annotations to train a screening model; and obtain sample data to be processed, and perform initial annotations on the sample data to be processed; the server 105 can also output the processing results of the samples to be processed through the screening model, screen the samples to be processed according to the processing results, and obtain available samples from the samples to be processed; and use the initial annotations to perform secondary annotations on the available samples, and expand the available samples containing the secondary annotations into the training data set.

[0047] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as needed. For example, the server 105 may be a server cluster consisting of multiple servers.

[0048] The sample data processing method provided in the embodiments of the present disclosure is generally executed by the server 105, and accordingly, the sample data processing device is generally disposed in the server 105. However, it will be readily understood by those skilled in the art that the sample data processing method provided in the embodiments of the present disclosure may also be executed by the terminal devices 101, 102, and 103, and accordingly, the sample data processing device may also be disposed in the terminal devices 101, 102, and 103, and this is not particularly limited in the present exemplary embodiment.

[0049] Based on this, the embodiment of the present disclosure provides a technical solution for a sample data processing method, which can automatically label sample data and use a screening model to filter the labels to obtain usable samples that meet the requirements. By performing a secondary labeling on the usable samples, the accuracy of the labeling can be improved, thereby improving the efficiency and accuracy of model training.

[0050] like Figure 2 As shown, the sample data processing method provided by the embodiment of the present disclosure may include the following steps:

[0051] Step S1: Use the labeled training data set to train and obtain a screening model.

[0052] Step S2: Obtain sample data to be processed, and perform initial labeling on the sample data to be processed.

[0053] Step S3: Outputting the processing results of the sample data to be processed through the screening model, wherein the processing results include confidence, and extracting available samples from the samples to be processed based on the processing results, wherein the confidence output by the screening model for the available samples is less than a first threshold.

[0054] Step S4: perform secondary annotation on the available samples, and expand the training data set with the available samples containing the secondary annotations.

[0055] The specific implementation of each step in this embodiment is described in detail below.

[0056] In step S1, a screening model is obtained by training using a training dataset containing annotations.

[0057] A training dataset can contain data of various categories, such as images, text, and audio. For example, images can be included in a training dataset, ranging from hundreds of thousands to millions, and even tens of millions of images. Each image can contain a target object and its annotations. Target objects can include road signs, such as traffic lights and directional arrows drawn on the road; they can also include buildings, vehicles, and roads. Annotations can include, for example, the target object's category, its location in the image, or its outline, all of which are not specifically limited in this embodiment.

[0058] A screening model can be obtained by using a machine learning algorithm using a training data set containing annotations. The screening model can be used to recognize images, or can be used to recognize the position of a specific target in an image, or can be used to recognize the outline of a specific target in an image; depending on the actual training objectives, the screening model can also be used for other recognition tasks, for example, the screening model can be used to recognize text in an image, etc. The method of training the screening model can use clustering, random forest, convolutional neural network, etc., but this embodiment is not limited to this. For example, an image containing road signs is collected as a training data set, where the annotation can be the category of the road sign, and the clustering model is trained using the training data set. After the training is completed, the screening model can be obtained.

[0059] In step S2, the sample data to be processed is obtained and the sample data to be processed is initially labeled.

[0060] Sample data to be processed refers to data that does not contain annotations, such as images of unannotated road signs. For example, initial annotation can first obtain map features corresponding to the target object from a high-precision map. These map features are then used to reverse-identify the target object in the sample data to be processed, thereby annotating the target object and completing the automatic initial annotation. After annotation, the labeled sample data to be processed is obtained.

[0061] Next, in step S3, the processing results of the sample data to be processed are outputted through the screening model, and then available samples are extracted from the sample data to be processed according to the processing results.

[0062] In this embodiment, the sample data to be processed is used as the input of the screening model, and the sample to be processed is identified by the screening model to obtain the processing result of the sample data to be processed. The processing result includes the confidence level of the output of the sample to be processed. The confidence level can refer to the probability that the screening model correctly identifies the sample data to be processed. The higher the confidence level, the more reliable the processing result of the screening model for the sample data to be processed. After obtaining the processing result of the sample data to be processed, the confidence level contained in the processing result can be used to screen the sample to be processed, and the data with a confidence level less than a first threshold value in the sample data to be processed is extracted as a usable sample. For example, if the processing result output by the screening model is the category of the target object in the sample data to be processed, then the confidence level in the processing result is the probability that the target object in the sample data to be processed is a specific category, and the higher the confidence level, the more accurate the processing result. For the sample data to be processed, if the confidence level of the screening model in its output is higher, the mapping relationship learned by the screening model is more accurate, and the optimization value of the sample data to be processed for the screening model is less; if the confidence level of the screening model in the sample data to be processed is lower, there may be a large difference between the sample data to be processed and the data in the training data set of the screening model. The sample data to be processed has a higher optimization value for the screening model and can be used as an available sample. The first threshold value can be 0.5, 0.4, 0.3, or 0.2, 0.35, etc., and this embodiment is not limited thereto.

[0063] In an exemplary embodiment, the processing results output by the screening model may also include the location or outline of the target object in the sample data to be processed. Accordingly, the initial annotation of the sample data to be processed may include: annotating the location of the target object in the sample data to be processed; or annotating the outline of the target object in the sample data to be processed.

[0064] When the initial annotation is to mark the position of the target object in the sample data to be processed, the processing result output by the screening model may include the position of the target object. When the processing result includes the position of the target object, the method of extracting the available samples from the sample to be processed may include: extracting the sample data to be processed whose deviation between the position of the target object in the processing result and the position in the initial annotation of the sample to be processed is greater than a second threshold, as available samples. If the position in the processing result identified by the screening model for the sample data to be processed deviates greatly from the position of the initial annotation of the sample data to be processed, the sample data to be processed may be used as data with optimization value for the screening model, and therefore the sample data to be processed whose position in the processing result deviates greatly from the initial annotation may be used as available samples. Among them, the second threshold may include 0.5, 0.4, etc., and may also include other values, such as 0.6, 0.55, 0.45, etc., which is not limited in this embodiment.

[0065] When the initial annotation is to mark the outline of the target object in the sample data to be processed, the processing result may include the outline output by the screening model for the target object in the sample data to be processed. When the processing result includes the outline of the target object, the method of extracting the available samples in the sample to be processed may include: extracting the sample data whose deviation between the outline in the processing result of the sample data to be processed and the outline of the target object in the initial annotation is greater than a third threshold as the available samples. Among them, the third threshold may include 0.4, 0.3, etc., and may also be determined according to actual needs, such as 0.45, 0.55, etc. It should be understood that the second threshold and the third threshold may be the same, for example, both are 0.5; or they may be different, for example, the second threshold value is 0.55, the third threshold value is 0.5, etc.; this embodiment does not limit this.

[0066] Continue to refer Figure 2 ,In step S4, the available samples are annotated again using the initial annotation, and the available samples including the secondary annotation are expanded to the training data set.

[0067] The available samples are sample data with a lower confidence level output by the screening model, which can be understood as the data missing from the training data set of the screening model. Therefore, the available samples are annotated again and added to the training data set, which can improve the training data set. Thus, a model with higher accuracy can be obtained by using the expanded and improved training data set.

[0068] When performing secondary annotation on available samples, the initial annotation can be used to optimize the initial annotation to improve the accuracy of the annotation. For example, the initial annotation takes the outline of the target object as an example. The outline of the target object marked in the initial annotation is used as the initial region, and the level set segmentation model is used to perform curve evolution to optimize the initial annotation outline to obtain the secondary annotation. When the annotation is a position, the position in the initial annotation can be used as the basis, and level set evolution can be performed to obtain the precise outline of the target object in the available sample, thereby calculating the position of the target object in the available sample.

[0069] After optimizing the initial annotations to obtain the secondary annotations, the available samples containing the secondary annotations can be added to the training dataset to obtain more comprehensive sample data.

[0070] In an exemplary embodiment, after the available samples are secondary labeled, the available samples can be screened again, and eligible available samples can be extracted and expanded into the training dataset. Since the initial labeling can label the category of the target object, based on this, secondary labeling of the available samples to expand the training dataset can include the following methods: first, a secondary labeling result can be obtained; and based on the category of the target object in the initial labeling, a template corresponding to the category can be obtained; then, a similarity evaluation index between the template and the secondary labeling result can be calculated, and available samples with a similarity evaluation index greater than a fourth threshold can be extracted and expanded into the training dataset.

[0071] The template can be pre-set for each category of the target object. For example, if the target object category is a car, an image of a car can be pre-acquired as a template. For another example, if the target object category is a truck, an image of a truck can be saved as a template, thereby obtaining a template corresponding to each category. A similarity evaluation index between the template and the secondary annotation result can be calculated using a similarity calculation algorithm such as the cosine similarity algorithm and the Euclidean distance. In addition, other algorithms can be used to calculate the similarity evaluation index, such as the Mahalanobis distance, etc., but the present embodiment is not limited thereto.

[0072] In an exemplary embodiment, after the available samples are annotated twice to expand the training data set, step S5 and step S6 may also be included. Figure 3 As shown. Among them:

[0073] Step S5: Update the screening model using the expanded training data set and update the samples to be processed.

[0074] Step S6: cyclically execute steps S1 to S5 using the updated screening model and the updated samples to be processed.

[0075] The screening model is retrained using the expanded training dataset. Since the training dataset includes newly annotated sample data, using the updated training dataset not only improves model training efficiency but also the accuracy of the trained model, whether for the screening model or other machine learning models. Simultaneously, a new batch of data can be acquired as sample data to be processed, thereby updating the sample data; or available samples can be deleted from the sample data to be processed, thereby updating the sample data. Steps S1 to S5 are then cyclically executed using the updated sample data to be processed and the updated screening model, thereby continuously updating the screening model and continuously selecting available samples from the updated sample data to be processed and adding them to the training dataset. This makes the data in the training dataset increasingly comprehensive, thereby improving the effectiveness of the training dataset. During the machine learning process, the training dataset can be used to train various different models, such as image recognition models, text recognition models, road recognition models, and so on, depending on actual needs. Since the training dataset contains sample data with relatively high quality after secondary annotation, it provides a good sample foundation for various machine learning models, thereby shortening the training cycle and improving training efficiency and model accuracy.

[0076] After each update of the screening model, the evaluation index of the screening model can be calculated, and the evaluation index can be used to determine whether to stop the loop. For example, the accuracy of the screening model can be calculated and the quality of the screening model can be evaluated by the accuracy. If the accuracy exceeds a certain threshold, it can be determined that the screening model meets the conditions, and the loop can be exited to obtain and save the final training data set. If the screening model meets the preset conditions, it can be understood that the data in the training data set meets the labeling requirements, thereby obtaining data with the required labeling accuracy, saving the time cost of manual labeling and improving the accuracy of automatic labeling.

[0077] Furthermore, this embodiment also provides a sample data processing device that can be used to execute the sample data processing method disclosed above. Figure 4 The sample data processing device 40 provided in the embodiment of the present disclosure may include: a model training module 41, a sample labeling module 42, a data screening module 43 and a sample data determination module 44.

[0078] The model training module 41 is used to train a screening model using a training data set containing annotations.

[0079] The sample labeling module 42 is used to obtain the sample data to be processed and perform initial labeling on the sample data to be processed.

[0080] The data screening module 43 is used to output the processing results of the sample data to be processed through the screening model, wherein the processing results include confidence levels, and extract available samples from the samples to be processed based on the processing results, wherein the confidence levels output by the screening model for the available samples are less than a first threshold value.

[0081] The sample data determination module 44 is configured to perform secondary annotation on the available samples using the initial annotation, and expand the training data set with the available samples including the secondary annotation.

[0082] In an exemplary embodiment of the present disclosure, the sample data processing apparatus further includes a data updating module 45 and a circulation module 46 .

[0083] The data updating module 45 is used to update the screening model using the expanded training data set and to update the sample data to be processed.

[0084] The loop module 46 is used to loop the model training module 41 to the data updating module 45 through the updated screening model and the updated sample data to be processed.

[0085] In an exemplary embodiment of the present disclosure, the loop module 46 may be configured to stop executing the loop when the evaluation index of the screening model meets a preset condition.

[0086] In an exemplary embodiment of the present disclosure, the processing result further includes the position or outline of the target object in the sample data to be processed.

[0087] In an exemplary embodiment of the present disclosure, the initial labeling includes labeling the position of the target object in the sample data to be processed. When the processing result includes the position of the target object in the sample data to be processed, the data screening module 43 is used to: extract the sample data to be processed whose deviation between the position of the processing result output by the screening model and the position of the initial labeling is greater than a second threshold as the available sample.

[0088] In an exemplary embodiment of the present disclosure, the initial labeling includes labeling the outline of the target object in the sample data to be processed. When the processing result includes the outline of the target object in the sample data to be processed, the data screening module 43 is used to: extract the sample data to be processed whose deviation between the outline of the processing result output by the screening model and the outline of the initial labeling is greater than a third threshold as the available sample.

[0089] In an exemplary embodiment of the present disclosure, the processing result includes the category of the target object in the sample data to be processed; the sample data determination module 44 can be used to: obtain a template corresponding to the target object according to the category of the target object; perform secondary annotation on the available samples to obtain secondary annotation results; calculate the similarity evaluation index between the template and the secondary annotation results, and expand the training data set with available samples whose similarity evaluation index is greater than a fourth threshold.

[0090] In an exemplary embodiment of the present disclosure, the initial labeling includes labeling the category of the target object in the sample data to be processed, and the data screening module is used to: when the category of the target object in the processing result is inconsistent with the category of the initially labeled target object, the corresponding sample to be processed is used as the available sample.

[0091] Since the functional modules of the sample data processing apparatus of the exemplary embodiment of the present disclosure correspond to the steps of the exemplary embodiment of the sample data processing method described above, for details not disclosed in the apparatus embodiment of the present disclosure, please refer to the exemplary embodiment of the sample data processing method described above.

[0092] Reference below Figure 5 , which shows a schematic structural diagram of a computer system 500 suitable for implementing an electronic device of an embodiment of the present disclosure. Figure 5 The computer system 500 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0093] like Figure 5 As shown, the computer system 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 502 or the program loaded from the storage unit 508 to the random access memory (RAM) 503. Various programs and data required for system operation are also stored in the RAM 503. The CPU 501, ROM 502, and RAM 503 are connected to each other via a bus 804. An input / output (I / O) interface 505 is also connected to the bus 504.

[0094] The following components are connected to the I / O interface 505: an input section 506 including a keyboard, a mouse, and the like; an output section 507 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 508 including a hard disk; and a communication section 509 including a network interface card such as a LAN card or a modem. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is installed in the drive 510 as needed, so that computer programs read therefrom can be installed into the storage section 508 as needed.

[0095] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 509, and / or installed from a removable medium 511. When the computer program is executed by the central processing unit (CPU) 501, the above-mentioned functions defined in the system of the present application are executed.

[0096] It should be noted that the computer-readable medium described in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wireline, optical fiber cable, RF, or any suitable combination thereof.

[0097] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0098] The units involved in the embodiments described in this disclosure may be implemented in software or hardware, and the units described may also be provided in a processor. In some cases, the names of these units do not constitute limitations on the units themselves.

[0099] As another aspect, the present application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiments; or may exist independently without being incorporated into the electronic device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by an electronic device, the electronic device implements the method for determining the effectiveness of epidemic prevention and control as described in the above embodiments.

[0100] For example, the electronic device can implement Figure 2 As shown in: Step S1, a screening model is obtained by training a training data set containing annotations; Step S2, sample data to be processed is obtained, and the sample data to be processed is initially annotated; Step S3, a processing result of the sample data to be processed is output through the screening model, wherein the processing result includes a confidence level, and available samples from the samples to be processed are extracted according to the processing result, wherein the confidence level output by the screening model for the available samples is less than a first threshold value; Step S4, the available samples are annotated again using the initial annotations, and the training data set is expanded with the available samples containing the secondary annotations.

[0101] For example, the electronic device can achieve the following Figure 3 The steps shown.

[0102] It should be noted that although several modules or units of the device for action execution are mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be concretized in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be concretized.

[0103] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.

[0104] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0105] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A sample data processing method, characterized in that: include: Step S1: using the training data set containing annotations to train a screening model; Step S2: obtaining sample data to be processed, and performing initial labeling on the sample data to be processed; the initial labeling includes labeling the category of the target object in the sample data to be processed; Step S3: Outputting the processing result of the sample data to be processed through the screening model, wherein the processing result includes a confidence level and a category of the target object in the sample data to be processed. When the category of the target object in the processing result is inconsistent with the category of the initially labeled target object, or the confidence level is less than a first threshold, the corresponding sample to be processed is treated as an available sample; Step S4: performing secondary annotation on the available samples using the primary annotation, obtaining a template corresponding to the target object according to the category of the target object in the primary annotation; performing secondary annotation on the available samples to obtain a secondary annotation result; calculating a similarity evaluation index between the template and the secondary annotation result, and expanding the training data set with available samples having the similarity evaluation index greater than a fourth threshold; Wherein, when the initial annotation is the outline of the target object, the secondary annotation of the available samples using the initial annotation includes: performing curve evolution on the outline of the target object using a level set segmentation model to obtain the secondary annotation; When the initial annotation is the position of the target object, the secondary annotation of the available samples is performed using the initial annotation, including: performing level set evolution on the position of the target object to obtain the precise outline of the target object, and calculating the position of the target object in the available samples to obtain the secondary annotation.

2. The method according to claim 1, characterized in that The method further comprises: Step S5: updating the screening model using the expanded training data set, and updating the sample data to be processed; Step S6: cyclically executing steps S3-S5 using the updated screening model and the updated sample data to be processed.

3. The method according to claim 2, characterized in that In step S6, when the evaluation index of the screening model meets the preset conditions, the execution loop is stopped.

4. The method according to claim 1, wherein The processing result in step S3 also includes the position or outline of the target object in the sample data to be processed.

5. The method according to claim 4, characterized in that The initial marking includes marking the position of the target object in the sample data to be processed. When the processing result includes the position of the target object in the sample data to be processed, extracting available samples from the samples to be processed according to the processing result includes: Extract the sample data to be processed whose deviation between the position of the processing result output by the screening model and the position of the initial annotation is greater than a second threshold as the available sample.

6. The method according to claim 4, characterized in that The initial labeling includes labeling the outline of the target object in the sample data to be processed. When the processing result includes the outline of the target object in the sample data to be processed, extracting the available samples from the samples to be processed according to the processing result includes: Extract the sample data to be processed whose deviation between the contour of the processing result output by the screening model and the contour of the initial annotation is greater than a third threshold as the available sample.

7. A sample data processing device, characterized in that: include: A model training module is used to train a screening model using a training data set containing annotations; A sample labeling module is used to obtain sample data to be processed and perform initial labeling on the sample data to be processed; the initial labeling includes labeling the category of the target object in the sample data to be processed; a data screening module, configured to output a processing result of the sample data to be processed through the screening model, wherein the processing result includes a confidence level and a category of a target object in the sample data to be processed, and when the category of the target object in the processing result is inconsistent with the category of the initially labeled target object, or when the confidence level is less than a first threshold, the corresponding sample to be processed is treated as an available sample; A sample data determination module is used to perform secondary annotation on the available samples using the initial annotation, obtain a template corresponding to the target object according to the category of the target object in the initial annotation; perform secondary annotation on the available samples to obtain a secondary annotation result; calculate a similarity evaluation index between the template and the secondary annotation result, and expand the training data set with available samples whose similarity evaluation index is greater than a fourth threshold; wherein, when the initial annotation is the outline of the target object, the secondary annotation of the available samples using the initial annotation includes: performing curve evolution on the outline of the target object using a level set segmentation model to obtain a secondary annotation; when the initial annotation is the position of the target object, the secondary annotation of the available samples using the initial annotation includes: performing level set evolution on the position of the target object to obtain the precise outline of the target object, and calculating the position of the target object in the available samples to obtain a secondary annotation.

8. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the sample data processing method according to any one of claims 1 to 7.

9. A computer-readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the sample data processing method according to any one of claims 1 to 6 is implemented.

Images