Image labeling method, image processing method, device and computer equipment

Abstract

The present disclosure provides an image labeling method, an image processing method, an apparatus and a computer device, wherein the method comprises: obtaining a to-be-labeled image; pre-labeling the to-be-labeled image by a pre-trained target network to obtain a labeling result; the target network can complete a specific task processing; performing at least one level of checking operation on the to-be-labeled image and the labeling result, and in the case that the checking operation is passed, determining a training sample according to the to-be-labeled image and the labeling result. The present disclosure performs pre-labeling processing on the to-be-labeled image by the target network, and performs at least one level of checking operation on the labeling result, which can save a large amount of labeling cost, speed up the data labeling speed, and ensure the data labeling quality.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of data processing, and in particular, to an image labeling method, an image processing method, an apparatus, and a computer device. BACKGROUND

[0002] In recent years, with the development of automatic driving and assisted driving technologies, and the continuous optimization of deep learning solutions, the precision requirements for automatic driving or assisted driving products are also increasingly high. In order to improve the precision of automatic driving or assisted driving products, a large amount of labeled data is needed for model training. However, the cost of data labeling is very high; in addition, since the labeling resources (e.g., the number of labeling personnel) are limited, for example, for some urgent tasks, the labeling task needs to be completed by a deadline, but in the case of limited number of labeling personnel, it is impossible to guarantee both the completion time and the completion quality of data labeling.

[0003] As can be seen from the above description, the data labeling speed is an important factor restricting product iteration, and the data labeling quality is another important factor determining whether the product has competitiveness in the market. According to labeling experience, the labeling level of labeling personnel is uneven, and there are many ambiguous or unquantifiable data labeling standards in the labeling documents used to guide the labeling personnel to label, which will result in that the labeling personnel cannot fully meet the expected requirements when labeling. Therefore, how to guarantee the data labeling quality is also an important link in the product generation process. SUMMARY

[0004] The embodiments of the present disclosure at least provide an image labeling method, an image processing method, an apparatus, and a computer device.

[0005] In a first aspect, the embodiments of the present disclosure provide an image labeling method, comprising: obtaining a to-be-labeled image; pre-labeling the to-be-labeled image by a pre-trained target network to obtain a labeling result; the target network can complete a specific task processing; performing at least one level of verification operation on the to-be-labeled image and the labeling result thereof, and in the case that the verification operation passes, determining a training sample according to the to-be-labeled image and the labeling result thereof.

[0006] As can be seen from the above description, in the embodiments of the present disclosure, the to-be-labeled image is pre-labeled by the target network, which can realize automatic labeling of the to-be-labeled image, thereby saving the workload of labeling personnel, accelerating the speed of data labeling, and through at least one level of verification operation on the labeling result, semi-supervised labeling can be realized, thereby guaranteeing the accuracy of the labeling result. Through the above processing mode, a large amount of labeling cost can be saved, the speed of data labeling can be accelerated, and the quality of data labeling can be guaranteed.

[0007] In an optional implementation, the obtaining the to-be-labeled image for the image labeling task comprises: obtaining a plurality of original images; and extracting a plurality of target original images from the plurality of original images according to a preset frame extraction frequency; and performing a deduplication processing on the plurality of extracted target original images to obtain the to-be-labeled image.

[0008] As can be seen from the above description, by means of video frame extraction processing on a plurality of original images, the number of labeled data can be reduced, and the speed of data labeling can be further improved. By means of deduplication processing on the plurality of extracted target original images, images with high repetition can be removed, thereby removing a large number of redundant images. By means of labeling the images after deduplication processing and training the corresponding network model based on the labeled images, the training accuracy of the network model can be improved.

[0009] In an optional implementation, the deduplication processing on the plurality of extracted target original images to obtain the to-be-labeled image comprises: in a case where it is determined that the similarity values between adjacent target original images in the plurality of target original images are greater than a preset threshold, discarding one frame of image in the adjacent target original images; in a case where it is determined that the similarity values between adjacent target original images in the plurality of target original images after discarding the images are greater than the preset threshold, repeatedly performing the step of discarding one frame of image in the adjacent target original images in a case where it is determined that the similarity values between adjacent target original images in the plurality of target original images are greater than the preset threshold, until the similarity values between adjacent target original images in the plurality of target original images after discarding the images are less than or equal to the preset threshold, and determining the plurality of target original images after discarding the images as the to-be-labeled image.

[0010] As can be seen from the above description, by means of deduplication processing on the plurality of target original images based on the similarity values, repeated images can be automatically removed from a large number of images, thereby removing a large number of redundant images. By means of labeling the images after deduplication processing and training the corresponding network model based on the labeled images, the training accuracy of the network model can be improved.

[0011] In an optional implementation, before the at least one level of verification operation is performed on the to-be-labeled image and the labeling result thereof, the method comprises: converting the labeling result of the to-be-labeled image into a specific format file.

[0012] In the embodiments of the present disclosure, by converting the labeling result of the to-be-labeled image into a specific format file, at least one level of manual verification can be performed on the labeling result automatically predicted by the target network, thereby realizing labeling of the to-be-labeled image by means of semi-supervised labeling to obtain a labeling result.

[0013] In an alternative implementation, the at least one level of verification operation on the to-be-labeled image and the labeling result thereof comprises: screening, according to data screening requirements, to-be-verified labeling results used for performing the verification operation and target to-be-labeled images corresponding to the to-be-verified labeling results from the labeling result; updating target labeling results that do not meet preset labeling requirements and target to-be-labeled images corresponding to the target labeling results in the to-be-verified labeling results; and performing at least one level of verification operation on all to-be-verified labeling results containing the updated target labeling results and target to-be-labeled images corresponding to the to-be-verified labeling results.

[0014] In an alternative implementation, the method further comprises: in a case where the to-be-verified labeling result and the target to-be-labeled image corresponding thereto meet the preset labeling requirements, extracting, according to a preset extraction ratio, part of the to-be-verified labeling results to perform the verification operation; and in a case where the part of the to-be-verified labeling results pass the verification, taking the target to-be-labeled image and the to-be-verified labeling result corresponding thereto as a training sample.

[0015] As described above, in a case where the to-be-verified labeling result and the labeling result thereof do not meet the preset labeling requirements, the to-be-verified labeling result can be updated, and at least one level of verification operation can be performed on the to-be-verified labeling result after the update; in a case where the to-be-verified labeling result meets the preset labeling requirements, the labeling result can be subjected to spot check. Through the above processing mode, the data labeling process can be refined, and multi-level verification of data labeling can be implemented, so as to ensure the labeling quality of the to-be-labeled image.

[0016] In an alternative implementation, the updating of the target labeling result that does not meet the preset labeling requirements and the target to-be-labeled image corresponding thereto in the to-be-verified labeling result and the performing of at least one level of verification operation on all to-be-verified labeling results containing the target labeling result after the update and target to-be-labeled images corresponding to the to-be-verified labeling results comprises: obtaining an updated labeling result of the target labeling result; performing a verification operation of a plurality of verification levels on all to-be-verified labeling results containing the updated labeling result and target to-be-labeled images corresponding to the to-be-verified labeling results, to obtain a verification operation result of each verification level; and in a case where the verification operation result of each verification level is verification pass, taking the to-be-labeled image and the labeling result thereof as a training sample.

[0017] In an alternative implementation, the method further comprises: for each check level, returning re-check information to a specified check level before the check level to the specified check level in case that the check operation result of any check level is a check failure, so as to make the specified check level re-execute the check operation until the check operation result of the check operation of each check level is a check pass, wherein the re-check information is used to indicate a label result of a check failure.

[0018] As can be seen from the above description, in the embodiments of the present disclosure, the label quality of the label result can be improved through the above layer-by-layer quality inspection, so that the labeled data with higher quality guarantee is obtained, and the entire data labeling process can be controlled layer by layer, and the responsibility can be refined to each labeling module of the data labeling system, so that the sense of responsibility of each labeling module is enhanced, the data labeling rework is reduced, and the labeling cost is greatly reduced.

[0019] In an alternative implementation, the method further comprises: after determining the training sample according to the to-be-labeled image and the label result thereof, retraining the target network to obtain the target network after retraining.

[0020] As can be seen from the above description, through repeated training of the target network, a target network with higher prediction accuracy can be obtained, so that the accuracy and quality of the pre-labeling processing can be improved when the pre-labeling processing is performed through the target network.

[0021] In a second aspect, the embodiments of the present disclosure further provide an image processing method, comprising: acquiring a to-be-processed image collected by a target vehicle in a driving process; performing a specific processing task on the to-be-processed image through an image processing network to obtain an image processing result, wherein the image processing network is a network obtained after a training sample determined through any one of the methods in the first aspect is trained; and controlling a driving state of the target vehicle based on the image processing result.

[0022] In a third aspect, the embodiments of the present disclosure provide an image labeling apparatus, comprising: a first acquisition unit configured to acquire a to-be-labeled image; a pre-labeling processing unit configured to perform pre-labeling on the to-be-labeled image through a target network pre-trained, to obtain a label result; the target network can complete a specific task processing; and a check unit configured to perform at least one level of check operation on the to-be-labeled image and the label result thereof, and determine a training sample according to the to-be-labeled image and the label result thereof in case that the check operation passes.

[0023] In a fourth aspect, the embodiments of the present disclosure provide an image processing apparatus, comprising: a second acquisition unit configured to acquire a to-be-processed image collected by a target vehicle during driving; an image processing unit configured to perform a specific processing task on the to-be-processed image by using an image processing network to obtain an image processing result, wherein the image processing network is obtained after the training sample determined by the method in the first aspect is trained; and a control unit configured to control a driving state of the target vehicle based on the image processing result.

[0024] In a fifth aspect, the embodiments of the present disclosure further provide a computer device, comprising: a processor, a memory and a bus, the memory stores machine readable instructions executable by the processor, when the computer device is running, the processor and the memory communicate through the bus, and the machine readable instructions are executed by the processor to perform the steps of any one of the first aspect to the second aspect.

[0025] In a sixth aspect, the embodiments of the present disclosure further provide a computer readable storage medium, the computer readable storage medium stores a computer program, and the computer program is executed by the processor to perform the steps of any one of the first aspect to the second aspect.

[0026] In order to make the above objectives, features and advantages of the present disclosure more apparent, clear and easy to understand, the following will specifically describe the preferred embodiments with reference to the accompanying drawings, and the specific description is as follows. BRIEF DESCRIPTION OF DRAWINGS

[0027] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following will briefly introduce the drawings needed to be used in the embodiments, the drawings herein are incorporated into the description and form a part of the description, the drawings show the embodiments consistent with the present disclosure, and are used to illustrate the technical solutions of the present disclosure together with the description. It should be understood that the following drawings only show some embodiments of the present disclosure, and therefore should not be considered as a limitation to the scope, and for those skilled in the art, other related drawings can also be obtained without creative labor.

[0028] Figure 1 A flowchart of an image labeling method provided by the embodiments of the present disclosure is shown;

[0029] Figure 2 A flowchart of a specific method for obtaining a to-be-labeled image for an image labeling task in the image labeling method provided by the embodiments of the present disclosure is shown;

[0030] Figure 3 A flowchart of a specific method for performing at least one level of verification operation on the to-be-labeled image and the labeling result thereof in the image labeling method provided by the embodiments of the present disclosure is shown;

[0031] Figure 4 A flow chart of another image labeling method provided by the embodiments of the present disclosure is shown;

[0032] Figure 5 A flow chart of a data training method provided by the embodiments of the present disclosure is shown;

[0033] Figure 6 A flow chart of an image processing method provided by the embodiments of the present disclosure is shown;

[0034] Figure 7 A schematic diagram of an image labeling device provided by the embodiments of the present disclosure is shown;

[0035] Figure 8 A schematic diagram of an image processing device provided by the embodiments of the present disclosure is shown;

[0036] Figure 9 A schematic diagram of a first optional computer device provided by the embodiments of the present disclosure is shown;

[0037] Figure 10 A schematic diagram of a second optional computer device provided by the embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0038] In order to make the objects, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be described clearly and completely below with reference to the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments of the present disclosure. The components of the embodiments of the present disclosure described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the claimed present disclosure, but only represents selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present disclosure.

[0039] It should be noted that: similar reference numerals and letters represent similar items in the following drawings, therefore, once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0040] The term "and / or", merely describes an associated relationship, which means that there can be three relationships, for example, A and / or B, which can represent: A exists alone, A and B exist together, and B exists alone. In addition, the term "at least one" herein means any one of a plurality or any combination of at least two of a plurality, for example, including at least one of A, B, and C, which means including any one or more elements selected from the set consisting of A, B, and C.

[0041] It is found through research that in the existing image labeling method, the to-be-labeled data is mainly labeled by manual labeling. However, when the amount of to-be-labeled data is large and the number of labeling personnel is limited, the image labeling task is difficult to complete as scheduled, and the accuracy of the labeled data cannot be guaranteed. In addition, due to the uneven labeling level of data labelers and the presence of many ambiguous or unquantifiable data labeling standards in the labeling documents, the labelers may not be able to fully meet the expected requirements during labeling. Therefore, how to ensure the data labeling quality is also an important part of the product generation process.

[0042] Based on the above research, the present disclosure provides an image labeling method, an image processing method, an apparatus, and a computer device. In the embodiment of the present disclosure, first, the to-be-labeled image for the image labeling task is obtained, then the to-be-labeled image is pre-labeled by a pre-trained target network matched with the image labeling task to obtain a labeling result, and finally, at least one level of verification operation is performed on the to-be-labeled image and its labeling result, and in the case that the verification operation is passed, the training sample is determined according to the to-be-labeled image and its labeling result. As can be seen from the above description, in the embodiment of the present disclosure, the to-be-labeled image is pre-labeled by the target network, which can realize automatic labeling of the to-be-labeled image, thereby saving the workload of the labeling personnel and speeding up the data labeling. Through at least one level of verification operation on the labeling result, semi-supervised labeling can be realized, thereby ensuring the accuracy of the labeling result. Through the above processing method, a large amount of labeling cost can be saved, the speed of data labeling can be accelerated, and the quality of data labeling can be ensured.

[0043] In order to facilitate the understanding of the present embodiment, first, a kind of image labeling method disclosed by the embodiment of the present disclosure will be introduced in detail, and the execution subject of the image labeling method provided by the embodiment of the present disclosure is generally a computer device with certain computing power, wherein a data labeling system is pre-installed in the computer device, and the method executed by the data labeling system is the image labeling method provided by the embodiment of the present disclosure.

[0044] Referring to Figure 1 The flowchart of the image labeling method provided by the embodiment of the present disclosure is shown in FIG. 1, and the method comprises steps S101-S105, wherein:

[0045] S101: Obtain an image to be labeled.

[0046] In the embodiments of the present disclosure, the image to be labeled is an image to be labeled for one or more image processing tasks, wherein the image processing task is associated with a network to be trained, for example, the image processing task is target detection, and the network to be trained is a target detection network, wherein the network to be trained is a model to be trained by a training sample in the following step S105.

[0047] S103: Pre-labeling the image to be labeled by using a pre-trained target network to obtain a labeling result, wherein the target network can complete a specific task processing.

[0048] In the embodiments of the present disclosure, for different image labeling tasks, a matching pre-trained target network is pre-set, wherein the target network is a large-scale network model pre-trained by labeled data. Since the large-scale network model has high precision, the labeling quality of the pre-labeling processing can be improved by selecting a large-scale network to pre-label the image to be labeled, thereby obtaining a more accurate labeling result.

[0049] S105: At least one level of verification operation is performed on the image to be labeled and the labeling result thereof, and in the case that the verification operation is passed, a training sample is determined according to the image to be labeled and the labeling result thereof.

[0050] After pre-labeling the image to be labeled to obtain a labeling result, at least one level of verification operation is performed on the labeling result. After at least one level of verification operation is performed on the labeling result, a verification result of each verification level operation is obtained. In the case that the verification result of each verification level operation is passed, a training sample is determined according to the image to be labeled and the labeling result thereof.

[0051] Through at least one level of verification operation, the labeling quality of the labeling result can be improved, the time of data labeling can be shortened, and the cost of data labeling can be saved.

[0052] As can be seen from the above description, in the embodiments of the present disclosure, the image to be labeled is pre-labeled by using a target network, which can realize automatic labeling of the image to be labeled, thereby saving the workload of the labeling personnel, accelerating the speed of data labeling, through at least one level of verification operation on the labeling result, semi-supervised labeling can be realized, thereby ensuring the accuracy of the labeling result, through the above processing method, a large amount of labeling cost can be saved, the speed of data labeling can be accelerated, and the quality of data labeling can be ensured.

[0053] As can be known from the above description, in the embodiment of the present disclosure, a plurality of original images are first acquired, wherein the plurality of original images can be a plurality of continuous image frames in a certain video sequence; then, the plurality of original images can be subjected to video frame extraction and deduplication processing to obtain the to-be-labeled images.

[0054] In an optional implementation, as shown in Figure 2 The video frame extraction and deduplication processing of the plurality of original images, i.e., the above step S101, acquiring the to-be-labeled images, includes the following process:

[0055] Step S201: acquiring a plurality of original images; and extracting a plurality of target original images from the plurality of original images according to a preset frame extraction frequency;

[0056] Step S202: performing deduplication processing on the extracted plurality of target original images to obtain the to-be-labeled images.

[0057] In the embodiment of the present disclosure, the data labeling manner adopted is single-frame labeling, i.e., each image frame is labeled in sequence. At this time, in order to reduce the number of labeled data and further improve the speed of data labeling, the plurality of original images need to be subjected to frame extraction processing. Specifically, the original images can be extracted as target original images from the plurality of original images according to a preset frame extraction frequency.

[0058] It should be noted that in the embodiment of the present disclosure, different preset frame extraction frequencies can be set according to different image labeling tasks. For example, if the image labeling task is a target detection task, the preset frame extraction frequency can be set to be relatively sparse, for example, the preset frame extraction frequency can be set to be between [3-5] seconds, for example, the preset frame extraction frequency can be set to be 3 seconds per frame, which means that an original image is extracted from the plurality of original images every 3 seconds. If the image labeling task is a lane line detection task, the preset frame extraction frequency can be set to be more dense, for example, the preset frame extraction frequency can be set to be between [1-3] seconds, for example, the preset frame extraction frequency can be set to be 1 second per frame, which means that an original image is extracted from the plurality of original images every 1 second. It should be noted that in the embodiment of the present disclosure, the size of the preset frame extraction frequency can be set according to actual needs of a user, which is not limited here.

[0059] After the plurality of original images are subjected to frame extraction processing, the plurality of target original images extracted can have some images with relatively large repetition degrees.

[0060] For example, for an autonomous vehicle, a sensor installed on the autonomous vehicle collects a corresponding video sequence, at this time, the multiple original images can be consecutive image frames in the video sequence. When the vehicle is waiting for a traffic light, or the vehicle is in a stopped state for a long time, the image content of some of the multiple original images is substantially the same, at this time, the multiple original images contain some images with high repetition.

[0061] At this time, it is necessary to perform deduplication processing on the extracted multiple target original images, and the deduplication processing obtains the to-be-labeled images. In the embodiment of the present disclosure, the multiple target original images can be deduplicated by similarity comparison.

[0062] As can be seen from the above description, by performing video frame extraction processing on the multiple original images, the number of labeled data can be reduced, and the speed of data labeling can be further improved. By performing deduplication processing on the extracted multiple target original images, images with high repetition can be removed, thereby removing a large number of redundant images. By labeling the images after deduplication processing and training the corresponding network model based on the labeled images, the training accuracy of the network model can be improved.

[0063] In the embodiment of the present disclosure, the multiple target original images are deduplicated to obtain the to-be-labeled images, and the specific process includes the following steps:

[0064] (1) If it is determined that the similarity between adjacent target original images in the multiple target original images is greater than a preset threshold, one frame of image in the adjacent target original images is discarded.

[0065] (2) If it is determined that the similarity between adjacent target original images in the multiple target original images after discarding images is greater than a preset threshold, the step of discarding one frame of image in the adjacent target original images is repeated until the similarity between adjacent target original images in the multiple target original images after discarding images is less than or equal to the preset threshold, and the multiple target original images after discarding images are finally obtained are determined as the to-be-labeled images.

[0066] In the embodiment of the present disclosure, the above steps can be described in detail as follows:

[0067] Stage one:

[0068] Firstly, a plurality of target original images to be processed in an initial state are acquired, wherein the plurality of target original images are denoted as an image set A1. The similarity values between adjacent target original images in the plurality of target original images A1 are determined by using a deduplication network to obtain a plurality of similarity values B1. In a case where the similarity values B1 between adjacent target original images are greater than a preset threshold, one frame of image in the adjacent target original images is discarded, for example, the second frame of image in the adjacent target original images can be discarded.

[0069] Stage two:

[0070] After performing the image discarding operation on all adjacent target original images in the image set A1, a plurality of target original images after discarding images are obtained, denoted as an image set A2. Then, the similarity values between adjacent target original images in the plurality of target original images A2 after discarding images are determined by using the deduplication network to obtain a plurality of similarity values B2. It is judged whether the plurality of similarity values B2 contain a similarity value greater than the preset threshold. If it is judged that the plurality of similarity values B2 contain a similarity value greater than the preset threshold, the similarity value C2 greater than the preset threshold is determined in the plurality of similarity values B2, and the adjacent target original images corresponding to the similarity value C2 in the image set A2 are determined, and then the image discarding operation is performed on the determined adjacent target original images, for example, the second frame of image in the adjacent target original images can be discarded. If it is judged that the plurality of similarity values B2 do not contain a similarity value greater than the preset threshold, the image set A2 is determined as the image to be labeled.

[0071] After performing the image discarding operation on all adjacent target original images in the image set A1, a plurality of target original images after discarding images are obtained, denoted as an image set A3. For the image set A3, the image set A3 can be processed by using the processing procedure of the image set A2 described in the above stage two until the image to be labeled is determined.

[0072] Here, the deduplication network can be a scene classification network, such as a VGG (Visual Geometry Group) network. It should be noted that in the embodiments of the present disclosure, in addition to the VGG network, other scene classification networks can also be selected, for example, other networks can be used instead of the VGG network, and the scene classification network is not limited here.

[0073] Before the deduplication network is used to perform the deduplication processing on the plurality of target original images, the deduplication network can also be trained by using an ImageNet data set, and after the training is completed, the image features between adjacent target original images are extracted by using the trained deduplication network, and then the similarity values between the image features of adjacent target original images are calculated.

[0074] In the embodiments of the present disclosure, the preset threshold value can be any value between 0.6 and 0.9, for example, the preset threshold value can be set to 0.6.

[0075] At this time, if the similarity value exceeds a certain preset threshold T = 0.6, it is considered that the similarity between adjacent target original images is relatively high, at this time, the first frame target original image in the adjacent target original images is retained, and the second frame target original image is discarded, and so on. The multiple target original images extracted by the frame are de-duplicated, and the multiple target original images after de-duplication are determined as the to-be-labeled images.

[0076] In an optional implementation, the process of calculating the similarity value between adjacent original images in the multiple target original images can be described as follows:

[0077] Suppose the number of multiple target original images is N. First, the similarity value between the first target original image and the second target original image is calculated, and in the case that the similarity value is greater than the preset threshold value, the second target original image is discarded. And continue to calculate the similarity value between the first target original image and the third target original image, if the similarity value is less than or equal to the preset threshold value, continue to calculate the similarity value between the third target original image and the fourth target original image, and so on, until N target original images are calculated and de-duplicated, and the N target original images after de-duplication are determined as the to-be-labeled images.

[0078] In an optional implementation, the de-duplication process can be described as follows:

[0079] Suppose the number of multiple target original images is N. First, the similarity value between any two adjacent target original images in the N target original images is calculated, for example, the similarity value between the first target original image and the second target original image is calculated, the similarity value between the third target original image and the fourth target original image is calculated, and so on. The similarity value between the N-1th target original image and the Nth target original image is calculated. After calculating the corresponding similarity value, the similarity value can be compared with the preset threshold value, if it is greater than the preset threshold value, the last target original image is retained, otherwise, the two target original images are retained.

[0080] For the target original images after the first de-duplication processing, the de-duplication processing is performed again according to the method described above, until the similarity value between any two adjacent target original images after de-duplication processing is less than or equal to the preset threshold value, and the target original images after de-duplication processing are determined as the to-be-labeled images.

[0081] It can be known from the above description that the similarity value is used to remove the duplicate images from a large number of images, so as to remove a large number of redundant images. The image after the deduplication processing is labeled, and the image after the labeling is used to train the corresponding network model, so as to improve the training accuracy of the network model.

[0082] In the embodiment of the present disclosure, after the to-be-labeled image is obtained by the above-described method, the to-be-labeled image is pre-labeled by the pre-trained target network matched with the image labeling task to obtain a labeling result. Then, at least one level of verification operation is performed on the to-be-labeled image and the labeling result.

[0083] It can be known from the above description that the similarity between the prediction result of the large-scale network model and the artificial labeling result is relatively high. Therefore, in the embodiment of the present disclosure, the target network used is a large-scale network model.

[0084] Another reason for selecting a large-scale network model is that the to-be-labeled image is generally used to train a small-scale network model. Compared with a large-scale network model, the structure and calculation process of a small-scale network model are simpler. Because the prediction accuracy of a small-scale network model is also lower than that of a large-scale network model. Therefore, the labeling result obtained by pre-labeled processing of the to-be-labeled image by the large-scale network model not only has relatively high similarity with the artificial labeling result, but also has an accuracy that can meet the training accuracy requirement of the small-scale network model. Therefore, after the to-be-labeled image is pre-processed by the large-scale network model, not only an accurate labeling result can be obtained, but also manpower can be saved, the time for data labeling can be shortened, and the process of data labeling can be accelerated.

[0085] In an optional embodiment, before the at least one level of verification operation is performed on the to-be-labeled image and the labeling result, the step S105 includes the following steps:

[0086] The labeling result of the to-be-labeled image is converted into a specific format file.

[0087] In the embodiment of the present disclosure, after the to-be-labeled image is pre-labeled by the target network to obtain a labeling result, the labeling result can be converted into a specific format file, such as a Json file, which can be recognized by a data labeling system. The information in the specific format file is used to represent the labeling result of the to-be-labeled image by the target network.

[0088] In an optional embodiment, as shown in FIG. 5, the at least one level of verification operation performed on the to-be-labeled image and the labeling result includes the following steps: Figure 3 the at least one level of verification operation performed on the to-be-labeled image and the labeling result includes the following steps:

[0089] Step S301, converting the labeling result of the image to be labeled into a target Json file, wherein information in the target Json file is used to represent the labeling result of the image to be labeled by the target network;

[0090] Step S302, performing a multi-level checking operation on the image to be labeled and the target Json file.

[0091] For example, if the image labeling task is a target detection task, the target Json file can include the position information of each target labeled in each image to be labeled and the category to which the target belongs.

[0092] In the embodiments of the present disclosure, by converting the pre-labeled data into a target Json file, at least one level of manual checking of the labeling result automatically predicted by the target network can be realized, so as to realize labeling of the image to be labeled by a semi-supervised labeling method and obtain a labeling result.

[0093] In an optional embodiment, the step S105 of performing at least one level of checking operation on the image to be labeled and the labeling result thereof includes the following steps:

[0094] Step S1051, screening out, according to data screening requirements, a to-be-checked labeling result used for performing a checking operation and a target image to be labeled corresponding to the to-be-checked labeling result from the labeling result.

[0095] In the embodiments of the present disclosure, a screening document is pre-set, and the data screening requirements are included in the screening document. After obtaining the image to be labeled and the labeling result thereof, the image to be labeled and the labeling result thereof can be uploaded to a data screening module of a data labeling system.

[0096] In an optional embodiment, the data screening module of the data labeling system can be set to determine, according to the data screening requirements, the to-be-checked labeling result used for performing the checking operation from the labeling result.

[0097] In another optional embodiment, an auditor can manually determine, according to the data screening requirements, the to-be-checked labeling result used for performing the checking operation from the labeling result in the data screening module of the data labeling system.

[0098] Step S1052, updating a target labeling result that does not meet a preset labeling requirement in the to-be-checked labeling result and a target image to be labeled corresponding to the target labeling result, and performing at least one level of checking operation on all to-be-checked labeling results including the target labeling result after the updating and target images to be labeled corresponding to the to-be-checked labeling results.

[0099] After the to-be-verified annotation result and the target to-be-labeled image corresponding thereto are screened in the manner described above, it can be determined whether the to-be-verified annotation result and the target to-be-labeled image corresponding thereto meet the preset annotation requirement. The preset annotation requirement is a requirement determined based on a model training standard, and the model is a model to be trained by the training sample.

[0100] For example, the training standard of the model is to detect target objects, i.e., obstacles, from the training sample. At this time, it can be determined whether the to-be-verified annotation result contains the annotation result of all obstacles in the target to-be-labeled image. If yes, it is determined that the to-be-verified annotation result meets the preset annotation requirement, otherwise, it does not meet the preset annotation requirement. Further, if it is determined that the to-be-verified annotation result contains the annotation result of all obstacles in the target to-be-labeled image, but the annotation result of part of the obstacles is not accurate, it is determined that the to-be-verified annotation result does not meet the preset annotation requirement, otherwise, it meets the preset annotation requirement.

[0101] In the case where it is determined that the to-be-verified annotation result and the target to-be-labeled object corresponding thereto do not meet the preset annotation requirement, the target annotation result in the to-be-verified annotation result that does not meet the preset annotation requirement and the target to-be-labeled image corresponding thereto can be updated, and all to-be-verified annotation results containing the updated target annotation result and the target to-be-labeled images corresponding thereto can be subjected to at least one level of verification operation. Each verification level operation in the at least one level of verification operation has a different verification process, and the to-be-verified annotation result and the target to-be-labeled image verified by each verification level operation are the same or different.

[0102] In the embodiments of the present disclosure, by setting the verification process of each verification level operation in the at least one level of verification operation to be different, and the to-be-verified annotation result and the target to-be-labeled image verified by each verification level operation to be the same or different, the data labeling process can be further simplified while ensuring the data labeling quality, thereby saving the cost of data labeling, shortening the time of data labeling, and improving the efficiency of data labeling.

[0103] Step S1053, in the case where the to-be-verified annotation result and the target to-be-labeled image corresponding thereto meet the preset annotation requirement, a part of the to-be-verified annotation result is extracted from the to-be-verified annotation result according to a preset extraction ratio to perform a verification operation.

[0104] Step S1054, in the case where the part of the to-be-verified annotation result passes the verification, the target to-be-labeled image and the to-be-verified annotation result corresponding thereto are taken as a training sample.

[0105] In the embodiments of the present disclosure, if the to-be-verified annotation result and the target to-be-labeled image corresponding thereto meet the preset annotation requirement described above, a part of the to-be-verified annotation results can be extracted according to a preset extraction ratio to perform a verification operation. For example, the to-be-verified annotation results can be randomly inspected according to the preset extraction ratio, where the preset extraction ratio can be set to 15%-35%, for example, 20%.

[0106] It should be noted that in the embodiments of the present disclosure, the user can arbitrarily set the value range and specific value of the preset extraction ratio according to actual needs, and the present disclosure does not make specific limitations thereon.

[0107] In an optional implementation, the annotation results can be automatically and randomly inspected according to a preset extraction ratio by an inspection module in the data labeling system.

[0108] In another optional implementation, the annotation results can be manually and randomly inspected according to a preset extraction ratio by an inspection module in the data labeling system.

[0109] It should be understood that inspection refers to randomly extracting a part of the to-be-verified annotation results from all to-be-verified annotation results, and verifying the extracted part of the to-be-verified annotation results and the target to-be-labeled image corresponding thereto. In the case of verification passing, the target to-be-labeled image and the corresponding to-be-verified annotation result are used as training samples. If the verification result of the extracted part of the to-be-verified annotation results and the target to-be-labeled image corresponding thereto is verification failure, the above steps S1051 to S1054 are returned to execute until the inspection result is inspection passing, and the to-be-verified to-be-labeled image and the corresponding target annotation result are used as training samples.

[0110] As described above, in the case that the to-be-verified annotation result does not meet the preset annotation requirement, the to-be-verified annotation result can be updated, and at least one level of verification operation can be performed on the to-be-verified annotation result after the update. In the case that the to-be-verified annotation result meets the preset annotation requirement, the to-be-verified annotation result can be inspected. Through the above processing mode, the data labeling process can be refined, and multi-level verification of data labeling can be realized, so as to ensure the annotation quality of the to-be-labeled image.

[0111] In the embodiments of the present disclosure, the step S1052 described above, updating the target annotation result and the target to-be-labeled image corresponding thereto in the to-be-verified annotation result that does not meet the preset annotation requirement, and performing at least one level of verification operation on all to-be-verified annotation results containing the target annotation result after the update and the target to-be-labeled image corresponding thereto, includes the following processes:

[0112] (1) obtaining an updated annotation result of the target annotation result;

[0113] (2) performing a plurality of check levels of check operations on all to-be-verified annotation results containing the updated annotation result and target to-be-annotated images corresponding to the to-be-verified annotation results, to obtain a check operation result of each check level.

[0114] (3) in the case where the check operation result of each check level is verified, the to-be-annotated image and the annotation result thereof are taken as training samples.

[0115] Specifically, in the embodiment of the present disclosure, in the case where it is determined that any one to-be-verified annotation result and the target to-be-annotated image corresponding thereto do not meet the preset annotation requirement, a modification prompt information can be sent to the annotator to make the annotator modify and update the target annotation result that does not meet the preset annotation requirement, so as to obtain the updated annotation result of the target annotation result.

[0116] After obtaining the updated annotation result, a plurality of check levels of check operations can be performed on all to-be-verified annotation results containing the updated annotation result and target to-be-annotated images corresponding to the to-be-verified annotation results, for example, three levels of check operations can be performed, which are: quality inspection-audit-sampling. For each check level operation, there is a corresponding check operation result. In the case where the check operation result of each check level is verified, the data annotation process can be completed, so as to determine the to-be-annotated image and the annotation result thereof as training samples.

[0117] For the check operation result of each check level, in the case where the check operation result of any check level is failed, re-checking information is returned to a specified check level before the check level, so that the specified check level re-executes the check operation, until the check operation result of each check level is verified. The re-checking information is used to indicate the annotation result that is not verified.

[0118] For example, the check operations of the plurality of check levels are: quality inspection-audit-sampling. For all to-be-verified annotation results containing the updated target annotation result, first, the quality inspection is performed on all to-be-verified annotation results, in the case where the quality inspection is passed, the audit is performed on all to-be-verified annotation results, and in the case where the audit is passed, the sampling is performed on all to-be-verified annotation results.

[0119] It should be noted that if the verification operation at any of the above verification levels fails, a re-verification message can be returned to a specified verification level preceding that level via a designated instruction. This allows the specified verification level to re-perform the verification operation for the failed annotations, or to re-perform the verification operation for all annotations to be verified, including those that failed. The designated instruction includes the hierarchical information of the specified verification level.

[0120] For example, considering the above multiple verification levels: quality inspection, audit, and sampling inspection, if the sampling inspection fails, a re-verification message can be returned to the quality inspection level (or audit level) through a specified instruction. This allows the quality inspection level (or audit level) to re-perform the verification operation for the failed annotation results, or to re-perform the verification operation for all annotation results to be verified, including those that failed the verification.

[0121] For example, if the review fails, the specified instructions above can be used to return re-verification information to the annotation or quality control level, so that the annotation level can perform a re-annotation operation based on the re-verification information. Specifically, if the verification level indicated by the specified instructions is the annotation level, the annotator can modify the annotation result that failed verification. Then, following the quality control-review-sampling verification operation described above, the updated annotation result can be re-verified.

[0122] As can be seen from the above description, in this embodiment of the disclosure, the quality of the annotation results can be improved through the above-mentioned multi-layer quality inspection, thereby obtaining annotation data with high quality assurance. At the same time, the entire data annotation process can be controlled at each level, and the responsibility can be refined to each annotation module of the data annotation system. This enhances the sense of responsibility of each annotation module, reduces data annotation rework, and greatly reduces the cost of data annotation.

[0123] In this embodiment of the disclosure, after determining the training samples based on the image to be labeled and the labeling results, the target network can be retrained to obtain the retrained target network.

[0124] As described above, by repeatedly training the target network, a target network with higher prediction accuracy can be obtained. Therefore, when the target network is used for pre-labeling, the accuracy and quality of the pre-labeling process can be improved.

[0125] like Figure 4 The flowchart shown is for another alternative image annotation method, such as... Figure 4 As shown, the method mainly includes the following processes:

[0126] Step S401: Acquire video sequence;

[0127] Step S402, frame extraction and deduplication processing is performed on the image frames in the video sequence to obtain to-be-labeled images;

[0128] Step S403, pre-labeling processing is performed on the to-be-labeled images by using a target network to obtain labeling results;

[0129] Step S404, to-be-checked labeling results and their corresponding target to-be-labeled images used for performing a checking operation are determined in the labeling results, and the to-be-checked labeling results are checked;

[0130] Step S405, it is judged whether the checking result of the to-be-checked labeling result is passed, wherein if yes, step S406 is executed, otherwise step S408 is executed;

[0131] Step S406, the to-be-checked labeling result and its corresponding target to-be-labeled image are spot-checked, wherein if the spot-checking result is passed, step S407 is executed, otherwise step S404 is returned to be executed;

[0132] Step S407, the target to-be-labeled image and its corresponding to-be-checked labeling result are taken as training samples;

[0133] Step S408, target labeling results in the to-be-checked labeling results that do not meet preset labeling requirements are updated to obtain updated labeling results;

[0134] Step S409, all to-be-checked labeling results containing the updated labeling results are quality inspected, wherein in the case that the quality inspection is passed, step S410 is executed, otherwise step S408 is returned to be executed;

[0135] Step S410, all to-be-checked labeling results after the quality inspection are audited, wherein in the case that the audit is passed, step S411 is executed, otherwise step S409 is returned to be executed;

[0136] Here, in the case that the audit is not passed, in addition to returning to execute step S409, step S408 can also be returned to be executed, and the specific return identifier is not shown in Figure 4 .

[0137] Step S411, all to-be-checked labeling results after the audit are spot-checked, wherein in the case that the spot-checking is passed, step S407 is executed, otherwise step S410 is returned to be executed;

[0138] Here, in the case that the audit is not passed, in addition to returning to execute step S410, step S409 or step S408 can also be returned to be executed, and the specific return identifier is not shown in Figure 4 .

[0139] The above steps S404 to S411 can also be described as the following process:

[0140] In the embodiment of the present disclosure, the data screening requirements can be used to determine the to-be-verified annotation result for performing the verification operation in the annotation result, and the target to-be-labeled image corresponding to the to-be-verified annotation result. Then, each to-be-verified annotation result is screened and verified, and in the case that each to-be-verified annotation result screening verification passes, the corresponding random inspection is performed by the inspector, and generally 20% of the annotation results are inspected for verification. If the random inspection result is completely qualified, it means that the quality of the annotation result after the target network pre-annotation processing is high, and can be used as a training sample for subsequent small model training. If the screening verification result of any to-be-verified annotation result is not passed, the unqualified to-be-verified annotation result and the corresponding target to-be-labeled image can be modified by the annotator, and after the modification, the to-be-verified annotation result after the modification is inspected one by one by the quality inspector. If the quality inspection is unqualified, it is returned to the corresponding annotator for modification; if the quality inspection is qualified, the next process is performed, and the auditor continues to audit one by one. If the auditor determines that the to-be-verified annotation result does not meet the audit requirements, it will be returned to the quality inspector for re-inspection or returned to the annotator for modification; if the audit requirements are met, the next process is performed, and the inspector performs the inspection; if the inspection passes, it is used as a training sample for subsequent small model training, and if it does not pass, it can be returned to the auditor for re-audit, returned to the quality inspector for re-inspection, or returned to the annotator for modification, until the inspection passes.

[0141] As can be seen from the above description, the present disclosure proposes a set of data semi-supervised screening-annotation-auditing integrated annotation data production process, each module can be checked layer by layer, the division of labor is clear, which speeds up the data annotation speed, and at the same time can control the annotation quality of each link. Therefore, through the method described above, the annotation data can be accurately and quickly produced, and through the data verification operation of multiple verification levels, it can be ensured that the entire data annotation process can reach 100% data quality. Through actual operation verification, the scheme can save more than 60% of the annotation cost, which greatly reduces the cost of the company, and can speed up the annotation speed of the urgent task, ensure the annotation quality, and greatly reduce the data annotation period and speed up the product iteration.

[0142] Referring to Figure 5 The flowchart of the data training method provided by the embodiment of the present disclosure is shown, and the method comprises steps S501-S503.

[0143] S501, obtaining a training sample, wherein the training sample is a training sample determined by the above data annotation method.

[0144] Specifically, in the embodiments of the present disclosure, the training sample can be determined by the data labeling method described above, and the specific labeling process of the data labeling method will not be described one by one.

[0145] S503, training the network model to be trained by using the training sample to obtain a trained network model, wherein the trained network model is used for image processing on the to-be-processed image.

[0146] In the embodiments of the present disclosure, the trained network model can be a network model applied in the fields of autonomous driving and assisted driving, can also be a network model applied in the field of security and protection, and can also be a network model applied in the field of medical treatment, and the application scenarios of the network model are not limited in the present disclosure.

[0147] In the embodiments of the present disclosure, the training sample labeled by the image labeling method described above has higher quality, and after the training sample is used to train the network model to be trained, the training precision of the network model to be trained can be improved.

[0148] Referring to Figure 6 FIG. 6 shows a flowchart of an image processing method provided by an embodiment of the present disclosure, and the method includes steps S601-S605, wherein:

[0149] S601, obtaining a to-be-processed image collected by a target vehicle during driving;

[0150] S603, performing a specific processing task on the to-be-processed image by using an image processing network to obtain an image processing result, wherein the image processing network is a network obtained by training a training sample determined by the image labeling method described above.

[0151] S605, controlling a driving state of the target vehicle based on the image processing result.

[0152] In the embodiments of the present disclosure, the training sample labeled by the image labeling method described above has higher quality, and after the training sample is used to train the image processing network to be trained, the training precision of the image processing network to be trained can be improved. When the image processing network is trained, the image processing precision of the network can be improved, thereby improving the reliability of the image processing network.

[0153] When the image processing network is used to analyze the to-be-processed image collected by the target vehicle during driving, a more accurate image processing result can be obtained, and then the driving state of the target vehicle can be accurately controlled, thereby further ensuring the safety of the driver.

[0154] Those skilled in the art can understand that, in the above method of the specific embodiment, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process, and the specific execution order of each step should be determined according to its function and possible internal logic.

[0155] Based on the same inventive concept, the image labeling method in the embodiments of the present disclosure is also provided. Since the principle of solving problems of the device in the embodiments of the present disclosure is similar to the above-mentioned image labeling method, the implementation of the device can be referred to the implementation of the method, and the repeated parts will not be described here.

[0156] Referring to Figure 7 As shown in FIG. 7, a schematic diagram of an image labeling device provided by the embodiments of the present disclosure is shown, which comprises a first acquisition unit 71, a pre-labeling processing unit 72, and a verification unit 73; wherein,

[0157] The first acquisition unit 71 is configured to acquire a to-be-labeled image.

[0158] The pre-labeling processing unit 72 is configured to pre-label the to-be-labeled image by using a pre-trained target network to obtain a labeling result; the target network can complete a specific task processing.

[0159] The verification unit 73 is configured to perform at least one level of verification operation on the to-be-labeled image and its labeling result, and in the case that the verification operation is passed, determine a training sample according to the to-be-labeled image and the labeling result thereof.

[0160] As can be known from the above description, in the embodiments of the present disclosure, by using the target network to pre-label the to-be-labeled image, a semi-supervised labeling of the to-be-labeled image can be realized, thereby saving the workload of the labeling personnel and accelerating the speed of data labeling. By performing at least one level of verification operation on the labeling result, the accuracy of the labeling result can be ensured. Through the above processing method, a large amount of labeling cost can be saved, the speed of data labeling can be accelerated, and the quality of data labeling can be ensured.

[0161] In a possible implementation, the first acquisition unit 71 is further configured to acquire a plurality of original images; extract a plurality of target original images from the plurality of original images according to a preset frame extraction frequency; and perform a deduplication processing on the extracted plurality of target original images to obtain the to-be-labeled image.

[0162] In a possible implementation, the first acquisition unit 71 is further configured to: in a case where the similarity value between adjacent target original images in the plurality of target original images is greater than a preset threshold, discard one image in the adjacent target original images; in a case where the similarity value between adjacent target original images in the plurality of target original images after discarding images is greater than the preset threshold, repeatedly perform the step of discarding one image in the adjacent target original images in a case where the similarity value between adjacent target original images in the plurality of target original images is greater than the preset threshold until the similarity value between adjacent target original images in the plurality of target original images after discarding images is less than or equal to the preset threshold, and determine the plurality of target original images after discarding images as the to-be-labeled images.

[0163] In a possible implementation, the checking unit 73 is further configured to: convert the labeling result of the to-be-labeled image into a specific format file.

[0164] In a possible implementation, the checking unit 73 is further configured to: according to a data screening requirement, screen out to-be-checked labeling results and target to-be-labeled images corresponding to the to-be-checked labeling results from the labeling result, for performing a checking operation; update a target labeling result that does not meet a preset labeling requirement and a target to-be-labeled image corresponding to the target labeling result in the to-be-checked labeling result, and perform at least one level of checking operation on all to-be-checked labeling results containing the target labeling result after the update and target to-be-labeled images corresponding to the to-be-checked labeling results.

[0165] In a possible implementation, the checking unit 73 is further configured to: in a case where the to-be-checked labeling result and the target to-be-labeled image corresponding to the to-be-checked labeling result meet the preset labeling requirement, extract a part of to-be-checked labeling results from the to-be-checked labeling result according to a preset extraction ratio, and perform a checking operation on the part of to-be-checked labeling results; in a case where the part of to-be-checked labeling results pass the checking, use the target to-be-labeled image and the to-be-checked labeling result corresponding to the target to-be-labeled image as a training sample.

[0166] In a possible implementation, the checking unit 73 is further configured to: acquire an updated labeling result of the target labeling result; perform a checking operation of a plurality of checking levels on all to-be-checked labeling results containing the updated labeling result and target to-be-labeled images corresponding to the to-be-checked labeling results, to obtain a checking operation result of each checking level; in a case where the checking operation result of each checking level is a checking pass, use the to-be-labeled image and the labeling result as a training sample.

[0167] In a possible implementation, the checking unit 73 is further configured to, for the checking operation result of each checking level, return re-checking information to a specified checking level before the checking level to the specified checking level, in a case where the checking operation result of any checking level is a checking failure, so that the specified checking level re-executes the checking operation, until the checking operation result of the checking operation of each checking level is a checking success, wherein the re-checking information is used to indicate a label result of a checking failure.

[0168] In a possible implementation, the apparatus is further configured to: after determining the training sample according to the to-be-labeled image and the label result of the to-be-labeled image, retrain the target network to obtain the target network after retraining.

[0169] Based on the same inventive concept, the embodiments of the present disclosure further provide an image processing apparatus corresponding to the image processing method. Since the principle of the apparatus in the embodiments of the present disclosure for solving the problem is similar to the image processing method in the embodiments of the present disclosure, the implementation of the apparatus can be referred to the implementation of the method, and the repeated parts will not be described herein.

[0170] With reference to Figure 8 FIG. 1 is a schematic diagram of an image processing apparatus provided by an embodiment of the present disclosure, which includes a second obtaining unit 81, an image processing unit 82, and a control unit 83.

[0171] The second obtaining unit 81 is configured to obtain a to-be-processed image collected by a target vehicle during driving.

[0172] The image processing unit 82 is configured to perform a specific processing task on the to-be-processed image by using an image processing network to obtain an image processing result, wherein the image processing network is a network obtained after training of a training sample determined by the image labeling method.

[0173] The control unit 83 is configured to control a driving state of the target vehicle based on the image processing result.

[0174] In the embodiments of the present disclosure, the training sample labeled by the image labeling method described above has higher quality, and after the image processing network to be trained is trained by using the training sample, the training precision of the image processing network to be trained can be improved. When the image processing network after training is used for image processing, the image processing precision of the network can be improved, thereby improving the reliability of the image processing network.

[0175] Corresponding to the image labeling method in Figure 1 FIG. 2, the embodiments of the present disclosure further provide a computer device 900, as shown in Figure 9 FIG. 3 is a structural schematic diagram of the computer device 900 provided by an embodiment of the present disclosure, which includes:

[0176] The processor 91, the memory 92, and the bus 93; the memory 92 is used for storing execution instructions, including the memory 921 and the external memory 922; the memory 921 here is also called the internal memory, used for temporarily storing the operation data in the processor 91 and the data exchanged with the external memory 922 such as a hard disk, the processor 91 exchanges data with the external memory 922 through the memory 921, when the computer equipment 900 runs, the processor 91 and the memory 92 communicate through the bus 93, so that the processor 91 executes the following instructions:

[0177] Obtain an image to be labeled;

[0178] Pre-label the image to be labeled through a pre-trained target network to obtain a labeling result; the target network can complete a specific task processing;

[0179] At least one verification operation is performed on the image to be labeled and the labeling result thereof, and in the case that the verification operation passes, a training sample is determined according to the image to be labeled and the labeling result thereof.

[0180] Corresponding to Figure 6 the image processing method in the embodiment of the disclosure, the embodiment of the disclosure also provides a computer equipment 1000, as shown in Figure 10 the structural schematic diagram of the computer equipment 1000 provided by the embodiment of the disclosure, which comprises:

[0181] The processor 101, the memory 102, and the bus 103; the memory 102 is used for storing execution instructions, including the memory 1021 and the external memory 1022; the memory 1021 here is also called the internal memory, used for temporarily storing the operation data in the processor 101 and the data exchanged with the external memory 1022 such as a hard disk, the processor 101 exchanges data with the external memory 1022 through the memory 1021, when the computer equipment 1000 runs, the processor 101 and the memory 102 communicate through the bus 103, so that the processor 101 executes the following instructions:

[0182] Obtain an image to be processed collected by a target vehicle in a driving process;

[0183] Perform a specific processing task on the image to be processed through an image processing network to obtain an image processing result, wherein the image processing network is a network obtained after training of a training sample determined through the above image labeling method;

[0184] Control the driving state of the target vehicle based on the image processing result.

[0185] The embodiment of the present disclosure further provides a computer readable storage medium, and the computer readable storage medium stores a computer program. When the computer program is run by a processor, steps of the data labeling, data training and image processing method described in the above method embodiment are executed. The storage medium can be a volatile or non-volatile computer readable storage medium.

[0186] The embodiment of the present disclosure further provides a computer program product, and the computer program product carries a program code. The program code includes instructions that can be used to execute steps of the data labeling, data training and image processing method described in the above method embodiment. For details, refer to the above method embodiment, which will not be described here.

[0187] The computer program product can be specifically implemented by means of hardware, software or a combination thereof. In an optional embodiment, the computer program product is specifically embodied as a computer storage medium, and in another optional embodiment, the computer program product is specifically embodied as a software product, such as a software development kit (SDK) and the like.

[0188] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system and device can refer to the corresponding process in the foregoing method embodiment, which will not be described here. In several embodiments provided by the present disclosure, it should be understood that the disclosed device and method can be implemented by other means. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division, and actual implementation can have another division manner. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some communication interface, device or unit, which can be electrical, mechanical or other forms.

[0189] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place or distributed on a plurality of network units. According to actual needs, some or all of the units can be selected to achieve the purpose of the embodiment.

[0190] In addition, each functional unit in each embodiment of the present disclosure can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit.

[0191] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a nonvolatile computer readable storage medium executable by a processor. Based on this understanding, the technical solutions of the present disclosure essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present disclosure. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0192] Finally, it should be noted that: the above-described embodiments are only specific embodiments of the present disclosure, used to illustrate the technical solutions of the present disclosure, and not to limit them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can make modifications or easily think of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed by the present disclosure, or make equivalent replacements to some of the technical features. The modifications, changes or replacements do not cause the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be covered within the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims.

Claims

1. An image annotation method, characterized in that, include: Obtaining the image to be labeled includes: obtaining multiple original images, extracting multiple target original images from the multiple original images according to a preset frame extraction frequency, and performing deduplication processing on the extracted multiple target original images to obtain the image to be labeled. The image to be labeled is pre-labeled by a pre-trained target network to obtain the labeling result; the target network can perform specific task processing. A three-level verification operation is performed on the image to be labeled and its labeling results. The three-level verification operation includes quality inspection, review and sampling inspection in sequence. Each level of verification operation of quality inspection, review and sampling inspection corresponds to a verification operation result. If the previous level of verification operation passes, the next level of verification operation is entered. If all three levels of verification operation pass, training samples are determined based on the image to be labeled and its labeling results. Before performing a three-level verification operation on the image to be labeled and its labeling results, the method further includes: converting the labeling results of the image to be labeled into a specific format file, wherein the specific format file is a format file that the data labeling system can recognize.

2. The method according to claim 1, characterized in that, The process of deduplicating the extracted multiple original target images to obtain the image to be labeled includes: If it is determined that the similarity value between adjacent target original images in the plurality of target original images is greater than a preset threshold, one frame of the adjacent target original images is discarded. If it is determined that the similarity value between adjacent target original images in multiple target original images after discarding the image is greater than a preset threshold, the step of discarding one frame of the adjacent target original images is repeated until the similarity value between adjacent target original images in multiple target original images after discarding the image is less than or equal to the preset threshold, and the final multiple target original images after discarding the image are determined as the images to be labeled.

3. The method according to claim 2, characterized in that, The three-level verification operation for the image to be labeled and its labeling results includes: According to the data filtering requirements, filter out the annotation results to be verified for the verification operation, and the target image to be annotated corresponding to the annotation results to be verified from the annotation results; The target annotation results that do not meet the preset annotation requirements and their corresponding target annotation images in the annotation results to be verified are updated, and a three-level verification operation is performed on all the annotation results to be verified and their corresponding target annotation images, including the updated target annotation results.

4. The method according to claim 3, characterized in that, The method further includes: If the annotation results to be verified and the corresponding target image to be annotated meet the preset annotation requirements, a portion of the annotation results to be verified are extracted from the annotation results to be verified according to a preset extraction ratio to perform a verification operation. If the verification of the partial annotation results passes, the target image to be annotated and its corresponding annotation results to be verified are used as training samples.

5. The method according to claim 4, characterized in that, The step involves updating the target annotation results that do not meet the preset annotation requirements and their corresponding target annotation images in the annotation results to be verified, and performing a three-level verification operation on all annotation results to be verified and their corresponding target annotation images, including the updated target annotation results. Obtain the updated annotation result of the target annotation result; Three levels of verification operations are performed on all the annotation results to be verified, including the updated annotation results, and their corresponding target images to be annotated, to obtain the verification operation results of each verification level. If the verification operation at each verification level passes, the image to be labeled and its labeling result are used as training samples.

6. The method according to claim 5, characterized in that, The method further includes: For each verification level, if the verification operation result at any verification level is a verification failure, re-verification information is returned to the specified verification level preceding that verification level, so that the specified verification level re-executes the verification operation until the verification operation result of each verification level is a verification success. The re-verification information is used to indicate the labeling results of verification failures.

7. The method according to any one of claims 1 to 6, characterized in that, The method further includes: After determining the training samples based on the image to be labeled and the labeling results, the target network is retrained to obtain the retrained target network.

8. An image processing method, characterized in that, include: Acquire images of the target vehicle during driving; The image to be processed is subjected to a specific processing task by an image processing network to obtain an image processing result, wherein the image processing network is a network obtained by training the training samples determined by the method of any one of claims 1 to 7 above; The driving state of the target vehicle is controlled based on the image processing results.

9. An image annotation device, characterized in that, include: The first acquisition unit is used to acquire an image to be labeled. The acquisition of the image to be labeled includes: acquiring multiple original images, extracting multiple target original images from the multiple original images according to a preset frame extraction frequency, and performing deduplication processing on the extracted multiple target original images to obtain the image to be labeled. The pre-labeling processing unit is used to pre-label the image to be labeled using a pre-trained target network to obtain the labeling result; the target network is capable of performing specific task processing. The verification unit performs a three-level verification operation on the image to be labeled and its labeling results. The three-level verification operation includes quality inspection, review and sampling inspection in sequence. Each level of verification operation of quality inspection, review and sampling inspection corresponds to a verification operation result. If the previous level of verification operation passes, the next level of verification operation is entered. If all three levels of verification operation pass, the training samples are determined based on the image to be labeled and its labeling results. Before performing a three-level verification operation on the image to be labeled and its labeling results, the verification unit is further configured to: convert the labeling results of the image to be labeled into a specific format file, wherein the specific format file is a format file that the data labeling system can recognize.

10. An image processing apparatus, characterized in that, include: The second acquisition unit is used to acquire the images to be processed collected by the target vehicle during driving. An image processing unit is configured to perform a specific processing task on the image to be processed through an image processing network to obtain an image processing result, wherein the image processing network is a network obtained after training with training samples determined by the method of any one of claims 1 to 7. A control unit is used to control the driving state of the target vehicle based on the image processing results.

11. A computer device, characterized in that, include: The computer device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the computer device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, they perform the steps of the image annotation method as described in any one of claims 1 to 7, or the steps of the image processing method as described in claim 8.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the image annotation method as described in any one of claims 1 to 7, or the steps of the image processing method as described in claim 8.

Images