Image annotation method, device, electronic device and storage medium

Abstract

The present disclosure relates to an image annotation method, device, electronic device and storage medium, the method comprising: obtaining an image to be processed containing an object to be annotated; inputting the image to be processed into a first generator in a pre-trained generative adversarial network to obtain a processed image; the processed image is an image generated by the first generator after converting the object to be annotated in the image to be processed from a first form to a second form; the first form and the second form are different forms of the object to be annotated; and generating an annotation result of the object to be annotated in the image to be processed according to the morphological difference between the processed image and the image to be processed. The present disclosure can improve the speed and accuracy of image annotation.

Description

Technical Field

[0001] The present disclosure relates to the technical field of machine learning, and in particular to an image annotation method, device, electronic device, and storage medium. Background Art

[0002] In the field of machine learning technology, people often use labeled sample images to train deep learning models so that the trained deep learning models can complete certain tasks (for example, face recognition, vehicle recognition, pedestrian tracking, image segmentation, etc.).

[0003] In the process of obtaining labeled sample images, related technologies often require a large amount of professionals and time for manual labeling, and the labeling error rate is high and the complex work is cumbersome.

[0004] However, current image annotation methods suffer from low efficiency. Summary of the invention

[0005] The present disclosure provides an image annotation method, device, electronic device and storage medium to at least solve the problem of low image annotation efficiency in related technologies. The technical solution of the present disclosure is as follows:

[0006] According to a first aspect of an embodiment of the present disclosure, there is provided an image annotation method, comprising:

[0007] Obtaining an image to be processed containing an object to be labeled;

[0008] Input the image to be processed into the first generator in the pre-trained generative adversarial network to obtain a processed image; the processed image is an image generated by the first generator after converting the object to be labeled in the image to be processed from a first form to a second form; the first form and the second form are different forms of the object to be labeled;

[0009] According to the morphological difference between the processed image and the image to be processed, a labeling result of the object to be labeled in the image to be processed is generated.

[0010] In one possible implementation, the training method used by the pre-trained generative adversarial network includes:

[0011] Obtain a first training sample set and a second training sample set; the objects to be labeled in each first sample image of the first training sample set are all in the first form; the objects to be labeled in each second sample image of the second training sample set are all in the second form;

[0012] The first training sample set and the second training sample set are used to iteratively train the generative adversarial network to be trained, so as to obtain the pre-trained generative adversarial network.

[0013] In a possible implementation, the using the first training sample set and the second training sample set to iteratively train the generative adversarial network to be trained to obtain the pre-trained generative adversarial network includes:

[0014] Inputting the first sample image into the first generator in the generative adversarial network to be trained to obtain a first processed sample image; the first processed sample image is an image generated after the first generator in the generative adversarial network to be trained transforms the object to be labeled in the first sample image from the first form to the second form;

[0015] Inputting the first processed sample image to the second generator in the generative adversarial network to be trained to obtain a first restored sample image; the first restored sample image is an image generated after the second generator in the generative adversarial network to be trained transforms the object to be labeled in the first processed sample image from the second form to the first form;

[0016] According to the first sample image, the first processed sample image and the first restored sample image, the first generator in the to-be-trained generative adversarial network, the second generator in the to-be-trained generative adversarial network and the first discriminator in the to-be-trained generative adversarial network are trained until the trained generative adversarial network meets a preset first training termination condition, thereby obtaining a first trained generative adversarial network;

[0017] Inputting the second sample image into the second generator in the first trained generative adversarial network to obtain a second processed sample image; the second processed sample image is an image generated after the second generator in the first trained generative adversarial network transforms the object to be labeled in the second sample image from the second form to the first form;

[0018] Inputting the second processed sample image into the first generator in the first trained generative adversarial network to obtain a second restored sample image; the second restored sample image is an image generated by the first generator in the first trained generative adversarial network converting the object to be labeled in the second processed sample image from the first form to the second form;

[0019] According to the second sample image, the second processed sample image and the second restored sample image, the first generator in the first trained generative adversarial network, the second generator in the first trained generative adversarial network and the second discriminator in the first trained generative adversarial network are trained until the trained generative adversarial network meets a preset second training termination condition, thereby obtaining a second trained generative adversarial network;

[0020] Repeat the above steps until the second trained generative adversarial network meets the preset third training termination condition, thereby obtaining the pre-trained generative adversarial network.

[0021] In a possible implementation, the training of the first generator in the to-be-trained generative adversarial network, the second generator in the to-be-trained generative adversarial network, and the first discriminator in the to-be-trained generative adversarial network according to the first sample image, the first processed sample image, and the first restored sample image includes:

[0022] Based on the first sample image and the first restored sample image, training the first generator in the to-be-trained generative adversarial network and the second generator in the to-be-trained generative adversarial network until the difference between the first sample image and the first restored sample image is less than a preset difference threshold;

[0023] Inputting the first processed sample image to a first discriminator in the generative adversarial network to be trained to obtain a first discrimination result, and training the first generator in the generative adversarial network to be trained based on the first discrimination result until the first discrimination result indicates that the first processed sample image is derived from the second training sample set;

[0024] The first discriminator in the generative adversarial network to be trained is trained until the first discriminant result is that the first processed sample image does not come from the second training sample set.

[0025] In a possible implementation, the training of the first generator in the first trained generative adversarial network, the second generator in the first trained generative adversarial network, and the second discriminator in the first trained generative adversarial network according to the second sample image, the second processed sample image, and the second restored sample image includes:

[0026] Based on the second sample image and the second restored sample image, training the first generator in the first trained generative adversarial network and the second generator in the first trained generative adversarial network until the difference between the second sample image and the second restored sample image is less than a preset difference threshold;

[0027] Inputting the second processed sample image to the second discriminator in the first trained generative adversarial network to obtain a second discrimination result, and training the second generator in the first trained generative adversarial network based on the second discrimination result until the second discrimination result indicates that the second processed sample image is derived from the first training sample set;

[0028] The second discriminator in the first trained generative adversarial network is trained until the second discrimination result is that the second processed sample image does not originate from the first training sample set.

[0029] In a possible implementation, generating a labeling result of the object to be labeled in the image to be processed according to the difference between the processed image and the image to be processed includes:

[0030] Subtracting pixel values ​​of corresponding pixels between the image to be processed and the processed image to obtain pixel difference values ​​of the pixels;

[0031] A labeling result of the object to be labeled in the image to be processed is generated according to the pixel difference value of each pixel point.

[0032] In a possible implementation, generating a labeling result of the object to be labeled in the image to be processed according to the pixel difference value of each pixel point includes:

[0033] The area composed of the pixel points whose pixel difference is greater than the preset threshold is used as the target marking area;

[0034] Performing labeling processing on the target labeling area to obtain the labeling result.

[0035] According to a second aspect of an embodiment of the present disclosure, there is provided an image annotation device, comprising:

[0036] An acquisition unit is configured to acquire an image to be processed containing an object to be annotated;

[0037] A generating unit is configured to input the image to be processed into a first generator in a pre-trained generative adversarial network to obtain a processed image; the processed image is an image generated after the first generator transforms the object to be annotated in the image to be processed from a first form to a second form; the first form and the second form are different forms of the object to be annotated;

[0038] The labeling unit is configured to generate a labeling result of the object to be labeled in the image to be processed according to the morphological difference between the processed image and the image to be processed.

[0039] In one possible implementation, the device also includes: a sample acquisition unit, configured to execute acquisition of a first training sample set and a second training sample set; the objects to be labeled in each first sample image of the first training sample set are all in the first form; the objects to be labeled in each second sample image of the second training sample set are all in the second form; and a training unit, configured to execute iterative training of a generative adversarial network to be trained using the first training sample set and the second training sample set to obtain the pre-trained generative adversarial network.

[0040] In one possible implementation, the training unit is specifically configured to execute inputting the first sample image into the first generator in the generative adversarial network to be trained to obtain a first processed sample image; the first processed sample image is an image generated after the first generator in the generative adversarial network to be trained transforms the object to be labeled in the first sample image from the first form to the second form; inputting the first processed sample image into the second generator in the generative adversarial network to be trained to obtain a first restored sample image; the first restored sample image is an image generated after the second generator in the generative adversarial network to be trained transforms the object to be labeled in the first processed sample image from the second form to the first form; according to the first sample image, the first processed sample image and the first restored sample image, the first generator in the generative adversarial network to be trained, the second generator in the generative adversarial network to be trained and the first discriminator in the generative adversarial network to be trained are trained until the trained generative adversarial network meets a preset first training termination condition to obtain a first trained generative adversarial network; inputting the second sample image into the first training The second generator in the first trained generative adversarial network is used to obtain a second processed sample image; the second processed sample image is an image generated by the second generator in the first trained generative adversarial network converting the object to be labeled in the second sample image from the second form to the first form; the second processed sample image is input into the first generator in the first trained generative adversarial network to obtain a second restored sample image; the second restored sample image is an image generated by the first generator in the first trained generative adversarial network converting the object to be labeled in the second processed sample image from the first form to the second form; according to the second sample image, the second processed sample image and the second restored sample image, the first generator in the first trained generative adversarial network, the second generator in the first trained generative adversarial network and the second discriminator in the first trained generative adversarial network are trained until the trained generative adversarial network meets the preset second training termination condition, thereby obtaining the second trained generative adversarial network; the above steps are repeated until the second trained generative adversarial network meets the preset third training termination condition, thereby obtaining the pre-trained generative adversarial network.

[0041] In one possible implementation, the training unit is specifically configured to perform training on the first generator in the generative adversarial network to be trained and the second generator in the generative adversarial network to be trained based on the first sample image and the first restored sample image until the difference between the first sample image and the first restored sample image is less than a preset difference threshold; input the first processed sample image to the first discriminator in the generative adversarial network to be trained to obtain a first discrimination result, and based on the first discrimination result, train the first generator in the generative adversarial network to be trained until the first discrimination result is that the first processed sample image originates from the second training sample set; and train the first discriminator in the generative adversarial network to be trained until the first discrimination result is that the first processed sample image does not originate from the second training sample set.

[0042] In one possible implementation, the training unit is specifically configured to perform training on the first generator in the first trained generative adversarial network and the second generator in the first trained generative adversarial network based on the second sample image and the second restored sample image until the difference between the second sample image and the second restored sample image is less than a preset difference threshold; input the second processed sample image to the second discriminator in the first trained generative adversarial network to obtain a second discrimination result, and based on the second discrimination result, train the second generator in the first trained generative adversarial network until the second discrimination result is that the second processed sample image is derived from the first training sample set; and train the second discriminator in the first trained generative adversarial network until the second discrimination result is that the second processed sample image is not derived from the first training sample set.

[0043] In one possible implementation, the labeling unit is specifically configured to perform a pixel value subtraction operation on corresponding pixel points between the image to be processed and the processed image to obtain a pixel difference value of each pixel point; and generate a labeling result of the object to be labeled in the image to be processed according to the pixel difference value of each pixel point.

[0044] In a possible implementation, the labeling unit is specifically configured to execute the region composed of pixels whose pixel difference is greater than a preset threshold as a target labeling region; and perform labeling processing on the target labeling region to obtain the labeling result.

[0045] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the image annotation method as described in the first aspect or any possible implementation manner of the first aspect is implemented.

[0046] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the image annotation method as described in the first aspect or any possible implementation manner of the first aspect is implemented.

[0047] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, the program product comprising a computer program, the computer program being stored in a readable storage medium, at least one processor of a device reading and executing the computer program from the readable storage medium, so that the device executes the image annotation method described in any possible implementation manner of the first aspect.

[0048] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects: by obtaining an image to be processed containing an object to be labeled; and inputting the image to be processed into the first generator in a pre-trained generative adversarial network to obtain a processed image; the processed image is an image generated after the first generator transforms the object to be labeled in the image to be processed from a first form to a second form; the first form and the second form are different forms of the object to be labeled; finally, based on the difference between the processed image and the image to be processed, a labeling result of the object to be labeled in the image to be processed is generated, which can realize that the first generator in the pre-trained generative adversarial network transforms the area to be labeled in the image to be processed from the first form to the second form; then, based on the difference between the first form and the second form, the area to be labeled in the image to be processed is automatically labeled, and the whole process is manually intervened in advance, which not only ensures the accuracy of the labeling, but also avoids the need to consume a large amount of professionals and time for manual labeling, thereby improving the labeling efficiency of the image labeling method.

[0049] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0051] Figure 1 The figure is a flowchart of an image annotation method according to an exemplary embodiment.

[0052] Figure 2The figure is a schematic diagram showing the effect of an image annotation method according to an exemplary embodiment.

[0053] Figure 3 The present invention is a schematic diagram of a training process of a generative adversarial network according to an exemplary embodiment.

[0054] Figure 4 The figure is a flowchart of another image annotation method according to an exemplary embodiment.

[0055] Figure 5 The figure is a block diagram of an image annotation device according to an exemplary embodiment.

[0056] Figure 6 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0057] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0058] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0059] It should also be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data for analysis, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties.

[0060] Figure 1 is a flowchart of an image annotation method according to an exemplary embodiment. Figure 1 As shown, the method can be used in an electronic device, and includes the following steps.

[0061] In step S210, an image to be processed containing an object to be labeled is obtained.

[0062] Among them, the object to be labeled may refer to an object that needs to be labeled in the image to be processed. In practical applications, the object to be labeled may meet the following conditions: the labeled object needs to have two or more common forms, such as single / double eyelids, black / yellow / brown hair, presence / absence of eye bags, and presence / absence of wrinkles. Taking the object to be labeled as the eyelids of human eyes as an example, the eyelids of human eyes may include a first form and a second form. Among them, if the first form is a double eyelid form, the second form is a single eyelid form; if the first form is a single eyelid form, the second form is a double eyelid form. That is, the first form and the second form are different forms of the object to be labeled.

[0063] In a specific implementation, the electronic device obtains an image to be processed that contains an object to be annotated. For example, the electronic device may obtain an image that needs to have a double eyelid area annotated or an image that needs to have a hair area annotated.

[0064] In step S220, the image to be processed is input into the first generator in the pre-trained generative adversarial network to obtain a processed image; the processed image is an image generated after the first generator transforms the object to be annotated in the image to be processed from the first form to the second form;

[0065] In a specific implementation, after the electronic device acquires the image to be processed, the electronic device inputs the image to be processed into the first generator in the pre-trained generative adversarial network, and transforms the object to be labeled in the image to be processed from the first form to the second form through the first generator to generate a processed image.

[0066] For example, assuming that the image to be processed is a double eyelid image that needs to have the double eyelid area marked, the electronic device inputs the double eyelid image into the first generator in the pre-trained generative adversarial network, and the first generator in the pre-trained generative adversarial network will convert the eyelid area in the double eyelid image from a double eyelid form to a single eyelid form, thereby obtaining a single eyelid image virtualized (forged) by the first generator. In actual applications, the single eyelid image is similar to the image to be processed except that the eyelid area is in a single eyelid form.

[0067] In step S230, a labeling result of the object to be labeled in the image to be processed is generated according to the difference between the processed image and the image to be processed.

[0068] In a specific implementation, the electronic device may perform image subtraction processing on the processed image and the image to be processed to obtain the difference between the processed image and the image to be processed; finally, the electronic device may generate a labeling result of the object to be labeled in the image to be processed based on the difference. In practical applications, the labeling result may refer to a region labeling Mask (mask image) that carries the position information of the target labeling area in the image to be processed, or it may be a segmentation label. In practical applications, the electronic device may use the labeling result and the image to be processed as a paired original image and region labeling mask, and then use them to train the deep learning network used for the segmentation task later. To facilitate the understanding of those skilled in the art, Figure 2 An effect schematic diagram of an image annotation method is provided by way of example; wherein 210 is a picture to be processed, 220 is a processed picture, and 230 is an annotation result.

[0069] In the above-mentioned image annotation method, an image to be processed containing an object to be annotated is obtained; and the image to be processed is input into the first generator in a pre-trained generative adversarial network to obtain a processed image; the processed image is an image generated after the first generator transforms the object to be annotated in the image to be processed from a first form to a second form; the first form and the second form are different forms of the object to be annotated; finally, based on the difference between the processed image and the image to be processed, an annotation result of the object to be annotated in the image to be processed is generated, which can realize that the first generator in the pre-trained generative adversarial network transforms the area to be annotated in the image to be processed from the first form to the second form; then, based on the difference between the first form and the second form, the area to be annotated in the image to be processed is automatically annotated, and the whole process is manually intervened in advance, which not only ensures the accuracy of the annotation, but also avoids the need to consume a large amount of professionals and time for manual annotation, thereby improving the annotation efficiency of the image annotation method.

[0070] In an exemplary embodiment, before the step of obtaining an image to be processed containing an object to be labeled, the method further includes: obtaining a first training sample set and a second training sample set; the objects to be labeled in each first sample image of the first training sample set are all in the first form; the objects to be labeled in each second sample image of the second training sample set are all in the second form; and the first training sample set and the second training sample set are used to iteratively train the generative adversarial network to be trained to obtain a pre-trained generative adversarial network.

[0071] In a specific implementation, the electronic device can obtain a first training sample set and a second training sample set. Among them, the objects to be labeled in each first sample image of the first training sample set are all in the first form; the objects to be labeled in each second sample image of the second training sample set are all in the second form. In practical applications, the electronic device can collect a large number of pictures containing areas to be labeled. Taking the object to be labeled as human eyelids, the first form of human eyelids is double eyelids, and the second form of human eyelids is single eyelids as an example, the electronic device can collect a large number of original portrait pictures. These pictures come from channels such as mobile phones, camera collection, and downloading of public pictures on the Internet. Then, according to the characteristics of the area to be labeled, the electronic device summarizes the area into two different categories A and B, such as single eyelids / double eyelids. The area to be labeled is labeled A and B according to the set classification, and all pictures are classified into two categories. The area is cropped and sorted into a specific size. The classified A and B data sets are formed.

[0072] The technical solution of this embodiment adopts a first training sample set in which all objects to be labeled in the first sample image are in the first form, and a second training sample set in which all objects to be labeled in the second sample image are in the second form; iterative training is performed on the generative adversarial network to be trained, thereby realizing classification that only requires labeled images, without the need for complex area / edge labeling, and greatly improving efficiency.

[0073] In an exemplary embodiment, a first training sample set and a second training sample set are used to iteratively train a generative adversarial network to be trained to obtain a pre-trained generative adversarial network, including: inputting a first sample image into a first generator in the generative adversarial network to be trained to obtain a first processed sample image; the first processed sample image is an image generated after the first generator in the generative adversarial network to be trained transforms the object to be labeled in the first sample image from a first form to a second form; inputting the first processed sample image into the second generator in the generative adversarial network to be trained to obtain a first restored sample image; the first restored sample image is an image generated after the second generator in the generative adversarial network to be trained transforms the object to be labeled in the first processed sample image from a second form to a first form; according to the first sample image, the first processed sample image and the first restored sample image, the first generator in the generative adversarial network to be trained, the second generator in the generative adversarial network to be trained and the first discriminator in the generative adversarial network to be trained are trained until the trained generative adversarial network meets a preset first training termination condition, thereby obtaining a first trained generative adversarial network. network; input the second sample image to the second generator in the first trained generative adversarial network to obtain a second processed sample image; the second processed sample image is an image generated after the second generator in the first trained generative adversarial network transforms the object to be labeled in the second sample image from the second form to the first form; input the second processed sample image to the first generator in the first trained generative adversarial network to obtain a second restored sample image; the second restored sample image is an image generated after the first generator in the first trained generative adversarial network transforms the object to be labeled in the second processed sample image from the first form to the second form; according to the second sample image, the second processed sample image and the second restored sample image, the first generator in the first trained generative adversarial network, the second generator in the first trained generative adversarial network and the second discriminator in the first trained generative adversarial network are trained until the trained generative adversarial network meets the preset second training termination condition, thereby obtaining the second trained generative adversarial network; repeat the above steps until the second trained generative adversarial network meets the preset third training termination condition, thereby obtaining the pre-trained generative adversarial network.

[0074] In an exemplary embodiment, a first generator in a generative adversarial network to be trained, a second generator in a generative adversarial network to be trained, and a first discriminator in a generative adversarial network to be trained are trained according to a first sample image, a first processed sample image, and a first restored sample image, including: based on the first sample image and the first restored sample image, the first generator in the generative adversarial network to be trained and the second generator in the generative adversarial network to be trained are trained until the difference between the first sample image and the first restored sample image is less than a preset difference threshold; the first processed sample image is input into the first discriminator in the generative adversarial network to be trained to obtain a first discrimination result, and based on the first discrimination result, the first generator in the generative adversarial network to be trained is trained until the first discrimination result is that the first processed sample image originates from the second training sample set; the first discriminator in the generative adversarial network to be trained is trained until the first discrimination result is that the first processed sample image does not originate from the second training sample set.

[0075] In an exemplary embodiment, the first generator in the first trained generative adversarial network, the second generator in the first trained generative adversarial network and the second discriminator in the first trained generative adversarial network are trained according to the second sample image, the second processed sample image and the second restored sample image, including: based on the second sample image and the second restored sample image, the first generator in the first trained generative adversarial network and the second generator in the first trained generative adversarial network are trained until the difference between the second sample image and the second restored sample image is less than a preset difference threshold; the second processed sample image is input into the second discriminator in the first trained generative adversarial network to obtain a second discrimination result, and based on the second discrimination result, the second generator in the first trained generative adversarial network is trained until the second discrimination result is that the second processed sample image originates from the first training sample set; the second discriminator in the first trained generative adversarial network is trained until the second discrimination result is that the second processed sample image does not originate from the first training sample set.

[0076] Among them, taking the human eyelid as the object to be labeled as an example, the first training sample set A is an image set containing a large number of double-eyelid pictures A1, A2, A3...; the second training sample set B is an image set containing a large number of single-eyelid pictures B1, B2, B3...

[0077] Among them, the first generator G, the second generator F, the first discriminator D_G and the second discriminator D_F in the generative adversarial network can all be deep neural networks D_F to be trained. Specifically, the first generator G may refer to a neural network G used to convert image A1 (double eyelids) into A1' (corresponding single eyelids). The second generator F may refer to a neural network F used to convert image B1 (single eyelids) into B1' (corresponding double eyelids). The first discriminator D_G may refer to a discriminator D_G used to distinguish between a real B1 and a generated fake A1'. The second discriminator may refer to a discriminator D_G used to distinguish between a real A1 and a generated fake B1'.

[0078] In a specific implementation, taking the human eyelid as an object to be labeled, the electronic device can input the first sample image A1 into the first generator G in the generative adversarial network to be trained to obtain the first processed sample image A1'; the first processed sample image A1' is an image generated by the first generator G after converting the object to be labeled in the first sample image A1 from the first form (i.e., double eyelids) to the second form (i.e., single eyelids); the first processed sample image A1' is input into the second generator F in the generative adversarial network to be trained to obtain the first restored sample image A1"; the first restored sample image A1" is an image generated by the second generator F after converting the object to be labeled in the first processed sample image A1' from the second form to the first form. In this way, by using the first training sample set in which the object to be labeled in each first sample image is in the first form, and the second training sample set in which the object to be labeled in each second sample image is in the second form; the generative adversarial network to be trained is iteratively trained, so that the classification of images that only need to be labeled is realized, without manual intervention, and without complex edge labeling, while improving the recognition accuracy of the model.

[0079] The electronic device inputs the first processed sample image A1' into the first discriminator D in the generative adversarial network to be trained. G , get the first discrimination result; the first discrimination result is the first discriminator D G Determine whether the sample image after the first processing is derived from the result obtained by the second training sample set B; input the second sample image B1 into the first discriminator D in the generative adversarial network to be trained G , get the second discrimination result; the second discrimination result is the first discriminator D G The electronic device determines whether the second sample image B1 is derived from the result of the second training sample set B; then, based on the difference between the first sample image A1 and the first restored sample image A", the first determination result, and the second determination result, the electronic device determines whether the first generator G, the second generator F, and the first discriminator D in the generative adversarial network to be trained are derived from the result of the second training sample set B; GIterative training is performed until the trained generative adversarial network meets the preset training termination conditions.

[0080] Specifically, the electronic device can use the stochastic gradient descent method to first fix the first discriminator D G The network parameters of the first generator G and the second generator F are optimized until A1" and A1 are consistent, that is, the similarity between image A1" and image A1 is less than the preset similarity threshold. Then, the electronic device uses the stochastic gradient descent method to fix the first discriminator D G The network parameters of the first generator G are optimized so that A1' generated by the first generator G is G Then, the electronic device uses the stochastic gradient descent method to fix the network parameters of the first generator G and the first discriminator D G The network parameters are optimized so that A1' generated by the first generator G is G Under the judgment of , it is correctly judged as coming from non-B data set.

[0081] In this way, by adopting the first training sample set in which the objects to be labeled in each first sample image are all in the first form, and the second training sample set in which the objects to be labeled in each second sample image are all in the second form; the generative adversarial network to be trained is iteratively trained, and classification is achieved that only requires labeled images, without the need for complex area labeling, thus saving the cost of manual labeling and improving the recognition accuracy of the model.

[0082] The electronic device inputs the second sample image Bi to the second generator F in the generative adversarial network to be trained, and obtains a second processed sample image Bi'; the second processed sample image Bi' is an image generated by the second generator F after converting the object to be labeled in the second sample image Bi from the second form (i.e., single eyelids) to the first form (i.e., double eyelids); then, the electronic device inputs the second processed sample image Bi' to the first generator G in the generative adversarial network to be trained, and obtains a second restored sample image Bi"; the second restored sample image Bi" is an image generated by the first generator G after converting the object to be labeled in the second processed sample image Bi' from the first form (i.e., double eyelids) to the second form (i.e., single eyelids); then, the electronic device inputs the second processed sample image Bi' to the first generator G in the generative adversarial network to be trained, and obtains a second restored sample image Bi". The second discriminator D_F in the generative adversarial network to be trained is used to obtain a third discrimination result; the third discrimination result is the result obtained by the second discriminator D_F to discriminate whether the second processed sample image Bi' is derived from the first training sample set A; the first sample image Ai is input into the second discriminator D_F in the generative adversarial network to be trained to obtain a fourth discrimination result; the fourth discrimination result is the result obtained by the second discriminator D_F to discriminate whether the first sample image Ai is derived from the first training sample set A; then, the electronic device iteratively trains the first generator G, the second generator F and the second discriminator D_F in the generative adversarial network to be trained based on the difference between the second sample image and the second restored sample image, the third discrimination result, and the fourth discrimination result, until the trained generative adversarial network meets the preset training termination condition. Specifically, the electronic device adopts a stochastic gradient descent method to first fix the second discriminator D F The network parameters of the first generator G and the second generator F are optimized until B1” and B1 tend to be consistent, that is, the similarity between image B1” and image B1 is less than the preset similarity threshold.

[0083] Then, the electronic device uses the stochastic gradient descent method to fix the second discriminator D F The network parameters of the second generator F are optimized so that B1' generated by the second generator F is F It is misjudged as coming from data set A under the judgment of .

[0084] Then, the electronic device uses the stochastic gradient descent method to fix the network parameters of the second generator F and the second discriminator D F The network parameters are optimized so that B1' generated by the second generator F is in the second discriminator D F In order to facilitate the understanding of those skilled in the art, Figure 3 A schematic diagram of the training process of a generative adversarial network is also provided by way of example.

[0085] The technical solution of this embodiment adopts a first training sample set in which the objects to be labeled in each first sample image are all in the first form, and a second training sample set in which the objects to be labeled in each second sample image are all in the second form; iterative training is performed on the generative adversarial network to be trained, thereby realizing classification that only requires labeled images, without the need for complex area / edge labeling, and greatly improving the training efficiency of the model.

[0086] In an exemplary embodiment, a labeling result of the object to be labeled in the image to be processed is generated based on the difference between the processed image and the image to be processed, including: subtracting the pixel values ​​of corresponding pixel points between the image to be processed and the processed image to obtain the pixel difference value of each pixel point; and generating the labeling result of the object to be labeled in the image to be processed based on the pixel difference value of each pixel point.

[0087] In a specific implementation, when the electronic device generates the annotation result of the object to be annotated in the image to be processed based on the difference between the processed image and the image to be processed, the electronic device can perform a subtraction operation on the pixel values ​​of corresponding pixel points between the image to be processed and the processed image to obtain the pixel difference value of each pixel point; then, the electronic device can quickly and accurately generate the annotation result of the object to be annotated in the image to be processed based on the pixel difference value of each pixel point.

[0088] For example, after determining the image Ai to be processed and the processed image Ai', the electronic device calculates the image interpolation between the image Ai to be processed and the processed image Ai', and uses the pixel set corresponding to the area |Ai-Ai'|>T as the object Ai_m to be labeled of the image Ai to be processed.

[0089] The technical solution of this embodiment is to obtain the pixel difference of each pixel by subtracting the pixel values ​​of corresponding pixels between the image to be processed and the processed image; and generate the annotation result of the object to be annotated in the image to be processed according to the pixel difference of each pixel, so as to quickly determine the difference between the image to be processed and the processed image, and determine the annotation result based on the difference.

[0090] In an exemplary embodiment, a labeling result of the object to be labeled in the image to be processed is generated according to the pixel difference of each pixel point, including: taking the area composed of pixel points whose pixel difference is greater than a preset threshold as the target labeling area; and labeling the target labeling area to obtain the labeling result.

[0091] In a specific implementation, when the electronic device generates the annotation result of the object to be annotated in the image to be processed according to the pixel difference of each pixel point, the electronic device can respectively determine whether each pixel point is greater than a preset threshold value T; the electronic device determines the pixel points whose pixel difference value T is greater than the preset threshold value, and uses the area composed of these pixel points as the target annotation area; finally, the electronic device performs annotation processing on the target annotation area to obtain the annotation result. For example, a regional annotation Mask (mask map) carrying the position information of the target annotation area in the image to be processed is generated as the annotation result.

[0092] The technical solution of this embodiment, by annotating the target annotation area with the area composed of pixels whose pixel difference is greater than a preset threshold, can accurately annotate the object to be annotated in the image to be processed based on the difference between the image to be processed and the processed image.

[0093] Figure 4 is a flowchart of another image annotation method according to an exemplary embodiment. Figure 4 As shown, the following steps are included.

[0094] In step S410, an image to be processed containing an object to be labeled is obtained.

[0095] In step S420, the image to be processed is input into the first generator in the pre-trained generative adversarial network to obtain a processed image; the processed image is an image generated after the first generator converts the object to be labeled in the image to be processed from a first form to a second form; the first form and the second form are different forms of the object to be labeled.

[0096] In step S430, a subtraction operation is performed on the pixel values ​​of corresponding pixels between the image to be processed and the processed image to obtain a pixel difference value of each pixel.

[0097] In step S440, the area consisting of pixels whose pixel difference values ​​are greater than a preset threshold is taken as the target marking area.

[0098] In step S450, the target annotation area is annotated to generate an annotation result of the object to be annotated in the image to be processed.

[0099] It should be noted that the specific limitations of the above steps can be found in the specific limitations of an image annotation method mentioned above, which will not be repeated here.

[0100] It should be understood that although Figure 1 and Figure 4The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 and Figure 4 At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0101] It can be understood that the same / similar parts between the various embodiments of the above method in this specification can refer to each other, and each embodiment focuses on the differences from other embodiments. For related points, please refer to the description of other method embodiments.

[0102] Figure 5 is a block diagram of an image annotation device according to an exemplary embodiment. Figure 5 , the image annotation device comprises:

[0103] An acquisition unit 510 is configured to acquire an image to be processed that contains an object to be annotated;

[0104] The generating unit 520 is configured to input the image to be processed into a first generator in a pre-trained generative adversarial network to obtain a processed image; the processed image is an image generated after the first generator transforms the object to be annotated in the image to be processed from a first form to a second form; the first form and the second form are different forms of the object to be annotated;

[0105] The labeling unit 530 is configured to generate a labeling result of the object to be labeled in the image to be processed according to the morphological difference between the processed image and the image to be processed.

[0106] In one possible implementation, the device also includes: a sample acquisition unit, configured to execute acquisition of a first training sample set and a second training sample set; the objects to be labeled in each first sample image of the first training sample set are all in the first form; the objects to be labeled in each second sample image of the second training sample set are all in the second form; and a training unit, configured to execute iterative training of a generative adversarial network to be trained using the first training sample set and the second training sample set to obtain the pre-trained generative adversarial network.

[0107] In one possible implementation, the training unit is specifically configured to execute inputting the first sample image into the first generator in the generative adversarial network to be trained to obtain a first processed sample image; the first processed sample image is an image generated after the first generator in the generative adversarial network to be trained transforms the object to be labeled in the first sample image from the first form to the second form; inputting the first processed sample image into the second generator in the generative adversarial network to be trained to obtain a first restored sample image; the first restored sample image is an image generated after the second generator in the generative adversarial network to be trained transforms the object to be labeled in the first processed sample image from the second form to the first form; according to the first sample image, the first processed sample image and the first restored sample image, the first generator in the generative adversarial network to be trained, the second generator in the generative adversarial network to be trained and the first discriminator in the generative adversarial network to be trained are trained until the trained generative adversarial network meets a preset first training termination condition to obtain a first trained generative adversarial network; inputting the second sample image into the first training The second generator in the first trained generative adversarial network is used to obtain a second processed sample image; the second processed sample image is an image generated by the second generator in the first trained generative adversarial network converting the object to be labeled in the second sample image from the second form to the first form; the second processed sample image is input into the first generator in the first trained generative adversarial network to obtain a second restored sample image; the second restored sample image is an image generated by the first generator in the first trained generative adversarial network converting the object to be labeled in the second processed sample image from the first form to the second form; according to the second sample image, the second processed sample image and the second restored sample image, the first generator in the first trained generative adversarial network, the second generator in the first trained generative adversarial network and the second discriminator in the first trained generative adversarial network are trained until the trained generative adversarial network meets the preset second training termination condition, thereby obtaining the second trained generative adversarial network; the above steps are repeated until the second trained generative adversarial network meets the preset third training termination condition, thereby obtaining the pre-trained generative adversarial network.

[0108] In one possible implementation, the training unit is specifically configured to perform training on the first generator in the generative adversarial network to be trained and the second generator in the generative adversarial network to be trained based on the first sample image and the first restored sample image until the difference between the first sample image and the first restored sample image is less than a preset difference threshold; input the first processed sample image to the first discriminator in the generative adversarial network to be trained to obtain a first discrimination result, and based on the first discrimination result, train the first generator in the generative adversarial network to be trained until the first discrimination result is that the first processed sample image originates from the second training sample set; and train the first discriminator in the generative adversarial network to be trained until the first discrimination result is that the first processed sample image does not originate from the second training sample set.

[0109] In one possible implementation, the training unit is specifically configured to perform training on the first generator in the first trained generative adversarial network and the second generator in the first trained generative adversarial network based on the second sample image and the second restored sample image until the difference between the second sample image and the second restored sample image is less than a preset difference threshold; input the second processed sample image to the second discriminator in the first trained generative adversarial network to obtain a second discrimination result, and based on the second discrimination result, train the second generator in the first trained generative adversarial network until the second discrimination result is that the second processed sample image is derived from the first training sample set; and train the second discriminator in the first trained generative adversarial network until the second discrimination result is that the second processed sample image is not derived from the first training sample set.

[0110] In one possible implementation, the labeling unit 530 is specifically configured to perform a pixel value subtraction operation on corresponding pixel points between the image to be processed and the processed image to obtain a pixel difference value of each pixel point; and generate a labeling result of the object to be labeled in the image to be processed according to the pixel difference value of each pixel point.

[0111] In a possible implementation, the labeling unit 530 is specifically configured to execute the region composed of the pixel points whose pixel difference is greater than a preset threshold as the target labeling region; and perform labeling processing on the target labeling region to obtain the labeling result.

[0112] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0113] Figure 6 1 is a block diagram of an electronic device 600 for performing image annotation according to an exemplary embodiment. For example, the electronic device 600 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0114] Reference Figure 6 , the electronic device 600 may include one or more of the following components: a processing component 602 , a memory 604 , a power component 606 , a multimedia component 608 , an audio component 610 , an input / output (I / O) interface 612 , a sensor component 614 , and a communication component 616 .

[0115] The processing component 602 generally controls the overall operation of the electronic device 600, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 602 may include one or more processors 620 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 602 may include one or more modules to facilitate the interaction between the processing component 602 and other components. For example, the processing component 602 may include a multimedia module to facilitate the interaction between the multimedia component 608 and the processing component 602.

[0116] The memory 604 is configured to store various types of data to support operations on the electronic device 600. Examples of such data include instructions for any application or method operating on the electronic device 600, contact data, phone book data, messages, pictures, videos, etc. The memory 604 may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, optical disk, or graphene memory.

[0117] The power supply component 606 provides power to the various components of the electronic device 600. The power supply component 606 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 600.

[0118] The multimedia component 608 includes a screen that provides an output interface between the electronic device 600 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 608 includes a front camera and / or a rear camera. When the electronic device 600 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0119] The audio component 610 is configured to output and / or input audio signals. For example, the audio component 610 includes a microphone (MIC), and when the electronic device 600 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 604 or sent via the communication component 616. In some embodiments, the audio component 610 also includes a speaker for outputting an audio signal.

[0120] I / O interface 612 provides an interface between processing component 602 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0121] The sensor assembly 614 includes one or more sensors for providing various aspects of status assessment for the electronic device 600. For example, the sensor assembly 614 can detect the open / closed state of the electronic device 600, the relative positioning of the components, such as the display and keypad of the electronic device 600, and the sensor assembly 614 can also detect the position change of the electronic device 600 or the electronic device 600 component, the presence or absence of contact between the user and the electronic device 600, the device 600 orientation or acceleration / deceleration and the temperature change of the electronic device 600. The sensor assembly 614 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 614 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 614 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0122] The communication component 616 is configured to facilitate wired or wireless communication between the electronic device 600 and other devices. The electronic device 600 can access a wireless network based on a communication standard, such as WiFi, a carrier network (such as 2G, 3G, 4G or 5G), or a combination thereof. In an exemplary embodiment, the communication component 616 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 616 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0123] In an exemplary embodiment, the electronic device 600 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0124] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory 604 including instructions, and the above instructions can be executed by the processor 620 of the electronic device 600 to perform the above method. For example, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0125] In an exemplary embodiment, a computer program product is further provided. The computer program product includes instructions. The instructions can be executed by the processor 620 of the electronic device 600 to complete the above method.

[0126] It should be noted that the above-mentioned devices, electronic devices, computer-readable storage media, computer program products, etc. may also include other implementation methods according to the description of the method embodiments. The specific implementation methods can refer to the description of the relevant method embodiments, which will not be described one by one here.

[0127] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0128] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An image annotation method, characterized in that: include: Obtaining an image to be processed containing an object to be labeled; Inputting the image to be processed into a first generator in a pre-trained generative adversarial network to obtain a processed image; The processed image is an image generated after the first generator transforms the object to be labeled in the image to be processed from a first form to a second form; the first form and the second form are different forms of the object to be labeled; Subtracting pixel values ​​of corresponding pixels between the image to be processed and the processed image to obtain pixel difference values ​​of the pixels; A labeling result of the object to be labeled in the image to be processed is generated according to the pixel difference value of each pixel point.

2. The image annotation method according to claim 1, characterized in that: The training method adopted by the pre-trained generative adversarial network includes: Obtain a first training sample set and a second training sample set; the objects to be labeled in each first sample image of the first training sample set are all in the first form; the objects to be labeled in each second sample image of the second training sample set are all in the second form; The first training sample set and the second training sample set are used to iteratively train the generative adversarial network to be trained, so as to obtain the pre-trained generative adversarial network.

3. The image annotation method according to claim 2, characterized in that: The step of using the first training sample set and the second training sample set to iteratively train the generative adversarial network to be trained to obtain the pre-trained generative adversarial network includes: Inputting the first sample image into the first generator in the generative adversarial network to be trained to obtain a first processed sample image; the first processed sample image is an image generated after the first generator in the generative adversarial network to be trained transforms the object to be labeled in the first sample image from the first form to the second form; Inputting the first processed sample image to the second generator in the generative adversarial network to be trained to obtain a first restored sample image; the first restored sample image is an image generated after the second generator in the generative adversarial network to be trained transforms the object to be labeled in the first processed sample image from the second form to the first form; According to the first sample image, the first processed sample image and the first restored sample image, the first generator in the to-be-trained generative adversarial network, the second generator in the to-be-trained generative adversarial network and the first discriminator in the to-be-trained generative adversarial network are trained until the trained generative adversarial network meets a preset first training termination condition, thereby obtaining a first trained generative adversarial network; Inputting the second sample image into the second generator in the first trained generative adversarial network to obtain a second processed sample image; the second processed sample image is an image generated after the second generator in the first trained generative adversarial network transforms the object to be labeled in the second sample image from the second form to the first form; Inputting the second processed sample image into the first generator in the first trained generative adversarial network to obtain a second restored sample image; the second restored sample image is an image generated by the first generator in the first trained generative adversarial network converting the object to be labeled in the second processed sample image from the first form to the second form; According to the second sample image, the second processed sample image and the second restored sample image, the first generator in the first trained generative adversarial network, the second generator in the first trained generative adversarial network and the second discriminator in the first trained generative adversarial network are trained until the trained generative adversarial network meets a preset second training termination condition, thereby obtaining a second trained generative adversarial network; Repeat the above steps until the second trained generative adversarial network meets the preset third training termination condition, thereby obtaining the pre-trained generative adversarial network.

4. The image annotation method according to claim 3, characterized in that: The step of training the first generator in the to-be-trained generative adversarial network, the second generator in the to-be-trained generative adversarial network, and the first discriminator in the to-be-trained generative adversarial network according to the first sample image, the first processed sample image, and the first restored sample image comprises: Based on the first sample image and the first restored sample image, training the first generator in the to-be-trained generative adversarial network and the second generator in the to-be-trained generative adversarial network until the difference between the first sample image and the first restored sample image is less than a preset difference threshold; Inputting the first processed sample image to a first discriminator in the generative adversarial network to be trained to obtain a first discrimination result, and training the first generator in the generative adversarial network to be trained based on the first discrimination result until the first discrimination result indicates that the first processed sample image is derived from the second training sample set; The first discriminator in the generative adversarial network to be trained is trained until the first discriminant result is that the first processed sample image does not come from the second training sample set.

5. The image annotation method according to claim 3, characterized in that: The step of training the first generator in the first trained generative adversarial network, the second generator in the first trained generative adversarial network, and the second discriminator in the first trained generative adversarial network according to the second sample image, the second processed sample image, and the second restored sample image comprises: Based on the second sample image and the second restored sample image, training the first generator in the first trained generative adversarial network and the second generator in the first trained generative adversarial network until the difference between the second sample image and the second restored sample image is less than a preset difference threshold; Inputting the second processed sample image to the second discriminator in the first trained generative adversarial network to obtain a second discrimination result, and training the second generator in the first trained generative adversarial network based on the second discrimination result until the second discrimination result indicates that the second processed sample image is derived from the first training sample set; The second discriminator in the first trained generative adversarial network is trained until the second discrimination result is that the second processed sample image does not originate from the first training sample set.

6. The image annotation method according to claim 1, characterized in that: Generating a labeling result of the object to be labeled in the image to be processed according to the pixel difference value of each pixel point includes: The area composed of the pixel points whose pixel difference is greater than the preset threshold is used as the target marking area; Performing labeling processing on the target labeling area to obtain the labeling result.

7. An image annotation device, characterized in that: include: An acquisition unit is configured to acquire an image to be processed containing an object to be annotated; A generating unit is configured to input the image to be processed into a first generator in a pre-trained generative adversarial network to obtain a processed image; The processed image is an image generated after the first generator transforms the object to be labeled in the image to be processed from a first form to a second form; the first form and the second form are different forms of the object to be labeled; a marking unit configured to perform a pixel value subtraction operation on corresponding pixel points between the image to be processed and the processed image to obtain a pixel difference value of each pixel point; A labeling result of the object to be labeled in the image to be processed is generated according to the pixel difference value of each pixel point.

8. The image annotation device according to claim 7, characterized in that: The device also includes: a sample acquisition unit, configured to execute acquisition of a first training sample set and a second training sample set; the objects to be labeled in each first sample image of the first training sample set are all in the first form; the objects to be labeled in each second sample image of the second training sample set are all in the second form; and a training unit, configured to execute iterative training of a generative adversarial network to be trained using the first training sample set and the second training sample set to obtain the pre-trained generative adversarial network.

9. The image annotation device according to claim 8, characterized in that: The training unit is specifically configured to input the first sample image into a first generator in the generative adversarial network to be trained to obtain a first processed sample image; the first processed sample image is an image generated after the first generator in the generative adversarial network to be trained transforms the object to be annotated in the first sample image from the first form to the second form; the first processed sample image is input into a second generator in the generative adversarial network to be trained to obtain a first restored sample image; The first restored sample image is an image generated by a second generator in the generative adversarial network to be trained that transforms the object to be labeled in the first processed sample image from the second form to the first form; According to the first sample image, the first processed sample image and the first restored sample image, the first generator in the to-be-trained generative adversarial network, the second generator in the to-be-trained generative adversarial network and the first discriminator in the to-be-trained generative adversarial network are trained until the trained generative adversarial network meets a preset first training termination condition, thereby obtaining a first trained generative adversarial network; Input the second sample image to the second generator in the first trained generative adversarial network to obtain a second processed sample image; the second processed sample image is an image generated after the second generator in the first trained generative adversarial network transforms the object to be labeled in the second sample image from the second form to the first form; input the second processed sample image to the first generator in the first trained generative adversarial network to obtain a second restored sample image; The second restored sample image is an image generated after the first generator in the first trained generative adversarial network transforms the object to be labeled in the second processed sample image from the first form to the second form; According to the second sample image, the second processed sample image and the second restored sample image, the first generator in the first trained generative adversarial network, the second generator in the first trained generative adversarial network and the second discriminator in the first trained generative adversarial network are trained until the trained generative adversarial network meets the preset second training termination condition, thereby obtaining the second trained generative adversarial network; the above steps are repeated until the second trained generative adversarial network meets the preset third training termination condition, thereby obtaining the pre-trained generative adversarial network.

10. The image annotation device according to claim 9, characterized in that: The training unit is specifically configured to perform training on the first generator in the to-be-trained generative adversarial network and the second generator in the to-be-trained generative adversarial network based on the first sample image and the first restored sample image until the difference between the first sample image and the first restored sample image is less than a preset difference threshold; input the first processed sample image to the first discriminator in the to-be-trained generative adversarial network to obtain a first discrimination result, and based on the first discrimination result, train the first generator in the to-be-trained generative adversarial network until the first discrimination result is that the first processed sample image is derived from the second training sample set; The first discriminator in the generative adversarial network to be trained is trained until the first discriminant result is that the first processed sample image does not come from the second training sample set.

11. The image annotation device according to claim 9, characterized in that: The training unit is specifically configured to train the first generator in the first trained generative adversarial network and the second generator in the first trained generative adversarial network based on the second sample image and the second restored sample image until the difference between the second sample image and the second restored sample image is less than a preset difference threshold; Inputting the second processed sample image to the second discriminator in the first trained generative adversarial network to obtain a second discrimination result, and training the second generator in the first trained generative adversarial network based on the second discrimination result until the second discrimination result indicates that the second processed sample image is derived from the first training sample set; The second discriminator in the first trained generative adversarial network is trained until the second discrimination result is that the second processed sample image does not originate from the first training sample set.

12. The image annotation device according to claim 7, characterized in that: The labeling unit is specifically configured to execute the process of taking the area composed of the pixels whose pixel difference is greater than a preset threshold as the target labeling area; and performing labeling processing on the target labeling area to obtain the labeling result.

13. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the image annotation method according to any one of claims 1 to 6.

14. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the image annotation method according to any one of claims 1 to 6.

15. A computer program product, comprising instructions, characterized in that: When the instruction is executed by a processor of an electronic device, the electronic device is enabled to execute the image annotation method according to any one of claims 1 to 6.

Images