Image annotation method, device, equipment and storage medium

Abstract

The present disclosure provides an image annotation method, device, equipment and storage medium, which relate to the field of artificial intelligence technology, and in particular to the technical fields of deep learning, data annotation, natural language processing, large language models and image processing. The specific implementation scheme is: using a preset annotation model to perform initial annotation processing on the target image to be annotated, obtaining the target position information and the description information to be verified of the target object in the target image, inputting the target image and the prompt information into the GPT model, obtaining the object search result, the prompt information is used to prompt the GPT model to search for the target object and / or the fictitious object associated with the target object in the target image according to the target description information, the target description information is determined according to the description information to be verified, if the description information to be verified is determined according to the object search result, the annotation information of the target image is determined according to the description information to be verified and the target position information. This technical solution can improve the accuracy of machine annotation.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to technical fields such as deep learning, data annotation, natural language processing, large language models, and image processing. Background Art

[0002] In recent years, with the rapid increase in the number of images, people are in urgent need of efficient annotation of image content to achieve effective retrieval and management of large-scale images. From the perspective of pattern recognition, the image annotation problem is regarded as assigning a set of labels to images based on their content. How to select appropriate features to represent the image content will greatly affect the annotation performance.

[0003] Currently, there are two main ways to annotate images, including manual annotation and automatic annotation. Summary of the invention

[0004] The present disclosure provides an image annotation method, apparatus, device and storage medium.

[0005] According to one aspect of the present disclosure, there is provided an image annotation method, comprising:

[0006] Performing initial annotation processing on the target image to be annotated using a preset annotation model to obtain target position information and description information to be verified of the target object in the target image;

[0007] Inputting the target image and the target prompt information into a preset GPT model to obtain an object search result, wherein the target prompt information is used to prompt the preset GPT model to search for the target object and / or a fictitious object associated with the target object in the target image according to the target description information, and the target description information is determined according to the description information to be verified;

[0008] When it is determined according to the object search result that the description information to be verified has passed verification, the labeling information of the target image is determined according to the description information to be verified and the target position information.

[0009] According to another aspect of the present disclosure, there is provided an image annotation apparatus, comprising:

[0010] An initial annotation module, used to perform initial annotation processing on the target image to be annotated using a preset annotation model, and obtain target position information and description information to be verified of the target object in the target image;

[0011] A search module, used for inputting the target image and the target prompt information into a preset GPT model to obtain an object search result, wherein the target prompt information is used to prompt the preset GPT model to search the target object and / or a fictitious object associated with the target object in the target image according to the target description information, and the target description information is determined according to the description information to be verified;

[0012] The annotation information determination module is used to determine the annotation information of the target image according to the description information to be verified and the target position information when it is determined according to the object search result that the description information to be verified has passed the verification.

[0013] According to another aspect of the present disclosure, there is provided an electronic device, comprising:

[0014] at least one processor; and

[0015] a memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the embodiment of the present disclosure.

[0017] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method described in the embodiment of the present disclosure.

[0018] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, wherein when the computer program is executed by a processor, the method described in any embodiment of the present disclosure is implemented.

[0019] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0021] Figure 1 is a flowchart of an image annotation method provided according to an embodiment of the present disclosure;

[0022] Figure 2 is a flowchart of another image annotation method provided according to an embodiment of the present disclosure;

[0023] Figure 3is a flowchart of another image annotation method provided according to an embodiment of the present disclosure;

[0024] Figure 4 is a flowchart of another image annotation method provided according to an embodiment of the present disclosure;

[0025] Figure 5 is a schematic diagram of a labeling process of an image labeling method provided according to an embodiment of the present disclosure;

[0026] Figure 6 is a structural schematic diagram of an image annotation device provided according to an embodiment of the present disclosure;

[0027] Figure 7 It is a block diagram of an electronic device used to implement the image annotation method of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0028] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0029] Figure 1 1 is a flowchart of an image annotation method provided according to an embodiment of the present disclosure. The embodiment of the present disclosure can be applied to the case of automatically annotating images. The method can be executed by an image annotation device, which can be implemented in hardware and / or software and can be configured in an electronic device. Figure 1 , the method specifically includes the following:

[0030] S101, performing initial annotation processing on a target image to be annotated using a preset annotation model to obtain target position information and description information to be verified of a target object in the target image;

[0031] S102, inputting the target image and target prompt information into a preset GPT model to obtain an object search result, wherein the target prompt information is used to prompt the GPT model to search for the target object and / or a fictitious object associated with the target object in the target image according to target description information, and the target description information is determined according to the description information to be verified;

[0032] S103: When it is determined according to the object search result that the description information to be verified has passed verification, determine the labeling information of the target image according to the description information to be verified and the target position information.

[0033] Among them, the preset annotation model can be a machine learning model with open domain object positioning and understanding capabilities, for example, it can be a Generative Region-to-text Transformer for Object Understanding (GRIT) model, or a Generative Image-to-text Transformer for Vision and Language (GIT), etc.

[0034] Exemplarily, the target image is an image that needs to be annotated, and the specific source is not limited. The target image can be preliminarily automatically annotated using a preset annotation model, such as dense caption, which aims to locate objects in the image and describe the located objects in a short sentence or phrase. The target object may include objects identified from the target image by the preset annotation model, which may include people, animals, plants, food, buildings, and various other types of objects. The number of target objects may be one or more.

[0035] Exemplarily, the target image to be annotated is initially annotated using a preset annotation model, which may be inputting the target image into a preset annotation model, and the preset annotation model may output the initial position information and initial description information of the target object. The position information in the disclosed embodiment may be represented by a rectangular frame, and the description information may be text information describing the target object. The specific representation method of the rectangular frame is not limited. For example, the left vertex of the target image may be used as the origin, the length direction of the target image may be used as the horizontal axis, and the width direction of the target image may be used as the vertical axis to construct a coordinate system, and the rectangular frame may be characterized by the left vertex horizontal coordinate (x), the left vertex vertical coordinate (y), the width (w) of the rectangular frame, and the height (h) of the rectangular frame, such as box (x, y, w, h). The target position information may be understood as the position information of the target object that needs to be annotated in the target image, and the initial position information may be used as the target position information, or the target position information may be obtained after processing the initial position information. The description information to be verified may be understood as the description information that needs to be further verified, and the initial description information may be used as the description information to be verified, or the description information to be verified may be obtained after processing the initial description information. For a single target object, there may be one target location information and one or more description information to be verified. If there are multiple description information to be verified, each description information to be verified may be verified respectively.

[0036] In the disclosed embodiment, a preset generative pre-trained transformer (GPT) model, that is, a preset GPT model, is used to verify the description information to be verified. The specific type of the preset GPT model is not limited, and it can be a multimodal large language model, such as an LLaVA1.5 model, a Shikra model, or a Qwen model.

[0037] Exemplarily, the target description information is determined according to the description information to be verified. The target description information can be understood as the description information of the object to be searched, and the object to be searched includes the target object and / or a fictitious object associated with the target object. The fictitious object associated with the target object can be understood as an object fictitious based on the target object. The description information of the fictitious object can be obtained by modifying the description information to be verified, and the modification method may include vocabulary replacement, etc. According to the target description information, a prompt information (prompt) for input into the preset GPT model is constructed, for example, "Can you find the 'target description information' in the picture?". The target image and the target prompt information are input into the preset GPT model, and the prompt information is used to prompt the preset GPT model to search for an object that meets the target description information in the target image according to the target description information, that is, to search for the target object and / or a fictitious object associated with the target object. The object search result may include found and not found, and the output of the preset GPT model may be, for example, true (corresponding to found) or false (corresponding to not found).

[0038] Exemplarily, the description information to be verified can be verified according to the object search result, that is, the accuracy of the description information to be verified can be verified using the GPT model. If the verification passes, it can be considered that the description information to be verified is accurate. The annotation information of the target image is determined according to the description information to be verified and the target position information, such as associating the target position information with the corresponding description information to be verified to obtain the annotation information of the target image. In the case where there are multiple target objects in the target image, the annotation information corresponding to each target object can be determined separately.

[0039] The technical solution provided by the embodiment of the present disclosure uses a preset annotation model to perform initial annotation processing on the target image to be annotated, obtains the target position information and the description information to be verified of the target object in the target image, inputs the target image and the target prompt information into the preset GPT model, and obtains the object search result, wherein the prompt information is used to prompt the preset GPT model to search for the target object and / or the fictitious object associated with the target object in the target image according to the target description information, the target description information is determined according to the description information to be verified, and when it is determined according to the object search result that the description information to be verified has passed the verification, the annotation information of the target image is determined according to the description information to be verified and the target position information. By adopting the above technical solution, after the image is automatically and preliminarily annotated using the preset annotation model, the prompt information is constructed according to the description information to be verified, the prompt information is used to instruct the preset GPT model to search for the preliminarily annotated object and / or the fictitious object from the image, and the description information to be verified is verified according to the search result, so as to improve the accuracy of machine annotation.

[0040] Optionally, when it is determined that the description information to be verified has not passed the verification according to the object search result, it is determined that there is an abnormality in the description information to be verified, and abnormal prompt information of the corresponding target position information can be output. Further, after checking the abnormal prompt information, the relevant personnel can correct the description information to be verified through manual intervention and other methods to obtain more accurate target image annotation information.

[0041] In an optional implementation, the target prompt information includes the target position information. The advantage of such a setting is that it prompts the preset GPT model to search for the target object and / or virtual object at a position in the target image corresponding to the target position information, thereby improving the efficiency and accuracy of the output result of the preset GPT model.

[0042] In an optional embodiment, the target image and the target prompt information are input into a preset GPT model to obtain an object search result, including: inputting the target image and the first prompt information into a first GPT model to obtain a first object search result, wherein the first GPT model belongs to the preset GPT model, the first prompt information belongs to the target prompt information, the first prompt information is used to prompt the first GPT model to search for a fictitious object associated with the target object in the target image according to the first description information, the first description information is the description information of the fictitious object, the first description information is obtained by replacing the first word in the description information to be verified with the second word, the first word and the second word have the same part of speech, and the similarity is less than a preset similarity threshold; wherein, if the first object search result is not found, then the first verification result of the description information to be verified is determined to be passed. The advantage of such a setting is that the first word in the description information to be verified is replaced with a second word with the same part of speech and a lower similarity to it, so as to construct the description information of the virtual object, and the GPT model is used to search for the fictitious object. If it is not found, it can be quickly proved that the description of the description information to be verified is more accurate, thereby improving the verification efficiency and accuracy, and further improving the accuracy of image annotation. If the first verification result of the description information to be verified is passed, it is determined that the verification of the description information to be verified is passed.

[0043] Exemplarily, the first vocabulary may be a vocabulary of the target part of speech, and the target part of speech may be, for example, an adjective, or a noun. The preset similarity threshold may be set according to actual needs. For example, if the description information to be verified is "a red apple", the first vocabulary may be "red", and "red" may be replaced by "green" (the second vocabulary), wherein "red" and "green" are both adjectives, have the same part of speech, and have a low similarity. Optionally, the first vocabulary and the second vocabulary may be antonyms, for example, if the description information to be verified is "a tall horse", the first vocabulary may be "tall", and "tall" may be replaced by "short" (the second vocabulary), wherein "tall" and "short" are both adjectives, have the same part of speech, and are antonyms.

[0044] As in the above example, assuming that the first description information is "a green apple" and the first prompt information is "Can you find 'a green apple' in the picture?", if the actual target object in the target image is a red apple, the first GPT model will output false, that is, no green apple can be found, which can prove that the description information to be verified "a red apple" is more accurate, and the first verification result is passed.

[0045] In an optional embodiment, the target image and target prompt information are input into a preset GPT model to obtain an object search result, including: inputting the target image and the second prompt information into a second GPT model to obtain a second object search result, wherein the second GPT model belongs to the preset GPT model, the second prompt information belongs to the target prompt information, and the second prompt information is used to prompt the second GPT model to search for the target object in the target image according to the second description information, and the second description information is the same as the description information to be verified; wherein, if the second object search result is found, then the second verification result of the description information to be verified is determined to be passed. The advantage of such a setting is that the description information to be verified is used as the second description information, and the target object is searched using the GPT model. If it is found, it can be quickly proved that the description of the description information to be verified is more accurate, thereby improving the verification efficiency and accuracy, and further improving the accuracy of image annotation. wherein, if the second verification result of the description information to be verified is passed, then it is determined that the description information to be verified has passed the verification.

[0046] For example, as in the above example, the description information to be verified is "a red apple", the second description information is "a red apple", and the second prompt information is "Can you find 'a red apple' in the picture?" If the real target object in the target image is a red apple, the second GPT model will output true, that is, the red apple is found, which can prove that the description information to be verified "a red apple" is relatively accurate, and the second verification result is passed.

[0047] Optionally, in order to improve the reliability of the verification, the self-learning property of the GPT model in the answering process can be used to repeatedly ask multiple times, and the output results of multiple times can be combined to determine the second verification result. For example, the target image and the second prompt information are input into the second GPT model for a preset number of times, such as 3 times. If the second object search results for the preset number of times are all found, then the second verification result of the description information to be verified can be determined to be passed.

[0048] In an optional implementation, the number of the second GPT models is at least two, and if the second object search results corresponding to the second GPT models are all found, it is determined that the second verification result of the description information to be verified is passed.

[0049] In an optional implementation, determining that the description information to be verified has passed verification according to the object search result includes: if the first verification result and the second verification result of the description information to be verified are both passed, determining that the description information to be verified has passed verification. The advantage of such a setting is that different verification methods are used to verify the description information to be verified, thereby further improving the accuracy of automatic image annotation.

[0050] Exemplarily, the first GPT model may be used to verify the description information to be verified to obtain a first verification result, and then the second GPT model may be used to verify the description information to be verified to obtain a second verification result; or the second GPT model may be used to verify the description information to be verified to obtain a second verification result, and then the first GPT model may be used to verify the description information to be verified to obtain a first verification result. Optionally, if the first verification result is passed, the subsequent verification may be performed. For example, the first verification result is obtained first, and if the first verification result is passed, the second GPT model may be used to verify the description information to be verified to obtain a second verification result, and whether the verification of the description information to be verified has passed may be determined according to the second verification result.

[0051] In an optional embodiment, the method of using a preset annotation model to perform initial annotation processing on the target image to be annotated to obtain the target position information and description information to be verified of the target object in the target image includes: using a preset annotation model to perform initial annotation processing on the target image to be annotated to obtain the initial position information and initial description information of the target object in the target image; inputting the target image and the third prompt information into the third GPT model to obtain the newly added description information of the target object, wherein the third prompt information is used to prompt the third GPT model to search for the target object in the target image according to the initial description information, and output the description information of the target object different from the initial description information; determining the target position information according to the initial position information; and determining the description information to be verified according to the initial description information and the newly added description information. The advantage of such a setting is that the newly added description information is automatically generated according to the target image and the initial description information by using the GPT model as the enhanced description information corresponding to the initial description information, which can improve the annotation success rate, accuracy and richness of the target image.

[0052] Exemplarily, for the description information to be verified determined based on the initial description information and the newly added description information, the first GPT model can be used to perform the aforementioned corresponding verification to obtain a first verification result, or the second GPT model can be used to perform the aforementioned corresponding verification to obtain a second verification result, or the first GPT model and the second GPT model can be used respectively to perform the aforementioned corresponding verification to obtain a first verification result and a second verification result.

[0053] Figure 2 : is a flowchart of another image annotation method provided according to an embodiment of the present disclosure. This embodiment is optimized based on the optional solutions in the above embodiments, and adds relevant steps for generating new description information, which can enrich the description information to be verified. The method may include:

[0054] S201 , performing initial annotation processing on a target image to be annotated using a preset annotation model to obtain initial position information and initial description information of a target object in the target image.

[0055] Exemplarily, the initial position information and the initial description information may be information directly output by the preset annotation model after the target image is input into the preset annotation model. Taking the preset annotation model as the GRIT model as an example, the target image is input into the GRIT model, and the GRIT model may output one or more capitons (initial description information) with boxes (initial position information), and the initial description information may be recorded as capiton_init.

[0056] S202. Input the target image and the third prompt information into the third GPT model to obtain newly added description information of the target object, wherein the third prompt information is used to prompt the third GPT model to search for the target object in the target image according to the initial description information, and output description information of the target object that is different from the initial description information.

[0057] Exemplarily, a third prompt information can be constructed, and the third prompt information can be used to prompt the third GPT model to search for the target object in the target image according to the initial description information, and output description information of the target object that is different from the initial description information, and the description information output by the third GPT model is used as the newly added description information of the target object, thereby enriching the description of the target object. The third prompt information can be, for example, "find the 'initial description information' in the image and provide different description information for it". Optionally, the number of newly added description information can be a preset number, and the preset number can be set according to actual needs, such as 2 or 4, etc., and the preset number can be included in the third prompt information. For example, the third prompt information prompt = "find" + caption_init + "in this image, provide two diverse description for it".

[0058] Exemplarily, the description information output by the third GPT model is recorded as the newly added description information of the target object. For example, the initial description information is "a red apple", and the newly added description information may be "a round apple" and "a big apple".

[0059] Optionally, the third prompt information includes target position information, and the third prompt information is used to prompt the third GPT model to search for the target object in the image area corresponding to the target position information in the target image according to the initial description information, and output description information of the target object that is different from the initial description information. In this way, the search efficiency and accuracy of the target object can be improved, thereby improving the output efficiency of the newly added description information, and ensuring that the initial description information and the newly added description information correspond to the same target object.

[0060] S203: Determine target location information according to the initial location information.

[0061] Exemplarily, the initial position information may be determined as the target position information.

[0062] S204: Determine the description information to be verified according to the initial description information and the newly added description information.

[0063] Exemplarily, both the initial description information and the newly added description information are determined as the description information to be verified, that is, the number of description information to be verified is multiple. By generating newly added description information, the same target location information corresponds to multiple description information to be verified, which can increase the probability of passing the verification, thereby ensuring that the same target location information can correspond to accurate description information, thereby improving the success rate, accuracy and richness of the target image annotation.

[0064] S205: Input the target image and the first prompt information into the first GPT model to obtain a first object search result.

[0065] Optionally, the third GPT model is the same as the first GPT model, for example, both are LLaVA1.5 models. Thus, for the newly added description information, internal calibration of the same GPT model can be achieved, for example, using the LLaVA1.5 model to generate newly added description information for the target object, constructing description information of the fictitious object according to the newly added description information, and using the LLaVA1.5 model to search for the fictitious object in the target image according to the description information of the fictitious object, achieving internal calibration based on the LLaVA1.5 model and improving the accuracy of verification.

[0066] Optionally, the first prompt information includes target position information, and the first prompt information is used to prompt the first GPT model to search for a fictitious object of the target object in an image area corresponding to the target position information in the target image according to the first description information.

[0067] For a plurality of description information to be verified, corresponding first prompt information may be generated according to each description information to be verified, that is, a plurality of first prompt information is generated, and the first prompt information and the description information to be verified are in a one-to-one relationship.

[0068] S206: Determine whether the first object search result is found, if so, execute S210; otherwise, execute S207.

[0069] Exemplarily, if a fictitious object is found in the image area corresponding to the target position information, it means that the current description information to be verified is not accurate enough, and S210 may be executed to determine that the verification has failed.

[0070] S207: Input the target image and the second prompt information into the second GPT model to obtain a second object search result.

[0071] Optionally, the second prompt information includes target position information, and the second prompt information is used to prompt the second GPT model to search for the target object in the image area corresponding to the target position information in the target image according to the second description information. Thus, the verification efficiency and accuracy can be improved.

[0072] Optionally, the number of the second GPT models is at least two, and if the second object search results corresponding to the second GPT models are all found, then the second verification result of the description information to be verified is determined to be passed. Thus, verification using different second GPT models can improve verification accuracy.

[0073] Optionally, the third GPT model is different from the second GPT model, and the two have different object recognition capabilities and / or input content understanding capabilities. Therefore, for the newly added description information, the generation model and the verification model are different. For example, the LLaVA1.5 model is used to generate the newly added description information for the target object, and then a GPT model different from the LLaVA1.5 model, such as the Qwen model, is used to search for the target object in the target image according to the newly added description information, so as to realize external calibration of the newly added description information and improve the verification accuracy and credibility.

[0074] Optionally, the first GPT model is different from the second GPT model, and both have different object recognition capabilities and / or input content understanding capabilities. Thus, different GPT models are used to perform different verification methods on the description information to be verified, further improving the verification accuracy and credibility.

[0075] Exemplarily, the second GPT model may include a Shikra model and a Qwen model. The Shikra model has strong object recognition capability (frame recognition capability), and Qwen has a relatively strong understanding of content input (input content understanding capability). LLaVA1.5 is a compromise between the two, and can generate more diverse outputs. By reasonably selecting the GPT model, utilizing the mutual verification between different multimodal large language models, and making full use of the advantages of different models, the annotation accuracy can be effectively improved.

[0076] S208, determining whether the second object search result is found, if so, executing S209; otherwise, executing S210.

[0077] S209: Determine whether the description information to be verified has been verified, and determine the labeling information of the target image according to the description information to be verified and the target position information.

[0078] Exemplarily, if there are at least two description information to be verified that pass verification, the target location information can be associated with the at least two description information to be verified to obtain the labeling information of the target object corresponding to the target location information, that is, there can be multiple description information in the labeling information of the same target object.

[0079] S210: Determine whether the description information to be verified has failed verification.

[0080] Exemplarily, if it is determined that the current description information to be verified has not passed the verification, it can be determined whether there is any unverified description information to be verified. If so, it can return to execute S205 until all the description information to be verified that needs to be verified is verified.

[0081] The image annotation method provided by the embodiment of the present disclosure obtains the initial position information and initial description information of the target object after automatically and preliminarily annotating the image using a preset annotation model, generates new description information of the target object according to the initial description information using a GPT model, and verifies the initial description information and the new description information in two ways using the GPT model, respectively, to improve the verification accuracy, and can improve the annotation success rate, accuracy, credibility and richness of the target image.

[0082] In an optional embodiment, the method of using a preset annotation model to perform initial annotation processing on the target image to be annotated to obtain the target position information and the description information to be verified of the target object in the target image includes: using a preset annotation model to perform initial annotation processing on the target image to be annotated to obtain the initial position information and the initial description information of the target object in the target image; for the target initial position information that meets the preset fusion conditions, performing a first fusion processing on the target initial position information, and performing a second fusion processing on the initial description information corresponding to the target initial position information; determining the target position information and the description information to be verified of the target object in the target image according to the fusion processing result. The advantage of such a setting is that before verifying the description information, the initial position information and the initial description information output by the preset annotation model are first fused, thereby reducing the duplication or high similarity annotations in the initial annotations and improving the accuracy of the annotations.

[0083] Exemplarily, for the description information to be verified determined according to the fusion processing results, the first GPT model can be used to perform the aforementioned corresponding verification to obtain a first verification result, or the second GPT model can be used to perform the aforementioned corresponding verification to obtain a second verification result, or the first GPT model and the second GPT model can be used respectively to perform the aforementioned corresponding verification to obtain a first verification result and a second verification result.

[0084] Figure 3This is a flowchart of another image annotation method provided according to an embodiment of the present disclosure. This embodiment is optimized based on the optional solutions in the above embodiments, and adds relevant steps for fusing the initial annotation results, which can improve the accuracy of the annotation. The method may include:

[0085] S301 , performing initial annotation processing on a target image to be annotated using a preset annotation model to obtain initial position information and initial description information of a target object in the target image.

[0086] S302: For target initial position information that meets preset fusion conditions, perform a first fusion process on the target initial position information, and perform a second fusion process on initial description information corresponding to the target initial position information.

[0087] The preset fusion condition can be set according to the actual situation. Different initial position information and initial description information may correspond to target objects with similar attributes. For example, there may be multiple apples on a fruit stand at the same time, and each apple may be identified as a target object. In order to improve the labeling efficiency and accuracy, the initial position information and initial description information corresponding to multiple apples can be fused.

[0088] Optionally, the initial position information is represented by a rectangular frame, and the target initial position information that meets the preset fusion condition is determined in the following manner: extracting a target vocabulary of a preset part of speech from the initial description information; for each of the initial position information, constructing a sample to be determined based on the current initial position information and the target vocabulary in the corresponding initial description information; for each pair of samples to be determined, calculating the intersection-and-union ratio of the rectangular frame areas corresponding to the two initial position information in the current pair of samples to be determined, if the intersection-and-union ratio is greater than a preset threshold and the target vocabulary corresponding to the same preset part of speech in the current pair of samples to be determined is the same, then the two initial position information are determined as the target initial position information that meets the preset fusion condition. The advantage of this setting is that it can accurately determine the initial annotation results that need to be merged.

[0089] Exemplarily, the preset parts of speech may include nouns and adjectives. For example, for a red apple, { <red> , <apple>}. Optionally, if there are multiple nouns and adjectives in the initial description information, all of them can be extracted or the main words can be extracted. For example, for a red apple on the table, { <red> , <apple>The initial description information corresponding to the target image can generate several adjective and noun pairs. Using the adjective and noun pairs and the corresponding initial position information (box), the corresponding samples to be determined can be constructed to obtain multiple samples to be determined. The multiple samples to be determined can exist in the form of a set or a list. For example, adj represents an adjective, n represents a noun, and a list [{<adj.1> ,<n.1> ,box1},{adj.2>,<n.2> ,box2}…].

[0090] Exemplarily, the above list is looped, and every two samples to be determined are determined as a sample pair to be determined. For each sample pair to be determined, the intersection over union (IOU) of the two boxes is calculated. If the IOU is greater than a preset value (preset threshold) and the corresponding adj and n are the same, it can be considered that fusion is required, that is, the preset fusion condition is met.

[0091] Optionally, before constructing the sample to be determined, it may also include: determining the category of the target object according to the target vocabulary, and if the determined category is a preset category, determining the initial description information to which the target vocabulary belongs as the initial position information of the sample to be determined. Accordingly, for each of the initial position information, constructing the sample to be determined according to the current initial position information and the target vocabulary in the corresponding initial description information, including: for each of the initial position information of the sample to be determined, constructing the sample to be determined according to the current initial position information and the target vocabulary in the corresponding initial description information. Among them, the preset category can be set according to actual needs, such as food and people.

[0092] Optionally, the initial position information is represented by a rectangular frame; the first fusion processing of the target initial position information includes: determining a minimum rectangular frame that includes the rectangular frame corresponding to the target initial position information; wherein, determining the target position information of the target object in the target image according to the fusion processing result includes: determining the minimum rectangular frame as the target position information of the target object in the target image. The advantage of such a setting is that the fused rectangular frame can accurately represent the position information of multiple target objects before fusion.

[0093] Exemplarily, the initial position information of the two targets to be fused is recorded as box1 and box2 respectively, and the fused rectangular box is recorded as box_new, box_new = [min(box1.x, box2.x), min(box1.y, box2.y), w_new, h_new), where x represents the horizontal coordinate of the left vertex, y represents the vertical coordinate of the left vertex, w represents the width of the rectangular box, h represents the height of the rectangular box, w_new is the width of the union of box1 and box2, and h_new is the height of the union of box1 and box2.

[0094] Optionally, the initial description information corresponding to the target initial position information is subjected to a second fusion process, including: comparing the initial description information corresponding to the target initial position information, and determining the initial description information with the richest semantics as the fused description information; wherein, the description information to be verified of the target object in the target image is determined according to the fusion process result, including: determining the fused description information as the description information to be verified. For example, the initial description information to be fused is subjected to semantic segmentation process respectively, and a corresponding plurality of candidate words (such as nouns and adjectives) are obtained, and the initial description information with the largest number of candidate words is determined as the fused description information.

[0095] S303: Determine the target position information and the description information to be verified of the target object in the target image according to the fusion processing result.

[0096] Exemplarily, for initial position information that does not meet preset fusion conditions, the initial position information may be determined as target position information of the target object, and the corresponding initial description information may be determined as description information to be verified of the target object.

[0097] Optionally, the rectangular frame size corresponding to the fused initial position information is determined according to the fusion processing result, the fused initial position information whose rectangular frame size is smaller than a preset size threshold is filtered out, and the fused initial position information remaining after the filtering processing is determined as the target position information of the target object.

[0098] S304. Input the target image and target prompt information into the GPT model to obtain an object search result, wherein the prompt information is used to prompt the GPT model to search for the target object and / or a fictitious object associated with the target object in the target image according to the target description information, and the target description information is determined according to the description information to be verified.

[0099] For example, the specific verification process can refer to the relevant content in the previous text, which will not be repeated here.

[0100] S305 . When it is determined according to the object search result that the description information to be verified has passed verification, the labeling information of the target image is determined according to the description information to be verified and the target position information.

[0101] The image annotation method provided by the embodiment of the present disclosure obtains the initial position information and initial description information of the target object after automatically and preliminarily annotating the image using a preset annotation model, and fuses the initial position information and the initial description information to reduce duplication in the initial annotation or the annotation of objects with high similarity, thereby improving the accuracy of the annotation.

[0102] Figure 4 is a flow chart of another image annotation method provided according to an embodiment of the present disclosure. This embodiment is optimized based on the optional solutions in the above embodiments. Figure 5 is a schematic diagram of the annotation process of an image annotation method provided according to an embodiment of the present disclosure, which can be combined with Figure 5 The method may include:

[0103] S401 , performing initial annotation processing on a target image to be annotated using a preset annotation model to obtain initial position information and initial description information of a target object in the target image.

[0104] like Figure 5 As shown, taking the preset annotation model as the GRIT model as an example, the target image is input into the GRIT model, and the GRIT model can output a box and a capiton.

[0105] S402: For target initial position information that meets preset fusion conditions, perform a first fusion process on the target initial position information, and perform a second fusion process on initial description information corresponding to the target initial position information.

[0106] S403: Determine target position information and first description information to be verified of the target object in the target image according to the fusion processing result.

[0107] S404: Input the target image and the third prompt information into the third GPT model to obtain newly added description information of the target object.

[0108] Among them, the third prompt information is used to prompt the third GPT model to search for the target object in the target image according to the first description information to be verified, and output description information of the target object that is different from the first description information to be verified.

[0109] Exemplarily, if the initial position information and the initial description information are fused before generating the newly added description information, the third prompt information can be constructed based on the fused position information (target position information) and description information (first description information to be verified).

[0110] like Figure 5 As shown, the third GPT model is used to generate enhanced capiton, that is, to add new description information.

[0111] S405: Determine the second description information to be verified according to the newly added description information.

[0112] S406: Input the target image and the first prompt information into the first GPT model to obtain a first object search result.

[0113] Among them, the first prompt information is used to prompt the first GPT model to search for a fictional object of the target object in the target image according to the first description information, wherein the first description information is the description information of the fictional object, and the first description information is obtained by replacing the first word in the description information to be verified (the first description information to be verified or the second description information to be verified) with the second word, the first word and the second word have the same part of speech, and the similarity is less than a preset similarity threshold.

[0114] For example, Figure 5 As shown, the description information to be verified is verified using the third GPT model.

[0115] S407 , determining whether the first object search result is found, if so, executing S411 ; otherwise, executing S408 .

[0116] S408: Input the target image and the second prompt information into the second GPT model to obtain a second object search result.

[0117] Among them, the second prompt information is used to prompt the second GPT model to search for the target object in the target image according to the second description information, and the second description information is the same as the description information to be verified (the first description information to be verified or the second description information to be verified).

[0118] For example, Figure 5 As shown, if GPT1 passes the verification, two different second GPT models are used to verify the description information to be verified.

[0119] S409, determining whether the second object search result is found, if so, executing S410; otherwise, executing S411.

[0120] S410: Determine whether the description information to be verified has been verified, and determine the labeling information of the target image according to the description information to be verified and the target position information.

[0121] like Figure 5 As shown, if two different second GPT models pass the verification, the description information to be verified can be output as the annotation information of the target image.

[0122] Exemplarily, if there are at least two description information to be verified that pass verification, the target location information can be associated with the at least two description information to be verified to obtain the labeling information of the target object corresponding to the target location information.

[0123] S411, determining that the description information to be verified has failed verification.

[0124] The image annotation method provided by the embodiment of the present disclosure obtains the initial position information and initial description information of the target object after automatically and preliminarily annotating the image by using a preset annotation model, fuses the initial position information and the initial description information, reduces the duplication in the initial annotation or the annotation of objects with high similarity, improves the accuracy of the annotation, uses the GPT model to generate new description information of the target object according to the fused description information, uses the GPT model to verify the fused description information and the new description information in two ways, respectively, to improve the verification accuracy, and can improve the annotation success rate, accuracy, credibility and richness of the target image.

[0125] Figure 6 1 is a schematic diagram of the structure of an image annotation device provided according to an embodiment of the present disclosure. The embodiment of the present disclosure can be applied to the case of automatically annotating images. The device can be implemented in hardware and / or software and can be configured in an electronic device. Figure 6 , the image annotation device 600 comprises:

[0126] An initial annotation module 601 is used to perform initial annotation processing on a target image to be annotated using a preset annotation model, and obtain target position information and description information to be verified of a target object in the target image;

[0127] A search module 602 is used to input the target image and the target prompt information into a preset GPT model to obtain an object search result, wherein the target prompt information is used to prompt the preset GPT model to search the target object and / or a fictitious object associated with the target object in the target image according to the target description information, and the target description information is determined according to the description information to be verified;

[0128] The annotation information determination module 603 is used to determine the annotation information of the target image according to the description information to be verified and the target position information when it is determined according to the object search result that the description information to be verified has passed the verification.

[0129] The image annotation device provided by the embodiment of the present disclosure, after automatically and preliminarily annotating the image using a preset annotation model, constructs prompt information according to the description information to be verified, uses the prompt information to instruct the GPT model to search for the preliminarily annotated objects and / or fictitious objects from the image, and verifies the description information to be verified according to the search results, thereby improving the accuracy of machine annotation.

[0130] In an optional implementation, the search module includes:

[0131] a first search unit, configured to input the target image and the first prompt information into a first GPT model to obtain a first object search result, wherein the first GPT model belongs to the preset GPT model, the first prompt information belongs to the target prompt information, the first prompt information is used to prompt the first GPT model to search for a fictitious object associated with the target object in the target image according to first description information, the first description information is description information of the fictitious object, the first description information is obtained by replacing a first word in the description information to be verified with a second word, the first word has the same part of speech as the second word, and the similarity is less than a preset similarity threshold;

[0132] If the first object search result is not found, it is determined that the first verification result of the description information to be verified is passed.

[0133] In an optional implementation, the search module includes:

[0134] A second search unit is used to input the target image and the second prompt information into a second GPT model to obtain a second object search result, wherein the second GPT model belongs to the preset GPT model, the second prompt information belongs to the target prompt information, and the second prompt information is used to prompt the second GPT model to search for the target object in the target image according to the second description information, and the second description information is the same as the description information to be verified;

[0135] If the second object search result is found, it is determined that the second verification result of the description information to be verified is passed.

[0136] In an optional implementation manner, determining, according to the object search result, that the description information to be verified has passed verification includes:

[0137] If both the first verification result and the second verification result of the description information to be verified are passed, it is determined that the verification of the description information to be verified is passed.

[0138] In an optional implementation, the initial marking module includes:

[0139] A first initial annotation unit, configured to perform initial annotation processing on a target image to be annotated using a preset annotation model, and obtain initial position information and initial description information of a target object in the target image;

[0140] A newly added description generating unit is used to input the target image and the third prompt information into a third GPT model to obtain newly added description information of the target object, wherein the third prompt information is used to prompt the third GPT model to search for the target object in the target image according to the initial description information, and output description information of the target object that is different from the initial description information;

[0141] a target position determining unit, configured to determine target position information according to the initial position information;

[0142] The description determining unit is used to determine the description information to be verified according to the initial description information and the newly added description information.

[0143] In an optional implementation, the initial marking module includes:

[0144] A second initial annotation unit, configured to perform initial annotation processing on the target image to be annotated using a preset annotation model, and obtain initial position information and initial description information of the target object in the target image;

[0145] A fusion processing unit, configured to perform a first fusion processing on the target initial position information that meets a preset fusion condition, and perform a second fusion processing on the initial description information corresponding to the target initial position information;

[0146] The determination unit is used to determine the target position information and the description information to be verified of the target object in the target image according to the fusion processing result.

[0147] In an optional implementation manner, the initial position information is represented by a rectangular frame, and the target initial position information satisfying the preset fusion condition is determined by:

[0148] Extracting a target word with a preset part of speech from the initial description information;

[0149] For each of the initial position information, construct a sample to be determined according to the current initial position information and the target vocabulary in the corresponding initial description information;

[0150] For each sample pair to be determined, the intersection-and-union ratio of the rectangular frame areas corresponding to the two initial position information in the current sample pair to be determined is calculated. If the intersection-and-union ratio is greater than a preset threshold and the target words corresponding to the same preset part of speech in the current sample pair to be determined are the same, then the two initial position information are determined as the target initial position information that meets the preset fusion condition.

[0151] In an optional implementation manner, the initial position information is represented by a rectangular frame; and the fusion processing unit is specifically used to:

[0152] Determine a minimum rectangular frame that includes the rectangular frame corresponding to the target initial position information;

[0153] Wherein, when determining the target position information of the target object in the target image according to the fusion processing result, the fusion unit is specifically used to:

[0154] The minimum rectangular frame is determined as target position information of the target object in the target image.

[0155] In an optional embodiment, at least one of the following is satisfied:

[0156] The third GPT model is the same as the first GPT model;

[0157] The number of the second GPT models is at least two, and if the second object search results corresponding to the second GPT models are all found, determining that the second verification result of the description information to be verified is passed;

[0158] The third GPT model and the second GPT model have different object recognition capabilities and / or input content understanding capabilities.

[0159] In an optional implementation, the target prompt information includes the target location information.

[0160] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0161] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0162] Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0163] like Figure 7 As shown, the device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0164] A number of components in the device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a disk, an optical disk, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0165] The computing unit 701 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 701 performs the various methods and processes described above, such as image annotation methods. For example, in some embodiments, the image annotation method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 700 via ROM 702 and / or communication unit 709. When the computer program is loaded into RAM 703 and executed by the computing unit 701, one or more steps of the image annotation method described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform the image annotation method in any other appropriate manner (e.g., by means of firmware).

[0166] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0167] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0168] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0169] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0170] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0171] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services. The server may also be a server of a distributed system, or a server combined with a blockchain.

[0172] Artificial intelligence is a discipline that studies how to use computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.). It includes both hardware-level and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies mainly include computer vision technology, speech recognition technology, natural language processing technology, machine learning / deep learning technology, big data processing technology, knowledge graph technology, and other major directions.

[0173] Cloud computing refers to a technology system that uses network access to elastically scalable shared physical or virtual resource pools. Resources can include servers, operating systems, networks, software, applications, and storage devices, and can be deployed and managed on demand and in a self-service manner. Cloud computing technology can provide efficient and powerful data processing capabilities for technical applications such as artificial intelligence and blockchain, as well as model training.

[0174] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution provided by this disclosure can be achieved, and this document does not limit this.

[0175] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.< / apple> < / red> < / apple> < / red>

Claims

1. An image annotation method, comprising: Performing initial annotation processing on the target image to be annotated using a preset annotation model to obtain target position information and description information to be verified of the target object in the target image; The target image and the first prompt information are input into the first GPT model to obtain a first object search result. If the first object search result is not found, the first verification result of the description information to be verified is determined to be passed; wherein the first GPT model belongs to a preset GPT model, and the first prompt information belongs to target prompt information; the first prompt information is used to prompt the first GPT model to search for a fictitious object associated with the target object in the target image according to the first description information; the first description information belongs to target description information, the first description information is the description information of the fictitious object, the first description information is obtained by replacing the first word in the description information to be verified with the second word, the first word has the same part of speech as the second word, and the similarity is less than a preset similarity threshold; wherein the target prompt information is used to prompt the preset GPT model to search for the target object and / or the fictitious object associated with the target object in the target image according to the target description information, and the target description information is determined according to the description information to be verified; Input the target image and the second prompt information into the second GPT model to obtain a second object search result, and if the second object search result is found, determine that the second verification result of the description information to be verified is passed; wherein the second GPT model belongs to the preset GPT model, and the second prompt information belongs to the target prompt information; the second prompt information is used to prompt the second GPT model to search for the target object in the target image according to the second description information, the second description information belongs to the target description information, and the second description information is the same as the description information to be verified; If both the first verification result and the second verification result of the descriptive information to be verified are passed, it is determined that the descriptive information to be verified has passed the verification; when it is determined that the descriptive information to be verified has passed the verification according to the object search result, the annotation information of the target image is determined according to the descriptive information to be verified and the target position information.

2. The method according to claim 1, wherein: The method of using a preset annotation model to perform initial annotation processing on the target image to be annotated to obtain target position information and description information to be verified of the target object in the target image includes: Performing initial annotation processing on the target image to be annotated using a preset annotation model to obtain initial position information and initial description information of the target object in the target image; Inputting the target image and the third prompt information into the third GPT model to obtain newly added description information of the target object, wherein the third prompt information is used to prompt the third GPT model to search for the target object in the target image according to the initial description information, and output description information of the target object that is different from the initial description information; Determine target location information according to the initial location information; Determine the description information to be verified according to the initial description information and the newly added description information.

3. The method according to any one of claims 1, wherein: The method of using a preset annotation model to perform initial annotation processing on the target image to be annotated to obtain target position information and description information to be verified of the target object in the target image includes: Performing initial annotation processing on the target image to be annotated using a preset annotation model to obtain initial position information and initial description information of the target object in the target image; For target initial position information that meets a preset fusion condition, performing a first fusion process on the target initial position information, and performing a second fusion process on initial description information corresponding to the target initial position information; The target position information and the description information to be verified of the target object in the target image are determined according to the fusion processing result.

4. The method according to claim 3, wherein: The initial position information is represented by a rectangular frame, and the target initial position information that meets the preset fusion condition is determined by the following method: Extracting a target word with a preset part of speech from the initial description information; For each of the initial position information, construct a sample to be determined according to the current initial position information and the target vocabulary in the corresponding initial description information; For each sample pair to be determined, the intersection-and-union ratio of the rectangular frame areas corresponding to the two initial position information in the current sample pair to be determined is calculated. If the intersection-and-union ratio is greater than a preset threshold and the target words corresponding to the same preset part of speech in the current sample pair to be determined are the same, then the two initial position information are determined as the target initial position information that meets the preset fusion condition.

5. The method according to claim 3, wherein: The initial position information is represented by a rectangular frame; the first fusion processing of the target initial position information includes: Determine a minimum rectangular frame that includes the rectangular frame corresponding to the target initial position information; Wherein, determining the target position information of the target object in the target image according to the fusion processing result includes: The minimum rectangular frame is determined as target position information of the target object in the target image.

6. The method according to claim 2, wherein at least one of the following is satisfied: The third GPT model is the same as the first GPT model; The number of the second GPT models is at least two, and if the second object search results corresponding to the second GPT models are all found, determining that the second verification result of the description information to be verified is passed; The third GPT model and the second GPT model have different object recognition capabilities and / or input content understanding capabilities.

7. The method according to claim 1, wherein: The target prompt information includes the target location information.

8. An image annotation device, comprising: An initial annotation module, used to perform initial annotation processing on the target image to be annotated using a preset annotation model, and obtain target position information and description information to be verified of the target object in the target image; A search module, comprising a first search unit and a second search unit; The first search unit is used to input the target image and the first prompt information into the first GPT model to obtain a first object search result. If the first object search result is not found, it is determined that the first verification result of the description information to be verified is passed; wherein, the first GPT model belongs to a preset GPT model, and the first prompt information belongs to target prompt information; the first prompt information is used to prompt the first GPT model to search for a fictitious object associated with the target object in the target image according to the first description information; the first description information belongs to target description information, the first description information is the description information of the fictitious object, the first description information is obtained by replacing the first word in the description information to be verified with the second word, the first word and the second word have the same part of speech, and the similarity is less than a preset similarity threshold; wherein, the target prompt information is used to prompt the preset GPT model to search for the target object and / or the fictitious object associated with the target object in the target image according to the target description information, and the target description information is determined according to the description information to be verified; The second search unit is used to input the target image and the second prompt information into the second GPT model to obtain a second object search result, and if the second object search result is found, it is determined that the second verification result of the description information to be verified is passed; wherein the second GPT model belongs to the preset GPT model, and the second prompt information belongs to the target prompt information; the second prompt information is used to prompt the second GPT model to search for the target object in the target image according to the second description information, the second description information belongs to the target description information, and the second description information is the same as the description information to be verified; A labeling information determination module, configured to determine the labeling information of the target image according to the description information to be verified and the target position information, when it is determined according to the object search result that the description information to be verified has passed verification; The step of determining whether the description information to be verified has passed verification according to the object search result includes: if both the first verification result and the second verification result of the description information to be verified have passed verification, determining that the description information to be verified has passed verification.

9. The device according to claim 8, wherein: The initial annotation module comprises: A first initial annotation unit, configured to perform initial annotation processing on a target image to be annotated using a preset annotation model, and obtain initial position information and initial description information of a target object in the target image; A newly added description generating unit is used to input the target image and the third prompt information into a third GPT model to obtain newly added description information of the target object, wherein the third prompt information is used to prompt the third GPT model to search for the target object in the target image according to the initial description information, and output description information of the target object that is different from the initial description information; a target position determining unit, configured to determine target position information according to the initial position information; The description determining unit is used to determine the description information to be verified according to the initial description information and the newly added description information.

10. The device according to claim 8, wherein: The initial annotation module comprises: A second initial annotation unit, configured to perform initial annotation processing on the target image to be annotated using a preset annotation model, and obtain initial position information and initial description information of the target object in the target image; A fusion processing unit, configured to perform a first fusion processing on the target initial position information that meets a preset fusion condition, and perform a second fusion processing on the initial description information corresponding to the target initial position information; The determination unit is used to determine the target position information and the description information to be verified of the target object in the target image according to the fusion processing result.

11. The device according to claim 10, wherein: The initial position information is represented by a rectangular frame, and the target initial position information that meets the preset fusion condition is determined by the following method: Extracting a target word with a preset part of speech from the initial description information; For each of the initial position information, construct a sample to be determined according to the current initial position information and the target vocabulary in the corresponding initial description information; For each sample pair to be determined, the intersection-and-union ratio of the rectangular frame areas corresponding to the two initial position information in the current sample pair to be determined is calculated. If the intersection-and-union ratio is greater than a preset threshold and the target words corresponding to the same preset part of speech in the current sample pair to be determined are the same, then the two initial position information are determined as the target initial position information that meets the preset fusion condition.

12. The device according to claim 10, wherein: The initial position information is represented by a rectangular frame; the fusion processing unit is specifically used to perform the first fusion processing on the target initial position information: Determine a minimum rectangular frame that includes the rectangular frame corresponding to the target initial position information; Wherein, when determining the target position information of the target object in the target image according to the fusion processing result, the fusion processing unit is specifically used to: The minimum rectangular frame is determined as target position information of the target object in the target image.

13. The device according to claim 9, wherein at least one of the following is satisfied: The third GPT model is the same as the first GPT model; The number of the second GPT models is at least two, and if the second object search results corresponding to the second GPT models are all found, determining that the second verification result of the description information to be verified is passed; The third GPT model and the second GPT model have different object recognition capabilities and / or input content understanding capabilities.

14. The device according to claim 8, wherein: The target prompt information includes the target location information.

15. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

16. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-7.

Images