Training method, device, equipment and storage medium for attribute recognition model

Abstract

The embodiment of the present application discloses a training method, device, equipment and storage medium for an attribute recognition model. The method includes: obtaining a first training sample; calling an attribute recognition model, visually encoding each candidate target contained in the first training image in the first training sample, and obtaining the visual encoding features of each candidate target; text encoding each attribute word contained in the text description information of the first training image, and obtaining the text encoding features of each attribute word; based on the visual encoding features of each candidate target and the text encoding features of each attribute word, determining the matching results of each candidate target and each attribute word; training the attribute recognition model in the direction of reducing the difference between the matching results of each determined candidate target and each attribute word and the matching labels of the marked corresponding candidate target and the corresponding attribute word, and obtaining the trained attribute recognition model. The embodiment of the present application can realize the attribute recognition task for open sets.

Description

Technical Field

[0001] The present application relates to the field of computer application technology, and in particular to a training method, device, equipment and storage medium for an attribute recognition model. Background Art

[0002] The description of target attributes is not only conducive to the accuracy of target recognition, but also enables a clearer and more accurate understanding of the target. The diversity of attributes in nature brings great difficulties to attribute labeling. Early work mainly focused on target attribute recognition in fixed fields, such as clothes, shoes, animals, faces, etc., which would limit the number of attribute categories. However, if attribute recognition is extended from a fixed single field to the entire field, the number of attributes in the existing data set is far from enough to achieve the arbitrary description of objects in the real world, and large-scale attribute labeling of different targets is extremely resource-intensive. Based on this, how to train the attribute recognition model to ensure that the trained attribute recognition model can achieve open-set attribute recognition tasks is a technical problem that needs to be solved urgently. Summary of the invention

[0003] The embodiments of the present application provide a training method, apparatus, device and storage medium for an attribute recognition model, which can realize open-set oriented attribute recognition tasks.

[0004] On the one hand, an embodiment of the present application provides a method for training an attribute recognition model, the method comprising:

[0005] Acquire a first training sample; wherein the first training sample includes image-text pair data, and the image-text pair data includes a first training image and text description information of the first training image;

[0006] Calling the attribute recognition model to perform visual coding on each candidate target included in the first training image to obtain a visual coding feature of each candidate target;

[0007] Performing text encoding on each attribute word included in the text description information to obtain text encoding features of each attribute word;

[0008] Based on the visual coding features of the candidate objects and the text coding features of the attribute words, determining the matching results of the candidate objects and the attribute words; wherein the matching results are used to indicate whether the candidate objects and the attribute words have a matching relationship;

[0009] The attribute recognition model is trained in the direction of reducing the difference between the matching results of the determined candidate targets and the attribute words and the matching labels of the marked corresponding candidate targets and the corresponding attribute words to obtain a trained attribute recognition model; wherein the trained attribute recognition model is used to perform attribute recognition on the image to be recognized.

[0010] In one embodiment, the method further comprises:

[0011] Acquire a second training sample, where the second training sample includes a second training image and an attribute annotation label of the second training image, where the attribute label is used to indicate a target attribute word of a target contained in the second training image;

[0012] Based on the attribute annotation label, a partial image is obtained by cropping from the second training image; wherein the partial image includes the target described by the target attribute word indicated by the attribute label;

[0013] Calling a pre-trained attribute recognition model to perform visual encoding on the local image to obtain visual encoding features of the local image;

[0014] Performing text encoding on the target attribute word to obtain text encoding features of the target attribute word;

[0015] Based on the visual coding features of the partial image and the text coding features of the target attribute word, obtaining a matching probability between the partial image and the target attribute word;

[0016] The pre-trained attribute recognition model is optimized with the matching probability being higher than a first probability threshold as an optimization goal to obtain the attribute recognition model.

[0017] In one embodiment, the step of cropping the second training image to obtain a partial image based on the attribute label includes:

[0018] Slicing the second training image to obtain a candidate image; wherein the candidate image at least includes the target described by the attribute word indicated by the attribute tag, and the size of the candidate image is larger than the size of the detection box of the target in the second training image;

[0019] The candidate image is randomly cropped to obtain the partial image; wherein the size of the partial image is smaller than the size of the candidate image, and the partial image contains part or all of the content of the target described by the target attribute word.

[0020] In one embodiment, the pre-trained attribute recognition model includes a visual encoder and a text encoder, the visual encoder is used to perform visual encoding on the local image, and the text encoder is used to perform text encoding on the target attribute word;

[0021] The optimizing the pre-trained attribute recognition model by taking the matching probability higher than the first probability threshold as the optimization target to obtain the attribute recognition model includes:

[0022] Taking the matching probability being higher than a first probability threshold as an optimization goal, the text encoder is optimized to obtain the attribute recognition model, and the attribute recognition model includes the visual encoder and the optimized text encoder.

[0023] In one embodiment, obtaining the matching probability between the local image and the attribute word indicated by the attribute annotation tag based on the visual encoding feature of the local image and the text encoding feature of the target attribute word includes:

[0024] Acquire the similarity between the visual coding feature of the local image and the text coding feature of the target attribute word;

[0025] Based on the similarity, a matching probability between the partial image and the target attribute word is obtained; wherein the matching probability between the partial image and the target attribute word is positively correlated with the similarity.

[0026] In one embodiment, the method further comprises:

[0027] Calling the attribute recognition model to perform visual encoding on the first training image to obtain visual encoding features of the first training image;

[0028] Performing text encoding on the text description information to obtain text encoding features of the text description information;

[0029] Determining a matching result between the first training image and the text description information based on the visual coding features of the first training image and the text coding features of the text description information;

[0030] The attribute recognition model is trained in a direction of reducing the difference between the matching results of the determined candidate targets and the attribute words and the matching labels of the marked corresponding candidate targets and the corresponding attribute words to obtain the trained attribute recognition model, including:

[0031] The attribute recognition model is trained in a direction of reducing the difference between the determined matching result of the first training image and the text description information and the preset matching label of the first training image and the text description information, and the difference between the matching result of each candidate target and each attribute word and the matching label of the marked corresponding candidate target and the corresponding attribute word to obtain the trained attribute recognition model; wherein the preset matching label of the first training image and the text description information is used to indicate that the first training image and the text description information have a matching relationship.

[0032] In one embodiment, the method further comprises:

[0033] Calling the attribute recognition model to perform visual encoding on the target unit image to obtain visual encoding features of the target unit image; wherein the target unit image refers to a unit image corresponding to a target detection frame in the first training image, and the target detection frame refers to: a detection frame with the largest size among the detection frames of at least one candidate target contained in the first training image;

[0034] Determining a matching result between the target unit image and each attribute word based on a visual coding feature of the target unit image and a text coding feature of each attribute word;

[0035] The attribute recognition model is trained in a direction of reducing the difference between the matching results of the determined candidate targets and the attribute words and the matching labels of the marked corresponding candidate targets and the corresponding attribute words to obtain the trained attribute recognition model, including:

[0036] The attribute recognition model is trained in the direction of reducing the difference between the determined matching results of the target unit image and the respective attribute words and the preset matching labels of the target unit image and the respective attribute words, as well as the difference between the matching results of the respective candidate targets and the respective attribute words and the marked matching labels of the corresponding candidate targets and the corresponding attribute words, to obtain the trained attribute recognition model; wherein the preset matching labels of the target unit image and the respective attribute words are used to indicate that the target unit image and the respective attribute words have a matching relationship.

[0037] In one embodiment, the method further comprises:

[0038] Calling the attribute recognition model to perform visual encoding on the target unit image to obtain visual encoding features of the target unit image; wherein the target unit image refers to a unit image corresponding to a target detection frame in the first training image, and the target detection frame refers to: a detection frame with the largest size among the detection frames of at least one candidate target contained in the first training image;

[0039] Performing text encoding on each noun phrase included in the text description information to obtain text encoding features of each noun phrase;

[0040] Determining matching results between the target unit image and each noun phrase based on visual coding features of the target unit image and text coding features of each noun phrase;

[0041] The attribute recognition model is trained in a direction of reducing the difference between the matching results of the determined candidate targets and the attribute words and the matching labels of the marked corresponding candidate targets and the corresponding attribute words to obtain the trained attribute recognition model, including:

[0042] The attribute recognition model is trained in the direction of reducing the difference between the determined matching results of the target unit image and the respective noun phrases and the preset matching labels of the target unit image and the respective noun phrases, and the difference between the matching results of the respective candidate targets and the respective attribute words and the marked matching labels of the corresponding candidate targets and the corresponding attribute words to obtain the trained attribute recognition model; wherein the preset matching labels of the target unit image and the respective noun phrases are used to indicate that the target unit image and the respective noun phrases have a matching relationship.

[0043] In one embodiment, the method further comprises:

[0044] Calling the attribute recognition model to obtain the matching probability of each candidate target and each attribute word based on the visual coding feature of each candidate target and the text coding feature of each attribute word;

[0045] The matching probabilities of the candidate targets and the attribute words are compared with a second probability threshold, and matching labels of the candidate targets and the attribute words are obtained according to the comparison results.

[0046] On the other hand, an embodiment of the present application provides a training device for an attribute recognition model, the training device for the attribute recognition model comprising:

[0047] An acquisition unit, configured to acquire a first training sample; wherein the first training sample includes image-text pair data, and the image-text pair data includes a first training image and text description information of the first training image;

[0048] A visual encoding unit, used to call an attribute recognition model to perform visual encoding on each candidate target included in the first training image to obtain a visual encoding feature of each candidate target;

[0049] A text encoding unit, used to perform text encoding on each attribute word included in the text description information to obtain text encoding features of each attribute word;

[0050] A determination unit, configured to determine a matching result between each candidate target and each attribute word based on the visual coding feature of each candidate target and the text coding feature of each attribute word; wherein the matching result is used to indicate whether each candidate target and each attribute word have a matching relationship;

[0051] The model training unit is used to train the attribute recognition model in the direction of reducing the difference between the matching results of each determined candidate target and each attribute word and the matching labels of the marked corresponding candidate target and the corresponding attribute word, so as to obtain a trained attribute recognition model; wherein the trained attribute recognition model is used to perform attribute recognition on the image to be recognized.

[0052] On the other hand, an embodiment of the present application provides a computer device, including a processor, a storage device, and a communication interface, wherein the processor, the storage device, and the communication interface are connected to each other, wherein the storage device is used to store a computer program that supports the computer device to execute the above method, the computer program includes program instructions, and the processor is configured to call the program instructions to perform the following steps:

[0053] Acquire a first training sample; wherein the first training sample includes image-text pair data, and the image-text pair data includes a first training image and text description information of the first training image;

[0054] Calling the attribute recognition model to perform visual coding on each candidate target included in the first training image to obtain a visual coding feature of each candidate target;

[0055] Performing text encoding on each attribute word included in the text description information to obtain text encoding features of each attribute word;

[0056] Based on the visual coding features of the candidate objects and the text coding features of the attribute words, determining the matching results of the candidate objects and the attribute words; wherein the matching results are used to indicate whether the candidate objects and the attribute words have a matching relationship;

[0057] The attribute recognition model is trained in the direction of reducing the difference between the matching results of the determined candidate targets and the attribute words and the matching labels of the marked corresponding candidate targets and the corresponding attribute words to obtain a trained attribute recognition model; wherein the trained attribute recognition model is used to perform attribute recognition on the image to be recognized.

[0058] On the other hand, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. The computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes the above-mentioned attribute recognition model training method.

[0059] On the other hand, an embodiment of the present application provides a computer program product, which includes a computer program, and the computer program is suitable for being loaded by a processor and executing the above-mentioned attribute recognition model training method.

[0060] In an embodiment of the present application, weakly supervised training can be performed using image-text pair data. Specifically, a first training sample is obtained, and the first training sample includes image-text pair data. The image-text pair data includes a first training image and text description information of the first training image. Then, an attribute recognition model is called to visually encode each candidate target contained in the first training image to obtain a visual encoding feature of each candidate target. Text encoding is performed on each attribute word contained in the text description information to obtain a text encoding feature of each attribute word. Based on the visual encoding feature of each candidate target and the text encoding feature of each attribute word, a matching result of each candidate target and each attribute word is determined. The matching result is used to indicate whether each candidate target and each attribute word have a matching relationship. The attribute recognition model is trained in a direction of reducing the difference between the matching results of each determined candidate target and each attribute word and the matching labels of the marked corresponding candidate target and the corresponding attribute word to obtain a trained attribute recognition model. It can be seen that the embodiment of the present application extracts a finer-grained target-attribute matching relationship from the coarse-grained matching relationship between images and texts, and then introduces noise learning to obtain a good representation of the attributes. In other words, the trained attribute recognition model obtained by the embodiment of the present application can realize open-set oriented attribute recognition tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0062] Figure 1 It is a structural diagram of a training system for an attribute recognition model provided in an embodiment of the present application;

[0063] Figure 2 It is a flowchart of a method for training an attribute recognition model provided in an embodiment of the present application;

[0064] Figure 3It is a structural diagram of another attribute recognition model training system provided in an embodiment of the present application;

[0065] Figure 4 It is a flowchart of another method for training an attribute recognition model provided in an embodiment of the present application;

[0066] Figure 5 It is a flowchart of another method for training an attribute recognition model provided in an embodiment of the present application;

[0067] Figure 6 It is a structural schematic diagram of a training device for an attribute recognition model provided in an embodiment of the present application;

[0068] Figure 7 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0069] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0070] The description of target attributes is not only conducive to the accuracy of target recognition, but also enables a clearer and more accurate understanding of the target. That is, learning the attribute representation of the target in the field of computer vision can not only supplement the information loss caused by the category description alone, but also help the intelligent agent deploying artificial intelligence algorithms to better understand the real world. Therefore, attribute recognition should be extended from a fixed single field to the whole field. For example, for a target, in addition to describing the category it belongs to, there should also be attributes such as color, material, texture, and shape in practical applications. For example, for a tree, it can be tall or short, green or withered. However, the existing attribute recognition model can only recognize the attributes that already exist in the database, but the number of attributes already in the database is far from reaching the arbitrary description of objects in the real world, and large-scale attribute labeling of different targets is extremely resource-consuming.

[0071] Based on this, an embodiment of the present application provides a training method for an attribute recognition model, obtains a first training sample, the first training sample includes image-text pair data, the image-text pair data includes a first training image and text description information of the first training image, then calls the attribute recognition model, visually encodes each candidate target contained in the first training image, and obtains the visual encoding features of each candidate target, and textually encodes each attribute word contained in the text description information to obtain the text encoding features of each attribute word, and determines the matching results of each candidate target and each attribute word based on the visual encoding features of each candidate target and the text encoding features of each attribute word, and the matching results are used to indicate whether each candidate target and each attribute word have a matching relationship, and trains the attribute recognition model in the direction of reducing the difference between the matching results of each determined candidate target and each attribute word and the matching labels of the marked corresponding candidate targets and corresponding attribute words to obtain a trained attribute recognition model. The embodiment of the present application can use the image and text to perform weakly supervised training on the data, extract a finer-grained target-attribute matching relationship from the coarse-grained matching relationship between the image and text, and then introduce noise learning to obtain a good representation of the attribute. In other words, the trained attribute recognition model obtained by the embodiment of the present application can realize the attribute recognition task for the open set. In other words, the trained attribute recognition model in the embodiment of the present application can recognize attributes that do not appear in the database, that is, the trained attribute recognition model can determine whether a target has any visual attribute.

[0072] The training method of the attribute recognition model provided in the embodiment of the present application can be applied in a computer device, and the computer device may include a terminal device or a server, etc. The computer device includes but is not limited to a smart phone, a camera, a wearable device or a computer, etc. It is understandable that the computer device used to train the attribute recognition model and the computer device that calls the trained attribute recognition model can be the same computer device or different computer devices. The computer device calls the trained attribute recognition model to perform attribute recognition on the image to be recognized, and the recognized attributes can be applied to the scene of target recognition, such as identifying whether there is a driver who is making a phone call or smoking in a traffic image, and whether there are objects or people that endanger public safety in a captured image.

[0073] See also Figure 1 , Figure 1It is a structural diagram of a training system for an attribute recognition model provided in an embodiment of the present application. First, a first training sample can be obtained, wherein the first training sample may include image-text pair data, and the image-text pair data may include a first training image 101 and text description information 102 of the first training image. Furthermore, the first training image 101 can be subjected to target recognition by an object detector to obtain at least one candidate target 103 contained in the first training image 101, and the text description information 102 can also be segmented to obtain at least one attribute word 104 contained in the text description information 102. Then, at least one candidate target 103 is input into a visual encoder (Visual Encoder), and each candidate target is visually encoded by the visual encoder to obtain visual encoding features of each candidate target. At least one attribute word 104 is input into a text encoder (Text Encoder), and each attribute word is text-encoded by the text encoder to obtain text encoding features of each attribute word. Then, based on the visual coding features of each candidate target and the text coding features of each attribute word, the matching results of each candidate target and each attribute word can be determined, and the matching results are used to indicate whether each candidate target and each attribute word have a matching relationship. The attribute recognition model is trained in the direction of reducing the difference between the determined matching results of each candidate target and each attribute word and the marked matching labels of the corresponding candidate target and the corresponding attribute word to obtain a trained attribute recognition model.

[0074] based on Figure 1 The training system for the attribute recognition model shown in Figure 2 , Figure 2 is a flow chart of a method for training an attribute recognition model provided in an embodiment of the present application. The method for training an attribute recognition model can be executed by a computer device; Figure 2 The training scheme of the attribute recognition model shown includes but is not limited to steps S201 to S205, wherein:

[0075] S201, obtaining a first training sample, where the first training sample includes image-text pair data, where the image-text pair data includes a first training image and text description information of the first training image.

[0076] In one implementation, at least one existing image-text pair data may be obtained from a database, and the obtained at least one image-text pair data may be used as a first training sample. For example, the image-text pair data in the database may be derived from COCO (Common Objects in COntext), which is a data set provided by the Microsoft team that can be used for image recognition. The COCO data set may include at least one image-text pair data, and each image-text pair data includes object instances (target instances, i.e., the first training image in the embodiment of the present application) and image captions (picture description, i.e., text description information in the embodiment of the present application).

[0077] In another implementation, the first training samples can be obtained by manual annotation, for example, first obtaining at least one first training image from a memory, or downloading at least one first training image from the Internet, and then manually annotating each first training image to obtain text description information of each first training image.

[0078] S202, calling the attribute recognition model to perform visual coding on each candidate target included in the first training image to obtain a visual coding feature of each candidate target.

[0079] In one example, the attribute recognition model in the embodiment of the present application may include a CLIP model or an ALIGN model, etc. Among them, the full name of the CLIP model is "Contrastive Language-Image Pretraining", which means using contrastive language-image pretraining. The basic idea of ​​the CLIP model is to let the machine learn the potential semantic structure from text and images. It establishes a model that compares and connects the concepts in the image and text corpus, so that the machine gradually learns and masters the "correct" relationship between images and texts. After the model is trained, the entire system can process new text and image instances with higher accuracy, such as the correspondence between images and texts, semantic analysis and other issues. Large-scale Image and Noisy-text embedding (ALIGN) model, the ALIGN model is a model that uses large-scale noisy image text data to expand the learning of visual and visual language representations. The ALIGN model can search cross-modally from image to text, text to image, and even jointly search for image + text queries.

[0080] In the embodiment of the present application, the CLIP model or the ALIGN model first uses the visual encoder and the text encoder to extract the features of the scene image and the text description respectively, and then learns based on the noise contrast estimation loss (InfoNCE loss). Thanks to the support of up to 100 million images and texts for data, this pre-trained model (i.e., the CLIP model or the ALIGN model) has a strong zero-sample classification capability for both categories and attributes, but the effect of directly applying the model to attribute recognition is not satisfactory. Based on this, the embodiment of the present application uses the zero-sample classification capability of the CLIP model or the ALIGN model to extract a finer-grained target-attribute matching relationship from the coarse-grained matching relationship of the image and text, and then introduces noise learning to obtain a good representation of the attributes, thereby enabling the open set-oriented attribute recognition task.

[0081] In another example, the attribute recognition model in the embodiment of the present application may refer to Figure 4 The attribute recognition model is obtained by training the attribute recognition model according to the training method of the attribute recognition model shown in FIG. Figure 5 The description of the embodiments of the present application will not be repeated here.

[0082] In one implementation, after the first training sample is acquired, a target detector may be used to perform target recognition on the first training image to obtain at least one candidate target.

[0083] Exemplarily, you can use target detection data, such as COCO, to train a category-independent target detector. The detector follows the design of Faster RCNN, but changes its classification head to binary classification, that is, it only classifies the background and the target, and does not give a specific category name. The category-independent target detector can have a certain recall ability for all targets in the first training image. In order to obtain the target in each first training image, we use the target detector to perform forward calculations on the images in COCO Caption to obtain the targets contained in each first training image. Optionally, in order to ensure the accuracy of the target, the top 20 targets with a probability greater than 0.8 can be taken as candidate targets that can be extracted from the first training image.

[0084] In one implementation, the attribute recognition model may include a visual encoder, which may input a unit image corresponding to at least one candidate target contained in the first training image into the visual encoder, and then perform feature extraction on the unit image corresponding to each candidate target through the visual encoder to obtain the visual coding features of each candidate target.

[0085] S203, performing text encoding on each attribute word included in the text description information to obtain a text encoding feature of each attribute word.

[0086] In one implementation, after the first training sample is obtained, part-of-speech tagging and word segmentation may be performed on the text description information to obtain at least one attribute word, and then text encoding may be performed on each attribute word to obtain text encoding features of each attribute word.

[0087] For example, the open source Part-of-Speech Tagging tool TextBlob can be used to split the text description information. For example, for a text description information, the sentence itself (c), noun phrase (p), and attribute word (a) can be obtained. For example, the text description information "Astriped zebra is eating greengrass" can be processed into c: Astriped zebra is eating green grass; p: striped zebra, green grass; a: striped, green.

[0088] In one implementation, the attribute recognition model may further include a text encoder, and each attribute word may be input into the text encoder, and then the text encoder may perform text encoding on each attribute word to obtain a text encoding feature of each attribute word.

[0089] For example, in order to obtain the text encoding of attribute words for attribute classification, each attribute word can be filled into a pre-designed template to adapt to the input of the text encoder. For example, 16 learnable prompt vectors can be designed, and then the attribute words are inserted into the middle of these 16 prompt vectors, and then this compound sentence is sent to the text encoder to extract the text encoding features of each attribute word.

[0090] S204, based on the visual coding features of each candidate target and the text coding features of each attribute word, determine the matching results of each candidate target and each attribute word, and the matching results are used to indicate whether each candidate target and each attribute word have a matching relationship.

[0091] Specifically, for any candidate target among at least one candidate target included in the first training image and any attribute word among at least one attribute word included in the text description information of the first training image, the matching result between any candidate target and any attribute word can be determined through an attribute recognition model based on the visual coding features of any candidate target and the text coding features of any attribute word, and the matching result is used to indicate whether any candidate target and any attribute word have a matching relationship. For example, assuming that the first training image includes M candidate targets and the text description information of the first training image includes N attribute words, then the matching result between the mth candidate target and the nth attribute word can be determined through an attribute recognition model based on the visual coding features of the mth candidate target and the text coding features of the nth attribute word, where m and n are both positive integers, 1≤m≤M, 1≤n≤N, and finally M*N matching results can be obtained.

[0092] S205 , training the attribute recognition model in a direction of reducing the difference between the determined matching results of each candidate target and each attribute word and the marked matching labels of the corresponding candidate target and the corresponding attribute word, to obtain a trained attribute recognition model.

[0093] Exemplarily, pseudo-labeling supervision can be used to obtain a BCE loss value, and the attribute recognition model can be trained in the direction of reducing the BCE loss value to obtain a trained attribute recognition model. Specifically, the difference between the matching results of each determined candidate target and each attribute word and the matching labels of the labeled corresponding candidate target and the corresponding attribute word can be obtained, and the BCE loss value can be determined based on the difference, wherein the BCE loss value is positively correlated with the sum of the differences between the matching results of each determined candidate target and each attribute word and the matching labels of the labeled corresponding candidate target and the corresponding attribute word.

[0094] The trained attribute recognition model is used to perform attribute recognition on the image to be recognized. For example, the trained attribute recognition model can not only recognize that the image to be recognized contains a tree, but also recognize that the tree is tall, green, and so on.

[0095] In one implementation, an attribute recognition model can be called to obtain the matching probability of each candidate target and each attribute word based on the visual encoding features of each candidate target and the text encoding features of each attribute word, and the matching probability of each candidate target and each attribute word is compared with a second probability threshold, and matching labels of each candidate target and each attribute word are annotated according to the comparison results.

[0096] Specifically, any candidate target in at least one candidate target and any attribute word in at least one attribute word may have a matching relationship or may not have a matching relationship. For example, the candidate target "dog" and the attribute word "withered and yellow" do not have a matching relationship, but the candidate target "tree" and the attribute word "withered and yellow" have a matching relationship. Then the attribute recognition model can be called to obtain the matching probability of each candidate target and each attribute word based on the visual encoding features of each candidate target and the text encoding features of each attribute word. If the matching probability of a candidate target and a certain attribute word is greater than the second probability threshold, it is considered that the candidate target and the attribute word are matched, and the matching label of the candidate target and the attribute word can be marked as a positive match, and the matching label is used to indicate that the candidate target and the attribute word have a matching relationship; if the matching probability of a candidate target and a certain attribute word is less than or equal to the second probability threshold, it is considered that the candidate target and the attribute word are not matched, and the matching label of the candidate target and the attribute word can be marked as a negative match, and the matching label is used to indicate that the candidate target and the attribute word do not have a matching relationship.

[0097] For example, the attribute recognition model can be called to perform pseudo-label annotation, for example, to obtain the probability of each candidate target containing each attribute word, and to regard attribute words with a probability greater than 0.8, or attribute words with a probability greater than 0.55 and existing in the text description information as positive attributes, and other attribute words as negative attributes. The matching label of the candidate target and the positive attribute is a positive match, and the matching label of the candidate target and the negative attribute is a negative match.

[0098] Optionally, the similarity between the visual coding features of any candidate target and the text coding features of any attribute word can be obtained, and the matching probability of any candidate target and any attribute word can be obtained based on the similarity. The matching probability of any candidate target and any attribute word is positively correlated with the similarity between the visual coding features of any candidate target and the text coding features of any attribute word, that is, the higher the similarity between the visual coding features of any candidate target and the text coding features of any attribute word, the greater the matching probability of any candidate target and any attribute word.

[0099] Exemplarily, the similarity between the visual encoding features of the candidate target and the text encoding features of the attribute word can be calculated by the following formula (1).

[0100]

[0101] Among them, s can represent the similarity between the visual encoding features of the candidate target and the text encoding features of the attribute word, σ refers to the Sigmoid function, v can represent the visual encoding features of the candidate target, for example, it can be the normalized visual encoding features, a can represent the text encoding features of the attribute word, for example, it can be the normalized text encoding features, and τ can be used to adjust the similarity, for example, it can be set to 100.

[0102] In one implementation, the attribute recognition model can also be called to perform visual encoding on the first training image to obtain the visual encoding features of the first training image, perform text encoding on the text description information to obtain the text encoding features of the text description information, and determine the matching result of the first training image and the text description information based on the visual encoding features of the first training image and the text encoding features of the text description information. Then, the attribute recognition model is trained in the direction of reducing the difference between the determined matching result of the first training image and the text description information and the preset matching labels of the first training image and the text description information, and the difference between the matching results of each candidate target and each attribute word and the matching labels of the marked corresponding candidate target and the corresponding attribute word, to obtain the trained attribute recognition model. Among them, the preset matching labels of the first training image and the text description information are used to indicate that the first training image and the text description information have a matching relationship.

[0103] In the embodiment of the present application, for the matching of the first training image and the text description information, the noise contrast learning method of CLIP can be followed, and the infoNCE loss can be used for supervision. Among them, since the first training image and the text description information are image-text pair data, the first training image and the text description information are matched, so the preset matching label of the first training image and the text description information is used to indicate that the first training image and the text description information have a matching relationship. Based on this, the attribute recognition model can be called to determine the matching result of the first training image and the text description information based on the visual coding features of the first training image and the text coding features of the text description information. The matching result is used to indicate whether the first training image and the text description information have a matching relationship, and then the infoNCE loss is obtained based on the matching result and the preset matching label of the first training image and the text description information. For example, the smaller the difference between the matching result and the preset matching label of the first training image and the text description information, the smaller the infoNCE loss; the greater the difference between the matching result and the preset matching label of the first training image and the text description information, the greater the infoNCE loss. The smaller the infoNCE loss is, the more accurate the matching result between the first training image and the text description information is; the larger the infoNCE loss is, the less accurate the matching result between the first training image and the text description information is.

[0104] For example, in order to obtain the text encoding of the text description information, the text description information may be directly input into a text encoder to extract text encoding features of the text description information.

[0105] In one implementation, the attribute recognition model can also be called to perform visual encoding on the target unit image to obtain the visual encoding features of the target unit image, where the target unit image refers to the unit image corresponding to the target detection frame in the first training image, and the target detection frame refers to: the largest detection frame in the detection frame of at least one candidate target contained in the first training image. Then, based on the visual encoding features of the target unit image and the text encoding features of each attribute word, the matching results of the target unit image and each attribute word are determined. Then, the attribute recognition model can be trained in the direction of reducing the difference between the matching results of the determined target unit image and each attribute word and the preset matching labels of the target unit image and each attribute word, and the difference between the matching results of each candidate target and each attribute word and the marked matching labels of the corresponding candidate target and the corresponding attribute word, to obtain the trained attribute recognition model. Among them, the preset matching labels of the target unit image and each attribute word are used to indicate that the target unit image and each attribute word have a matching relationship.

[0106] In the embodiment of the present application, for the matching of the target unit image and each attribute word, since the target unit image and each attribute word have a matching relationship, the MIL-NCE loss can be used for supervision. For example, the smaller the difference between the determined matching result of the target unit image and each attribute word and the preset matching label of the target unit image and the corresponding attribute word, the smaller the MIL-NCE loss; the greater the difference between the determined matching result of the target unit image and each attribute word and the preset matching label of the target unit image and the corresponding attribute word, the greater the MIL-NCE loss. The smaller the MIL-NCE loss, the more accurate the matching result of the determined target unit image and each attribute word; the greater the MIL-NCE loss, the less accurate the matching result of the determined target unit image and each attribute word.

[0107] Exemplarily, the MIL-NCE loss can be calculated by the following formula (2).

[0108]

[0109] Among them, L MIL-NCEIt can represent the MIL-NCE loss, that is, the sum of the differences between the matching results of the target unit image and each attribute word and the matching labels of the preset target unit image and the corresponding attribute words. v can represent the visual encoding features of the target unit image, t can represent the text encoding features of the attribute words, τ is an adjustment factor, P is a positive match, P indicates that the target unit image and the attribute word have a matching relationship, N is a negative match, N indicates that the target unit image and the attribute word do not have a clearly given matching relationship.

[0110] In one implementation, the attribute recognition model can also be called to perform visual encoding on the target unit image to obtain the visual encoding features of the target unit image, where the target unit image refers to the unit image corresponding to the target detection frame in the first training image, and the target detection frame refers to: the largest detection frame in the detection frame of at least one candidate target contained in the first training image, and then the noun phrases contained in the text description information can be text-encoded to obtain the text encoding features of each noun phrase, and the matching results of the target unit image and each noun phrase are determined based on the visual encoding features of the target unit image and the text encoding features of each noun phrase. Then, the attribute recognition model is trained in the direction of reducing the difference between the matching results of the determined target unit image and each noun phrase and the preset matching labels of the target unit image and each noun phrase, and the difference between the matching results of each candidate target and each attribute word and the marked matching labels of the corresponding candidate target and the corresponding attribute word, to obtain the trained attribute recognition model. Among them, the preset matching labels of the target unit image and each noun phrase are used to indicate that the target unit image and each noun phrase have a matching relationship.

[0111] For example, in order to obtain the text encoding of noun phrases, each noun phrase can be filled into a pre-designed template to adapt to the input of the text encoder. For example, 8 learnable prompt vectors can be designed, and then the noun phrase is inserted in the middle of these 8 prompt vectors, and then the compound sentence is sent to the text encoder to extract the text encoding features of each noun phrase.

[0112] In the embodiment of the present application, for the matching of the target unit image and each noun phrase, since the target unit image and each noun phrase have a matching relationship, the MIL-NCE loss can be used for supervision. For example, the smaller the difference between the determined matching result of the target unit image and each noun phrase and the preset matching label of the target unit image and the corresponding noun phrase, the smaller the MIL-NCE loss; the greater the difference between the determined matching result of the target unit image and each noun phrase and the preset matching label of the target unit image and the corresponding noun phrase, the greater the MIL-NCE loss. The smaller the MIL-NCE loss, the more accurate the matching result of the determined target unit image and each noun phrase; the greater the MIL-NCE loss, the less accurate the matching result of the determined target unit image and each noun phrase.

[0113] It can be understood that: in the process of training the attribute recognition model, one or more of the above-mentioned training methods can be combined. For example, the attribute recognition model can be trained in the direction of reducing the difference between the matching result of the determined first training image and the text description information and the matching label of the preset first training image and the text description information, the difference between the matching result of each candidate target and the attribute words and the matching label of the marked corresponding candidate target and the corresponding attribute words, and the difference between the matching result of the determined target unit image and each noun phrase and the matching label of the preset target unit image and each noun phrase, to obtain a trained attribute recognition model. For another example, the attribute recognition model can be trained in the direction of reducing the difference between the matching result of the determined first training image and the text description information and the preset matching label of the first training image and the text description information, the difference between the matching result of each candidate target and the attribute words and the matching label of the marked corresponding candidate target and the corresponding attribute word, the difference between the matching result of the determined target unit image and each noun phrase and the matching label of the preset target unit image and each noun phrase, and the difference between the matching result of the determined target unit image and each noun phrase and the matching label of the preset target unit image and the corresponding noun phrase, to obtain the trained attribute recognition model.

[0114] Optionally, the text encoder for text encoding attribute words, the text encoder for text encoding text description information, and the text encoder for text encoding noun phrases can be the same text encoder or different text encoders, which is not limited by the embodiments of the present application.

[0115] In the case where the attribute recognition model includes a text encoder and a visual encoder, the specific process of training the attribute recognition model may be: adjusting the parameters of the text encoder and the parameters of the visual encoder respectively.

[0116] Through the training method of the above attribute recognition model, it is possible to effectively obtain beneficial attribute representations in these noisy data and improve the open set attribute classification performance for the real world.

[0117] In an embodiment of the present application, a first training sample is obtained, the first training sample includes image-text pair data, the image-text pair data includes a first training image and text description information of the first training image, and then an attribute recognition model is called to visually encode each candidate target contained in the first training image to obtain a visual encoding feature of each candidate target, and text encoding is performed on each attribute word contained in the text description information to obtain a text encoding feature of each attribute word. Based on the visual encoding feature of each candidate target and the text encoding feature of each attribute word, a matching result of each candidate target and each attribute word is determined, and the matching result is used to indicate whether each candidate target and each attribute word have a matching relationship. The attribute recognition model is trained in the direction of reducing the difference between the matching results of each determined candidate target and each attribute word and the matching labels of the marked corresponding candidate target and the corresponding attribute word to obtain a trained attribute recognition model, which can realize open set-oriented attribute recognition tasks.

[0118] See also Figure 3 , Figure 3 It is a structural diagram of a training system for another attribute recognition model provided in an embodiment of the present application. First, a second training sample can be obtained, wherein the second training sample can include a second training image 301 and an attribute annotation label 302 of the second training image, and the attribute label is used to indicate the target attribute word of the target contained in the second training image. Furthermore, a local image 303 can be cropped from the second training image based on the attribute annotation label. Then, a pre-trained attribute recognition model is called to visually encode the local image to obtain a visual encoding feature v of the local image. In addition, the target attribute word can be text-encoded to obtain a text encoding feature of the target attribute word. Then, based on the visual encoding feature of the local image and the text encoding feature of the target attribute word, the matching probability of the local image and the target attribute word is obtained. Then, with the matching probability being higher than the first probability threshold as the optimization goal, the pre-trained attribute recognition model is optimized to obtain an attribute recognition model.

[0119] Optionally, the attribute recognition model may include a visual encoder, and the method of visually encoding the local image may be: extracting features of the local image through the visual encoder to obtain visual encoding features of the local image.

[0120] Optionally, the attribute recognition model may include a text encoder (Text Encoder), and the method of performing text encoding on the target attribute word may be: performing feature extraction on the target attribute word through the text encoder to obtain text encoding features of the target attribute word.

[0121] Optionally, after obtaining the matching probability between the local image and the target attribute word, the loss value L can be obtained based on the matching probability and the first probability threshold. cls For example, the ratio of the first probability threshold and the matching probability can be used as the loss value. If the matching probability is higher than the first probability threshold, the ratio of the first probability threshold and the matching probability is less than 1. In other words, the smaller the loss value, the greater the matching probability between the local image and the target attribute word obtained by the attribute recognition model, and the more accurate the attribute recognition model.

[0122] Optionally, when the attribute recognition model includes a text encoder and a visual encoder, the specific process of training the attribute recognition model may be: adjusting the parameters of the text encoder.

[0123] Based on the above Figure 3 For a description of Figure 4 , Figure 4 is a flow chart of another attribute recognition model training method provided in an embodiment of the present application, and the attribute recognition model training method can be executed by a computer device; Figure 4 The training scheme of the attribute recognition model shown includes but is not limited to steps S401 to S406, wherein:

[0124] S401, obtaining a second training sample, where the second training sample includes a second training image and an attribute annotation label of the second training image, where the attribute label is used to indicate a target attribute word of a target contained in the second training image.

[0125] In one implementation, the existing second training image and the attribute annotation labels of the second training image can be obtained from the database, and the obtained attribute annotation labels of the second training image and the second training image are used as the first training sample. For example, the attribute annotation labels of the second training image and the second training image in the database can be derived from VAW, wherein VAW includes the attribute annotations in 621, which can be directly used for the training of the attribute recognition model.

[0126] In another implementation, the second training samples can be obtained by manual annotation, for example, first obtaining at least one second training image from a memory, or downloading at least one second training image from the Internet, and then manually annotating each second training image to obtain an attribute annotation label for each second training image.

[0127] S402: Based on the attribute annotation label, a partial image is obtained by cropping from the second training image, where the partial image includes a target described by a target attribute word indicated by the attribute label.

[0128] In one implementation, the second training image can be sliced ​​to obtain a candidate image, which at least contains the target described by the attribute word indicated by the attribute label, and the size of the candidate image is larger than the size of the detection box of the target in the second training image. The candidate image is then randomly cropped to obtain a local image, which has a smaller size than the candidate image and contains part or all of the content of the target described by the target attribute word.

[0129] For example, the local image can be obtained by cropping from the second training image using the following formula (3).

[0130] Crop(I,b i )=RandomCrop(RandomScaleCrop(I,b i ,α)β) Formula (3)

[0131] Among them, formula (3) can be expressed as cutting out the target b from the image I i In the process, first b i The detection frame is randomly enlarged by 0 to α times, cropped from image I, and then sub-slices of β to 1 times are randomly cropped from the obtained image slices for training. For example, α can be 0.2 and β can be 0.8.

[0132] In the embodiment of the present application, since the detection frame of the target is the smallest rectangular frame of the target, the detection frame of the target is enlarged and then randomly cropped. The cropped local image may contain part of the target. Because target occlusion may occur in existing target recognition scenarios, this cropping method has good robustness against occluded targets and can significantly improve the diversity of targets.

[0133] S403, calling a pre-trained attribute recognition model to perform visual encoding on the local image to obtain visual encoding features of the local image.

[0134] For example, the pre-trained attribute recognition model may be CLIP. After acquiring the local image, the local image may be scaled to a size of 224×244 and then sent to the visual encoder of CLIP to obtain the visual coding features of the local image, that is, to obtain the visual coding features of the target.

[0135] S404, performing text encoding on the target attribute word to obtain text encoding features of the target attribute word.

[0136] For example, in order to obtain the text encoding of the attribute word for attribute classification, the target attribute word can be filled into a pre-designed template to adapt to the input of the text encoder. For example, 16 learnable prompt vectors can be designed, and then the attribute word is inserted into the middle of these 16 prompt vectors, and then this compound sentence is sent to the text encoder to extract the text encoding features of the target attribute word.

[0137] S405 , based on the visual coding features of the local image and the text coding features of the target attribute word, obtain the matching probability between the local image and the target attribute word.

[0138] In one implementation, the similarity between the visual coding features of the local image and the text coding features of the target attribute word can be obtained, and based on the similarity, the matching probability of the local image and the target attribute word is obtained, and the matching probability of the local image and the target attribute word is positively correlated with the similarity. That is, the higher the similarity between the visual coding features of the local image and the text coding features of the target attribute word, the greater the matching probability of the local image and the target attribute word.

[0139] Exemplarily, the similarity between the visual encoding features of the local image and the text encoding features of the target attribute word can be calculated using the above formula (1).

[0140] S406, taking the matching probability higher than the first probability threshold as the optimization goal, optimizing the pre-trained attribute recognition model to obtain an attribute recognition model.

[0141] Optionally, after obtaining the matching probability between the local image and the target attribute word, the loss value L can be obtained based on the matching probability and the first probability threshold. cls For example, the ratio of the first probability threshold and the matching probability can be used as the loss value. If the matching probability is higher than the first probability threshold, the ratio of the first probability threshold and the matching probability is less than 1. In other words, the smaller the loss value, the greater the matching probability between the local image and the target attribute word obtained by the attribute recognition model, and the more accurate the attribute recognition model.

[0142] In one implementation, a pre-trained attribute recognition model may include a visual encoder and a text encoder. The visual encoder is used to visually encode a local image, and the text encoder is used to textually encode a target attribute word. Based on this, the text encoder can be optimized with a matching probability higher than a first probability threshold as the optimization goal to obtain an attribute recognition model. The attribute recognition model includes a visual encoder and an optimized text encoder.

[0143] Exemplarily, the pre-trained attribute recognition model can be CLIP. The embodiment of the present application loads the pre-trained CLIP, extracts the visual coding features of the local image and the text coding features of the target attribute words by the method provided in the embodiment of the present application, and fine-tunes the CLIP based on the visual coding features and the text coding features. During fine-tuning, in order to obtain stronger open vocabulary capabilities, only the text encoder part can be fine-tuned, and the parameters of the visual encoder can be frozen. Through this fine-tuning, a CLIP model that is more suitable for open set attribute recognition can be obtained. Because the attributes of each target example are supervised in this training process, we use a multi-label classification loss function (binary cross entropy loss, BCE) for training.

[0144] In an embodiment of the present application, a second training sample is obtained, the second training sample includes a second training image and an attribute annotation label of the second training image, the attribute label is used to indicate a target attribute word of a target contained in the second training image, based on the attribute annotation label, a local image is cropped from the second training image, the local image includes a target described by the target attribute word indicated by the attribute label, a pre-trained attribute recognition model is called, the local image is visually encoded to obtain a visual encoding feature of the local image, the target attribute word is textually encoded to obtain a text encoding feature of the target attribute word, based on the visual encoding feature of the local image and the text encoding feature of the target attribute word, the matching probability of the local image and the target attribute word is obtained, the pre-trained attribute recognition model is optimized with the matching probability being higher than a first probability threshold as the optimization goal, to ensure that an attribute recognition model that is more suitable for open set attribute recognition is obtained.

[0145] See also Figure 5 , Figure 5 is a flow chart of another attribute recognition model training method provided in an embodiment of the present application, and the attribute recognition model training method can be executed by a computer device; Figure 5 The training scheme of the attribute recognition model shown includes but is not limited to steps S501 to S511, wherein:

[0146] S501, obtaining a second training sample, where the second training sample includes a second training image and an attribute annotation label of the second training image, where the attribute label is used to indicate a target attribute word of a target contained in the second training image.

[0147] S502: Based on the attribute annotation label, a partial image is obtained by cropping from the second training image, where the partial image includes a target described by a target attribute word indicated by the attribute label.

[0148] S503, calling a pre-trained attribute recognition model to perform visual encoding on the local image to obtain visual encoding features of the local image.

[0149] S504: Perform text encoding on the target attribute word to obtain text encoding features of the target attribute word.

[0150] S505 , based on the visual coding features of the local image and the text coding features of the target attribute word, obtain the matching probability between the local image and the target attribute word.

[0151] S506, taking the matching probability higher than the first probability threshold as the optimization goal, optimizing the pre-trained attribute recognition model to obtain an attribute recognition model.

[0152] The steps S501 to S506 in the embodiment of the present application can refer to the relevant descriptions of the steps S401 to S406 in the above embodiment, and will not be described in detail in the embodiment of the present application.

[0153] S507: Acquire a first training sample, where the first training sample includes image-text pair data, where the image-text pair data includes a first training image and text description information of the first training image.

[0154] S508, calling the attribute recognition model to perform visual coding on each candidate target included in the first training image to obtain a visual coding feature of each candidate target.

[0155] S509: Perform text encoding on each attribute word included in the text description information to obtain a text encoding feature of each attribute word.

[0156] S510, based on the visual coding features of each candidate target and the text coding features of each attribute word, determine the matching results of each candidate target and each attribute word, and the matching results are used to indicate whether each candidate target and each attribute word have a matching relationship.

[0157] S511, training the attribute recognition model in a direction of reducing the difference between the determined matching results of each candidate target and each attribute word and the marked matching labels of the corresponding candidate target and the corresponding attribute word to obtain a trained attribute recognition model.

[0158] The steps S507 to S511 in the embodiment of the present application can refer to the relevant descriptions of the steps S201 to S205 in the above embodiment, and will not be repeated in the embodiment of the present application.

[0159] The above steps S501 to S506 can only have good recognition capabilities for attributes that appear in the data set. The recognition capabilities for other unknown attributes are completely derived from the pre-trained attribute recognition model, and the performance is limited. Therefore, through steps S507 to S511, the attribute recognition model is further fine-tuned using the more fine-grained target-attribute pairs extracted from the image-text pair data.

[0160] In the embodiment of the present application, the first stage is trained on a limited attribute recognition data set, that is, the pre-trained attribute recognition model is fine-tuned using attribute recognition annotated data to better align the visual representation of the target with the attribute words and obtain higher open set attribute classification performance. The second stage is trained using image and text data that is easily accessible on the Internet, that is, weakly supervised training is performed using image and text data. First, a finer-grained target-attribute matching relationship is extracted from the coarse-grained matching relationship between image and text pairs, and then noise learning is introduced to obtain a good representation of the attributes, thereby ensuring that the trained attribute recognition model can achieve open set-oriented attribute recognition tasks.

[0161] An embodiment of the present application further provides a computer storage medium, in which program instructions are stored. When the program instructions are executed, they are used to implement the corresponding methods described in the above embodiments.

[0162] See also Figure 6 , Figure 6 It is a structural diagram of a training device for an attribute recognition model provided in an embodiment of the present application.

[0163] In one implementation of the attribute recognition model training device of the embodiment of the present application, the attribute recognition model training device includes the following structure.

[0164] The acquisition unit 601 is used to acquire a first training sample; wherein the first training sample includes image-text pair data, and the image-text pair data includes a first training image and text description information of the first training image;

[0165] A visual encoding unit 602 is used to call an attribute recognition model to perform visual encoding on each candidate target included in the first training image to obtain a visual encoding feature of each candidate target;

[0166] A text encoding unit 603 is used to perform text encoding on each attribute word included in the text description information to obtain text encoding features of each attribute word;

[0167] The determination unit 604 is used to determine the matching results of the candidate objects and the attribute words based on the visual coding features of the candidate objects and the text coding features of the attribute words; wherein the matching results are used to indicate whether the candidate objects and the attribute words have a matching relationship.

[0168] The model training unit 605 is used to train the attribute recognition model in the direction of reducing the difference between the matching results of each determined candidate target and each attribute word and the matching labels of the marked corresponding candidate target and the corresponding attribute word, so as to obtain a trained attribute recognition model; wherein the trained attribute recognition model is used to perform attribute recognition on the image to be recognized.

[0169] In one embodiment, the acquisition unit 601 is further used to acquire a second training sample, where the second training sample includes a second training image and an attribute annotation label of the second training image, where the attribute label is used to indicate a target attribute word of a target contained in the second training image;

[0170] The training device of the attribute recognition model may also include:

[0171] A cropping unit 606 is configured to crop a partial image from the second training image based on the attribute annotation label; wherein the partial image includes the target described by the target attribute word indicated by the attribute label;

[0172] The visual encoding unit 602 is further used to call a pre-trained attribute recognition model to perform visual encoding on the local image to obtain a visual encoding feature of the local image;

[0173] The text encoding unit 603 is further used to perform text encoding on the target attribute word to obtain a text encoding feature of the target attribute word;

[0174] The determination unit 604 is further configured to obtain a matching probability between the partial image and the target attribute word based on the visual coding feature of the partial image and the text coding feature of the target attribute word;

[0175] The model training unit 605 is further used to optimize the pre-trained attribute recognition model with the matching probability being higher than the first probability threshold as an optimization target to obtain the attribute recognition model.

[0176] In one embodiment, the cropping unit 606 crops the second training image to obtain a partial image based on the attribute label, including:

[0177] Slicing the second training image to obtain a candidate image; wherein the candidate image at least includes the target described by the attribute word indicated by the attribute tag, and the size of the candidate image is larger than the size of the detection box of the target in the second training image;

[0178] The candidate image is randomly cropped to obtain the partial image; wherein the size of the partial image is smaller than the size of the candidate image, and the partial image contains part or all of the content of the target described by the target attribute word.

[0179] In one embodiment, the pre-trained attribute recognition model includes a visual encoder and a text encoder, the visual encoder is used to perform visual encoding on the local image, and the text encoder is used to perform text encoding on the target attribute word;

[0180] The model training unit 605 optimizes the pre-trained attribute recognition model with the matching probability being higher than the first probability threshold as an optimization target to obtain the attribute recognition model, including:

[0181] Taking the matching probability being higher than a first probability threshold as an optimization goal, the text encoder is optimized to obtain the attribute recognition model, and the attribute recognition model includes the visual encoder and the optimized text encoder.

[0182] In one embodiment, the determining unit 604 obtains the matching probability between the partial image and the attribute word indicated by the attribute annotation tag based on the visual encoding feature of the partial image and the text encoding feature of the target attribute word, including:

[0183] Acquire the similarity between the visual coding feature of the local image and the text coding feature of the target attribute word;

[0184] Based on the similarity, a matching probability between the partial image and the target attribute word is obtained; wherein the matching probability between the partial image and the target attribute word is positively correlated with the similarity.

[0185] In one embodiment, the visual encoding unit 602 is further configured to call the attribute recognition model to perform visual encoding on the first training image to obtain visual encoding features of the first training image;

[0186] The text encoding unit 603 is further used to perform text encoding on the text description information to obtain text encoding features of the text description information;

[0187] The determining unit 604 is further configured to determine a matching result between the first training image and the text description information based on the visual coding feature of the first training image and the text coding feature of the text description information;

[0188] The model training unit 605 trains the attribute recognition model in a direction of reducing the difference between the matching results of the determined candidate targets and the attribute words and the matching labels of the marked corresponding candidate targets and the corresponding attribute words, to obtain a trained attribute recognition model, including:

[0189] The attribute recognition model is trained in a direction of reducing the difference between the determined matching result of the first training image and the text description information and the preset matching label of the first training image and the text description information, and the difference between the matching result of each candidate target and each attribute word and the matching label of the marked corresponding candidate target and the corresponding attribute word to obtain the trained attribute recognition model; wherein the preset matching label of the first training image and the text description information is used to indicate that the first training image and the text description information have a matching relationship.

[0190] In one embodiment, the visual encoding unit 602 is further used to call the attribute recognition model to perform visual encoding on the target unit image to obtain visual encoding features of the target unit image; wherein the target unit image refers to a unit image corresponding to a target detection frame in the first training image, and the target detection frame refers to: a detection frame with the largest size among detection frames of at least one candidate target contained in the first training image;

[0191] The determining unit 604 is further configured to determine a matching result between the target unit image and each attribute word based on a visual coding feature of the target unit image and a text coding feature of each attribute word;

[0192] The model training unit 605 trains the attribute recognition model in a direction of reducing the difference between the matching results of the determined candidate targets and the attribute words and the matching labels of the marked corresponding candidate targets and the corresponding attribute words, to obtain a trained attribute recognition model, including:

[0193] The attribute recognition model is trained in the direction of reducing the difference between the determined matching results of the target unit image and the respective attribute words and the preset matching labels of the target unit image and the respective attribute words, as well as the difference between the matching results of the respective candidate targets and the respective attribute words and the marked matching labels of the corresponding candidate targets and the corresponding attribute words, to obtain the trained attribute recognition model; wherein the preset matching labels of the target unit image and the respective attribute words are used to indicate that the target unit image and the respective attribute words have a matching relationship.

[0194] In one embodiment, the visual encoding unit 602 is further used to call the attribute recognition model to perform visual encoding on the target unit image to obtain visual encoding features of the target unit image; wherein the target unit image refers to a unit image corresponding to a target detection frame in the first training image, and the target detection frame refers to: a detection frame with the largest size among detection frames of at least one candidate target contained in the first training image;

[0195] The text encoding unit 603 is further used to perform text encoding on each noun phrase included in the text description information to obtain text encoding features of each noun phrase;

[0196] The determination unit 604 is further configured to determine a matching result between the target unit image and each noun phrase based on a visual coding feature of the target unit image and a text coding feature of each noun phrase;

[0197] The model training unit 605 trains the attribute recognition model in a direction of reducing the difference between the matching results of the determined candidate targets and the attribute words and the matching labels of the marked corresponding candidate targets and the corresponding attribute words, to obtain a trained attribute recognition model, including:

[0198] The attribute recognition model is trained in the direction of reducing the difference between the determined matching results of the target unit image and the respective noun phrases and the preset matching labels of the target unit image and the respective noun phrases, and the difference between the matching results of the respective candidate targets and the respective attribute words and the marked matching labels of the corresponding candidate targets and the corresponding attribute words to obtain the trained attribute recognition model; wherein the preset matching labels of the target unit image and the respective noun phrases are used to indicate that the target unit image and the respective noun phrases have a matching relationship.

[0199] In one embodiment, the determination unit 604 is also used to call the attribute recognition model to obtain the matching probability of each candidate target and each attribute word based on the visual encoding features of each candidate target and the text encoding features of each attribute word; compare the matching probability of each candidate target and each attribute word with the second probability threshold, and annotate the matching labels of each candidate target and each attribute word according to the comparison results.

[0200] In an embodiment of the present application, an acquisition unit 601 acquires a first training sample, the first training sample includes image-text pair data, the image-text pair data includes a first training image and text description information of the first training image, a visual encoding unit 602 calls an attribute recognition model, performs visual encoding on each candidate target contained in the first training image, and obtains visual encoding features of each candidate target, a text encoding unit 603 performs text encoding on each attribute word contained in the text description information, and obtains text encoding features of each attribute word, a determination unit 604 determines matching results of each candidate target and each attribute word based on the visual encoding features of each candidate target and the text encoding features of each attribute word, the matching results are used to indicate whether each candidate target and each attribute word have a matching relationship, a model training unit 605 trains the attribute recognition model in a direction of reducing the difference between the matching results of each determined candidate target and each attribute word and the matching labels of the marked corresponding candidate target and the corresponding attribute word, and obtains a trained attribute recognition model, which can realize open set-oriented attribute recognition tasks.

[0201] See also Figure 7 , Figure 7 7 is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. The computer device in the embodiment of the present application includes a power supply module and other structures, and includes a processor 701, a storage device 702, and a communication interface 703. The processor 701, the storage device 702, and the communication interface 703 can exchange data, and the processor 701 implements the training method of the corresponding attribute recognition model.

[0202] The storage device 702 may include a volatile memory, such as a random-access memory (RAM); the storage device 702 may also include a non-volatile memory, such as a flash memory, a solid-state drive (SSD), etc.; the storage device 702 may also include a combination of the above-mentioned types of memory.

[0203] The processor 701 may be a central processing unit (CPU). The processor 701 may also be a combination of a CPU and a GPU. A plurality of CPUs and GPUs may be included as needed to train the corresponding attribute recognition model. In one embodiment, the storage device 702 is used to store program instructions. The processor 701 may call program instructions to implement the various methods involved in the embodiments of the present application.

[0204] In a first possible implementation, the processor 701 of the computer device calls the program instructions stored in the storage device 702 to obtain a first training sample; wherein the first training sample includes image-text pair data, and the image-text pair data includes a first training image and text description information of the first training image; calls an attribute recognition model to visually encode each candidate target contained in the first training image to obtain the visual encoding features of each candidate target; textually encodes each attribute word contained in the text description information to obtain the text encoding features of each attribute word; based on the visual encoding features of each candidate target and the text encoding features of each attribute word, determines the matching results of each candidate target and each attribute word; wherein the matching results are used to indicate whether each candidate target and each attribute word have a matching relationship; trains the attribute recognition model in a direction of reducing the difference between the matching results of each determined candidate target and each attribute word and the matching labels of the marked corresponding candidate target and the corresponding attribute word to obtain a trained attribute recognition model; wherein the trained attribute recognition model is used to perform attribute recognition on the image to be recognized.

[0205] In one embodiment, the processor 701 is further configured to perform the following operations:

[0206] Acquire a second training sample, where the second training sample includes a second training image and an attribute annotation label of the second training image, where the attribute label is used to indicate a target attribute word of a target contained in the second training image;

[0207] Based on the attribute annotation label, a partial image is obtained by cropping from the second training image; wherein the partial image includes the target described by the target attribute word indicated by the attribute label;

[0208] Calling a pre-trained attribute recognition model to perform visual encoding on the local image to obtain visual encoding features of the local image;

[0209] Performing text encoding on the target attribute word to obtain text encoding features of the target attribute word;

[0210] Based on the visual coding features of the partial image and the text coding features of the target attribute word, obtaining a matching probability between the partial image and the target attribute word;

[0211] The pre-trained attribute recognition model is optimized with the matching probability being higher than a first probability threshold as an optimization goal to obtain the attribute recognition model.

[0212] In one embodiment, when the processor 701 crops the local image from the second training image based on the attribute label, it may perform the following operations:

[0213] Slicing the second training image to obtain a candidate image; wherein the candidate image at least includes the target described by the attribute word indicated by the attribute tag, and the size of the candidate image is larger than the size of the detection box of the target in the second training image;

[0214] The candidate image is randomly cropped to obtain the partial image; wherein the size of the partial image is smaller than the size of the candidate image, and the partial image contains part or all of the content of the target described by the target attribute word.

[0215] In one embodiment, the pre-trained attribute recognition model includes a visual encoder and a text encoder, the visual encoder is used to perform visual encoding on the local image, and the text encoder is used to perform text encoding on the target attribute word;

[0216] The processor 701 optimizes the pre-trained attribute recognition model with the matching probability being higher than the first probability threshold as the optimization target to obtain the attribute recognition model, and may perform the following operations:

[0217] Taking the matching probability being higher than a first probability threshold as an optimization goal, the text encoder is optimized to obtain the attribute recognition model, and the attribute recognition model includes the visual encoder and the optimized text encoder.

[0218] In one embodiment, when the processor 701 obtains the matching probability between the partial image and the attribute word indicated by the attribute annotation tag based on the visual encoding feature of the partial image and the text encoding feature of the target attribute word, the processor 701 may perform the following operations:

[0219] Acquire the similarity between the visual coding feature of the local image and the text coding feature of the target attribute word;

[0220] Based on the similarity, a matching probability between the partial image and the target attribute word is obtained; wherein the matching probability between the partial image and the target attribute word is positively correlated with the similarity.

[0221] In one embodiment, the processor 701 is further configured to perform the following operations:

[0222] Calling the attribute recognition model to perform visual encoding on the first training image to obtain visual encoding features of the first training image;

[0223] Performing text encoding on the text description information to obtain text encoding features of the text description information;

[0224] Determining a matching result between the first training image and the text description information based on the visual coding features of the first training image and the text coding features of the text description information;

[0225] When the processor 701 trains the attribute recognition model in a direction of reducing the difference between the matching results of the determined candidate targets and the attribute words and the matching labels of the marked corresponding candidate targets and the corresponding attribute words to obtain the trained attribute recognition model, the processor 701 may perform the following operations:

[0226] The attribute recognition model is trained in a direction of reducing the difference between the determined matching result of the first training image and the text description information and the preset matching label of the first training image and the text description information, and the difference between the matching result of each candidate target and each attribute word and the matching label of the marked corresponding candidate target and the corresponding attribute word to obtain the trained attribute recognition model; wherein the preset matching label of the first training image and the text description information is used to indicate that the first training image and the text description information have a matching relationship.

[0227] In one embodiment, the processor 701 is further configured to perform the following operations:

[0228] Calling the attribute recognition model to perform visual encoding on the target unit image to obtain visual encoding features of the target unit image; wherein the target unit image refers to a unit image corresponding to a target detection frame in the first training image, and the target detection frame refers to: a detection frame with the largest size among the detection frames of at least one candidate target contained in the first training image;

[0229] Determining a matching result between the target unit image and each attribute word based on a visual coding feature of the target unit image and a text coding feature of each attribute word;

[0230] When the processor 701 trains the attribute recognition model in a direction of reducing the difference between the matching results of the determined candidate targets and the attribute words and the matching labels of the marked corresponding candidate targets and the corresponding attribute words to obtain the trained attribute recognition model, the processor 701 may perform the following operations:

[0231] The attribute recognition model is trained in the direction of reducing the difference between the determined matching results of the target unit image and the respective attribute words and the preset matching labels of the target unit image and the respective attribute words, as well as the difference between the matching results of the respective candidate targets and the respective attribute words and the marked matching labels of the corresponding candidate targets and the corresponding attribute words, to obtain the trained attribute recognition model; wherein the preset matching labels of the target unit image and the respective attribute words are used to indicate that the target unit image and the respective attribute words have a matching relationship.

[0232] In one embodiment, the processor 701 is further configured to perform the following operations:

[0233] Calling the attribute recognition model to perform visual encoding on the target unit image to obtain visual encoding features of the target unit image; wherein the target unit image refers to a unit image corresponding to a target detection frame in the first training image, and the target detection frame refers to: a detection frame with the largest size among the detection frames of at least one candidate target contained in the first training image;

[0234] Performing text encoding on each noun phrase included in the text description information to obtain text encoding features of each noun phrase;

[0235] Determining matching results between the target unit image and each noun phrase based on visual coding features of the target unit image and text coding features of each noun phrase;

[0236] When the processor 701 trains the attribute recognition model in a direction of reducing the difference between the matching results of the determined candidate targets and the attribute words and the matching labels of the marked corresponding candidate targets and the corresponding attribute words to obtain the trained attribute recognition model, the processor 701 may perform the following operations:

[0237] The attribute recognition model is trained in the direction of reducing the difference between the determined matching results of the target unit image and the respective noun phrases and the preset matching labels of the target unit image and the respective noun phrases, and the difference between the matching results of the respective candidate targets and the respective attribute words and the marked matching labels of the corresponding candidate targets and the corresponding attribute words to obtain the trained attribute recognition model; wherein the preset matching labels of the target unit image and the respective noun phrases are used to indicate that the target unit image and the respective noun phrases have a matching relationship.

[0238] In one embodiment, the processor 701 is further configured to perform the following operations:

[0239] Calling the attribute recognition model to obtain the matching probability of each candidate target and each attribute word based on the visual coding feature of each candidate target and the text coding feature of each attribute word;

[0240] The matching probabilities of the candidate targets and the attribute words are compared with a second probability threshold, and matching labels of the candidate targets and the attribute words are obtained according to the comparison results.

[0241] In an embodiment of the present application, the processor 701 obtains a first training sample, the first training sample includes image-text pair data, the image-text pair data includes a first training image and text description information of the first training image, calls an attribute recognition model, visually encodes each candidate target contained in the first training image, and obtains a visual encoding feature of each candidate target, and textually encodes each attribute word contained in the text description information to obtain a text encoding feature of each attribute word. Based on the visual encoding feature of each candidate target and the text encoding feature of each attribute word, a matching result of each candidate target and each attribute word is determined, and the matching result is used to indicate whether each candidate target and each attribute word have a matching relationship. The attribute recognition model is trained in a direction of reducing the difference between the matching results of each determined candidate target and each attribute word and the matching labels of the marked corresponding candidate target and the corresponding attribute word to obtain a trained attribute recognition model, which can realize open set-oriented attribute recognition tasks.

[0242] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the computer-readable storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM). The computer-readable storage medium can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function, etc.; the data storage area can store data created according to the use of the blockchain node, etc.

[0243] The above disclosure is only part of the embodiments of the present application, which certainly cannot be used to limit the scope of rights of the present application. Ordinary technicians in this field can understand that all or part of the processes of implementing the above embodiments and making equivalent changes according to the claims of this application are still within the scope of the present invention.

Claims

1. A method for training an attribute recognition model, characterized in that: include: Acquire a second training sample, where the second training sample includes a second training image and an attribute annotation label of the second training image, where the attribute annotation label is used to indicate a target attribute word of a target contained in the second training image; Based on the attribute annotation label, a partial image is obtained by cropping from the second training image; wherein the partial image includes the target described by the target attribute word indicated by the attribute annotation label; Calling a pre-trained attribute recognition model to perform visual encoding on the local image to obtain visual encoding features of the local image; Performing text encoding on the target attribute word to obtain text encoding features of the target attribute word; Based on the visual coding features of the partial image and the text coding features of the target attribute word, obtaining a matching probability between the partial image and the target attribute word; Taking the matching probability being higher than a first probability threshold as an optimization goal, optimizing the pre-trained attribute recognition model to obtain an attribute recognition model; Acquire a first training sample; wherein the first training sample includes image-text pair data, and the image-text pair data includes a first training image and text description information of the first training image; Calling the attribute recognition model to perform visual coding on each candidate target included in the first training image to obtain a visual coding feature of each candidate target; Performing text encoding on each attribute word included in the text description information to obtain text encoding features of each attribute word; Based on the visual coding features of the candidate objects and the text coding features of the attribute words, determining the matching results of the candidate objects and the attribute words; wherein the matching results are used to indicate whether the candidate objects and the attribute words have a matching relationship; The attribute recognition model is trained in the direction of reducing the difference between the matching results of the determined candidate targets and the attribute words and the matching labels of the marked corresponding candidate targets and the corresponding attribute words to obtain a trained attribute recognition model; wherein the trained attribute recognition model is used to perform attribute recognition on the image to be recognized.

2. The method according to claim 1, characterized in that The step of cropping the second training image to obtain a partial image based on the attribute annotation label includes: Slicing the second training image to obtain a candidate image; wherein the candidate image at least includes the target described by the attribute word indicated by the attribute annotation label, and the size of the candidate image is larger than the size of the detection box of the target in the second training image; The candidate image is randomly cropped to obtain the partial image; wherein the size of the partial image is smaller than the size of the candidate image, and the partial image contains part or all of the content of the target described by the target attribute word.

3. The method according to claim 1, characterized in that The pre-trained attribute recognition model includes a visual encoder and a text encoder, wherein the visual encoder is used to perform visual encoding on the local image, and the text encoder is used to perform text encoding on the target attribute word; The optimizing the pre-trained attribute recognition model by taking the matching probability higher than the first probability threshold as the optimization target to obtain the attribute recognition model includes: Taking the matching probability being higher than a first probability threshold as an optimization goal, the text encoder is optimized to obtain the attribute recognition model, and the attribute recognition model includes the visual encoder and the optimized text encoder.

4. The method according to claim 1, characterized in that The obtaining, based on the visual coding feature of the partial image and the text coding feature of the target attribute word, a matching probability between the partial image and the attribute word indicated by the attribute annotation label includes: Acquire the similarity between the visual coding feature of the local image and the text coding feature of the target attribute word; Based on the similarity, a matching probability between the partial image and the target attribute word is obtained; wherein the matching probability between the partial image and the target attribute word is positively correlated with the similarity.

5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: Calling the attribute recognition model to perform visual encoding on the first training image to obtain visual encoding features of the first training image; Performing text encoding on the text description information to obtain text encoding features of the text description information; Determining a matching result between the first training image and the text description information based on the visual coding features of the first training image and the text coding features of the text description information; The attribute recognition model is trained in a direction of reducing the difference between the matching results of the determined candidate targets and the attribute words and the matching labels of the marked corresponding candidate targets and the corresponding attribute words to obtain the trained attribute recognition model, including: The attribute recognition model is trained in a direction of reducing the difference between the determined matching result of the first training image and the text description information and the preset matching label of the first training image and the text description information, and the difference between the matching result of each candidate target and each attribute word and the matching label of the marked corresponding candidate target and the corresponding attribute word to obtain the trained attribute recognition model; wherein the preset matching label of the first training image and the text description information is used to indicate that the first training image and the text description information have a matching relationship.

6. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: Calling the attribute recognition model to perform visual encoding on the target unit image to obtain visual encoding features of the target unit image; wherein the target unit image refers to a unit image corresponding to a target detection frame in the first training image, and the target detection frame refers to: a detection frame with the largest size among the detection frames of at least one candidate target contained in the first training image; Determining a matching result between the target unit image and each attribute word based on a visual coding feature of the target unit image and a text coding feature of each attribute word; The attribute recognition model is trained in a direction of reducing the difference between the matching results of the determined candidate targets and the attribute words and the matching labels of the marked corresponding candidate targets and the corresponding attribute words to obtain the trained attribute recognition model, including: The attribute recognition model is trained in the direction of reducing the difference between the determined matching results of the target unit image and the respective attribute words and the preset matching labels of the target unit image and the respective attribute words, as well as the difference between the matching results of the respective candidate targets and the respective attribute words and the marked matching labels of the corresponding candidate targets and the corresponding attribute words, to obtain the trained attribute recognition model; wherein the preset matching labels of the target unit image and the respective attribute words are used to indicate that the target unit image and the respective attribute words have a matching relationship.

7. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: Calling the attribute recognition model to perform visual encoding on the target unit image to obtain visual encoding features of the target unit image; wherein the target unit image refers to a unit image corresponding to a target detection frame in the first training image, and the target detection frame refers to: a detection frame with the largest size among the detection frames of at least one candidate target contained in the first training image; Performing text encoding on each noun phrase included in the text description information to obtain text encoding features of each noun phrase; Determining matching results between the target unit image and each noun phrase based on visual coding features of the target unit image and text coding features of each noun phrase; The attribute recognition model is trained in a direction of reducing the difference between the matching results of the determined candidate targets and the attribute words and the matching labels of the marked corresponding candidate targets and the corresponding attribute words to obtain the trained attribute recognition model, including: The attribute recognition model is trained in the direction of reducing the difference between the determined matching results of the target unit image and the respective noun phrases and the preset matching labels of the target unit image and the respective noun phrases, and the difference between the matching results of the respective candidate targets and the respective attribute words and the marked matching labels of the corresponding candidate targets and the corresponding attribute words to obtain the trained attribute recognition model; wherein the preset matching labels of the target unit image and the respective noun phrases are used to indicate that the target unit image and the respective noun phrases have a matching relationship.

8. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: Calling the attribute recognition model to obtain the matching probability between each candidate target and each attribute word based on the visual coding feature of each candidate target and the text coding feature of each attribute word; The matching probabilities of the candidate targets and the attribute words are compared with a second probability threshold, and matching labels of the candidate targets and the attribute words are obtained according to the comparison results.

9. A training device for an attribute recognition model, characterized in that: The device comprises: An acquisition unit, configured to acquire a second training sample, wherein the second training sample includes a second training image and an attribute annotation label of the second training image, wherein the attribute annotation label is used to indicate a target attribute word of a target contained in the second training image; A cropping unit, configured to crop a partial image from the second training image based on the attribute annotation label; wherein the partial image includes a target described by a target attribute word indicated by the attribute annotation label; A visual encoding unit, used to call a pre-trained attribute recognition model to perform visual encoding on the local image to obtain a visual encoding feature of the local image; A text encoding unit, used to perform text encoding on the target attribute word to obtain a text encoding feature of the target attribute word; A determination unit, configured to obtain a matching probability between the partial image and the target attribute word based on a visual coding feature of the partial image and a text coding feature of the target attribute word; A model training unit, configured to optimize the pre-trained attribute recognition model with the matching probability being higher than a first probability threshold as an optimization target, to obtain an attribute recognition model; The acquisition unit is further used to acquire a first training sample; wherein the first training sample includes image-text pair data, and the image-text pair data includes a first training image and text description information of the first training image; The visual encoding unit is further used to call the attribute recognition model to perform visual encoding on each candidate target included in the first training image to obtain visual encoding features of each candidate target; The text encoding unit is further used to perform text encoding on each attribute word included in the text description information to obtain text encoding features of each attribute word; The determination unit is further used to determine the matching results of the candidate targets and the attribute words based on the visual coding features of the candidate targets and the text coding features of the attribute words; wherein the matching results are used to indicate whether the candidate targets and the attribute words have a matching relationship; The model training unit is also used to train the attribute recognition model in the direction of reducing the difference between the matching results of the determined candidate targets and the attribute words and the matching labels of the marked corresponding candidate targets and the corresponding attribute words, so as to obtain a trained attribute recognition model; wherein the trained attribute recognition model is used to perform attribute recognition on the image to be recognized.

10. A computer device, characterized in that: The computer device comprises a processor, a storage device and a communication interface, wherein the processor, the storage device and the communication interface are connected to each other, wherein: The storage device is used to store a computer program, wherein the computer program includes program instructions; The processor is used to call the program instructions to execute the training method of the attribute recognition model as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed by a processor, the method for training an attribute recognition model as described in any one of claims 1 to 8 is executed.

12. A computer program product, characterized in that The computer program product comprises a computer program, which is suitable for being loaded by a processor and executing the training method for the attribute recognition model according to any one of claims 1 to 8.

Images