Attribute recognition model training method and device, storage medium and computer device

Abstract

The attribute recognition model training method and device, the storage medium and the computer device provided in the application, in the model training, first acquire sample character data, and an initial attribute recognition model composed of a multi-modal learning model, a visual transformation network and a classification layer. In the model training, the text classification tensor and the fusion feature tensor of the sample character data can be determined by using the multi-modal learning model, and the fusion feature tensor is aggregated by using the visual transformation network to fuse the multi-scale visual features and the text features. Therefore, the target classification tensor obtained by the classification layer for the aggregated fusion feature tensor has high accuracy. Then, the parameter loss value is determined based on the text classification tensor and the target classification tensor to update the model parameters and enter the next round of training until the model meets the training end condition. The model obtained by training can multi-scale aggregate the visual features and the text features, thereby improving the recognition ability of the character attributes.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to an attribute recognition model training method and device, a storage medium and a computer device. BACKGROUND

[0002] With the vigorous development of computer vision technology, as an advanced field of intelligent video monitoring, human attribute recognition is increasingly becoming an indispensable part. Relying on the powerful ability of deep learning and other advanced algorithms, this technology can automatically mine deep information of pedestrians from monitoring videos and realize accurate mapping to a high-level semantic level, including but not limited to gender recognition, age estimation, and clothing style analysis and other fine attribute classification.

[0003] However, the existing human attribute recognition performs poorly in dealing with complex scenes, for example, in the face of occlusion, illumination change and high background complexity, the model is difficult to effectively extract and utilize key human features, and the traditional method cannot fully fuse multi-scale visual features, resulting in that the features at different levels cannot effectively cooperate, thereby affecting the recognition accuracy, and the recognition ability of human attributes is poor. SUMMARY

[0004] The purpose of the present application is to at least solve one of the above technical defects, in particular, the technical defect that the prior art cannot fully fuse multi-scale visual features, resulting in that the features at different levels cannot effectively cooperate, thereby affecting the recognition accuracy, and the recognition ability of human attributes is poor.

[0005] The present application provides an attribute recognition model training method, which comprises:

[0006] Obtaining sample human data and an initial attribute recognition model, wherein the initial attribute recognition model comprises a multi-modal learning model, a visual transformation network and a classification layer;

[0007] Using the multi-modal learning model to determine a text classification tensor and a fusion feature tensor of the sample human data, using the visual transformation network to aggregate the fusion feature tensor, and using the classification layer to classify the aggregated fusion feature tensor to obtain a target classification tensor;

[0008] Determining a parameter loss value of the initial attribute recognition model based on the text classification tensor and the target classification tensor, updating the parameters of the initial attribute recognition model based on the parameter loss value, and returning the use of the multi-modal learning model to determine the text classification tensor and the fusion feature tensor of the sample human data and the subsequent steps until the initial attribute recognition model meets a preset training end condition, to obtain a target attribute recognition model.

[0009] Optionally, the sample character data is obtained, including:

[0010] determining a model attribute to be trained and a plurality of classification types under the model attribute;

[0011] obtaining a sample character image corresponding to each classification type in a proportion, and labeling a sample attribute label in each sample character image according to each classification type, to form the sample character data.

[0012] Optionally, the multi-modal learning model includes a feature output layer, a text encoding layer, and a tensor processing layer; the feature output layer includes a visual feature output layer and a prompt word feature output layer.

[0013] The multi-modal learning model is used to determine a text classification tensor and a fusion feature tensor of the sample character data, including:

[0014] The sample character data is input into the feature output layer to obtain a visual feature tensor output by the visual feature output layer and a prompt word feature tensor output by the prompt word feature output layer.

[0015] The sample attribute label of the sample character data is text encoded by using the text encoding layer to obtain a label feature.

[0016] The label feature is dimensionally augmented after using a full connection layer in the tensor processing layer to form a text feature, the text feature and the visual feature tensor are fused to obtain a fusion feature tensor, and a text classification tensor is generated based on the label feature and the prompt word feature tensor.

[0017] Optionally, the classification layer includes a convolution layer and a full connection layer.

[0018] The fusion feature tensor after aggregation is classified by the classification layer to obtain a target classification tensor, including:

[0019] The fusion feature tensor is feature mapped by using the convolution layer to obtain a fusion feature vector.

[0020] The fusion feature vector is classified by using the full connection layer to generate a target classification tensor.

[0021] Optionally, the parameter loss value of the initial attribute recognition model is determined based on the text classification tensor and the target classification tensor, including:

[0022] determining a first loss function of the text classification tensor and a second loss function of the target classification tensor;

[0023] The first loss value of the text classification tensor is calculated by using the first loss function, and the second loss value of the target classification tensor is calculated by using the second loss function, and a parameter loss value of the initial attribute recognition model is determined according to the first loss value and the second loss value.

[0024] Optionally, the updating the parameters of the initial attribute recognition model based on the parameter loss value comprises:

[0025] The parameters of the visual transformation network in the initial attribute recognition model and other network layers in the multi-modal learning model except the feature output layer are frozen.

[0026] The parameters of the feature output layer and the classification layer in the initial attribute recognition model are updated by backward propagation in the initial attribute recognition model based on the parameter loss value.

[0027] The application further provides an attribute recognition model training device, comprising:

[0028] A data acquisition module is configured to acquire sample person data and an initial attribute recognition model, wherein the initial attribute recognition model comprises a multi-modal learning model, a visual transformation network and a classification layer.

[0029] A tensor generation module is configured to determine a text classification tensor and a fusion feature tensor of the sample person data by using the multi-modal learning model, aggregate the fusion feature tensor by using the visual transformation network, and perform attribute classification on the aggregated fusion feature tensor by using the classification layer to obtain a target classification tensor.

[0030] A model training module is configured to determine a parameter loss value of the initial attribute recognition model based on the text classification tensor and the target classification tensor, update the parameters of the initial attribute recognition model based on the parameter loss value, and return the determination of the text classification tensor and the fusion feature tensor of the sample person data by using the multi-modal learning model and subsequent steps until the initial attribute recognition model meets a preset training end condition, so as to obtain a target attribute recognition model.

[0031] The application further provides a storage medium, wherein the storage medium stores computer readable instructions, and the computer readable instructions are executed by one or more processors to make the one or more processors execute the steps of the attribute recognition model training method in any one of the above embodiments.

[0032] The application further provides a computer device, comprising one or more processors and a memory.

[0033] The memory stores computer readable instructions that, when executed by the one or more processors, perform the steps of the attribute recognition model training method of any of the above embodiments.

[0034] From the above technical solutions, the embodiments of the present application have the following advantages:

[0035] The attribute recognition model training method, device, storage medium and computer equipment provided by the present application can obtain sample person data and an initial attribute recognition model when training the model. The initial attribute recognition model includes a multi-modal learning model, a visual transformation network and a classification layer. When training the initial attribute recognition model, the multi-modal learning model can be used to determine a text classification tensor and a fusion feature tensor of the sample person data. The text classification tensor reflects the category semantic information related to the person attribute in the sample person data, and provides rich and accurate context understanding to improve the accuracy of person attribute recognition. Then, the visual transformation network can be used to aggregate the fusion feature tensor to fuse multi-scale visual features and text features, so that the fusion features have higher recognition. Therefore, the target classification tensor obtained by the classification layer for attribute classification of the aggregated fusion feature tensor has high accuracy. Finally, the parameter loss value of the initial attribute recognition model can be determined based on the text classification tensor and the target classification tensor, and the parameters of the initial attribute recognition model are updated based on the parameter loss value. Then, the next round of model training is entered until the initial attribute recognition model meets the preset training end condition, and the target attribute recognition model is obtained. The target attribute recognition model obtained by training can multi-scale aggregate visual features and text features, and further improve the recognition ability of person attributes. BRIEF DESCRIPTION OF DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0037] Figure 1 A flowchart of an attribute recognition model training method provided by an embodiment of the present application;

[0038] Figure 2 A flowchart of a multi-modal learning model processing process provided by an embodiment of the present application;

[0039] Figure 3 A structural diagram of an attribute recognition model training device provided by an embodiment of the present application;

[0040] Figure 4 FIG. 1 is a schematic diagram of an internal structure of a computer device according to an embodiment of the present application. DETAILED DESCRIPTION

[0041] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work are within the scope of protection of the present application.

[0042] Existing person attribute recognition performs poorly in processing complex scenes. For example, in the face of occlusion, illumination change and high background complexity, the model is difficult to effectively extract and utilize key person features, and traditional methods cannot fully fuse multi-scale visual features, resulting in ineffective coordination of features at different levels, thereby affecting the recognition accuracy and making the person attribute recognition ability poor.

[0043] Based on this, the technical solutions are proposed as follows, please refer to the following:

[0044] In one embodiment, as shown in Figure 1 , Figure 1 FIG. 1 is a schematic diagram of an internal structure of a computer device according to an embodiment of the present application. The present application provides a person attribute recognition method and device, and a person attribute recognition model training method and device, and a person attribute recognition model and device.

[0045] S110: Obtain sample person data and an initial attribute recognition model, and the initial attribute recognition model includes a multi-modal learning model, a visual transformation network and a classification layer.

[0046] In this step, when training a model for type recognition of a certain attribute of a person in a passenger flow attribute analysis scene, the computer device can first obtain sample person data containing various types of person images, and can also obtain an initial attribute recognition model as a pre-training model for sample person data training. The initial attribute recognition model can include a multi-modal learning model, a visual transformation network and a classification layer.

[0047] It can be understood that the multi-modal learning model refers to a model capable of processing and fusing data from different modalities, aiming to improve the performance of the task by combining the information of multiple modalities in the input data through the multi-modal learning model. Here, the present application can adopt a CLIP (Contrastive Language-Image Pre-Training) model, which can map image information and text information in sample person data to the same vector space, enhance the understanding and processing ability of the model for visual concepts, and thus improve the recognition ability of the model.

[0048] In addition, the visual transformation network can capture global information in the image and perform well in the fusion of multi-scale features. In person attribute recognition, it can effectively extract and aggregate visual features of different scales, such as extracting key attributes of a person under different lighting, angle or occlusion conditions, thereby further improving the recognition ability of the model. The classification layer is the final layer of the initial attribute recognition model, mainly responsible for mapping the features extracted by the model to specific class labels. It can receive features output by the visual transformation network and map these features to predefined person attribute categories; for example, the model can classify the attribute type of the input person image as "male" or "female".

[0049] Further, the classification layer of the initial attribute recognition model in the present application can be one or multiple. When the model needs to recognize a person attribute, such as gender, the computer device can adopt a classification layer that has learned various type labels corresponding to the attribute of gender in advance when building the initial attribute recognition model, so that the recognition result output by the model in the training process corresponds to only gender; when the model needs to recognize multiple person attributes, such as gender and age, the computer device can adopt a classification layer corresponding to gender and a classification layer corresponding to age connected to the visual transformation network, so that the recognition result output by the model in the training process can contain both gender and age.

[0050] S120: Determine the text classification tensor and the fusion feature tensor of the sample person data using the multi-modal learning model, and aggregate the fusion feature tensor using the visual transformation network, and perform attribute classification on the aggregated fusion feature tensor through the classification layer to obtain a target classification tensor.

[0051] In this step, after obtaining the sample person data and the initial attribute recognition model in step S110, the computer device can first determine the text classification tensor and the fusion feature tensor of the sample person data by using the multi-modal learning model, then aggregate the fusion feature tensor by using the visual transformation network, and perform attribute classification on the aggregated fusion feature tensor by using the classification layer to obtain the target classification tensor. Finally, the computer device can perform attribute classification on the aggregated fusion feature tensor by using the classification layer to obtain the target classification tensor.

[0052] It can be understood that the text classification tensor contains the category information of the text features of the sample person data and can be used for classification, and the text features therein can reflect the semantic information related to the person attributes in the sample person data, which can supplement the semantic information that may be missed in the image visual features, provide more rich and accurate context understanding, and thus improve the accuracy of person attribute recognition. The fusion feature tensor directly fuses the data of two different modalities of the image and the text in the sample person data, and it covers various types of person images and their corresponding text descriptions in the sample person data, so that the model can learn very extensive visual concepts and thus improve the recognition accuracy of the model.

[0053] Further, the classification layer is the final layer of the initial attribute recognition model, and it has learned the feature distribution of each type of label under the corresponding attribute in advance, so it can directly perform feature classification on the input fusion feature tensor and output the target classification tensor to represent the final recognition result of the corresponding person image in the sample person data.

[0054] S130: Determine the parameter loss value of the initial attribute recognition model based on the text classification tensor and the target classification tensor, and update the parameters of the initial attribute recognition model based on the parameter loss value.

[0055] In this step, after obtaining the text classification tensor and the target classification tensor in step S120, the computer device can determine the parameter loss value of the initial attribute recognition model based on the text classification tensor and the target classification tensor, and then update the parameters of the initial attribute recognition model based on the parameter loss value to complete the training of the initial attribute recognition model in the current round.

[0056] It can be understood that the text classification tensor is a category semantic representation related to the person attributes in the sample person data, and the target classification tensor is a fusion representation of the visual and text of the person image in the sample person data. The parameter loss value calculated based on these two feature tensors is used to update the parameters of the initial attribute recognition model, which can ensure that the model can pay attention to both the text and the visual information during the training process, and thus improve the effective fusion degree of the text features and the visual features of the image to achieve more accurate attribute recognition.

[0057] Specifically, the parameter loss value provides a feedback signal for the model, indicating the performance of the model in the current round of training iteration. Therefore, after obtaining the parameter loss value, the computer device can derive the gradient of each parameter in the model after the parameter loss value, which indicates the direction and size of each parameter that should be adjusted; then the computer device can perform back propagation through the chain calculation method, so that each parameter of the model is updated iteratively in the direction of reducing the loss value.

[0058] S140: determining whether the initial attribute recognition model meets the preset training end condition.

[0059] S150: if not, returning to step S120; if yes, ending the training and taking the initial attribute recognition model as the target attribute recognition model.

[0060] In this embodiment, after updating the parameters of the initial attribute recognition model in step S130, the computer device can be considered as the end of the model training in the current round, and then determine whether the initial attribute recognition model meets the preset training end condition. If not, the model needs to enter the next round of training, so the computer device can continue to train the initial attribute recognition model using the sample character data; if yes, the model can be considered as completed, so the computer device can take the initial attribute recognition model as the target attribute recognition model.

[0061] The preset training end condition can be that the model meets a certain training condition or the convergence condition of the parameters, such as the number of iterations reaching a set value, which is not limited herein.

[0062] In the above embodiments, when training the model, the sample person data and the initial attribute recognition model can be obtained first, where the initial attribute recognition model includes the multi-modal learning model, the visual transformation network and the classification layer. When training the initial attribute recognition model, the multi-modal learning model can be used to determine the text classification tensor and the fusion feature tensor of the sample person data, where the text classification tensor reflects the category semantic information related to the person attribute in the sample person data, and provides rich and accurate context understanding to improve the accuracy of person attribute recognition. Then, the visual transformation network can be used to aggregate the fusion feature tensor to fuse the multi-scale visual features and text features, so that the fusion features have higher recognition degree. Therefore, the target classification tensor obtained by the classification layer for attribute classification of the aggregated fusion feature tensor has high accuracy. Finally, the parameter loss value of the initial attribute recognition model can be determined based on the text classification tensor and the target classification tensor, to update the parameters of the initial attribute recognition model based on the parameter loss value, and then enter the next round of model training until the initial attribute recognition model meets the preset training end condition to obtain the target attribute recognition model. In this way, the target attribute recognition model obtained by training can multi-scale aggregate visual features and text features, thereby improving the recognition ability of person attributes.

[0063] In one embodiment, obtaining the sample person data in step S110 can include:

[0064] S111: determining a model attribute to be trained and a plurality of classification types under the model attribute.

[0065] S112: obtaining sample person images corresponding to each classification type in equal proportions, and labeling sample attribute labels in each sample person image according to each classification type to form sample person data.

[0066] In this embodiment, when obtaining the sample person data, the computer device can first determine a model attribute to be trained and a plurality of classification types under the model attribute. For example, when the model attribute is gender, the classification types can be divided into male and female; when the model attribute is age, the classification types can be divided into youth, middle age and old age. Then, the computer device can obtain sample person images corresponding to each classification type in equal proportions, and label sample attribute labels in each sample person image according to each classification type, i.e., the corresponding classification type, and mix each labeled sample person image to form sample person data, so as to improve the efficiency and generalization ability of the model when training using the sample person data.

[0067] It can be understood that the present application can obtain sample person image corresponding to each classification type by equal proportion, so as to ensure that the number of samples of each classification type in the data set is approximately equal, so that the model can learn the feature distribution of each classification type in the training process, rather than being biased to one or more categories, thereby improving the generalization ability of the model; in addition, by mixing sample person images of various classification types, the model can be exposed to more diversified data in the training process, and can learn more generalized feature representation, thereby avoiding model overfitting.

[0068] Further, after obtaining the sample person image, the computer device can perform a preprocessing operation on the sample person image, which includes but is not limited to normalization processing, sharpening processing, denoising processing, etc. The normalization processing refers to mapping the data of each dimension of the data vector to the interval between (0, 1) or (-1, 1), or mapping a certain norm of the data vector to 1. It can be understood that the pixel value of the sample person image in the present application is an integer in the range of 0-255, although the pixel data of the original image can be directly used as the training data of the model, it may bring some problems to the training process of the model, such as causing the model training speed to slow down, too many iteration times causing slow convergence, etc. Therefore, the present application can improve the model training efficiency by normalization processing.

[0069] The sharpening processing here refers to compensating the outline of the sample person image, enhancing the edges and gray level jump parts of the image, and making the image clear, which can be divided into spatial domain processing and frequency domain processing. By highlighting the edges and outlines of the characters on the image, or the features of certain linear target elements, the contrast between the character edges and the surrounding pixels is improved. The denoising processing refers to the process of reducing noise in digital images. Generally, digital images are often affected by imaging devices and external environmental noise during digitalization and transmission, that is, the obtained image information generally includes noise, and these noises will become an important reason for image interference. By denoising the image, the noise in the image is removed, and the authenticity and accuracy of the obtained image are further improved.

[0070] The above preprocessing operation on the obtained sample person image can scale the sample person image to a suitable size and effectively improve the clarity of the image itself, facilitating subsequent processing of the sample person data.

[0071] In one embodiment, as shown in Figure 2 , a flowchart of a multi-modal learning model processing process provided by an embodiment of the present application is shown. Figure 2 Figure 2 ​In the step S120, the multi-modal learning model can include a feature output layer, a text encoding layer, and a tensor processing layer. The feature output layer includes a visual feature output layer and a prompt word feature output layer. In the step of determining the text classification tensor and the fusion feature tensor of the sample character data by using the multi-modal learning model, the following steps can be included:

[0072] S121: inputting the sample character data into the feature output layer to obtain a visual feature tensor output by the visual feature output layer and a prompt word feature tensor output by the prompt word feature output layer.

[0073] S122: performing text encoding on the sample attribute label of the sample character data by using the text encoding layer to obtain a label feature.

[0074] S123: performing dimension expansion on the label feature by using a full connection layer in the tensor processing layer to form a text feature, performing feature fusion on the text feature and the visual feature tensor to obtain a fusion feature tensor, and generating a text classification tensor based on the label feature and the prompt word feature tensor.

[0075] In the embodiment, when determining the text classification tensor and the fusion feature tensor of the sample character data by using the multi-modal learning model, the computer device can first input the sample character data into the feature output layer to obtain a visual feature tensor output by the visual feature output layer and a prompt word feature tensor output by the prompt word feature output layer, then perform text encoding on the sample attribute label of the sample character data by using the text encoding layer to obtain a label feature, and finally perform dimension expansion on the label feature by using a full connection layer in the tensor processing layer to form a text feature, perform feature fusion on the text feature and the visual feature tensor to obtain a fusion feature tensor, and generate a text classification tensor based on the label feature and the prompt word feature tensor.

[0076] The visual feature tensor is a pixel feature extracted from an image in the sample character data by using the multi-modal learning model, which plays a main role in model training. The prompt word feature tensor is a classification feature extracted from the image in the sample character data by using the multi-modal learning model, which plays an auxiliary role in model training. The text feature is a classification feature extracted from a sample attribute label in the sample character data by using the multi-modal learning model. However, unlike the prompt word feature tensor, the text feature is from a text and belongs to a soft label form, which can guide classification in model training to improve the recognition ability of the model.

[0077] Specifically, after the feature output layer extracts features from the sample character data, the shapes of the visual feature tensor and the prompt word feature tensor output by the feature output layer are [N, C1, D] and [N, C2, D] respectively, where N is the number of samples, C1 is the number of channels, C2 is the number of channels, and D is the feature dimension. Then, the computer device can first generate a label text of a sample attribute label in the text encoding layer, so that the label text can be text encoded to obtain a label feature with a shape of [Class, D], where Class is the number of classification types under the corresponding attribute. From the shape of the label feature [Class, D] and the shape of the visual feature tensor [N, C1, D], it can be seen that the visual feature tensor has a high dimension, and in the training process, it is input into the model in batches with N as the dimension. Therefore, when the computer device fuses the text feature into the visual feature tensor, the label feature needs to be processed by a full connection layer first, and then dimensionally expanded to [N, Class, D] to form a text feature that matches the visual feature tensor in dimension, so as to ensure that they can effectively fuse features. This alignment enables the fused feature tensor to contain rich information from both text and vision, which helps the model to more accurately understand and recognize character attributes. Next, in the tensor processing layer, the computer device can also generate a fused feature tensor based on the text feature and the visual feature tensor, which has a shape of [N, C3, D], where C3 = C1 + Class. At the same time, the computer device can also generate a text classification tensor based on the label feature and the prompt word feature tensor, which has a shape of [N, Class] and can be used for type classification.

[0078] In one embodiment, the classification layer in step S120 can include a convolution layer and a full connection layer; wherein the step of performing attribute classification on the aggregated fused feature tensor through the classification layer to obtain a target classification tensor can include:

[0079] S124: performing feature mapping on the fused feature tensor using the convolution layer to obtain a fused feature vector.

[0080] S125: performing attribute classification on the fused feature vector using the full connection layer to generate a target classification tensor.

[0081] In this embodiment, the classification layer can include a convolution layer and a full connection layer. When performing attribute classification on the fused feature tensor, the computer device can first perform feature mapping on the fused feature tensor using the convolution layer to obtain a fused feature vector, and then perform attribute classification on the fused feature vector using the full connection layer to generate a target classification tensor, which is taken as the final recognition result of the model.

[0082] It is understandable that the fused feature tensor of this application is a multi-channel multi-dimensional feature vector. To reduce the risk of model overfitting, the computer device can reduce the number of parameters of the tensor through the shared weights of the convolutional layer, that is, the fused feature vector mapping is refined into a multi-dimensional feature vector of one channel, and then converted into a two-dimensional fused feature vector. The fully connected layer can reduce the dimension of the two-dimensional fused feature vector to the number of classification types by integrating the information in the fused feature vector, generating the target classification tensor, which is the final output of the model.

[0083] In one embodiment, determining the parameter loss value of the initial attribute recognition model based on the text classification tensor and the target classification tensor in step S130 may include:

[0084] S131: Determine a first loss function for the text classification tensor and a second loss function for the target classification tensor.

[0085] S132: Using the first loss function to calculate the first loss value of the text classification tensor, and using the second loss function to calculate the second loss value of the target classification tensor, and determining the parameter loss value of the initial attribute recognition model based on the first loss value and the second loss value.

[0086] In this embodiment, when calculating the parameter loss value of the initial attribute recognition model, the computer device can first determine the first loss function of the text classification tensor and the second loss function of the target classification tensor, and then use the first loss function to calculate the first loss value of the text classification tensor, and use the second loss function to calculate the second loss value of the target classification tensor, and determine the parameter loss value of the initial attribute recognition model based on the first loss value and the second loss value.

[0087] It should be noted that both the first and second loss functions can be classification loss functions, including but not limited to LabelSmooth loss function, Focal loss function, and CrossEntropy loss function. The first and second loss functions can use the same classification loss function or different classification loss functions, depending on the actual needs of the passenger flow attribute analysis scenario and are not limited here.

[0088] Specifically, the calculation expression of the parameter loss value can be as follows:

[0089] Loss=aLoss1+bLoss2

[0090] In the formula, Loss represents the parameter loss value; Loss1 represents the first loss function; Loss2 represents the second loss function; a represents the adjustment weight of the first loss function; b represents the adjustment weight of the second loss function.

[0091] In one embodiment, updating the parameters of the initial attribute recognition model based on the parameter loss value in step S130 may include:

[0092] S133: Freeze parameters of the visual transformation network in the initial attribute recognition model and other network layers except the feature output layer in the multimodal learning model;

[0093] S134: Back propagation is performed in the initial attribute recognition model based on the parameter loss value to update the parameters of the feature output layer and the classification layer in the initial attribute recognition model.

[0094] In this embodiment, when the computer device updates the parameters of the initial attribute recognition model, it can first freeze the parameters of the visual transformation network in the initial attribute recognition model and the other network layers in the multimodal learning model except the feature output layer, and then perform backpropagation in the initial attribute recognition model based on the parameter loss value to update the parameters of the feature output layer and the classification layer in the initial attribute recognition model.

[0095] It is understandable that the visual transformation network in the initial attribute recognition model and other network layers in the multimodal learning model except the feature output layer have been fully trained on previous tasks or data sets and have good feature extraction capabilities. This application can freeze these parameters to retain the learned feature representations, avoid relearning these features in subsequent tasks, and reduce the risk of model overfitting due to overtraining.

[0096] In addition, when the computer device performs backpropagation in the initial attribute recognition model based on the parameter loss value, it can accurately calculate the gradient of the feature output layer and classification layer parameters, and then gradually reduce the value of the loss function by updating these parameters along the negative direction of the gradient, thereby learning the data distribution and feature representation that are more suitable for the current task to enhance the recognition performance of the model.

[0097] The attribute recognition model training device provided in an embodiment of the present application is described below. The attribute recognition model training device described below and the attribute recognition model training method described above can be referenced to each other.

[0098] In one embodiment, Figure 3 As shown, Figure 3 This is a schematic diagram of the structure of an attribute recognition model training device provided in an embodiment of the present application. The present application also provides an attribute recognition model training device, including a data acquisition module 210, a tensor generation module 220, and a model training module 230, specifically including the following:

[0099] The data acquisition module 210 is configured to acquire sample person data and an initial attribute recognition model, and the initial attribute recognition model comprises a multi-modal learning model, a visual transformation network and a classification layer.

[0100] The tensor generation module 220 is configured to determine a text classification tensor and a fusion feature tensor of the sample person data by using the multi-modal learning model, aggregate the fusion feature tensor by using the visual transformation network, and perform attribute classification on the aggregated fusion feature tensor by using the classification layer to obtain a target classification tensor.

[0101] The model training module 230 is configured to determine a parameter loss value of the initial attribute recognition model based on the text classification tensor and the target classification tensor, update the parameters of the initial attribute recognition model based on the parameter loss value, and return to the step of determining the text classification tensor and the fusion feature tensor of the sample person data by using the multi-modal learning model and subsequent steps until the initial attribute recognition model meets a preset training end condition to obtain a target attribute recognition model.

[0102] In the above embodiment, when training the model, the sample person data and the initial attribute recognition model can be acquired first, and the initial attribute recognition model comprises a multi-modal learning model, a visual transformation network and a classification layer. When training the initial attribute recognition model, the multi-modal learning model can be used to determine a text classification tensor and a fusion feature tensor of the sample person data, wherein the text classification tensor reflects the category semantic information related to the person attribute in the sample person data, and provides rich and accurate context understanding to improve the accuracy of person attribute recognition; then the visual transformation network can be used to aggregate the fusion feature tensor to fuse multi-scale visual features and text features, so that the fusion features have higher recognition degree; therefore, the target classification tensor obtained by performing attribute classification on the aggregated fusion feature tensor by using the classification layer has high accuracy. Finally, the parameter loss value of the initial attribute recognition model can be determined based on the text classification tensor and the target classification tensor, the parameters of the initial attribute recognition model can be updated based on the parameter loss value, and then the next round of model training is entered until the initial attribute recognition model meets the preset training end condition to obtain a target attribute recognition model. The target attribute recognition model obtained by training can multi-scale aggregate visual features and text features, and thus improve the recognition ability of person attributes.

[0103] In one embodiment, the data acquisition module 210 can comprise:

[0104] The attribute determination sub-module is configured to determine a model attribute to be trained and a plurality of classification types under the model attribute.

[0105] The image acquisition submodule is configured to acquire sample person images corresponding to each classification type in a same proportion, and mark sample attribute labels in each sample person image according to the respective classification types, to form sample person data.

[0106] In one embodiment, the multi-modal learning model can include a feature output layer, a text encoding layer, and a tensor processing layer, the feature output layer including a visual feature output layer and a prompt word feature output layer; and the tensor generation module 220 can include:

[0107] The feature output submodule is configured to input the sample person data into the feature output layer, to obtain a visual feature tensor output by the visual feature output layer and a prompt word feature tensor output by the prompt word feature output layer.

[0108] The text encoding submodule is configured to perform text encoding on sample attribute labels of the sample person data by using the text encoding layer, to obtain label features.

[0109] The feature fusion submodule is configured to perform dimension expansion on the label features after using a full connection layer in the tensor processing layer, to form text features, and perform feature fusion on the text features and the visual feature tensor, to obtain a fusion feature tensor, and generate a text classification tensor based on the label features and the prompt word feature tensor.

[0110] In one embodiment, the tensor generation module 220 can further include:

[0111] The feature mapping submodule is configured to perform feature mapping on the fusion feature tensor by using a convolution layer, to obtain a fusion feature vector.

[0112] The attribute classification submodule is configured to perform attribute classification on the fusion feature vector by using a full connection layer, to generate a target classification tensor.

[0113] In one embodiment, the model training module 230 can include:

[0114] The function determination submodule is configured to determine a first loss function of the text classification tensor and a second loss function of the target classification tensor.

[0115] The loss calculation submodule is configured to calculate a first loss value of the text classification tensor by using the first loss function, and calculate a second loss value of the target classification tensor by using the second loss function, and determine a parameter loss value of the initial attribute recognition model according to the first loss value and the second loss value.

[0116] In one embodiment, the model training module 230 can further include:

[0117] The parameter freezing submodule is configured to freeze parameters of the visual transformation network in the initial attribute recognition model and other network layers in the multi-modal learning model except the feature output layer.

[0118] The parameter updating submodule is configured to perform back propagation in the initial attribute recognition model based on the parameter loss value, so as to update parameters of the feature output layer and the classification layer in the initial attribute recognition model.

[0119] In one embodiment, the present application further provides a storage medium having computer readable instructions stored therein, which, when executed by one or more processors, cause the one or more processors to perform the steps of the attribute recognition model training method according to any one of the above embodiments.

[0120] In one embodiment, the present application further provides a computer device having computer readable instructions stored therein, which, when executed by one or more processors, cause the one or more processors to perform the steps of the attribute recognition model training method according to any one of the above embodiments.

[0121] As shown in Figure 4 , Figure 4 Fig. 3 is a schematic diagram of an internal structure of a computer device according to an embodiment of the present application. The computer device 300 can be provided as a server. As shown in Figure 4 , the computer device 300 includes a processing assembly 302, which further includes one or more processors, and a memory resource represented by a memory 301, for storing instructions executable by the processing assembly 302, such as an application program. The application program stored in the memory 301 can include one or more than one module each corresponding to a set of instructions. In addition, the processing assembly 302 is configured to execute the instructions to perform the attribute recognition model training method according to any one of the above embodiments.

[0122] The computer device 300 can further include a power supply assembly 303 configured to perform power management of the computer device 300, a wired or wireless network interface 304 configured to connect the computer device 300 to a network, and an input / output (I / O) interface 305. The computer device 300 can operate based on an operating system stored in the memory 301, such as Windows Server TM, Mac OS X TM, Unix TM, Linux TM, Free BSD TM, or the like.

[0123] Those skilled in the art can understand Figure 4 that the structure shown in the above embodiments is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. Specifically, the computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0124] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0125] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referenced to each other.

[0126] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for training an attribute recognition model, characterized in that: Methods include: Obtain sample person data and an initial attribute recognition model, which includes a multimodal learning model, a visual transformation network, and a classification layer; The multimodal learning model is used to determine the text classification tensor and fusion feature tensor of the sample character data, and the visual transformation network is used to aggregate the fusion feature tensor. The attributes of the aggregated fusion feature tensor are classified through the classification layer to obtain the target classification tensor. Determine the parameter loss value of the initial attribute recognition model based on the text classification tensor and the target classification tensor, update the parameters of the initial attribute recognition model based on the parameter loss value, and return to determine the text classification tensor and fusion feature tensor of the sample person data using the multimodal learning model and subsequent steps until the initial attribute recognition model meets the preset training end condition, thereby obtaining the target attribute recognition model; Get sample character data, including: Determine the model attributes to be trained and the multiple classification types under the model attributes; Obtain sample person images corresponding to each classification type in equal proportion, and mark sample attribute labels in each sample person image according to each classification type to form sample person data; The multimodal learning model includes a feature output layer, a text encoding layer, and a tensor processing layer; the feature output layer includes a visual feature output layer and a prompt word feature output layer; The multimodal learning model is used to determine the text classification tensor and fusion feature tensor of the sample character data, including: Input the sample person data into the feature output layer to obtain a visual feature tensor output by the visual feature output layer and a prompt word feature tensor output by the prompt word feature output layer. The visual feature tensor is the pixel feature extracted by the multimodal learning model from the image in the sample person data, and the prompt word feature tensor is the classification feature extracted by the multimodal learning model from the image in the sample person data. Use the text encoding layer to perform text encoding on the sample attribute labels of the sample character data to obtain label features; In the tensor processing layer, the label features are subjected to a fully connected layer and then dimensionally expanded to form text features. The text features are then fused with the visual feature tensors to obtain a fused feature tensor. Furthermore, a text classification tensor is generated based on the label features and the prompt word feature tensors. The text features reflect semantic information related to the attributes of the sample character data. The text classification tensor contains category information of the text features in the sample character data and is used for classification. The classification layer includes convolutional layer and fully connected layer; The attributes of the aggregated fusion feature tensor are classified through the classification layer to obtain the target classification tensor, including: Use the convolution layer to perform feature mapping on the fused feature tensor to obtain the fused feature vector; A fully connected layer is used to perform attribute classification on the fused feature vector to generate the target classification tensor.

2. The attribute recognition model training method according to claim 1, characterized in that: Determine the parameter loss value of the initial attribute recognition model based on the text classification tensor and the target classification tensor, including: Determine a first loss function for the text classification tensor and a second loss function for the target classification tensor; A first loss function is used to calculate a first loss value of the text classification tensor, and a second loss function is used to calculate a second loss value of the target classification tensor, and a parameter loss value of the initial attribute recognition model is determined based on the first loss value and the second loss value.

3. An attribute recognition model training device, characterized in that: include: The data acquisition module is used to obtain sample character data and an initial attribute recognition model, which includes a multimodal learning model, a visual transformation network, and a classification layer; A tensor generation module is used to determine the text classification tensor and fusion feature tensor of the sample person data using a multimodal learning model, aggregate the fusion feature tensor using a visual transformation network, and perform attribute classification on the aggregated fusion feature tensor through a classification layer to obtain a target classification tensor; A model training module is used to determine the parameter loss value of the initial attribute recognition model based on the text classification tensor and the target classification tensor, update the parameters of the initial attribute recognition model based on the parameter loss value, and return the text classification tensor and fusion feature tensor of the sample person data determined by the multimodal learning model and subsequent steps until the initial attribute recognition model meets the preset training end conditions, thereby obtaining the target attribute recognition model; The data acquisition module includes: The attribute determination submodule is used to determine the model attributes to be trained and the multiple classification types under the model attributes; The image acquisition submodule is used to acquire sample person images corresponding to each classification type in equal proportions, and mark sample attribute labels in each sample person image according to each classification type to form sample person data; The multimodal learning model includes a feature output layer, a text encoding layer, and a tensor processing layer; the feature output layer includes a visual feature output layer and a prompt word feature output layer; The multimodal learning model is used to determine the text classification tensor and fusion feature tensor of the sample character data, including: Input the sample person data into the feature output layer to obtain a visual feature tensor output by the visual feature output layer and a prompt word feature tensor output by the prompt word feature output layer. The visual feature tensor is the pixel feature extracted by the multimodal learning model from the image in the sample person data, and the prompt word feature tensor is the classification feature extracted by the multimodal learning model from the image in the sample person data. Use the text encoding layer to perform text encoding on the sample attribute labels of the sample character data to obtain text features; In the tensor processing layer, the label features are dimensionally expanded after using a fully connected layer to form label features, and the text features and visual feature tensors are fused to obtain a fused feature tensor. In addition, a text classification tensor is generated based on the label features and the prompt word feature tensors. The text features reflect semantic information related to the character attributes in the sample character data. The text classification tensor contains category information of the text features in the sample character data and is used for classification. The classification layer includes convolutional layer and fully connected layer; The attributes of the aggregated fusion feature tensor are classified through the classification layer to obtain the target classification tensor, including: Use the convolution layer to perform feature mapping on the fused feature tensor to obtain the fused feature vector; A fully connected layer is used to perform attribute classification on the fused feature vector to generate the target classification tensor.

4. A storage medium, characterized in that: The storage medium stores computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the attribute recognition model training method according to any one of claims 1 to 2.

5. A computer device, characterized in that: include: one or more processors, and memory; The memory stores computer-readable instructions, which, when executed by one or more processors, execute the steps of the attribute recognition model training method according to any one of claims 1 to 2.

Images