Attribute recognition model training method and device, storage medium and computer equipment

Abstract

According to the attribute recognition model training method and device, the storage medium and the computer equipment provided by the invention, during model training, the sample character data and the initial attribute recognition model composed of the multi-modal learning model, the visual transformation network and the classification layer are firstly acquired; when the model is trained, a text classification tensor and a fusion feature tensor of sample character data can be determined by using a multi-modal learning model, and the fusion feature tensor is aggregated by using a visual transformation network so as to fuse multi-scale visual features and text features; therefore, the target classification tensor obtained by carrying out attribute classification on the aggregated fusion feature tensor through the classification layer has relatively high accuracy; and then determining a parameter loss value based on the text classification tensor and the target classification tensor so as to update model parameters and enter next round of training until the model meets a training ending condition, so that the model obtained by training can aggregate visual features and text features in a multi-scale manner, and the character attribute recognition capability is further improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an attribute recognition model training method, device, storage medium and computer equipment. Background Art

[0002] With the rapid development of computer vision technology, human attribute recognition, as a frontier field of intelligent video surveillance, is becoming an indispensable component. Relying on the powerful capabilities of cutting-edge algorithms such as deep learning, this technology can automatically mine the deep information of pedestrians from surveillance videos and achieve accurate mapping to high-level semantic levels, including but not limited to refined attribute classification such as gender recognition, age estimation, and clothing style analysis.

[0003] However, existing person attribute recognition performs poorly when dealing with complex scenes. For example, when faced with occlusion, lighting changes, and high background complexity, the model finds it difficult to effectively extract and utilize key person features. In addition, traditional methods cannot fully integrate multi-scale visual features, resulting in the inability to effectively coordinate features at different levels, which affects recognition accuracy and makes the recognition ability of person attributes poor. Summary of the invention

[0004] The purpose of this application is to solve at least one of the above-mentioned technical defects, especially the technical defect that the prior art cannot fully integrate multi-scale visual features, resulting in the inability to effectively coordinate features at different levels, thereby affecting the recognition accuracy and making the recognition ability of person attributes poor.

[0005] The present application provides an attribute recognition model training method, the method comprising:

[0006] Acquire sample person data and an initial attribute recognition model, wherein the initial attribute recognition model includes a multimodal learning model, a visual transformation network, and a classification layer;

[0007] Determine the text classification tensor and fusion feature tensor of the sample character data by using the multimodal learning model, aggregate the fusion feature tensor by using the visual transformation network, and classify the attributes of the aggregated fusion feature tensor by using the classification layer to obtain a target classification tensor;

[0008] Based on the text classification tensor and the target classification tensor, the parameter loss value of the initial attribute recognition model is determined to update the parameters of the initial attribute recognition model based on the parameter loss value, and the text classification tensor and fusion feature tensor of the sample character data determined by using the multimodal learning model and subsequent steps are returned until the initial attribute recognition model meets the preset training end conditions to obtain the target attribute recognition model.

[0009] Optionally, obtaining sample person data includes:

[0010] Determining a model attribute to be trained and a plurality of classification types under the model attribute;

[0011] Sample person images corresponding to each classification type are obtained in equal proportion, and sample attribute labels are marked in each sample person image according to each classification type to form sample person data.

[0012] Optionally, the multimodal learning model includes a feature output layer, a text encoding layer, and a tensor processing layer; the feature output layer includes a visual feature output layer and a prompt word feature output layer;

[0013] The method of using the multimodal learning model to determine the text classification tensor and the fusion feature tensor of the sample character data includes:

[0014] Inputting the sample character data into the feature output layer to obtain a visual feature tensor output by the visual feature output layer and a prompt word feature tensor output by the prompt word feature output layer;

[0015] Using the text encoding layer to perform text encoding on the sample attribute labels of the sample person data to obtain label features;

[0016] In the tensor processing layer, the label feature is dimensionally expanded after using a fully connected layer to form a text feature, and the text feature is fused with the visual feature tensor to obtain a fused feature tensor, and a text classification tensor is generated based on the label feature and the prompt word feature tensor.

[0017] Optionally, the classification layer includes a convolutional layer and a fully connected layer;

[0018] The method of performing attribute classification on the aggregated fusion feature tensor through the classification layer to obtain a target classification tensor includes:

[0019] Performing feature mapping on the fused feature tensor using the convolutional layer to obtain a fused feature vector;

[0020] The fully connected layer is used to perform attribute classification on the fused feature vector to generate a target classification tensor.

[0021] Optionally, determining a parameter loss value of the initial attribute recognition model based on the text classification tensor and the target classification tensor includes:

[0022] Determining a first loss function of the text classification tensor and a second loss function of the target classification tensor;

[0023] The first loss function is used to calculate the first loss value of the text classification tensor, and the second loss function is used to calculate the second loss value of the target classification tensor, and the parameter loss value of the initial attribute recognition model is determined according to the first loss value and the second loss value.

[0024] Optionally, updating the parameters of the initial attribute recognition model based on the parameter loss value includes:

[0025] Freezing parameters of the visual transformation network in the initial attribute recognition model and other network layers in the multimodal learning model except the feature output layer;

[0026] Back propagation is performed in the initial attribute recognition model based on the parameter loss value to update the parameters of the feature output layer and the classification layer in the initial attribute recognition model.

[0027] The present application also provides an attribute recognition model training device, comprising:

[0028] A data acquisition module, used to acquire sample character data and an initial attribute recognition model, wherein the initial attribute recognition model includes a multimodal learning model, a visual transformation network, and a classification layer;

[0029] A tensor generation module, used to determine the text classification tensor and fusion feature tensor of the sample character data by using the multimodal learning model, aggregate the fusion feature tensor by using the visual transformation network, and perform attribute classification on the aggregated fusion feature tensor by using the classification layer to obtain a target classification tensor;

[0030] A model training module is used to determine the parameter loss value of the initial attribute recognition model based on the text classification tensor and the target classification tensor, to update the parameters of the initial attribute recognition model based on the parameter loss value, and to return the text classification tensor and fusion feature tensor of the sample character data determined by using the multimodal learning model and its subsequent steps, until the initial attribute recognition model meets the preset training end conditions to obtain the target attribute recognition model.

[0031] The present application also provides a storage medium, in which computer-readable instructions are stored. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the attribute recognition model training method described in any of the above embodiments.

[0032] The present application also provides a computer device, comprising: one or more processors, and a memory;

[0033] The memory stores computer-readable instructions, and when the computer-readable instructions are executed by the one or more processors, the steps of the attribute recognition model training method described in any one of the above embodiments are performed.

[0034] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0035] The attribute recognition model training method, device, storage medium and computer equipment provided by the present application can first obtain sample character data and an initial attribute recognition model when training the model, wherein the initial attribute recognition model includes a multimodal learning model, a visual transformation network and a classification layer. When training the initial attribute recognition model, the multimodal learning model can be used to first determine the text classification tensor and fusion feature tensor of the sample character data, wherein the text classification tensor reflects the category semantic information related to the character attributes in the sample character data, and improves the accuracy of character attribute recognition by providing rich and accurate contextual understanding; then the visual transformation network can be used to aggregate the fusion feature tensor to fuse multi-scale visual features and text features, so that the fusion features are more recognizable; therefore, the target classification tensor obtained by attribute classification of the aggregated fusion feature tensor through the classification layer has a higher accuracy. Finally, the present application can determine the parameter loss value of the initial attribute recognition model based on the text classification tensor and the target classification tensor, update the parameters of the initial attribute recognition model based on the parameter loss value, and then enter the next round of model training until the initial attribute recognition model meets the preset training end conditions to obtain the target attribute recognition model. The target attribute recognition model trained in this way can aggregate visual features and text features at multiple scales, thereby improving the recognition ability of character attributes. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0037] Figure 1 A flowchart of an attribute recognition model training method provided in an embodiment of the present application;

[0038] Figure 2 A flowchart of a multimodal learning model processing process provided in an embodiment of the present application;

[0039] Figure 3 A schematic diagram of the structure of an attribute recognition model training device provided in an embodiment of the present application;

[0040] Figure 4 A schematic diagram of the internal structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0042] Existing person attribute recognition performs poorly when dealing with complex scenes. For example, when faced with occlusion, lighting changes, and high background complexity, it is difficult for the model to effectively extract and utilize key person features. In addition, traditional methods cannot fully integrate multi-scale visual features, resulting in the inability to effectively coordinate features at different levels, which affects recognition accuracy and makes the recognition ability of person attributes poor.

[0043] Based on this, this application proposes the following technical solutions, see below for details:

[0044] In one embodiment, Figure 1 As shown, Figure 1 A flow chart of an attribute recognition model training method provided in an embodiment of the present application; the present application provides an attribute recognition model training method, which specifically includes the following:

[0045] S110: Obtain sample person data and an initial attribute recognition model, where the initial attribute recognition model includes a multimodal learning model, a visual transformation network, and a classification layer.

[0046] In this step, when the computer device is training a model for type recognition of a certain attribute of a person in a passenger flow attribute analysis scenario, it can first obtain sample person data containing various types of person images, and can also obtain an initial attribute recognition model as a pre-trained model for training the sample person data. The initial attribute recognition model can include a multimodal learning model, a visual transformation network, and a classification layer.

[0047] It can be understood that a multimodal learning model refers to a model that can process and fuse data from different modalities, and aims to improve the performance of tasks by combining information from multiple modalities in the input data through a multimodal learning model. Here, this application can adopt the CLIP (Contrastive Language-Image Pre-Training) model, which can map the image information and text information in the sample character data to the same vector space, enhance the model's understanding and processing capabilities of visual concepts, and thus improve the model's recognition ability.

[0048] In addition, the visual transformation network can capture global information in the image and excels in the fusion of multi-scale features. In person attribute recognition, it can effectively extract and aggregate visual features of different scales, such as extracting key attributes of people under different lighting, angles or occlusion conditions, thereby further improving the recognition ability of the model. The classification layer is the final layer of the initial attribute recognition model, which is mainly responsible for mapping the features extracted by the model to specific category labels. It can receive features output from the visual transformation network and map these features to predefined person attribute categories; for example, the model can classify the attribute type of the input person image as "male" or "female".

[0049] Furthermore, the classification layer of the initial attribute recognition model in the present application can be one or more. When the model needs to recognize only one attribute of a person, such as gender, the computer device can use a classification layer that has pre-learned various types of labels corresponding to the attribute gender to connect to the visual transformation network when constructing the initial attribute recognition model, so that the attribute corresponding to the recognition result output by the model during the training process is only gender; when the model needs to recognize multiple attributes of a person, such as gender and age, the computer device can use a classification layer corresponding to gender and a classification layer corresponding to age to connect to the visual transformation network respectively, so that the attribute corresponding to the recognition result output by the model during the training process can include both gender and age.

[0050] S120: Using a multimodal learning model to determine the text classification tensor and fusion feature tensor of the sample character data, and using a visual transformation network to aggregate the fusion feature tensor, and performing attribute classification on the aggregated fusion feature tensor through a classification layer to obtain a target classification tensor.

[0051] In this step, after obtaining the sample character data and the initial attribute recognition model through step S110, the computer device can first use the multimodal learning model to determine the text classification tensor and the fused feature tensor of the sample character data, and then use the visual transformation network to aggregate the fused feature tensor, and perform attribute classification on the aggregated fused feature tensor through the classification layer to obtain the target classification tensor. Finally, the computer device can perform attribute classification on the aggregated fused feature tensor through the classification layer to obtain the target classification tensor.

[0052] It can be understood that the text classification tensor contains the category information of the text features in the sample character data and can be used for classification, and the text features therein can reflect the semantic information related to the character attributes in the sample character data. It can supplement the semantic information that may be missed in the image visual features and provide richer and more accurate contextual understanding, thereby improving the accuracy of character attribute recognition; and the fused feature tensor directly fuses the two different modal data of image and text in the sample character data. It covers various types of character images and their corresponding text descriptions in the sample character data, so the model can learn a very wide range of visual concepts, thereby improving the recognition accuracy of the model.

[0053] Furthermore, as the final layer of the initial attribute recognition model, the classification layer has pre-learned the feature distribution of each type of label under the corresponding attribute. Therefore, it can directly perform feature classification on the input fused feature tensor, and then output the target classification tensor to represent the final recognition result of the corresponding person image in the sample person data.

[0054] S130: Determine a parameter loss value of the initial attribute recognition model based on the text classification tensor and the target classification tensor, so as to update the parameters of the initial attribute recognition model based on the parameter loss value.

[0055] In this step, after obtaining the text classification tensor and the target classification tensor through step S120, the computer device can determine the parameter loss value of the initial attribute recognition model based on the text classification tensor and the target classification tensor, and then update the parameters of the initial attribute recognition model based on the parameter loss value to complete the training of the initial attribute recognition model in the current round.

[0056] It can be understood that the text classification tensor is used as a semantic representation of the categories related to the character attributes in the sample character data, while the target classification tensor is used as a fusion representation of the visual and text of the character image in the sample character data. The parameter loss values ​​calculated by these two feature tensors are used to update the parameters of the initial attribute recognition model, which can ensure that the model can pay attention to both text and visual information during the training process, thereby improving the effective fusion of the text features and visual features of the image and achieving more accurate attribute recognition.

[0057] Specifically, the parameter loss value provides a feedback signal for the model, indicating the performance of the model in the current round of training iterations. Therefore, after obtaining the parameter loss value, the computer device can derive the parameter loss value to obtain the gradient of each parameter in the model. These gradients indicate the direction and size of each parameter that should be adjusted; then the computer device can perform back propagation through the chain calculation method, so that each parameter of the model is iteratively updated in the direction of reducing the loss value.

[0058] S140: Determine whether the initial attribute recognition model meets a preset training end condition.

[0059] S150: If not, return to step S120; if yes, end the training and use the initial attribute recognition model as the target attribute recognition model.

[0060] In this embodiment, after updating the parameters of the initial attribute recognition model in step S130, the computer device can regard the current round of model training as finished, and then determine whether the initial attribute recognition model meets the preset training end condition. If not, the model needs to enter the next round of training, so the computer device can continue to train the initial attribute recognition model using the sample person data; if so, the model can be regarded as trained, so the computer device can use the trained initial attribute recognition model as the target attribute recognition model.

[0061] Among them, the preset training end condition can be that the model meets certain training conditions or parameter convergence conditions, such as the number of iterations reaching a set value, etc., which is not limited here.

[0062] In the above embodiment, when training the model, sample character data and an initial attribute recognition model may be obtained first, where the initial attribute recognition model includes a multimodal learning model, a visual transformation network, and a classification layer. When training the initial attribute recognition model, the multimodal learning model may be used to determine the text classification tensor and fused feature tensor of the sample character data, wherein the text classification tensor reflects the category semantic information related to the character attributes in the sample character data, and improves the accuracy of character attribute recognition by providing rich and accurate contextual understanding; then the visual transformation network may be used to aggregate the fused feature tensor to fuse multi-scale visual features and text features, so that the fused features are more recognizable; therefore, the target classification tensor obtained by attribute classification of the aggregated fused feature tensor through the classification layer has a higher accuracy. Finally, the present application can determine the parameter loss value of the initial attribute recognition model based on the text classification tensor and the target classification tensor, update the parameters of the initial attribute recognition model based on the parameter loss value, and then enter the next round of model training until the initial attribute recognition model meets the preset training end conditions to obtain the target attribute recognition model. The target attribute recognition model trained in this way can aggregate visual features and text features at multiple scales, thereby improving the recognition ability of character attributes.

[0063] In one embodiment, obtaining sample person data in step S110 may include:

[0064] S111: Determine a model attribute to be trained and multiple classification types under the model attribute.

[0065] S112: Obtain sample person images corresponding to each classification type in equal proportion, and mark sample attribute labels in each sample person image according to each classification type to form sample person data.

[0066] In this embodiment, when the computer device acquires sample person data, it can first determine the model attribute to be trained and multiple classification types under the model attribute; for example, when the model attribute is gender, its classification type can be divided into male and female; when the model attribute is age, its classification type can be divided into teenagers, middle-aged, and elderly. Then, the computer device can acquire sample person images corresponding to each classification type in equal proportion, and then annotate the sample attribute label, i.e., the corresponding classification type, in each sample person image based on each classification type, and perform image mixing on each annotated sample person image to form sample person data, so as to improve the efficiency and generalization ability of the model when using the sample person data for training.

[0067] It can be understood that the present application obtains sample character images corresponding to each classification type in equal proportion, which can ensure that the number of samples of each classification type in the data set is roughly equal, so that the model can learn the feature distribution of each classification type during the training process, rather than being biased towards one or certain categories, thereby improving the generalization ability of the model; in addition, by mixing sample character images of each classification type, the model can be exposed to more diverse data during the training process, and can learn more generalized feature representations, avoiding model overfitting.

[0068] Furthermore, after acquiring the sample character image, the computer device may perform a preprocessing operation on the sample character image, and the preprocessing operation includes but is not limited to normalization, sharpening, denoising, etc. Among them, normalization refers to mapping the data of each dimension of the data vector to an interval between (0, 1) or (-1, 1), or mapping a norm of the data vector to 1. It can be understood that the pixel values ​​of the sample character images in the present application are integers in the range of 0 to 255. Although the pixel data of the original image can be directly used as the training data of the model, it may cause some problems in the training process of the model, such as causing the model training speed to slow down, too many iterations leading to slower convergence, etc. Therefore, the present application can improve the efficiency of model training by normalization.

[0069] The sharpening process here refers to compensating the contour of the sample character image, enhancing the edge of the image and the grayscale jump part, making the image clear. It can be divided into two categories: spatial domain processing and frequency domain processing. By highlighting the edges, contours, or features of certain linear target elements of the character on the image, the contrast between the edge of the character and the surrounding pixels can be improved. De-noising refers to the process of reducing noise in digital images. Generally, the image is often affected by the interference of imaging equipment and external environmental noise during digitization and transmission. That is, the image information received generally includes noise, and these noises will become an important cause of image interference. By denoising the image, the noise in the image is removed, and the authenticity and accuracy of the obtained image are further improved.

[0070] By performing the preprocessing operation on the acquired sample person image, the sample person image can be scaled to a suitable size and the clarity of the image itself can be effectively improved, so as to facilitate the subsequent processing of the sample person data.

[0071] In one embodiment, Figure 2 As shown, Figure 2 A flowchart of a multimodal learning model processing process provided in an embodiment of the present application; Figure 2In step S120, the multimodal learning model may include a feature output layer, a text encoding layer, and a tensor processing layer, and the feature output layer includes a visual feature output layer and a prompt word feature output layer; wherein the step of determining the text classification tensor and the fusion feature tensor of the sample character data by using the multimodal learning model may include:

[0072] S121: Inputting the sample person data into the feature output layer, obtaining a visual feature tensor output by the visual feature output layer and a prompt word feature tensor output by the prompt word feature output layer.

[0073] S122: Using the text encoding layer, text encoding is performed on the sample attribute labels of the sample person data to obtain label features.

[0074] S123: In the tensor processing layer, the label feature is dimensionally expanded after using the fully connected layer to form a text feature, and the text feature and the visual feature tensor are feature fused to obtain a fused feature tensor, and a text classification tensor is generated based on the label feature and the prompt word feature tensor.

[0075] In this embodiment, when the computer device uses a multimodal learning model to determine the text classification tensor and fusion feature tensor of sample character data, it can first input the sample character data into the feature output layer to obtain the visual feature tensor output by the visual feature output layer and the prompt word feature tensor output by the prompt word feature output layer, and then use the text encoding layer to perform text encoding on the sample attribute labels of the sample character data to obtain label features. Finally, the computer device can use a fully connected layer in the tensor processing layer to perform dimension expansion on the label features to form text features, and perform feature fusion on the text features and the visual feature tensor to obtain a fused feature tensor, and generate a text classification tensor based on the label features and the prompt word feature tensors.

[0076] Among them, the visual feature tensor is the pixel feature extracted by the multimodal learning model from the image in the sample character data, which plays a main role in the model training; the cue word feature tensor is the classification feature extracted by the multimodal learning model from the image in the sample character data, which plays an auxiliary role in the model training; the text feature is the classification feature extracted by the multimodal learning model from the sample attribute label in the sample character data. However, unlike the cue word feature tensor, the text feature comes from the text and is in the form of a soft label, which can guide classification in the model training to improve the recognition ability of the model.

[0077] Specifically, after the feature output layer extracts features from the sample person data, the shapes of the visual feature tensor and the prompt word feature tensor outputted by the feature output layer are [N, C1, D] and [N, C2, D], respectively, where N is the number of samples, C1 is the number of channels, C2 is the number of channels, and D is the feature dimension. Then, the computer device can first generate the label text of the sample attribute label in the text encoding layer, so that the label text can be text encoded to obtain the label feature, whose shape is shown as [Class, D], where Class is the number of classification types under the corresponding attribute. From the shape of the label feature [Class, D] and the shape of the visual feature tensor [N, C1, D], it can be seen that the visual feature tensor has a higher dimension and is input into the model in batches with N as the dimension during the training process. Therefore, when the computer device fuses the text feature into the visual feature tensor, it is necessary to first perform a full-connected layer processing on the label feature, and then expand the dimension to [N, Class, D] to form a text feature so that it matches the visual feature tensor in terms of dimension, ensuring that they can perform effective feature fusion. This alignment enables the fused fusion feature tensor to contain rich information from both text and vision, which helps the model understand and identify person attributes more accurately. Next, in the tensor processing layer, the computer device can also generate a fused feature tensor based on the text feature and visual feature tensors, with a shape of [N, C3, D], where C3 = C1 + Class. At the same time, the computer device can also generate a text classification tensor based on the label feature and prompt word feature tensors, with a shape of [N, Class], which can be used for type classification.

[0078] In one embodiment, the classification layer in step S120 may include a convolutional layer and a fully connected layer; wherein the step of performing attribute classification on the aggregated fusion feature tensor through the classification layer to obtain the target classification tensor may include:

[0079] S124: Use a convolutional layer to perform feature mapping on the fused feature tensor to obtain a fused feature vector.

[0080] S125: Use the fully connected layer to classify the attributes of the fused feature vector and generate the target classification tensor.

[0081] In this embodiment, the classification layer may include a convolutional layer and a fully connected layer. When the computer device performs attribute classification on the fused feature tensor, it may first use the convolutional layer to perform feature mapping on the fused feature tensor to obtain a fused feature vector, and then use the fully connected layer to perform attribute classification on the fused feature vector to generate a target classification tensor, so as to use the target classification tensor as the final recognition result of the model.

[0082] It is understandable that the fused feature tensor of the present application belongs to a multi-dimensional feature vector of multiple channels. To reduce the risk of overfitting of the model, the computer device can reduce the number of parameters of the tensor through the shared weights of the convolutional layer, that is, the fused feature vector mapping is refined into a multi-dimensional feature vector of one channel, and then equivalently converted into a two-dimensional fused feature vector. The fully connected layer can reduce the dimension of the two-dimensional fused feature vector to the number of classification types by comprehensively integrating the information in the fused feature vector, and generate the target classification tensor, that is, the final output of the model.

[0083] In one embodiment, determining the parameter loss value of the initial attribute recognition model based on the text classification tensor and the target classification tensor in step S130 may include:

[0084] S131: Determine a first loss function of a text classification tensor and a second loss function of a target classification tensor.

[0085] S132: A first loss function is used to calculate a first loss value of a text classification tensor, and a second loss function is used to calculate a second loss value of a target classification tensor, and a parameter loss value of an initial attribute recognition model is determined based on the first loss value and the second loss value.

[0086] In this embodiment, when calculating the parameter loss value of the initial attribute recognition model, the computer device can first determine the first loss function of the text classification tensor and the second loss function of the target classification tensor, and then use the first loss function to calculate the first loss value of the text classification tensor, and use the second loss function to calculate the second loss value of the target classification tensor, and determine the parameter loss value of the initial attribute recognition model based on the first loss value and the second loss value.

[0087] It should be noted that the first loss function and the second loss function here can both select classification loss functions, which include but are not limited to LabelSmooth loss function, Focal loss function, and CrossEntropy loss function. Among them, the first loss function and the second loss function can use the same classification loss function, or different classification loss functions can be selected, which can be determined according to the actual needs of the passenger flow attribute analysis scenario, and is not limited here.

[0088] Specifically, the calculation expression of the parameter loss value can be as follows:

[0089] Loss = aLoss1 + bLoss2

[0090] In the formula, Loss represents the parameter loss value; Loss1 represents the first loss function; Loss2 represents the second loss function; a represents the adjustment weight of the first loss function; b represents the adjustment weight of the second loss function.

[0091] In one embodiment, updating the parameters of the initial attribute recognition model based on the parameter loss value in step S130 may include:

[0092] S133: freezing parameters of the visual transformation network in the initial attribute recognition model and other network layers except the feature output layer in the multimodal learning model;

[0093] S134: Perform back propagation in the initial attribute recognition model based on the parameter loss value to update the parameters of the feature output layer and the classification layer in the initial attribute recognition model.

[0094] In this embodiment, when the computer device updates the parameters of the initial attribute recognition model, it can first freeze the parameters of the visual transformation network in the initial attribute recognition model and the other network layers in the multimodal learning model except the feature output layer, and then perform back propagation in the initial attribute recognition model based on the parameter loss value to update the parameters of the feature output layer and the classification layer in the initial attribute recognition model.

[0095] It is understandable that the visual transformation network in the initial attribute recognition model and other network layers in the multimodal learning model except the feature output layer have been fully trained on previous tasks or data sets and have good feature extraction capabilities. The present application can freeze these parameters, retain the learned feature representations, avoid relearning these features in subsequent tasks, and reduce the risk of over-fitting of the model due to over-training.

[0096] In addition, when the computer device performs backpropagation in the initial attribute recognition model based on the parameter loss value, it can accurately calculate the gradient of the feature output layer and classification layer parameters, and then gradually reduce the value of the loss function by updating these parameters along the negative direction of the gradient, thereby learning the data distribution and feature representation that are more suitable for the current task to enhance the recognition performance of the model.

[0097] The attribute recognition model training device provided in an embodiment of the present application is described below. The attribute recognition model training device described below and the attribute recognition model training method described above can be referenced to each other.

[0098] In one embodiment, Figure 3 As shown, Figure 3 A structural diagram of an attribute recognition model training device provided in an embodiment of the present application; the present application also provides an attribute recognition model training device, including a data acquisition module 210, a tensor generation module 220 and a model training module 230, specifically including the following:

[0099] The data acquisition module 210 is used to acquire sample character data and an initial attribute recognition model, which includes a multimodal learning model, a visual transformation network and a classification layer.

[0100] The tensor generation module 220 is used to determine the text classification tensor and fusion feature tensor of the sample character data using a multimodal learning model, aggregate the fusion feature tensor using a visual transformation network, and perform attribute classification on the aggregated fusion feature tensor through a classification layer to obtain a target classification tensor.

[0101] The model training module 230 is used to determine the parameter loss value of the initial attribute recognition model based on the text classification tensor and the target classification tensor, to update the parameters of the initial attribute recognition model based on the parameter loss value, and to return to the text classification tensor and fusion feature tensor of the sample character data determined by the multimodal learning model and its subsequent steps until the initial attribute recognition model meets the preset training end conditions to obtain the target attribute recognition model.

[0102] In the above embodiment, when training the model, sample character data and an initial attribute recognition model may be obtained first, where the initial attribute recognition model includes a multimodal learning model, a visual transformation network, and a classification layer. When training the initial attribute recognition model, the multimodal learning model may be used to determine the text classification tensor and fused feature tensor of the sample character data, wherein the text classification tensor reflects the category semantic information related to the character attributes in the sample character data, and improves the accuracy of character attribute recognition by providing rich and accurate contextual understanding; then the visual transformation network may be used to aggregate the fused feature tensor to fuse multi-scale visual features and text features, so that the fused features are more recognizable; therefore, the target classification tensor obtained by attribute classification of the aggregated fused feature tensor through the classification layer has a higher accuracy. Finally, the present application can determine the parameter loss value of the initial attribute recognition model based on the text classification tensor and the target classification tensor, update the parameters of the initial attribute recognition model based on the parameter loss value, and then enter the next round of model training until the initial attribute recognition model meets the preset training end conditions to obtain the target attribute recognition model. The target attribute recognition model trained in this way can aggregate visual features and text features at multiple scales, thereby improving the recognition ability of character attributes.

[0103] In one embodiment, the data acquisition module 210 may include:

[0104] The attribute determination submodule is used to determine the model attributes to be trained and multiple classification types under the model attributes.

[0105] The image acquisition submodule is used to acquire sample person images corresponding to each classification type in equal proportion, and mark sample attribute labels in each sample person image according to each classification type to form sample person data.

[0106] In one embodiment, the multimodal learning model may include a feature output layer, a text encoding layer, and a tensor processing layer, wherein the feature output layer includes a visual feature output layer and a prompt word feature output layer; the tensor generation module 220 may include:

[0107] The feature output submodule is used to input the sample character data into the feature output layer to obtain the visual feature tensor output by the visual feature output layer and the prompt word feature tensor output by the prompt word feature output layer.

[0108] The text encoding submodule is used to perform text encoding on the sample attribute labels of the sample character data using the text encoding layer to obtain label features.

[0109] The feature fusion submodule is used to perform dimension expansion on the label feature in the tensor processing layer after using the fully connected layer to form text features, and to fuse the text features and visual feature tensors to obtain a fused feature tensor, and to generate a text classification tensor based on the label features and the prompt word feature tensors.

[0110] In one embodiment, the tensor generation module 220 may further include:

[0111] The feature mapping submodule is used to perform feature mapping on the fused feature tensor using a convolutional layer to obtain a fused feature vector.

[0112] The attribute classification submodule is used to classify the attributes of the fused feature vector using a fully connected layer to generate a target classification tensor.

[0113] In one embodiment, the model training module 230 may include:

[0114] The function determination submodule is used to determine the first loss function of the text classification tensor and the second loss function of the target classification tensor.

[0115] The loss calculation submodule is used to calculate the first loss value of the text classification tensor using the first loss function, and to calculate the second loss value of the target classification tensor using the second loss function, and to determine the parameter loss value of the initial attribute recognition model based on the first loss value and the second loss value.

[0116] In one embodiment, the model training module 230 may further include:

[0117] The parameter freezing submodule is used to freeze the parameters of the visual transformation network in the initial attribute recognition model and other network layers except the feature output layer in the multimodal learning model;

[0118] The parameter updating submodule is used to perform back propagation in the initial attribute recognition model based on the parameter loss value to update the parameters of the feature output layer and the classification layer in the initial attribute recognition model.

[0119] In one embodiment, the present application also provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the attribute recognition model training method described in any of the above embodiments.

[0120] In one embodiment, the present application also provides a computer device having computer-readable instructions stored therein. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the attribute recognition model training method as described in any one of the above embodiments.

[0121] Indicatively, Figure 4 As shown, Figure 4 This is a schematic diagram of the internal structure of a computer device provided in an embodiment of the present application. The computer device 300 may be provided as a server. Figure 4 The computer device 300 includes a processing component 302, which further includes one or more processors, and a memory resource represented by a memory 301, for storing instructions executable by the processing component 302, such as an application. The application stored in the memory 301 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 302 is configured to execute instructions to perform the attribute recognition model training method of any of the above embodiments.

[0122] The computer device 300 may further include a power supply component 303 configured to perform power management of the computer device 300, a wired or wireless network interface 304 configured to connect the computer device 300 to a network, and an input / output (I / O) interface 305. The computer device 300 may operate based on an operating system stored in the memory 301, such as Windows Server TM, Mac OS X TM, Unix TM, Linux TM, Free BSD TM, or the like.

[0123] Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0124] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0125] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can refer to each other.

[0126] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for training an attribute recognition model, characterized in that: The method comprises: Acquire sample person data and an initial attribute recognition model, wherein the initial attribute recognition model includes a multimodal learning model, a visual transformation network, and a classification layer; Determine the text classification tensor and fusion feature tensor of the sample character data by using the multimodal learning model, aggregate the fusion feature tensor by using the visual transformation network, and classify the attributes of the aggregated fusion feature tensor by using the classification layer to obtain a target classification tensor; Based on the text classification tensor and the target classification tensor, the parameter loss value of the initial attribute recognition model is determined to update the parameters of the initial attribute recognition model based on the parameter loss value, and the text classification tensor and fusion feature tensor of the sample character data determined by using the multimodal learning model and subsequent steps are returned until the initial attribute recognition model meets the preset training end conditions to obtain the target attribute recognition model.

2. The attribute recognition model training method according to claim 1, characterized in that: The obtaining of sample person data comprises: Determining a model attribute to be trained and a plurality of classification types under the model attribute; Sample person images corresponding to each classification type are obtained in equal proportion, and sample attribute labels are marked in each sample person image according to each classification type to form sample person data.

3. The attribute recognition model training method according to claim 1, characterized in that: The multimodal learning model includes a feature output layer, a text encoding layer, and a tensor processing layer; the feature output layer includes a visual feature output layer and a prompt word feature output layer; The method of using the multimodal learning model to determine the text classification tensor and the fusion feature tensor of the sample character data includes: Inputting the sample character data into the feature output layer to obtain a visual feature tensor output by the visual feature output layer and a prompt word feature tensor output by the prompt word feature output layer; Using the text encoding layer to perform text encoding on the sample attribute labels of the sample character data to obtain text features; In the tensor processing layer, the label feature is dimensionally expanded after using a fully connected layer to form a text feature, and the text feature is fused with the visual feature tensor to obtain a fused feature tensor, and a text classification tensor is generated based on the label feature and the prompt word feature tensor.

4. The attribute recognition model training method according to claim 1, characterized in that: The classification layer includes a convolutional layer and a fully connected layer; The method of performing attribute classification on the aggregated fusion feature tensor through the classification layer to obtain a target classification tensor includes: Performing feature mapping on the fused feature tensor using the convolutional layer to obtain a fused feature vector; The fully connected layer is used to perform attribute classification on the fused feature vector to generate a target classification tensor.

5. The attribute recognition model training method according to claim 1, characterized in that: The determining the parameter loss value of the initial attribute recognition model based on the text classification tensor and the target classification tensor includes: Determining a first loss function of the text classification tensor and a second loss function of the target classification tensor; The first loss function is used to calculate the first loss value of the text classification tensor, and the second loss function is used to calculate the second loss value of the target classification tensor, and the parameter loss value of the initial attribute recognition model is determined according to the first loss value and the second loss value.

6. The attribute recognition model training method according to claim 1, characterized in that: The updating of the parameters of the initial attribute recognition model based on the parameter loss value comprises: Freezing parameters of the visual transformation network in the initial attribute recognition model and other network layers in the multimodal learning model except the feature output layer; Back propagation is performed in the initial attribute recognition model based on the parameter loss value to update the parameters of the feature output layer and the classification layer in the initial attribute recognition model.

7. An attribute recognition model training device, characterized in that: include: A data acquisition module, used to acquire sample character data and an initial attribute recognition model, wherein the initial attribute recognition model includes a multimodal learning model, a visual transformation network, and a classification layer; A tensor generation module, used to determine the text classification tensor and fusion feature tensor of the sample character data by using the multimodal learning model, aggregate the fusion feature tensor by using the visual transformation network, and perform attribute classification on the aggregated fusion feature tensor by using the classification layer to obtain a target classification tensor; A model training module is used to determine the parameter loss value of the initial attribute recognition model based on the text classification tensor and the target classification tensor, to update the parameters of the initial attribute recognition model based on the parameter loss value, and to return the text classification tensor and fusion feature tensor of the sample character data determined by using the multimodal learning model and its subsequent steps, until the initial attribute recognition model meets the preset training end conditions to obtain the target attribute recognition model.

8. The attribute recognition model training method according to claim 7, characterized in that: The data acquisition module comprises: An attribute determination submodule, used to determine the model attributes to be trained and multiple classification types under the model attributes; The image acquisition submodule is used to acquire sample person images corresponding to each classification type in equal proportion, and mark sample attribute labels in each sample person image according to each classification type to form sample person data.

9. A storage medium, characterized in that: The storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the attribute recognition model training method as described in any one of claims 1 to 6.

10. A computer device, characterized in that: include: one or more processors, and memory; The memory stores computer-readable instructions, and when the computer-readable instructions are executed by the one or more processors, the steps of the attribute recognition model training method according to any one of claims 1 to 6 are performed.

Images