Multi-attribute recognition and model training method, device, equipment and storage medium

Abstract

The present disclosure provides a multi-attribute recognition and model training method, apparatus, device and storage medium, which relates to the field of artificial intelligence technology, specifically to the technical fields of image processing, deep learning, computer vision, etc. The training method of the multi-attribute recognition model includes: obtaining training data, the training data including: multiple groups of image samples; using a backbone network to process the image samples to obtain image features; using a target branch network among multiple branch networks to process the image features corresponding to the target attribute information to obtain the predicted value of the target attribute information; constructing a total loss function based on the predicted value of the target attribute information; based on the total loss function, adjusting at least one of the model parameters of the backbone network and the model parameters of the branch network until the preset conditions are met. The present disclosure can obtain multi-attribute recognition results with high accuracy at low cost and high efficiency.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, specifically to technical fields such as image processing, deep learning, and computer vision, and more particularly to a multi-attribute recognition and model training method, apparatus, device, and storage medium. Background Art

[0002] Identification of personnel operating specifications is an important link in a safe production environment. Personnel operating specifications usually include multiple items. Therefore, it is necessary to solve the multi-attribute recognition problem of targets (such as personnel). Summary of the Invention

[0003] The present disclosure provides a multi-attribute recognition and model training method, apparatus, device and storage medium.

[0004] According to one aspect of the present disclosure, a training method for a multi-attribute recognition model is provided, wherein the model includes a backbone network and multiple branch networks, and the multiple branch networks correspond one-to-one to multiple attribute information. The method includes: obtaining training data, wherein the training data includes: multiple groups of image samples, and the multiple groups of image samples correspond one-to-one to the multiple attribute information; using the backbone network to process the image samples to obtain image features of the image samples; using a target branch network among the multiple branch networks to process the image features corresponding to the target attribute information to obtain a predicted value of the target attribute information; wherein the target attribute information is the attribute information corresponding to the target branch network; constructing a total loss function based on the predicted value of the target attribute information; and adjusting at least one of the model parameters of the backbone network and the model parameters of the branch network based on the total loss function until a preset condition is reached.

[0005] According to another aspect of the present disclosure, a multi-attribute recognition method is provided, comprising: acquiring an image to be recognized, wherein a target in the image has multiple attribute information; performing feature extraction processing on the image to obtain image features of the image; and determining a recognition result of target attribute information among the multiple attribute information based on the image features.

[0006] According to another aspect of the present disclosure, a training device for a multi-attribute recognition model is provided, wherein the model includes a backbone network and multiple branch networks, and the multiple branch networks correspond one-to-one to multiple attribute information. The device includes: an acquisition module for acquiring training data, and the training data includes: multiple groups of image samples, and the multiple groups of image samples correspond one-to-one to the multiple attribute information; an extraction module for using the backbone network to process the image samples to obtain image features of the image samples; a prediction module for using a target branch network among the multiple branch networks to process the image features corresponding to the target attribute information to obtain a predicted value of the target attribute information; wherein the target attribute information is the attribute information corresponding to the target branch network; a construction module for constructing a total loss function based on the predicted value of the target attribute information; and an adjustment module for adjusting at least one of the model parameters of the backbone network and the model parameters of the branch network based on the total loss function until a preset condition is reached.

[0007] According to another aspect of the present disclosure, a multi-attribute recognition device is provided, comprising: an acquisition module for acquiring an image to be recognized, wherein a target in the image has multiple attribute information; an extraction module for performing feature extraction processing on the image to obtain image features of the image; and a recognition module for determining a recognition result of target attribute information among the multiple attribute information based on the image features.

[0008] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute any one of the methods described in any one of the above aspects.

[0009] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute any one of the methods according to any one of the above aspects.

[0010] According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the method according to any one of the above aspects.

[0011] According to the technical solution of the present disclosure, a multi-attribute recognition result with high accuracy can be obtained at low cost and high efficiency.

[0012] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0014] Figure 1 is a schematic diagram according to a first embodiment of the present disclosure;

[0015] Figure 2 is a schematic diagram of an application scenario corresponding to an embodiment of the present disclosure;

[0016] Figure 3 is a schematic diagram of the overall architecture of a training method for a multi-attribute recognition model provided according to an embodiment of the present disclosure;

[0017] Figure 4 is a schematic diagram of an attribute header provided according to an embodiment of the present disclosure;

[0018] Figure 5 is a schematic diagram according to a second embodiment of the present disclosure;

[0019] Figure 6a-6b is a schematic diagram of a two-stage training method for incremental attribute heads provided according to an embodiment of the present disclosure;

[0020] Figure 7 is a schematic diagram according to a third embodiment of the present disclosure;

[0021] Figure 8 is a schematic diagram of the overall architecture of the multi-attribute recognition method provided according to an embodiment of the present disclosure;

[0022] Figure 9 is a schematic diagram according to a fourth embodiment of the present disclosure;

[0023] Figure 10 is a schematic diagram according to the fifth embodiment of the present disclosure

[0024] Figure 11 Schematic diagram of an electronic device for implementing the training method of a multi-attribute recognition model or the multi-attribute recognition method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0025] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0026] For multi-attribute recognition of targets, related technologies can train multiple models, each model is used to identify one attribute information. However, training multiple models is inefficient and costly. Alternatively, multi-label training of multi-task models can be used. However, since each sample needs to be labeled with labels of all categories, the labeling cost is high. In addition, since there are contradictions between some attribute information (for example, some attribute information requires a half-body image, while some attribute information requires a full-body image), the multi-label training method will lead to insufficient model accuracy.

[0027] In order to better obtain the multi-attribute recognition results of the target, the present disclosure provides the following embodiments.

[0028] Figure 1 is a schematic diagram according to the first embodiment of the present disclosure. This embodiment provides a training method for a multi-attribute recognition model. The model includes a backbone network and multiple branch networks. The multiple branch networks correspond one-to-one to multiple attribute information. The method includes:

[0029] 101. Acquire training data, where the training data includes: multiple groups of image samples, where the multiple groups of image samples correspond one-to-one to the multiple attribute information.

[0030] 102. Use the backbone network to process the image sample to obtain image features of the image sample.

[0031] 103. Use a target branch network among the multiple branch networks to process image features corresponding to target attribute information to obtain a predicted value of the target attribute information; wherein the target attribute information is attribute information corresponding to the target branch network.

[0032] 104. Construct a total loss function based on the predicted value corresponding to the target attribute information.

[0033] 105. Based on the total loss function, adjust at least one of the model parameters of the backbone network and the model parameters of the branch network until a preset condition is met.

[0034] During the training process, image samples may be collected in advance, and each group of image samples corresponds to one attribute information.

[0035] Taking the identification of personnel work specifications as an example, multiple attribute information of personnel work specification identification usually includes: personnel wearing safety helmets, personnel wearing work clothes, personnel wearing reflective clothing, personnel wearing life jackets, personnel wearing safety belts, personnel making phone calls / playing with mobile phones during work, personnel sleeping on duty, etc.

[0036] When collecting image samples, a set of image samples can be collected for each attribute information. For example, if multiple attribute information includes whether a user is making a phone call (first attribute information) and whether the user is wearing work clothes (second attribute information), a first set of image samples can be collected for the first attribute information, and a second set of image samples can be collected for the second attribute information.

[0037] Due to the image samples and attribute information, image samples suitable for each attribute information can be collected. For example, for the above-mentioned first attribute information, half-body images can be collected, and for the above-mentioned second attribute information, full-body images can be collected.

[0038] Furthermore, each group of image samples may be annotated manually or automatically to obtain label information corresponding to the group of image samples, and the label information is used to annotate the true value of the corresponding attribute information.

[0039] Taking the above-mentioned first attribute information and second attribute information as an example, a first group of image samples and a second group of image samples can be collected. In addition, a first group of label information can be annotated corresponding to the first group of image samples, and the first group of label information is used to annotate the true value of the first attribute information. A second group of label information can be annotated corresponding to the second group of image samples, and the second group of label information is used to annotate the true value of the second attribute information.

[0040] Since the label information also corresponds to the attribute information, it is not necessary to fully annotate the same sample (label the label information of all categories). You can only annotate the label value of the corresponding attribute information.

[0041] Taking an image sample in the first group of image samples as an example, if the person in the image sample is making a phone call, the corresponding label information can be marked as 1, and if not, the corresponding label information can be marked as 0. That is, in this embodiment, since the first group of image samples corresponds to whether the person is making a phone call, only the label information of whether the person is making a phone call can be marked.

[0042] Correspondingly, if full labeling is required, for the above image samples, in addition to labeling information indicating whether the image is making a phone call, label information indicating whether the image is wearing work clothes is also required.

[0043] After obtaining multiple groups of image samples, each group of image samples can be input into a backbone network. The backbone network is used to extract image features, and its output is the image features corresponding to the group of image samples.

[0044] The backbone network can be a convolutional neural network (CNN), specifically, a Convnext-tiny network. The Convnext-tiny network is an existing convolutional neural network developed based on ResNet.

[0045] Different attribute information shares the same backbone network, and different attribute information corresponds to different branch networks.

[0046] The target branch network refers to a branch network to be adopted among multiple branch networks. For example, each (each) branch network can be used as a target branch network.

[0047] Taking the case where the target branch network is each branch network in multiple branch networks as an example, after using the backbone network to obtain each group of image features corresponding to each group of image samples, each group of image features is input into each branch network respectively, and each branch network is used to obtain the predicted value of the corresponding attribute information.

[0048] Taking the above two attribute information as an example, the branch network includes a first branch network and a second branch network. After using the common backbone network to obtain image features, the first branch network is used to obtain a first prediction value based on the image features of the first group of image samples, and the second branch network is used to obtain a second prediction value based on the image features of the second group of image samples.

[0049] After obtaining the predicted value of the target attribute information, the total loss function can be constructed based on the predicted value. Specifically, the sub-loss function corresponding to the target attribute information can be constructed based on the predicted value of the target attribute information and the true value of the target attribute information (obtained from the label information), and then the total loss function can be constructed based on the sub-loss functions corresponding to different target attribute information.

[0050] For example, for the first branch network, a first sub-loss function can be constructed based on the first predicted value and the first true value, and for the second branch network, a second sub-loss function can be constructed based on the second predicted value and the second true value.

[0051] After obtaining each sub-loss function, the total loss function can be obtained based on each sub-loss function. For example, the first sub-loss function and the second sub-loss function mentioned above can be added together to obtain the total loss function.

[0052] After obtaining the total loss function, the model parameters can be adjusted based on the total loss function. For example, the back propagation (BP) algorithm can be used to adjust the model parameters.

[0053] The adjusted model parameters include: model parameters of the backbone network, and / or model parameters of the branch network.

[0054] Adjustment parameters can be calculated as follows: adjusted parameter = pre-adjustment parameter - learning rate * gradient value, where the gradient value can be obtained by taking the derivative of the total loss function with respect to the model parameters, and the learning rate is a preset hyperparameter. The initial values ​​of each parameter can be set, for example, by performing randomization to obtain random initial values ​​for each parameter.

[0055] The preset conditions are, for example, that the number of iterations reaches a preset value or meets the convergence condition (such as the absolute value of the difference between the loss function of the previous round and the loss function of the current round is less than the preset value). The model that meets the preset conditions can be used as the final multi-attribute recognition model, which can be used in the inference stage to obtain the multi-attribute recognition results of the target in the image based on the image.

[0056] In this embodiment, the multi-attribute recognition model to be trained includes a backbone network and multiple branch networks, each branch network corresponds to one attribute information (multiple branch networks correspond one-to-one to multiple attribute information). Since multiple attribute information share the backbone network, it is not necessary to train a separate model for each attribute information, which can reduce costs and improve efficiency. By having multiple groups of image samples correspond one-to-one to multiple attribute information, appropriate image samples can be selected based on different attribute information to improve the accuracy of the model. Therefore, a multi-attribute recognition model with high accuracy can be obtained at low cost and high efficiency, and then when reasoning is performed based on the model, a multi-attribute recognition result with high accuracy can be obtained at low cost and high efficiency. Furthermore, by having multiple groups of label information correspond one-to-one to multiple attribute information, corresponding annotations can be made for each attribute information without the need for full annotation of all attribute information, which can reduce the amount of annotations.

[0057] In order to better understand the present disclosure, the application scenarios of the embodiments of the present disclosure are described.

[0058] Figure 2 Schematic diagram of the application scenario corresponding to the embodiment of the present disclosure. Figure 2 As shown, a user can upload an image sample to electronic device 201, which then performs training based on the image sample to obtain a multi-attribute recognition model. Furthermore, during the inference phase, the input image can be processed based on the model to obtain multi-attribute recognition results for objects in the image. It is understood that the electronic devices corresponding to model training and model application can be the same or different. This embodiment uses model training as an example.

[0059] Among them, the electronic device 201 can be a user terminal, for example, a personal computer (Personal Computer), a laptop computer, a mobile device (such as a mobile phone), etc. Alternatively, the electronic device 201 can also be a server, and the server can be a cloud server or a local server. Alternatively, the user can also send the image sample to the server through the user terminal, and the server will train based on the image sample, obtain a multi-attribute recognition model, and then send the model to the user terminal, and the user terminal will perform multi-attribute recognition in the reasoning stage based on the model. Alternatively, the reasoning stage can also include: the user sends the image to be recognized to the server through the user terminal, and the server recognizes the image based on the pre-trained model, obtains the multi-attribute recognition result of the target in the image, and returns the multi-attribute recognition result to the user terminal, which is displayed to the user by the user terminal.

[0060] In the technical solutions disclosed herein, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0061] like Figure 3 As shown, the overall architecture of the training process of the multi-attribute recognition model may include: a backbone network and multiple branch networks, where the branch networks may also be referred to as attribute heads.

[0062] Assume that the number of attribute information is N (N is a positive integer), such as Figure 3 As shown in FIG, the multi-attribute recognition model includes a unified backbone network and N branch networks (attribute heads), each branch network corresponding to one attribute information.

[0063] Each piece of attribute information corresponds to a group of image samples. Therefore, N groups of image samples can be collected. The size of each group of image samples can be expressed as B1, B2, B3, ..., BN.

[0064] Among them, each group of image samples can come from different data sources, such as Figure 3 As shown, the first set of image samples (size B1) may come from data of source_1, and the second set of image samples (size B2) may come from data of source_2.

[0065] When selecting image samples from different groups, you can use a uniform sampling method between classes to achieve balanced training for each scene. For example, if the amount of data in source_1 is 100, select B1 = 10. If the amount of data in source_2 is 1000, select B2 = 100.

[0066] By uniformly sampling between classes, the problem of data imbalance between classes can be solved and the model training effect can be improved.

[0067] Each set of image samples is input into a unified backbone network, which is used to extract image features of the image samples. The backbone network is, for example, a Convnext-tiny network.

[0068] In a specific implementation, N groups of image samples can be concatenated and then input into the backbone network. Concatenation refers to merging different image samples together, such as the concatenation of the first image and the second image to form [first image, second image].

[0069] After being processed by the backbone network, the image features of the image samples can be obtained. The image features of each group of image samples are represented by F1, F2, F3, ..., FN respectively.

[0070] After different groups of image features are split, they can be input into the corresponding attribute headers, for example, inputting F1 into the attribute header of source_1. Splitting is the inverse of splicing, for example, splitting [F1, F2] into F1 and F2.

[0071] During the training phase, after different attribute heads process the image features, the predicted values ​​of the corresponding attribute information can be obtained, and sub-loss functions such as loss1 to lossN are constructed based on the predicted values ​​and the corresponding true values.

[0072] For example, Figure 3 As shown, after the backbone network obtains each set of image features, it can segment each set of image features and input the segmented image features (such as F1, F2, etc.) into the corresponding branch network (attribute header). For each attribute header, a prediction value of the corresponding attribute information is obtained based on the image features. For example, if the first attribute header (the attribute header of source_1) corresponds to the first attribute information (whether to make a phone call), then after processing the first attribute header, a prediction value (specifically, a probability value) of whether to make a phone call can be obtained; if the second attribute header (the attribute header of source_2) corresponds to the second attribute information (whether to wear work clothes), then after processing the second attribute header, a prediction value (such as a probability value) of whether to wear work clothes can be obtained.

[0073] For each attribute head, the predicted value and true value of the attribute head can be used to construct the corresponding sub-loss function. Figure 3 loss1~lossN.

[0074] The sub-loss function can be specifically a weighted focal loss function.

[0075] The calculation formula can be:

[0076] Among them, L fl is the sub-loss function, i.e. the weighted focal loss function;

[0077] y' is the predicted value, y is the true value;

[0078] α,γ are preset parameters;

[0079] Log is the logarithm operation.

[0080] By adopting a weighted focal loss function, the intra-class data imbalance problem can be solved.

[0081] After obtaining the sub-loss functions of each branch network, the total loss function can be obtained based on these sub-loss functions. For example, after adding up the sub-loss functions, the total loss function L is obtained.

[0082] The calculation formula is:

[0083] L = loss1 + loss2 + ... + lossN;

[0084] Where L is the total loss function;

[0085] loss1~lossN are the sub-loss functions.

[0086] It is understandable that obtaining the total loss function based on the sub-loss functions is not limited to direct addition, but can also be done in other calculation methods such as weighted addition.

[0087] After obtaining the total loss function L, the model parameters can be adjusted based on L, such as adjusting the model parameters of the backbone network and the model parameters of each branch network. The specific adjustment method can be the BP method, such as Figure 3 As shown by the dotted arrow.

[0088] For each branch network (attribute header), such as Figure 4 As shown, each attribute head can include: an attention layer (represented by a dynamic attention mechanism), a pooling layer (represented by an adaptive pooling), a perception layer (represented by a multi-layer perception module) and an output layer.

[0089] Accordingly, the adopting of a target branch network among the plurality of branch networks to process image features corresponding to target attribute information to obtain a predicted value of the target attribute information includes:

[0090] Using the attention layer, based on preset attention weights, the image features corresponding to the target attribute information are processed to obtain weighted image features;

[0091] Using the pooling layer, performing pooling processing on the weighted image features to obtain pooled image features;

[0092] Using the perception layer, obtaining classification features based on the pooled image features;

[0093] The output layer is used to obtain the predicted value based on the classification feature.

[0094] In this embodiment, by setting the attention layer, appropriate attention weights can be set in different branch networks based on the difference in attribute information, which is conducive to the joint training of mutually exclusive attribute information.

[0095] For each attribute information, the corresponding image sample is a group, and the corresponding image feature is also a group, that is, it may include one or more image features.

[0096] like Figure 4 As shown in Figure 2, after a set of image features corresponding to each attribute information is processed by the dynamic attention mechanism, a processed feature can be obtained, which can be called an attention feature. Specifically, a weight matrix of the same size as the image feature can be pre-set, and the attention feature can be obtained by multiplying the corresponding elements of the image feature with the weight matrix.

[0097] For example, the dimension of each image feature is M*M (M is a positive integer). Assuming that a group of image features is C (C is a positive integer), each group of image features is C M*M image features. A weight matrix of size M*M can be preset, and the value of each element in the weight matrix is ​​set. Then, for each M*M image feature in the C image features, the M*M weight matrix is ​​multiplied by the corresponding element of the M*M image feature to obtain an M*M attention feature. Multiply the corresponding elements, for example, multiply the elements in the first row and first column of the weight matrix by the elements in the first row and first column of the image feature. Since there are C image features in total, C M*M attention features can be obtained.

[0098] The dynamic attention mechanism means that the same or different weight matrices can be selected based on different attribute information.

[0099] For example, for the first attribute information (whether to make a phone call), since the focus is on the upper body, the elements of the upper part of the weight matrix (such as the 1 / 3 rows of the upper part) can be set to a larger value (such as 1.2), and the elements of the lower part of the weight matrix (such as the 2 / 3 rows of the lower part) can be set to a smaller value (such as 0.2).

[0100] For another example, regarding the second attribute information (whether or not wearing work clothes), since the whole body is the focus, the middle part of the weight matrix (which can be set based on experience) can be set to a larger value, and the remaining elements can be set to smaller values. The specific element values ​​can be set based on experience.

[0101] In the context of joint training of multiple attributes, mutually exclusive attributes may exist. For example, a half-body image is required for a person making a phone call, while a full-body image is required for a person wearing work clothes. Directly processing image features is not conducive to joint training of mutually exclusive attributes. However, in this embodiment, based on the dynamic attention mechanism, different attention weights can be set based on the different attribute information. For example, for a person making a phone call, an attention weight focusing on the upper body is added to the corresponding branch network, while for a person wearing work clothes, an attention feature focusing on the whole body is added to the corresponding branch network. This solves the problem of joint training of mutually exclusive attributes.

[0102] The adaptive pooling layer is used to convert the dimension of the input feature to a set size. For example, if it is converted to a 1*1 dimension, then C M*M attention features can be converted into C features with a dimension of 1 after passing through the adaptive pooling layer.

[0103] The multi-layer perception module can process the features after adaptive pooling to obtain classification features.

[0104] The output layer is used to obtain the predicted value of the corresponding attribute head based on the classification features (training phase) or the corresponding attribute recognition result (inference phase). The output layer can specifically include: a fully connected layer and a softmax function.

[0105] The model parameters of the branch networks adjusted during the training phase specifically refer to adjustments to the model parameters of the multi-layer perception module and the model parameters in the output layer. The model parameters in the multi-layer perception module can include the model parameters of the BN layer (the first BN layer and the second BN layer) and the fully connected layer. BN refers to batch normalization, and there are two BN layers. For differentiation, they can be referred to as the first BN layer and the second BN layer. The model parameters in the output layer can include the model parameters of the fully connected layer.

[0106] The multi-layer perception module may include: a first BN layer, a random dropout layer, a fully connected layer, a second BN layer and an activation layer.

[0107] Accordingly, the perception layer is used to obtain classification features based on the pooled image features, including:

[0108] Using the first normalization layer, performing a first normalization process on the pooled image features to obtain first normalized image features;

[0109] Using the random drop layer, performing random drop processing on the image features after the first normalization processing to obtain image features after random drop processing;

[0110] Using the fully connected layer, transforming the image features after the random discarding process to obtain transformed image features;

[0111] Using the second batch of normalization layers, performing a second batch of normalization processing on the converted image features to obtain second normalized image features;

[0112] The activation layer is used to perform activation processing on the image features after the second normalization processing to obtain the classification features.

[0113] In this embodiment, by setting a dropout layer, the accuracy of the classification results can be improved while taking into account the differences between attributes, thereby improving the accuracy of the model and improving the accuracy of attribute recognition.

[0114] Specifically, the BN layer (the first BN layer and the second BN layer) can use a BN function to perform BN processing on the input features to output BN features.

[0115] The dropout layer is used to randomly select some neurons in each iteration and stop them from working, that is, the output of the selected neurons is set to 0. Through dropout processing, the overfitting problem in the training phase can be avoided.

[0116] The fully connected layer can flatten the input features into a one-dimensional vector.

[0117] The activation layer is used to process the input features using an activation function to obtain activated features. The activation function is, for example, the ReLu function.

[0118] It is understandable that Figure 4 The composition of the training phase is shown. In the inference phase, the multi-layer perception module does not require a dropout layer, that is, it can include: the first BN layer, the fully connected layer, the second BN layer, and the activation layer.

[0119] In this embodiment, by sharing a backbone network for each attribute information, complexity can be reduced, costs can be lowered, and efficiency can be improved. By assigning each attribute information its own branch network, decoupling between the attribute information can be achieved, including training decoupling and labeling decoupling. Training decoupling can set different attention weights based on different attribute information, facilitating joint training of mutually exclusive attribute information. Labeling decoupling can avoid full labeling of samples, reducing the amount of labeling and facilitating data utilization. Therefore, the accuracy of the model and the training efficiency of the model can be improved, thereby improving the inference speed and accuracy when using the model for multi-attribute recognition.

[0120] The above shows a general training process for multiple attribute information. In some scenarios, new attribute information may appear. For example, if there are N attribute information, and one new attribute information is added, the number of attribute information becomes (N+1). The model corresponding to this (N+1) attribute information can be trained using the method described above for N attribute information. However, since the model corresponding to the N attribute information has already been trained, an incremental training method can also be used to reduce training costs, effectively utilize existing resources, and increase training speed.

[0121] In general, incremental training can be a two-stage training method, the first stage is used to train the incremental attribute head, and the second stage is used to train the entire network. Figure 5 and Figure 6a-6b .

[0122] Figure 5 is a schematic diagram according to a second embodiment of the present disclosure, which can be used for incremental training. The method of this embodiment includes:

[0123] 501. Obtain a backbone network and existing branch networks.

[0124] Among them, the branch network corresponding to the newly added attribute information can be called an incremental branch network, and the branch network that has been trained historically can be called a stock branch network.

[0125] The existing branch network and backbone network can be obtained by the above training method and have historical model parameters.

[0126] For example, Figure 3 As shown, it includes a backbone network and N existing branch networks.

[0127] 502. In the first stage, based on the training data of the first stage, obtain a total loss function of the first stage.

[0128] For newly added attribute information, such as the (N+1)th attribute information, an incremental training method may be used. The incremental training may include a first stage and a second stage.

[0129] Each stage (the first stage and the second stage) has its own training data and obtains the total loss function of its own stage based on its own training data.

[0130] Taking the first stage as an example, see Figure 6a, image samples corresponding to the (N+1)th attribute information can be collected, such as obtained from the data of source_(N+1). The size of this group of image samples can be expressed as B(N+1). This group of image samples can be spliced ​​with the image samples corresponding to the existing branch network and input into the backbone network. The backbone network can output each group of image features, i.e., F1~F(N+1). Each group of image features is divided and input into each branch network (attribute head), and the sub-loss function is calculated in each branch network. After adding the sub-loss functions of each branch network (a total of N+1), the total loss function of the first stage is obtained.

[0131] 503. In the first stage, the historical model parameters of the trunk network and the historical model parameters of the existing branch network are kept unchanged, and based on the total loss function of the first stage, the model parameters of the incremental branch network are adjusted until the preset conditions of the first stage are reached to obtain the local model parameters of the incremental branch network.

[0132] In the first stage, the model parameters of the backbone network and the existing branch network remain unchanged, and only the model parameters of the incremental branch network are adjusted, such as Figure 6a In the figure, the adjusted model parameters are represented by dashed arrows.

[0133] The number of iterations of the model parameters of the first stage (specifically, the model parameters of the incremental branch network) may be multiple times until the preset conditions of the first stage are met.

[0134] The preset condition of the first stage may be that the number of iterations reaches a preset number, or that a preset convergence condition is satisfied. The model parameters of the incremental branch network when the preset condition of the first stage is met may be referred to as local model parameters.

[0135] Therefore, through the first stage, the local model parameters of the incremental branch network can be obtained.

[0136] Generally speaking, the distribution of incremental training data and existing training data differs significantly. If the incremental training data and existing training data are directly combined for training, the loss of the incremental attribute head will disrupt the optimization direction of the backbone network. To this end, in this embodiment, the parameters of the backbone network and existing attribute head can be frozen (keeping the model parameters unchanged), and the incremental attribute head can be trained with a large learning rate (the specific learning rate value can be set) until the incremental attribute head converges to a stable range or reaches a preset number of times. This can prevent the impact of the newly added attribute scene on the existing attribute scene.

[0137] 504. In the second stage, based on the training data of the second stage, a total loss function of the second stage is obtained.

[0138] Among them, the process of obtaining the total loss function of the second stage based on the training data of the second stage is similar to the acquisition process of the above-mentioned first stage, and can be performed with reference to the principles of the above-mentioned first stage.

[0139] 505. In the second stage, based on the total loss function of the second stage, adjust the historical model parameters of the trunk network, the historical model parameters of the existing branch network, and the local model parameters of the incremental branch network until the preset conditions of the second stage are reached.

[0140] Among them, since the first stage only adjusts the model parameters of the incremental branch network, it can be considered as a local adjustment. Correspondingly, the second stage is a global adjustment, that is, adjusting the model parameters of the backbone network, existing branch network and incremental branch network.

[0141] In the second stage, as Figure 6b In the figure, the adjusted model parameters are represented by dotted arrows, including the model parameters of the backbone network, the model parameters of the existing branch network, and the model parameters of the incremental branch network.

[0142] The number of iterations of the model parameters in the second stage can be multiple times until the preset conditions of the second stage are met. The preset conditions of the second stage can be that the number of iterations reaches a preset number, or that a preset convergence condition is met. In addition, the preset number of times in the first and second stages can be the same or different. The convergence condition is usually that the absolute value of the difference between the loss function of the previous round and the loss function of the current round is less than a preset value. The preset value in the first and second stages can also be the same or different.

[0143] The model that meets the preset conditions of the second stage is the final model, which includes a backbone network and (N+1) branch networks. The final model can be used in the inference stage to identify the (N+1) attribute information of the target.

[0144] In the first stage, the incremental attribute head is basically converged. In order to ensure the effect of global attributes, the whole network can be fine-tuned in the second stage to achieve the optimization of all attributes.

[0145] In this embodiment, a two-stage training method is adopted for the newly added attribute information, which can quickly implement the training of the newly added attribute information, reduce training costs, and improve training efficiency.

[0146] The above embodiment involves a model training process. The trained model can also be used in the reasoning stage to realize the recognition of multiple attribute information of the target.

[0147] Figure 7 is a schematic diagram according to the third embodiment of the present disclosure, which provides a multi-attribute recognition method, such as Figure 7 As shown, the method includes:

[0148] 701. Acquire an image to be identified, where an object in the image has multiple attribute information.

[0149] 702. Perform feature extraction processing on the image to obtain image features of the image.

[0150] 703. Determine a recognition result of target attribute information among the multiple attribute information based on the image features.

[0151] Here, an image refers to an image containing an object, such as a person, and various attribute information of the person can be recognized based on the image of the person. For example, the person's work specifications can be recognized based on the image of the person.

[0152] After the image is obtained, characteristic features of the image may be extracted, and recognition results of target attribute information among the various attribute information of the person may be determined based on the image features.

[0153] Target attribute information refers to the attribute information to be identified among the plurality of attribute information. For example, each (individual) attribute information can be used as target attribute information.

[0154] Specifically, target attribute information includes: whether making a phone call, whether wearing work clothes, etc.

[0155] In this embodiment, by extracting image features and obtaining recognition results of target attribute information based on the image features, multi-attribute recognition can be achieved based on common image features, thereby reducing costs and improving efficiency.

[0156] In some embodiments, multi-attribute recognition may be performed based on a multi-attribute recognition model, and the multi-attribute recognition model may be trained using the above-mentioned embodiments.

[0157] Specifically, if Figure 8 As shown, the backbone network of the multi-attribute recognition model is used to perform feature extraction processing on the image to obtain the image features of the image; and the target branch network of the multi-attribute recognition model is used to process the image features to determine the recognition result of the target attribute information, and the target branch network is a branch network corresponding to the target attribute information in multiple branch networks, and the multiple branch networks correspond one-to-one to the multiple attribute information.

[0158] The multi-attribute recognition model can include a backbone network and multiple branch networks, each of which corresponds to multiple attribute information. Assuming the target attribute information is each of the multiple attribute information, each branch network can be used to process the image features extracted by the backbone network to obtain recognition results for each attribute information.

[0159] In this embodiment, each attribute information can share image features, reducing costs and improving efficiency; each attribute information has its own branch network, which can achieve decoupling of attribute information and improve the accuracy of recognition results of each attribute information.

[0160] In some embodiments, such as Figure 8 As shown, the target branch network includes: an attention layer, a pooling layer, a perception layer, and an output layer; the target branch network using the multi-attribute recognition model processes the image features to determine the recognition result of the target attribute information, including:

[0161] Using the attention layer, the image features are processed based on preset attention weights to obtain weighted image features;

[0162] Adopting the pooling layer to perform adaptive pooling processing on the weighted image features to obtain pooled image features;

[0163] Using the perception layer, obtaining classification features based on the pooled image features;

[0164] The output layer is used to obtain a recognition result of the target attribute information based on the classification features.

[0165] In this embodiment, by setting the attention layer, appropriate attention weights can be set in different branch networks based on different attribute information, thereby improving the recognition accuracy of the recognition results of each attribute information.

[0166] In some embodiments, such as Figure 8 As shown, the perception layer includes: a first batch of normalization layers, a fully connected layer, a second batch of normalization layers and an activation layer;

[0167] The method of using the perception layer to obtain classification features based on the pooled image features includes:

[0168] Using the first normalization layer, performing a first normalization process on the pooled image features to obtain first normalized image features;

[0169] Using the fully connected layer, transforming the first normalized image features to obtain transformed image features;

[0170] Using the second batch of normalization layers, performing a second batch of normalization processing on the converted image features to obtain second normalized image features;

[0171] The activation layer is used to perform activation processing on the image features after the second normalization processing to obtain the classification features.

[0172] In this embodiment, through the processing of the above two BN layers, the fully connected layer and the activation layer, classification features with better effects can be obtained, thereby improving the accuracy of the recognition results.

[0173] It is understandable that the model structure in the inference phase is similar to that in the training phase, except that the training phase includes a dropout layer, while the inference phase does not. The specific structure and description of the remaining layers can be found in the description of the model training phase above.

[0174] In this embodiment, based on the backbone network and multiple branch networks, it can be used in scenarios where multiple attributes of a target are identified, including but not limited to the identification of multiple attributes of personnel work specifications. After experiments, it is found that the classification accuracy and performance indicators are better than those of single model training and multi-label training methods, effectively improving the effect and performance of target multi-attribute recognition.

[0175] Figure 9 It is a schematic diagram according to the fourth embodiment of the present disclosure. This embodiment provides a training device for a multi-attribute recognition model, wherein the model includes a backbone network and multiple branch networks, and the multiple branch networks correspond one-to-one to multiple attribute information. The device 900 includes: an acquisition module 901, an extraction module 902, a prediction module 903, a construction module 904 and an adjustment module 905.

[0176] The acquisition module 901 is used to acquire training data, and the training data includes: multiple groups of image samples, and the multiple groups of image samples correspond one-to-one to the multiple attribute information; the extraction module 902 is used to use the backbone network to process the image samples to obtain the image features of the image samples; the prediction module 903 is used to use the target branch network among the multiple branch networks to process the image features corresponding to the target attribute information to obtain the predicted value of the target attribute information; wherein the target attribute information is the attribute information corresponding to the target branch network; the construction module 904 is used to construct a total loss function based on the predicted value of the target attribute information; the adjustment module 905 is used to adjust the model parameters of the backbone network and at least one of the model parameters of the branch network based on the total loss function until the preset conditions are met.

[0177] In this embodiment, the multi-attribute recognition model to be trained includes a backbone network and multiple branch networks, each branch network corresponding to one attribute information (multiple branch networks correspond one-to-one to multiple attribute information). Since multiple attribute information share the backbone network, it is not necessary to train a separate model for each attribute information, which can reduce costs and improve efficiency. By having multiple groups of image samples correspond one-to-one to multiple attribute information, appropriate image samples can be selected based on different attribute information to improve the accuracy of the model. Therefore, a multi-attribute recognition model with high accuracy can be obtained at low cost and high efficiency. Then, when reasoning based on this model, a multi-attribute recognition result with high accuracy can be obtained at low cost and high efficiency.

[0178] In some embodiments, the training data also includes: multiple groups of label information, the multiple groups of label information correspond one-to-one to the multiple attribute information, and the label information is used to mark the true value of the corresponding attribute information; the construction module 904 is further used to: obtain the true value of the target attribute information in the multiple groups of label information; construct the sub-loss function corresponding to the target attribute information based on the true value of the target attribute information and the predicted value of the target attribute information; and construct the total loss function based on the sub-loss function corresponding to the target attribute information.

[0179] In this embodiment, by corresponding multiple sets of label information to multiple pieces of attribute information one by one, corresponding annotation can be performed for each piece of attribute information without the need to fully annotate all the attribute information, thereby reducing the amount of annotation.

[0180] In some embodiments, the target branch network includes: an attention layer, a pooling layer, a perception layer and an output layer; the prediction module 903 is further used to: use the attention layer to process each group of image features based on preset attention weights to obtain weighted image features; use the pooling layer to pool the weighted image features to obtain pooled image features; use the perception layer to obtain classification features based on the pooled image features; use the output layer to obtain the prediction value based on the classification features.

[0181] In this embodiment, by setting the attention layer, appropriate attention weights can be set in different branch networks based on the difference in attribute information, which is conducive to the joint training of mutually exclusive attribute information.

[0182] In some embodiments, the perception layer includes: a first batch of normalization layers, a random drop layer, a fully connected layer, a second batch of normalization layers and an activation layer; the prediction module 903 is further used to: use the first batch of normalization layers to perform a first batch of normalization processing on the pooled image features to obtain first normalized image features; use the random drop layer to perform random drop processing on the first normalized image features to obtain randomly dropped image features; use the fully connected layer to perform conversion processing on the randomly dropped image features to obtain converted image features; use the second batch of normalization layers to perform a second batch of normalization processing on the converted image features to obtain second normalized image features; use the activation layer to perform activation processing on the second normalized image features to obtain the classification features.

[0183] In this embodiment, by setting a dropout layer, the accuracy of the classification results can be improved while taking into account the differences between attributes, thereby improving the accuracy of the model and improving the accuracy of attribute recognition.

[0184] In some embodiments, the multiple branch networks include: an existing branch network and an incremental branch network, the incremental branch network is a branch network to be trained, and the existing branch network and the trunk network have been historically trained and have historical model parameters; the adjustment module 905 is further used to: in the first stage, keep the historical model parameters of the trunk network and the historical model parameters of the existing branch network unchanged, and, based on the total loss function of the first stage, adjust the model parameters of the incremental branch network until the preset conditions of the first stage are reached to obtain the local model parameters of the incremental branch network; in the second stage, based on the total loss function of the second stage, adjust the historical model parameters of the trunk network, the historical model parameters of the existing branch network, and the local model parameters of the incremental branch network until the preset conditions of the second stage are reached.

[0185] In this embodiment, a two-stage training method is adopted for the newly added attribute information, which can quickly implement the training of the newly added attribute information, reduce training costs, and improve training efficiency.

[0186] Figure 10 According to the schematic diagram of the fifth embodiment of the present disclosure, this embodiment provides a multi-attribute recognition device. The device 1000 includes: an acquisition module 1001 , an extraction module 1002 , and a recognition module 1003 .

[0187] The acquisition module 1001 is used to acquire an image to be identified, where the target in the image has multiple attribute information; the extraction module 1002 is used to perform feature extraction processing on the image to obtain the image features of the image; the identification module 1003 is used to determine the identification result of the target attribute information in the multiple attribute information based on the image features.

[0188] In this embodiment, by extracting image features and obtaining various attribute information based on the image features, multi-attribute recognition can be achieved based on the common image features, thereby reducing costs and improving efficiency.

[0189] In some embodiments, the extraction module 1002 is further used to: use the backbone network of the multi-attribute recognition model to perform feature extraction processing on the image to obtain the image features of the image; the recognition module 1003 is further used to: use the target branch network of the multi-attribute recognition model to process the image features to determine the recognition result of the target attribute information, the target branch network is a branch network corresponding to the target attribute information among multiple branch networks, and the multiple branch networks correspond one-to-one to the multiple attribute information.

[0190] In this embodiment, multiple attribute information can share image features, reducing costs and improving efficiency; each attribute information has its own branch network, which can achieve decoupling of the attribute information and improve the accuracy of the recognition results of each attribute information.

[0191] In some embodiments, the target branch network includes: an attention layer, a pooling layer, a perception layer and an output layer; the recognition module 1003 is further used to: use the attention layer to process the image features based on preset attention weights to obtain weighted image features; use the pooling layer to adaptively pool the weighted image features to obtain pooled image features; use the perception layer to obtain classification features based on the pooled image features; use the output layer to obtain recognition results of the various attribute information based on the classification features.

[0192] In this embodiment, by setting the attention layer, appropriate attention weights can be set in different branch networks based on different attribute information, thereby improving the recognition accuracy of the recognition results of each attribute information.

[0193] In some embodiments, the perception layer includes: a first batch of normalization layers, a fully connected layer, a second batch of normalization layers and an activation layer; the recognition module 1003 is further used to: use the first batch of normalization layers to perform a first batch of normalization processing on the pooled image features to obtain first normalized image features; use the fully connected layer to perform conversion processing on the first normalized image features to obtain converted image features; use the second batch of normalization layers to perform a second batch of normalization processing on the converted image features to obtain second normalized image features; use the activation layer to perform activation processing on the second normalized image features to obtain the classification features.

[0194] In this embodiment, through the processing of the above two BN layers, the fully connected layer and the activation layer, classification features with better effects can be obtained, thereby improving the accuracy of the recognition results.

[0195] It can be understood that in the embodiments of the present disclosure, the same or similar contents in different embodiments can be referenced to each other.

[0196] It can be understood that the terms “first”, “second”, etc. in the embodiments of the present disclosure are only used for distinction and do not indicate the degree of importance, time sequence, etc.

[0197] In the technical solutions disclosed herein, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0198] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0199] Figure 11 A schematic block diagram of an example electronic device 1100 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, servers, blade servers, mainframe computers, and other suitable computers. The electronic device 1100 can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0200] like Figure 11As shown, the electronic device 1100 includes a computing unit 1101, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1102 or a computer program loaded from a storage unit 11011 into a random access memory (RAM) 1103. Various programs and data required for the operation of the electronic device 1100 can also be stored in the RAM 1103. The computing unit 1101, the ROM 1102, and the RAM 1103 are connected to each other via a bus 1104. An input / output (I / O) interface 1105 is also connected to the bus 1104.

[0201] Multiple components in the electronic device 1100 are connected to the I / O interface 1105, including an input unit 1106, such as a keyboard, a mouse, etc.; an output unit 1107, such as various types of displays, speakers, etc.; a storage unit 1108, such as a magnetic disk, an optical disk, etc.; and a communication unit 1109, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1109 allows the electronic device 1100 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0202] The computing unit 1101 can be a variety of general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the computing unit 1101 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 1101 performs the various methods and processes described above, such as the method for determining a video cover or the method for training a video cover determination model. For example, in some embodiments, the method for determining a video cover or the method for training a video cover determination model can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as the storage unit 1108. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 1100 via the ROM 1102 and / or the communication unit 1109. When the computer program is loaded into the RAM 1103 and executed by the computing unit 1101, one or more steps of the method for determining a video cover or the method for training a video cover determination model described above can be performed. Alternatively, in other embodiments, the computing unit 1101 may be configured to execute the method for determining a video cover or the method for training a video cover determination model in any other appropriate manner (for example, by means of firmware).

[0203] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0204] The program code for implementing the methods of the present disclosure can be written in any combination of one or more programming languages. Such program code can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable load balancing device, so that when the program code is executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0205] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0206] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0207] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0208] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact via a communication network. This client-server relationship is established by computer programs running on the respective computers, establishing a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host, a host product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosts and VPS services ("Virtual Private Servers" or simply "VPS"). The server may also be a server in a distributed system or a server integrated with blockchain.

[0209] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not a limitation herein.

[0210] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A method for training a multi-attribute recognition model, the model comprising a backbone network and multiple branch networks, the multiple branch networks corresponding one-to-one to multiple attribute information, the method comprising: Acquire training data, the training data comprising: multiple groups of image samples, the multiple groups of image samples corresponding to the multiple attribute information one-to-one; Using the backbone network, processing the image sample to obtain image features of the image sample; Using a target branch network among the multiple branch networks, processing image features corresponding to target attribute information to obtain a predicted value of the target attribute information; wherein the target attribute information is attribute information corresponding to the target branch network; Constructing a total loss function based on the predicted value of the target attribute information; Based on the total loss function, adjusting at least one of the model parameters of the backbone network and the model parameters of the branch network until a preset condition is met; The multiple branch networks include: existing branch networks and incremental branch networks, the incremental branch networks are branch networks to be trained, and the existing branch networks and the trunk network have been trained historically and have historical model parameters; The adjusting, based on the total loss function, at least one of the model parameters of the backbone network and the model parameters of the branch network until a preset condition is met, includes: In the first stage, the historical model parameters of the trunk network and the historical model parameters of the existing branch network are kept unchanged, and based on the total loss function of the first stage, the model parameters of the incremental branch network are adjusted until the preset conditions of the first stage are met, so as to obtain the local model parameters of the incremental branch network; In the second stage, based on the total loss function of the second stage, the historical model parameters of the backbone network, the historical model parameters of the existing branch network, and the local model parameters of the incremental branch network are adjusted until the preset conditions of the second stage are met; The preset condition includes satisfying a preset convergence condition, wherein the convergence condition includes that the absolute value of the difference between the loss function of the previous round and the loss function of the current round is less than a preset value, and the preset values ​​of the first stage and the second stage are the same or different; The distribution of incremental training data and existing training data is very different.

2. The method according to claim 1, wherein The training data further includes: multiple sets of label information, the multiple sets of label information corresponding to the multiple attribute information one-to-one, and the label information is used to mark the true value of the corresponding attribute information; The constructing of a total loss function based on the predicted value of the target attribute information includes: Obtaining the true value of the target attribute information from the multiple sets of label information; Constructing a sub-loss function corresponding to the target attribute information based on the true value of the target attribute information and the predicted value of the target attribute information; The total loss function is constructed based on the sub-loss function corresponding to the target attribute information.

3. The method according to claim 1, wherein The target branch network includes: an attention layer, a pooling layer, a perception layer, and an output layer; The using a target branch network among the multiple branch networks to process image features corresponding to target attribute information to obtain a predicted value of the target attribute information includes: Using the attention layer, based on preset attention weights, the image features corresponding to the target attribute information are processed to obtain weighted image features; Using the pooling layer, performing pooling processing on the weighted image features to obtain pooled image features; Using the perception layer, obtaining classification features based on the pooled image features; The output layer is used to obtain the predicted value based on the classification feature.

4. The method according to claim 3, wherein: The perception layer includes: a first batch of normalization layers, a random dropout layer, a fully connected layer, a second batch of normalization layers and an activation layer; The adopting the perception layer to obtain classification features based on the pooled image features includes: Using the first normalization layer, performing a first normalization process on the pooled image features to obtain first normalized image features; Using the random drop layer, performing random drop processing on the image features after the first normalization processing to obtain image features after random drop processing; Using the fully connected layer, transforming the image features after the random discarding process to obtain transformed image features; Using the second batch of normalization layers, performing a second batch of normalization processing on the converted image features to obtain second normalized image features; The activation layer is used to perform activation processing on the image features after the second normalization processing to obtain the classification features.

5. A multi-attribute recognition method comprising: Acquire an image to be identified, wherein an object in the image has multiple attribute information; performing feature extraction processing on the image to obtain image features of the image; determining, based on the image features, a recognition result of target attribute information among the plurality of attribute information; The performing feature extraction processing on the image to obtain image features of the image includes: Using a backbone network of a multi-attribute recognition model to perform feature extraction processing on the image to obtain image features of the image; The determining, based on the image features, a recognition result of the target attribute information among the plurality of attribute information includes: Processing the image features using a target branch network of the multi-attribute recognition model to determine a recognition result of the target attribute information, wherein the target branch network is a branch network corresponding to the target attribute information among a plurality of branch networks, and the plurality of branch networks correspond one-to-one to the plurality of attribute information; The multiple branch networks include: existing branch networks and incremental branch networks, the incremental branch networks are branch networks to be trained, and the existing branch networks and the trunk network have been trained historically and have historical model parameters; At least one of the model parameters of the backbone network and the model parameters of the branch network is adjusted in the following manner during the training phase: In the first stage, the historical model parameters of the trunk network and the historical model parameters of the existing branch network are kept unchanged, and based on the total loss function of the first stage, the model parameters of the incremental branch network are adjusted until the preset conditions of the first stage are met, so as to obtain the local model parameters of the incremental branch network; In the second stage, based on the total loss function of the second stage, the historical model parameters of the backbone network, the historical model parameters of the existing branch network, and the local model parameters of the incremental branch network are adjusted until the preset conditions of the second stage are met; The preset condition includes satisfying a preset convergence condition, wherein the convergence condition includes that the absolute value of the difference between the loss function of the previous round and the loss function of the current round is less than a preset value, and the preset values ​​of the first stage and the second stage are the same or different; The distribution of incremental training data and existing training data is very different.

6. The method according to claim 5, wherein: The target branch network includes: an attention layer, a pooling layer, a perception layer, and an output layer; The target branch network of the multi-attribute recognition model is used to process the image features to determine the recognition result of the target attribute information, including: Using the attention layer, the image features are processed based on preset attention weights to obtain weighted image features; Adopting the pooling layer to perform adaptive pooling processing on the weighted image features to obtain pooled image features; Using the perception layer, obtaining classification features based on the pooled image features; The output layer is used to obtain a recognition result of the target attribute information based on the classification features.

7. The method according to claim 6, wherein: The perception layer includes: a first batch of normalization layers, a fully connected layer, a second batch of normalization layers and an activation layer; The method of using the perception layer to obtain classification features based on the pooled image features includes: Using the first normalization layer, performing a first normalization process on the pooled image features to obtain first normalized image features; Using the fully connected layer, transforming the first normalized image features to obtain transformed image features; Using the second batch of normalization layers, performing a second batch of normalization processing on the converted image features to obtain second normalized image features; The activation layer is used to perform activation processing on the image features after the second normalization processing to obtain the classification features.

8. A training device for a multi-attribute recognition model, the model comprising a backbone network and multiple branch networks, the multiple branch networks corresponding one-to-one to multiple attribute information, the device comprising: An acquisition module, configured to acquire training data, wherein the training data includes: a plurality of groups of image samples, wherein the plurality of groups of image samples correspond one-to-one to the plurality of attribute information; an extraction module, configured to process the image sample using the backbone network to obtain image features of the image sample; a prediction module, configured to process image features corresponding to target attribute information using a target branch network among the plurality of branch networks to obtain a predicted value of the target attribute information; wherein the target attribute information is attribute information corresponding to the target branch network; A construction module, configured to construct a total loss function based on the predicted value of the target attribute information; An adjustment module, configured to adjust at least one of the model parameters of the backbone network and the model parameters of the branch network based on the total loss function until a preset condition is met; The multiple branch networks include: existing branch networks and incremental branch networks, the incremental branch networks are branch networks to be trained, and the existing branch networks and the trunk network have been trained historically and have historical model parameters; The adjustment module is further configured to: In the first stage, the historical model parameters of the trunk network and the historical model parameters of the existing branch network are kept unchanged, and the model parameters of the incremental branch network are adjusted based on the total loss function of the first stage until the preset conditions of the first stage are met, so as to obtain the local model parameters of the incremental branch network; In the second stage, based on the total loss function of the second stage, the historical model parameters of the backbone network, the historical model parameters of the existing branch network, and the local model parameters of the incremental branch network are adjusted until the preset conditions of the second stage are met; The preset condition includes satisfying a preset convergence condition, wherein the convergence condition includes that the absolute value of the difference between the loss function of the previous round and the loss function of the current round is less than a preset value, and the preset values ​​of the first stage and the second stage are the same or different; The distribution of incremental training data and existing training data is very different.

9. The device according to claim 8, wherein The training data further includes: multiple sets of label information, the multiple sets of label information corresponding to the multiple attribute information one-to-one, and the label information is used to mark the true value of the corresponding attribute information; The building blocks are further configured to: Obtaining the true value of the target attribute information from the multiple sets of label information; Constructing a sub-loss function corresponding to the target attribute information based on the true value of the target attribute information and the predicted value of the target attribute information; The total loss function is constructed based on the sub-loss function corresponding to the target attribute information.

10. The device according to claim 8, wherein The target branch network includes: an attention layer, a pooling layer, a perception layer, and an output layer; The prediction module is further configured to: Using the attention layer, processing each group of image features based on preset attention weights to obtain weighted image features; Using the pooling layer, performing pooling processing on the weighted image features to obtain pooled image features; Using the perception layer, obtaining classification features based on the pooled image features; The output layer is used to obtain the predicted value based on the classification feature.

11. The device according to claim 10, wherein The perception layer includes: a first batch of normalization layers, a random dropout layer, a fully connected layer, a second batch of normalization layers and an activation layer; The prediction module is further configured to: Using the first normalization layer, performing a first normalization process on the pooled image features to obtain first normalized image features; Using the random drop layer, performing random drop processing on the image features after the first normalization processing to obtain image features after random drop processing; Using the fully connected layer, transforming the image features after the random discarding process to obtain transformed image features; Using the second batch of normalization layers, performing a second batch of normalization processing on the converted image features to obtain second normalized image features; The activation layer is used to perform activation processing on the image features after the second normalization processing to obtain the classification features.

12. A multi-attribute recognition device, comprising: An acquisition module is used to acquire an image to be identified, wherein the target in the image has multiple attribute information; An extraction module, configured to perform feature extraction processing on the image to obtain image features of the image; an identification module, configured to determine an identification result of target attribute information among the plurality of attribute information based on the image features; The extraction module is further used to: use the backbone network of the multi-attribute recognition model to perform feature extraction processing on the image to obtain image features of the image; The recognition module is further configured to: process the image features using a target branch network of the multi-attribute recognition model to determine a recognition result of the target attribute information, wherein the target branch network is a branch network corresponding to the target attribute information among a plurality of branch networks, and the plurality of branch networks correspond one-to-one to the plurality of attribute information; The multiple branch networks include: existing branch networks and incremental branch networks, the incremental branch networks are branch networks to be trained, and the existing branch networks and the trunk network have been trained historically and have historical model parameters; At least one of the model parameters of the backbone network and the model parameters of the branch network is adjusted in the following manner during the training phase: In the first stage, the historical model parameters of the trunk network and the historical model parameters of the existing branch network are kept unchanged, and based on the total loss function of the first stage, the model parameters of the incremental branch network are adjusted until the preset conditions of the first stage are met, so as to obtain the local model parameters of the incremental branch network; In the second stage, based on the total loss function of the second stage, the historical model parameters of the backbone network, the historical model parameters of the existing branch network, and the local model parameters of the incremental branch network are adjusted until the preset conditions of the second stage are met; The preset condition includes satisfying a preset convergence condition, wherein the convergence condition includes that the absolute value of the difference between the loss function of the previous round and the loss function of the current round is less than a preset value, and the preset values ​​of the first stage and the second stage are the same or different; The distribution of incremental training data and existing training data is very different.

13. The device according to claim 12, wherein The target branch network includes: an attention layer, a pooling layer, a perception layer, and an output layer; The identification module is further configured to: Using the attention layer, the image features are processed based on preset attention weights to obtain weighted image features; Adopting the pooling layer to perform adaptive pooling processing on the weighted image features to obtain pooled image features; Using the perception layer, obtaining classification features based on the pooled image features; The output layer is used to obtain recognition results of the respective attribute information based on the classification features.

14. The device according to claim 13, wherein The perception layer includes: a first batch of normalization layers, a fully connected layer, a second batch of normalization layers and an activation layer; The identification module is further configured to: Using the first normalization layer, performing a first normalization process on the pooled image features to obtain first normalized image features; Using the fully connected layer, transforming the first normalized image features to obtain transformed image features; Using the second batch of normalization layers, performing a second batch of normalization processing on the converted image features to obtain second normalized image features; The activation layer is used to perform activation processing on the image features after the second normalization processing to obtain the classification features.

15. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

16. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 7.

17. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images