Person attribute classification model training method and person attribute classification method

Abstract

The application discloses a personnel attribute classification model training method and a personnel attribute classification method. The personnel attribute classification model is constructed. Personnel image samples and label texts are obtained, and the personnel image samples and the label texts are input into the personnel attribute classification model to determine a first classification score tensor, a second classification score tensor and a fusion feature vector corresponding to the personnel image samples. A target loss function is established based on the first classification score tensor, the second classification score tensor and the fusion feature vector. The personnel attribute classification model is trained to minimize the target loss function, and a trained personnel attribute classification model is obtained. The model training method provided in the application improves the optimization effect, takes into account multi-source data, and breaks through the limitation of single image classification. The personnel attribute classification realized by using the model can greatly improve the classification accuracy and obtain accurate classification results.

Description

Technical Field

[0001] This application relates to the field of attribute classification technology, specifically to a training method for a personnel attribute classification model and a personnel attribute classification method. Background Technology

[0002] Personnel attribute classification refers to the analysis and categorization of personnel characteristics in images. Personnel attribute classification has significant application value in fields such as business marketing and intelligent interaction. For example, the retail industry can use customer attribute information to achieve precise product recommendations; intelligent interactive devices can provide customized interactive experiences based on user attribute characteristics, and so on.

[0003] Existing methods for classifying personnel attributes only perform single-image recognition and classification, which cannot produce models with better classification results, resulting in low classification accuracy and affecting the classification outcome. Summary of the Invention

[0004] In view of this, this application provides a training method for a personnel attribute classification model and a personnel attribute classification method to solve the problem that existing personnel attribute classification methods only identify and classify single images, resulting in low classification accuracy and affecting classification results.

[0005] To achieve the above objectives, the following solution is proposed:

[0006] Firstly, a training method for a personnel attribute classification model includes:

[0007] Construct a personnel attribute classification model;

[0008] Obtain personnel image samples and label text, and input both the personnel image samples and the label text into the personnel attribute classification model to determine the first classification score tensor, the second classification score tensor, and the fused feature vector corresponding to the personnel image samples;

[0009] A target loss function is established based on the first classification score tensor, the second classification score tensor, and the fused feature vector;

[0010] The personnel attribute classification model is trained with the objective of minimizing the target loss function to obtain a trained personnel attribute classification model.

[0011] Preferably, the step of inputting both the personnel image sample and the label text into the personnel attribute classification model to determine the first classification score tensor, the second classification score tensor, and the fused feature vector corresponding to the personnel image sample includes:

[0012] The image text recognition module in the personnel attribute classification model is used to extract and fuse features from the personnel image samples and the label text to obtain a first classification score tensor and a first feature tensor.

[0013] The first feature tensor is aggregated using the tensor aggregation module in the personnel attribute classification model to obtain the second feature tensor;

[0014] The second feature tensor is convolved using the tensor convolution module in the personnel attribute classification model to obtain a fused feature vector.

[0015] The fused feature vector is classified using the classification module in the personnel attribute classification model to obtain a second classification score tensor.

[0016] Preferably, the image text recognition module includes a first feature extraction unit, a second feature extraction unit, a fusion classification unit, a dimension augmentation unit, and a connection unit;

[0017] The input terminals of the first feature extraction unit and the second feature extraction unit are both used as input terminals of the image text recognition module, and the output terminals of the fusion classification unit and the connection unit are both used as output terminals of the image text recognition module.

[0018] The first feature extraction unit includes a first output terminal and a second output terminal;

[0019] The second output terminal of the first feature extraction unit and the output terminal of the second feature extraction unit are both connected to the input terminal of the fusion classification unit;

[0020] The output of the second feature extraction unit is also connected to the input of the dimension expansion unit;

[0021] The output of the dimension augmentation unit and the first output of the first feature extraction unit are both connected to the input of the connection unit.

[0022] Preferably, the step of using the image text recognition module in the personnel attribute classification model to extract and fuse features from the personnel image samples and the label text to obtain a first classification score tensor and a first feature tensor includes:

[0023] The first feature extraction unit in the image text recognition module is used to extract features from the personnel image sample to obtain the visual feature tensor and prompt word feature tensor corresponding to the personnel image sample;

[0024] The second feature extraction unit in the image text recognition module is used to extract features from the text label to obtain a text feature tensor.

[0025] Using the fusion classification unit in the image text recognition module, attribute classification is performed on the prompt word feature tensor and the text feature tensor to obtain the first classification score tensor corresponding to the personnel image sample;

[0026] The dimension augmentation unit in the image text recognition module is used to augment the dimension of the text feature tensor to obtain a first text tensor.

[0027] The visual feature tensor and the first text tensor are connected using the connection unit in the image text recognition module to obtain the first feature tensor.

[0028] Preferably, the step of using the dimension augmentation unit in the image text recognition module to augment the dimension of the text feature tensor to obtain a first text tensor includes:

[0029] The text feature tensor is mapped using the fully connected layer in the dimension augmentation unit to obtain the initial text features;

[0030] The initial text features are expanded using the tensor expansion layer in the dimension expansion unit to obtain the first text tensor.

[0031] Preferably, the step of convolving the second feature tensor using the tensor convolution module in the personnel attribute classification model to obtain the fused feature vector includes:

[0032] The second feature tensor is mapped using the convolution units in the tensor convolution module to obtain the initial feature tensor;

[0033] The initial feature tensor is compressed using the dimension compression unit in the tensor convolution module to obtain a fused feature vector.

[0034] Preferably, the step of establishing the target loss function based on the first classification score tensor, the second classification score tensor, and the fused feature vector includes:

[0035] A first loss function is established based on the first classification score tensor;

[0036] A first loss function is established based on the second classification score tensor;

[0037] A third loss function is established based on the fused feature vectors;

[0038] The target loss function is constructed using the first loss function, the second loss function, and the third loss function.

[0039] Preferably, constructing the target loss function using the first loss function, the second loss function, and the third loss function includes:

[0040] Set a first weight value, a second weight value, and a third weight value corresponding to the first loss function, the second loss function, and the third loss function, respectively;

[0041] The first loss function, the second loss function, and the third loss function are weighted and fused based on the first weight value, the second weight value, and the third weight value to obtain the target loss function.

[0042] Preferably, the label text includes a gender label and an age label.

[0043] Secondly, a method for classifying personnel attributes includes:

[0044] Obtain images of the people to be classified;

[0045] The personnel image is input into a trained personnel attribute classification model to obtain a classification score tensor corresponding to the personnel image; the trained personnel attribute classification model is trained based on the training method of the personnel attribute classification model described in any of the first aspects;

[0046] Based on the classification score tensor, the attribute classification result of the personnel image is determined.

[0047] As can be seen from the above technical solution, this application constructs a personnel attribute classification model; obtains personnel image samples and label text, and inputs both the personnel image samples and the label text into the personnel attribute classification model to determine the first classification score tensor, the second classification score tensor, and the fusion feature vector corresponding to the personnel image samples; establishes a target loss function based on the first classification score tensor, the second classification score tensor, and the fusion feature vector; and trains the personnel attribute classification model with the goal of minimizing the target loss function to obtain a trained personnel attribute classification model. The starting point of this application is to train a personnel attribute classification model that can also process personnel images. However, in the training process of this model, both personnel image samples and label text are used as training samples and input into the initial personnel attribute classification model. After training and processing, a first classification score tensor and a second classification score tensor are obtained, along with a fused feature vector. This is a multimodal feature complementarity enhancement process. After determining these tensors and feature vectors, joint optimization is performed through a target loss function. The model corresponding to the minimum loss function is the trained personnel attribute classification model. This model training method greatly improves the optimization effect of the model, takes into account multi-source data, and breaks through the limitations of classifying images alone. Therefore, using this model for subsequent personnel attribute classification can significantly improve the classification accuracy and obtain accurate classification results. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0049] Figure 1 An optional flowchart of a training method for a personnel attribute classification model provided in an embodiment of this application;

[0050] Figure 2 This is a schematic diagram of the structure of an image text recognition module provided in an embodiment of this application;

[0051] Figure 3 A schematic diagram of the structure of a training device for a personnel attribute classification model provided in an embodiment of this application;

[0052] Figure 4 This is a schematic diagram of the structure of a training device for a personnel attribute classification model provided in an embodiment of this application. Detailed Implementation

[0053] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0054] This invention can be used in a wide variety of general-purpose or special-purpose computing environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor devices, distributed computing environments including any of the above devices, etc.

[0055] This invention provides a training method for a personnel attribute classification model. This method can be applied to various computer terminals or smart terminals, and its execution entity can be the processor or server of the computer terminal or smart terminal. The method flowchart is shown below. Figure 1 As shown, it specifically includes:

[0056] S1: Construct a personnel attribute classification model.

[0057] In many scenarios requiring personnel attribute classification, such as passenger flow attribute analysis, online attribute analysis can be performed using an algorithm server. The advantages include large data processing capacity, high speed, high algorithm accuracy, and accurate analysis of personnel attributes within passenger flow, covering a variety of different attribute types. Therefore, this application considers constructing and training a personnel attribute classification model, which can then be used for personnel attribute classification.

[0058] The training samples can be any number of images containing people. The model can combine CLIP large model, ViT visual network model, etc., to aggregate and enhance fusion features, and use classification layers to achieve true personnel attribute classification, which can achieve a high personnel attribute recognition rate.

[0059] S2: Obtain personnel image samples and label text, and input both the personnel image samples and the label text into the personnel attribute classification model to determine the first classification score tensor, the second classification score tensor, and the fused feature vector corresponding to the personnel image samples.

[0060] Existing attribute classification methods typically rely solely on the visual features of the image itself, such as color, texture, and shape, for classification. However, this application introduces label text, using both personnel image samples and label text as training samples input into the initial personnel attribute classification model. This allows for the fusion of visual and textual features of the image, such as forming contextual relationships, resulting in a more comprehensive fused feature vector.

[0061] In addition, the first and second classification score tensors can be used to indicate the classification status. Then, by using the first and second classification score tensors and fusing visual features, the classification effect of the personnel attribute classification model can be improved. Furthermore, cross-modal features can be used to complement each other, reduce the ambiguity of a single modality, and improve the classification accuracy.

[0062] S3: Establish the target loss function based on the first classification score tensor, the second classification score tensor, and the fused feature vector.

[0063] The target loss function takes into account the first classification score tensor, the second classification score tensor, and the fused feature vector simultaneously, forming a global loss function that can take into account the multi-source data features of the model, achieve global optimization, and obtain better model parameters. It can avoid the shortcomings of distributed training in existing model training methods and improve convergence efficiency.

[0064] S4: Train the personnel attribute classification model with the objective of minimizing the target loss function to obtain the trained personnel attribute classification model.

[0065] The training process guided by minimizing the objective loss function in this application essentially forces the model to learn a more comprehensive and discriminative representation of personnel attributes through the collaborative optimization of multimodal features. This not only breaks through the performance bottleneck of single image classification, but also endows the personnel attribute classification model with adaptive decision-making capabilities in complex scenarios, providing key technical support for high-precision and robust personnel attribute analysis in practical applications.

[0066] As can be seen from the above technical solution, this application constructs a personnel attribute classification model; obtains personnel image samples and label text, and inputs both the personnel image samples and the label text into the personnel attribute classification model to determine the first classification score tensor, the second classification score tensor, and the fusion feature vector corresponding to the personnel image samples; establishes a target loss function based on the first classification score tensor, the second classification score tensor, and the fusion feature vector; and trains the personnel attribute classification model with the goal of minimizing the target loss function to obtain a trained personnel attribute classification model. The starting point of this application is to train a personnel attribute classification model that can also process personnel images. However, in the training process of this model, both personnel image samples and label text are used as training samples and input into the initial personnel attribute classification model. After training and processing, a first classification score tensor and a second classification score tensor are obtained, along with a fused feature vector. This is a multimodal feature complementarity enhancement process. After determining these tensors and feature vectors, joint optimization is performed through a target loss function. The model corresponding to the minimum loss function is the trained personnel attribute classification model. This model training method greatly improves the optimization effect of the model, takes into account multi-source data, and breaks through the limitations of classifying images alone. Therefore, using this model for subsequent personnel attribute classification can significantly improve the classification accuracy and obtain accurate classification results.

[0067] The method provided in this embodiment of the invention involves inputting both the personnel image sample and the label text into the personnel attribute classification model to determine the first classification score tensor, the second classification score tensor, and the fused feature vector corresponding to the personnel image sample. The specific process is described below:

[0068] The image text recognition module in the personnel attribute classification model is used to extract and fuse features from the personnel image samples and the label text to obtain a first classification score tensor and a first feature tensor.

[0069] The first feature tensor is aggregated using the tensor aggregation module in the personnel attribute classification model to obtain the second feature tensor;

[0070] The second feature tensor is convolved using the tensor convolution module in the personnel attribute classification model to obtain a fused feature vector.

[0071] The fused feature vector is classified using the classification module in the personnel attribute classification model to obtain a second classification score tensor.

[0072] Specifically, the personnel attribute classification model includes an image text recognition module, a tensor aggregation module, a tensor convolution module, and a classification module.

[0073] The input of the image text recognition module is also the input of the personnel attribute classification model. The output of both the image text recognition module and the classification module are used as the output of the personnel attribute classification model. The output of the image text recognition module is also connected to the input of the tensor aggregation module, the output of the tensor aggregation module is connected to the input of the tensor convolution module, and the output of the tensor convolution module is connected to the input of the classification module.

[0074] After inputting personnel image samples and label text into the personnel attribute classification model, the data undergoes processing through multiple modules within the model. First, the image text recognition module extracts and fuses features from each personnel image sample and label text, obtaining a first classification score tensor and a first feature tensor. The first classification score tensor is then the output data of the entire personnel attribute classification model. Simultaneously, the first feature tensor is aggregated by the tensor aggregation module to obtain a second feature tensor, which is then convolved in the tensor convolution module to obtain a fused feature vector. Finally, the classification module classifies the fused feature tensor to obtain a second classification score tensor, which can then be used as the final classification result. The classification module is a standard classification layer that can reduce 2D features to the number of categories, outputting the probability value for each attribute.

[0075] Specifically, the CLIP (Contrastive Language–Image Pre-training) model with pre-defined network parameters can be used as the foundation for the image-text recognition module. It is a multimodal model that embeds images and text into the same semantic space through contrastive learning, achieving cross-modal understanding. Alternatively, the ViT (VisionTransformer) visual network model can be used as the foundation for the tensor aggregation module. It is a Transformer-based visual model that allows the pure Transformer structure to be applied to image classification tasks, enhancing and fusing visual feature information and textual information from images of people.

[0076] In each module of the personnel attribute classification model, assuming the CLIP large model and the ViT visual network model are selected, apart from the optimization of the parameters of the output layers of the CLIP large model and the ViT visual network model, the other internal parameters are frozen and remain unchanged, while the network parameters of the classification module are updated and optimized.

[0077] Optionally, the image text recognition module includes a first feature extraction unit, a second feature extraction unit, a fusion classification unit, a dimension augmentation unit, and a connection unit;

[0078] The input terminals of the first feature extraction unit and the second feature extraction unit are both used as input terminals of the image text recognition module, and the output terminals of the fusion classification unit and the connection unit are both used as output terminals of the image text recognition module.

[0079] The first feature extraction unit includes a first output terminal and a second output terminal;

[0080] The second output terminal of the first feature extraction unit and the output terminal of the second feature extraction unit are both connected to the input terminal of the fusion classification unit;

[0081] The output of the second feature extraction unit is also connected to the input of the dimension expansion unit;

[0082] The output of the dimension augmentation unit and the first output of the first feature extraction unit are both connected to the input of the connection unit.

[0083] Specifically, the structural diagram of the image recognition module is as follows: Figure 2 As shown, from Figure 2 As can be seen, the image text recognition module has undergone complex data processing. This module includes two inputs, corresponding to the person image sample and the label text, and finally has two outputs.

[0084] The following embodiments provide a detailed explanation of the steps in this application whereby the image text recognition module in the personnel attribute classification model extracts and fuses features from the personnel image samples and the label text to obtain the first classification score tensor and the first feature tensor.

[0085] The first feature extraction unit in the image text recognition module is used to extract features from the personnel image sample to obtain the visual feature tensor and prompt word feature tensor corresponding to the personnel image sample;

[0086] The second feature extraction unit in the image text recognition module is used to extract features from the text label to obtain a text feature tensor.

[0087] Using the fusion classification unit in the image text recognition module, attribute classification is performed on the prompt word feature tensor and the text feature tensor to obtain the first classification score tensor corresponding to the personnel image sample;

[0088] The dimension augmentation unit in the image text recognition module is used to augment the dimension of the text feature tensor to obtain a first text tensor.

[0089] The visual feature tensor and the first text tensor are connected using the connection unit in the image text recognition module to obtain the first feature tensor.

[0090] Specifically, the first feature extraction unit outputs a visual feature tensor and a prompt word feature tensor. The visual feature tensor is the pixel feature extracted from the personnel image sample, and the prompt word feature tensor is the classification feature extracted from the personnel image sample. In this application, the visual feature tensor is involved in the feature fusion process. Since it comes from image pixels, it can play a major role. The shape of the visual feature tensor is [N, C1, D], and the shape of the prompt word feature tensor is [N, C2, D], where N is the number of samples, C1 and C2 are the number of channels, and D is the feature dimension.

[0091] For the tag text, the second feature extraction unit performs dimensionality augmentation processing, outputting a first text tensor. This can be achieved by first generating a token, then encoding it to generate the first text tensor with a shape of [Class, D], where Class represents the number of attribute categories predefined in this application. Next, the fusion classification unit performs attribute classification on the prompt word feature tensor and the text feature tensor, outputting a first classification score tensor with a shape of [N, Class], which can be used for attribute classification. Then, the dimensionality augmentation unit processes the text feature tensor. This dimensionality augmentation unit can be considered a fully connected layer, performing dimensionality augmentation to obtain a new text feature tensor with a shape of [N, Class, D], which is used as the first text tensor. Finally, the connection unit connects the visual feature tensor corresponding to the personnel image sample to the first text tensor to obtain a new feature tensor, the first feature tensor, with a shape of [N, C3, D], where C3 = C1 + Class, ensuring that the first feature tensor contains not only visual information but also textual information.

[0092] In addition, the tensor aggregation module aggregates the first feature tensor, but the dimensions of the first feature tensor remain unchanged, and its shape is still [N, C3, D].

[0093] It's important to note that the label text is in soft label form, guiding classification. It serves both a categorization and label description function. Optionally, the label text includes gender and age labels. Gender labels are divided into two categories: male and female, with "a" representing male and "b" representing female. Age is divided into five types, labeled as "1", "2", "3", "4", and "5". "1" represents age 0-15 years old, "2" represents age 16-30 years old, "3" represents age 31-45 years old, "4" represents age 46-60 years old, and "5" represents age over 60 years old. In one example, the label text description could be: "the gender of person is man" or "the age of woman less than 15". This embodiment does not impose any restrictions on this.

[0094] In addition, facial attributes, pedestrian attributes, etc. can be defined, and personnel image samples can be selected from human body images, facial images, head and shoulder images, etc.

[0095] Furthermore, the process of using the dimension augmentation unit in the image text recognition module to augment the dimensions of the text feature tensor to obtain the first text tensor may include the following steps:

[0096] The text feature tensor is mapped using the fully connected layer in the dimension augmentation unit to obtain the initial text features;

[0097] The initial text features are expanded using the tensor expansion layer in the dimension expansion unit to obtain the first text tensor.

[0098] Specifically, the shape of the text feature tensor is [Class,D]. After mapping through a fully connected layer, the shape remains [Class,D]. Since the model is input with N samples per batch during training, the shape of the text feature tensor can be naturally copied N times per batch N, i.e., the dimensions are expanded to [N,Class,D]. The purpose is to enable subsequent feature fusion with the visual feature tensor vf1 along the channel direction, while the other two dimensions must be the same.

[0099] Optionally, the process of convolving the second feature tensor using the tensor convolution module in the personnel attribute classification model to obtain the fused feature vector in this application includes:

[0100] The second feature tensor is mapped using the convolution units in the tensor convolution module to obtain the initial feature tensor;

[0101] The initial feature tensor is compressed using the dimension compression unit in the tensor convolution module to obtain a fused feature vector.

[0102] Specifically, the tensor convolution module can extract multi-channel multi-dimensional feature tensors into single-channel multi-dimensional features. For example, multiple channels C3 are mapped to one channel, resulting in a shape of [N, 1, D]. This is then equivalently compressed into 2D, with a shape of [N, D], which is convenient for the next classification operation. The final second classification score tensor has a shape of [N, Class].

[0103] The following embodiments provide a detailed explanation of the steps in this application for establishing the target loss function based on the first classification score tensor, the second classification score tensor, and the fused feature vector:

[0104] A first loss function is established based on the first classification score tensor;

[0105] A first loss function is established based on the second classification score tensor;

[0106] A third loss function is established based on the fused feature vectors;

[0107] The target loss function is constructed using the first loss function, the second loss function, and the third loss function.

[0108] The step of constructing the target loss function using the first loss function, the second loss function, and the third loss function can be performed in the following manner:

[0109] Set a first weight value, a second weight value, and a third weight value corresponding to the first loss function, the second loss function, and the third loss function, respectively;

[0110] The first loss function, the second loss function, and the third loss function are weighted and fused based on the first weight value, the second weight value, and the third weight value to obtain the target loss function.

[0111] Specifically, the first, second, and third loss functions can all be LabelSmoothloss, Focal loss, or CrossEntropy loss, etc. The first loss function is built based on the first classification score tensor, from which attribute labels defined by the label text can be combined. The second loss function is built based on the second classification score tensor, and can also be combined with attribute labels. The third loss function is built based on the fused feature vector, and can be combined with pre-set ID labels. The ID labels are used for people counting to remove duplicate occurrences of the same person, such as 1, 200, etc., and are a type of integer 1-dimensional tensor.

[0112] By continuously optimizing the personnel attribute classification model with the goal of minimizing the objective loss function, backpropagation is performed to automatically update the network parameters of the personnel attribute classification model until the network parameters reach the optimal level, and the training is completed.

[0113] In one example, the formula for calculating the objective function Loss is:

[0114] Loss = aLoss1 + bLoss2 + cLoss3;

[0115] Where Loss1, Loss2, and Loss3 represent the first loss function, the second loss function, and the third loss function, respectively, and a, b, and c are the weight values ​​of the first loss function, the second loss function, and the third loss function, respectively.

[0116] In addition, the personnel attribute classification model trained in this application can also achieve the purpose of multi-attribute classification. For example, for two attributes, two different branches are set up during training, and their respective target loss functions are constructed. The two target loss functions are then combined to form a total target loss function for joint optimization. This makes the application scope of the trained personnel attribute classification model wider and improves its flexibility.

[0117] Understandably, it is also possible to train a separate personnel attribute classification model for each attribute, which can improve the classification accuracy of the model.

[0118] In one example, this application constructed two databases and used the training method of the personnel attribute model provided in this application to obtain models respectively. In the classification of age attribute labeled text, the average accuracy rates of age classification were 78.11% (528 / 676) and 70.70% (3499 / 4949), respectively. In contrast, the average accuracy rates of age classification models trained using the existing Swin_S backbone network were 71.01% (480 / 676) and 67.00% (3316 / 4949), respectively. Therefore, the training method of the personnel attribute model provided in this application can achieve better results.

[0119] and Figure 1 Corresponding to the method described above, this embodiment of the invention also provides a training device for a personnel attribute classification model, used for... Figure 1 In the specific implementation of the method, the training device for the personnel attribute classification model provided in this embodiment of the invention can be used on a computer terminal or various mobile devices, combined with Figure 3 The training device for the personnel attribute classification model is introduced, such as... Figure 3 As shown, the device may include:

[0120] Model building module 10 is used to build a personnel attribute classification model;

[0121] Model processing module 20 is used to acquire personnel image samples and label text, and input the personnel image samples and label text into the personnel attribute classification model to determine the first classification score tensor, the second classification score tensor and the fusion feature vector corresponding to the personnel image sample;

[0122] Loss function construction module 30 is used to establish a target loss function based on the first classification score tensor, the second classification score tensor, and the fused feature vector;

[0123] Training module 40 is used to train the personnel attribute classification model with the objective of minimizing the target loss function, so as to obtain the trained personnel attribute classification model.

[0124] As can be seen from the above technical solution, this application constructs a personnel attribute classification model; obtains personnel image samples and label text, and inputs both the personnel image samples and the label text into the personnel attribute classification model to determine the first classification score tensor, the second classification score tensor, and the fusion feature vector corresponding to the personnel image samples; establishes a target loss function based on the first classification score tensor, the second classification score tensor, and the fusion feature vector; and trains the personnel attribute classification model with the goal of minimizing the target loss function to obtain a trained personnel attribute classification model. The starting point of this application is to train a personnel attribute classification model that can also process personnel images. However, in the training process of this model, both personnel image samples and label text are used as training samples and input into the initial personnel attribute classification model. After training and processing, a first classification score tensor and a second classification score tensor are obtained, along with a fused feature vector. This is a multimodal feature complementarity enhancement process. After determining these tensors and feature vectors, joint optimization is performed through a target loss function. The model corresponding to the minimum loss function is the trained personnel attribute classification model. This model training method greatly improves the optimization effect of the model, takes into account multi-source data, and breaks through the limitations of classifying images alone. Therefore, using this model for subsequent personnel attribute classification can significantly improve the classification accuracy and obtain accurate classification results.

[0125] Furthermore, embodiments of this application provide a training device for a personnel attribute classification model. Optionally, Figure 4 The diagram shows the hardware structure of the training device for the personnel attribute classification model. (Refer to...) Figure 4 The hardware structure of the training device for the personnel attribute classification model may include: at least one processor 01, at least one communication interface 02, at least one memory 03, and at least one communication bus 04.

[0126] In this embodiment, the number of processor 01, communication interface 02, memory 03 and communication bus 04 is at least one, and processor 01, communication interface 02 and memory 03 communicate with each other through communication bus 04.

[0127] Processor 01 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention.

[0128] Memory 03 may include high-speed RAM, and may also include non-volatile memory, such as at least one disk storage device.

[0129] The memory stores a program, which the processor can call. The program is used to execute the training method for the following personnel attribute classification model, including:

[0130] Construct a personnel attribute classification model;

[0131] Obtain personnel image samples and label text, and input both the personnel image samples and the label text into the personnel attribute classification model to determine the first classification score tensor, the second classification score tensor, and the fused feature vector corresponding to the personnel image samples;

[0132] A target loss function is established based on the first classification score tensor, the second classification score tensor, and the fused feature vector;

[0133] The personnel attribute classification model is trained with the objective of minimizing the target loss function to obtain a trained personnel attribute classification model.

[0134] Optionally, the detailed and extended functions of the program can be found in the description of the training method for the personnel attribute classification model in the method embodiment.

[0135] This application embodiment also provides a storage medium that can store a program suitable for execution by a processor. When the program runs, it controls the device where the storage medium is located to execute a training method for a personnel attribute classification model, including:

[0136] Construct a personnel attribute classification model;

[0137] Obtain personnel image samples and label text, and input both the personnel image samples and the label text into the personnel attribute classification model to determine the first classification score tensor, the second classification score tensor, and the fused feature vector corresponding to the personnel image samples;

[0138] A target loss function is established based on the first classification score tensor, the second classification score tensor, and the fused feature vector;

[0139] The personnel attribute classification model is trained with the objective of minimizing the target loss function to obtain a trained personnel attribute classification model.

[0140] Specifically, the storage medium can be a computer-readable storage medium, which can be an electronic storage device such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM.

[0141] Optionally, the detailed and extended functions of the program can be found in the description of the training method for the personnel attribute classification model in the method embodiment.

[0142] In one embodiment provided in this application, a method for classifying personnel attributes is also provided, including:

[0143] Obtain images of the people to be classified;

[0144] The personnel image is input into the trained personnel attribute classification model to obtain the classification score tensor corresponding to the personnel image; the trained personnel attribute classification model is trained based on any of the training methods of the above personnel attribute classification model;

[0145] Based on the classification score tensor, the attribute classification result of the personnel image is determined.

[0146] Specifically, images containing people can be captured by surveillance cameras. When a person passes through a pre-defined area, the person is cut out. The image can be pre-processed to improve the accuracy of subsequent classification. Then, it can be input into the personnel attribute classification model trained by any of the above training methods to output the attribute classification result, and can also output the ID.

[0147] Furthermore, the functional modules in the various embodiments of this disclosure can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part. If the function is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a live streaming device, or a network device, etc.) to execute all or part of the steps of the methods in the various embodiments of this disclosure.

[0148] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0149] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0150] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A training method for a personnel attribute classification model, characterized in that, include: Construct a personnel attribute classification model; The process involves acquiring personnel image samples and tag text, and inputting both the personnel image samples and the tag text into the personnel attribute classification model to determine the first classification score tensor, the second classification score tensor, and the fusion feature vector corresponding to the personnel image samples. This includes: using the first feature extraction unit in the image text recognition module of the personnel attribute classification model to extract features from the personnel image samples, obtaining a visual feature tensor and a prompt word feature tensor corresponding to the personnel image samples; using the second feature extraction unit in the image text recognition module to extract features from the tag text, obtaining a text feature tensor; and using the fusion classification unit in the image text recognition module to perform attribute classification on the prompt word feature tensor and the text feature tensor to obtain the first classification score corresponding to the personnel image samples. The text feature tensor is augmented using the dimension augmentation unit in the image text recognition module to obtain a first text tensor. The visual feature tensor and the first text tensor are then connected using the connection unit in the image text recognition module to obtain a first feature tensor. The first feature tensor is aggregated using the tensor aggregation module in the personnel attribute classification model to obtain a second feature tensor. The second feature tensor is mapped using the convolution unit in the tensor convolution module of the personnel attribute classification model to obtain an initial feature tensor. The initial feature tensor is compressed using the dimension compression unit in the tensor convolution module to obtain a fused feature vector. The fused feature vector is then classified using the classification module in the personnel attribute classification model to obtain a second classification score tensor. A target loss function is established based on the first classification score tensor, the second classification score tensor, and the fused feature vector; The personnel attribute classification model is trained with the objective of minimizing the target loss function to obtain a trained personnel attribute classification model.

2. The method according to claim 1, characterized in that, The image text recognition module includes a first feature extraction unit, a second feature extraction unit, a fusion classification unit, a dimension augmentation unit, and a connection unit; The input terminals of the first feature extraction unit and the second feature extraction unit are both used as input terminals of the image text recognition module, and the output terminals of the fusion classification unit and the connection unit are both used as output terminals of the image text recognition module. The first feature extraction unit includes a first output terminal and a second output terminal; The second output terminal of the first feature extraction unit and the output terminal of the second feature extraction unit are both connected to the input terminal of the fusion classification unit; The output of the second feature extraction unit is also connected to the input of the dimension expansion unit; The output of the dimension augmentation unit and the first output of the first feature extraction unit are both connected to the input of the connection unit.

3. The method according to claim 1, characterized in that, The step of using the dimension augmentation unit in the image text recognition module to augment the dimension of the text feature tensor to obtain a first text tensor includes: The text feature tensor is mapped using the fully connected layer in the dimension augmentation unit to obtain the initial text features; The initial text features are expanded using the tensor expansion layer in the dimension expansion unit to obtain the first text tensor.

4. The method according to claim 1, characterized in that, The step of establishing the target loss function based on the first classification score tensor, the second classification score tensor, and the fused feature vector includes: A first loss function is established based on the first classification score tensor; A first loss function is established based on the second classification score tensor; A third loss function is established based on the fused feature vectors; The target loss function is constructed using the first loss function, the second loss function, and the third loss function.

5. The method according to claim 4, characterized in that, The construction of the target loss function using the first loss function, the second loss function, and the third loss function includes: Set a first weight value, a second weight value, and a third weight value corresponding to the first loss function, the second loss function, and the third loss function, respectively; The first loss function, the second loss function, and the third loss function are weighted and fused based on the first weight value, the second weight value, and the third weight value to obtain the target loss function.

6. The method according to claim 1, characterized in that, The label text includes gender and age labels.

7. A method for classifying personnel attributes, characterized in that, include: Obtain images of the people to be classified; The personnel image is input into a trained personnel attribute classification model to obtain a classification score tensor corresponding to the personnel image; the trained personnel attribute classification model is trained based on the training method of the personnel attribute classification model according to any one of claims 1 to 6; Based on the classification score tensor, the attribute classification result of the personnel image is determined.

Images