Target attribute recognition method, device, computer equipment and storage medium

Abstract

The present invention relates to the field of artificial intelligence technology, and the present invention discloses a target attribute recognition method, device, computer equipment and storage medium, the method comprising: obtaining a video to be recognized, and performing target tracking processing on the video to be recognized, obtaining a target identification of each target and an image set associated with the target identification; performing target attribute recognition on each target image in each image set, obtaining a target recognition result set corresponding to each image set and containing image attribute results, the image attribute results including branch attribute values ​​and branch confidences; performing confidence voting on all branch attribute values ​​and branch confidences in each target recognition result set, obtaining each final target attribute value and target confidence result of each target. Therefore, the present invention realizes automatic recognition of each final target attribute value and corresponding target confidence result of each target in the video to be recognized, and improves the accuracy and correctness of the confidence of target attribute recognition.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a target attribute recognition method, device, computer equipment and storage medium. Background Art

[0002] With the popularity of surveillance videos, it is of great value to accurately and effectively utilize the information of the target objects in the video and mine the relevant attributes of the target, such as pedestrian information in pedestrian surveillance videos, vehicle information in vehicle surveillance videos, etc. The existing target recognition method usually builds a deep network based on the attention mechanism, which can learn different attributes according to the attention allocation rules of the image to obtain the attribute results of each target in a single image. Because this scheme is for target attribute recognition of the target in the image, there will be some target attributes that are missampled or interfered / occluded, resulting in inaccurate target attribute recognition and low confidence in the recognition of target attributes. Therefore, this scheme cannot accurately recognize the target attributes in the video. Summary of the invention

[0003] The present invention provides a target attribute recognition method, device, computer equipment and storage medium, which realize automatic recognition of target identification and image sets in a video to be recognized, and can accurately recognize the final target attributes and target confidence results of each target, thereby improving the accuracy and correctness of target attribute recognition.

[0004] A target attribute recognition method, characterized by comprising:

[0005] Acquire a video to be identified, and perform target tracking processing on the video to be identified to obtain a target identifier of each target and an image set associated with the target identifier;

[0006] Performing target attribute recognition on each target image of each of the image sets to obtain a target recognition result set corresponding to each of the image sets, wherein the target recognition result set includes a plurality of image attribute results, and the image attribute results include branch attribute values ​​of a plurality of target attributes and branch confidences corresponding to the branch attribute values ​​one by one;

[0007] Confidence voting is performed on all the branch attribute values ​​and the branch confidences in each target recognition result set to obtain each final target attribute value of each target and a target confidence result corresponding to each final target attribute value.

[0008] A target attribute recognition device, comprising:

[0009] An acquisition module is used to acquire a video to be identified, and perform target tracking processing on the video to be identified to obtain a target identifier of each target and an image set associated with the target identifier;

[0010] an identification module, configured to perform target attribute identification on each target image of each of the image sets, and obtain a target identification result set corresponding to each of the image sets, wherein the target identification result set includes a plurality of image attribute results, and the image attribute results include branch attribute values ​​of a plurality of target attributes and branch confidences corresponding to the branch attribute values ​​one by one;

[0011] The voting module is used to perform confidence voting on all the branch attribute values ​​and the branch confidences in each target recognition result set to obtain each final target attribute value of each target and a target confidence result corresponding to each final target attribute value.

[0012] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the target attribute identification method when executing the computer program.

[0013] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the target attribute identification method are implemented.

[0014] The target attribute recognition method, device, computer equipment and storage medium provided by the present invention obtain a video to be recognized and perform target tracking processing on the video to be recognized to obtain a target identifier of each target and an image set associated with the target identifier; perform target attribute recognition on each target image in each of the image sets to obtain a target recognition result set corresponding to each of the image sets, wherein the target recognition result set includes multiple image attribute results, and the image attribute results include branch attribute values ​​of several target attributes and branch confidences corresponding to the branch attribute values; perform confidence voting on all the branch attribute values ​​and the branch confidences in each of the target recognition result sets to obtain the final target attributes of each target. The target attribute value and the target confidence result corresponding to each final target attribute value one by one, in this way, the target identification and image set of each target in the video to be identified are automatically identified, and the target attribute recognition and confidence voting are used to automatically and accurately identify the final target attribute values ​​and corresponding target confidence results of each target in the image set. Therefore, there is no need to manually extract a large number of target images from the video, and the situation of human missed recognition or misidentification is avoided. The target attribute recognition and confidence voting of the image set of each target are objectively and scientifically realized, and the final target attributes and target confidence results of each target can be accurately identified, which greatly saves labor costs and improves the accuracy and correctness of target attribute recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0016] Figure 1 is a schematic diagram of an application environment of a target attribute recognition method according to an embodiment of the present invention;

[0017] Figure 2 is a flow chart of a method for identifying target attributes in one embodiment of the present invention;

[0018] Figure 3 is a flow chart of step S10 of a method for identifying target attributes in one embodiment of the present invention;

[0019] Figure 4 is a flow chart of step S20 of the target attribute recognition method in one embodiment of the present invention;

[0020] Figure 5 is a principle block diagram of a target attribute recognition device in one embodiment of the present invention;

[0021] Figure 6 is a schematic diagram of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] The target attribute recognition method provided by the present invention can be applied to Figure 1 In an application environment, a client (computer device or terminal) communicates with a server via a network. The client (computer device or terminal) includes but is not limited to various personal computers, laptops, smart phones, tablet computers, and portable wearable devices. The server can be implemented as an independent server or a server cluster consisting of multiple servers.

[0024] In one embodiment, if Figure 2 As shown, a target attribute recognition method is provided, and its technical solution mainly includes the following steps S10-S30:

[0025] S10, obtaining a video to be identified, and performing target tracking processing on the video to be identified to obtain a target identifier of each target and an image set associated with the target identifier.

[0026] It can be understood that the video to be identified is a video that contains a target to be identified and from which target-related attributes need to be identified, and the video to be identified is a video captured by the same lens, which satisfies the same-lens tracking shooting, thereby reflecting the characteristic that the attribute value reflected by the target in the same lens does not change basically, wherein the target can be set according to demand, such as pedestrians, animals, vehicles or faces, etc., and the video to be identified can be obtained by uploading it to a server or database by a user and then downloading it from the server or database, wherein the target tracking processing includes multi-target detection processing and multi-target tracking processing, and the process of performing target tracking processing on the video to be identified is: first, performing frame processing on the video to be identified to split the video frame image of each frame in the video to be identified; secondly, performing multi-target detection processing on each of the video frame images to obtain the detection results of each of the video frame images; thirdly, performing multi-target tracking processing based on all the detection results to obtain the identification of each target in the video to be identified; finally, extracting target images containing different targets from each video frame image, and taking the target images identified with the same target identification as the image set associated with the target identification.

[0027] Among them, the multi-target detection processing process can be implemented by a trained target detection model, and the target detection model is used to identify the model of the coordinate area of ​​the target in each video frame image in the video to be identified. The network structure of the target detection model can be set according to the needs. For example, the network structure of the target detection model can be a network structure such as Faster R-CNN, SSD and YOLO. Preferably, the network structure of the target detection model is the network structure of CenterNet. The processing process of the target detection model is to preprocess the input video frame image and scale the video frame image to a preset size. The preprocessing includes image scaling processing, that is, using image scaling technology, scaling the long side and short side of the image to a preset size by scaling the long side and padding the short side with zeros, and then performing image channel separation on the scaled video frame image to separate the image to be processed containing three channels. The image channel separation is to separate the red, green and blue channels in the image. The image to be processed is input into the target detection model, and the target features are extracted through the ResNet50 network in the CenterNet-based target detection model. The target detection model detects the target as a point, that is, the center point of the target area is used to represent the target, and the center point offset (offset), width and height (size) of the target are predicted to obtain the actual area of ​​the object. The target features are related features unique to the target to be identified, such as human features, animal features or vehicle features, etc., and then the extracted target features are upsampled by the deconvolution module, that is, deconvolution, and upsampling can be performed three times. The feature map to be predicted is obtained by sampling, and finally the feature map to be predicted is predicted by three branch networks, namely, a heat map prediction network, a length and width prediction network, and a regional center offset prediction network. The target classification is performed on the feature map to be predicted by the heat map prediction network, and the center point of the target area of ​​each target and the radius of the Gaussian circle are predicted. With the center point as the center of the circle, the Gaussian function is decreased outward along the calculated radius to obtain a heat map corresponding to the feature map to be predicted. The length and width area prediction of multiple targets is performed by the length and width prediction network to obtain the length and width map of the target corresponding to the feature map to be predicted. The regional center offset prediction is used The network predicts the offset values ​​of multiple targets to obtain the center offset value of the target corresponding to the feature map to be predicted, and determines the area of ​​each target, that is, the detection result of the video frame image, according to the heat map, length and width map and center offset value, so as to obtain the detection result of each video frame image. The training process of the target detection model is to train the collected sample images associated with the target label, perform heat map prediction, length and width prediction and center offset value prediction on the sample images, obtain the prediction result, calculate the loss value of the prediction result and the target label, and the loss value includes the heat map loss value, the center point offset value loss value and the length and width prediction loss value.When it is detected that the loss value does not reach the convergence condition, the parameters in the target detection model are iteratively updated, and the heat map prediction, length and width prediction, and center offset value prediction are re-performed until the loss value reaches the convergence condition, and the training is stopped to obtain a trained target detection model.

[0028] Among them, the process of performing multi-target tracking processing based on all the detection results can be achieved by a trained target tracking model, and multi-target tracking processing is performed on each of the detection results through the target tracking model. The target tracking model is a model for predicting the action trajectory or movement trajectory of the target. The network structure of the target tracking model can be set according to demand. For example, the network structure of the target tracking model can be a network structure such as DeepSORT, Siammask, etc. Preferably, the network structure of the target tracking model is the network structure of DeepSORT. The DeepSORT (Deep Learning Simple Online and Realtime Tracking) algorithm is a multi-target tracking algorithm that integrates appearance information and focuses on simple and efficient algorithms. The process of performing multi-target tracking processing on each of the detection results by the target tracking model can be: using an IOU matching algorithm to perform IOU matching on the target areas of two adjacent video frame images, calculate the IOU value, and when the IOU value is greater than a preset threshold, determine the two target areas as associated with the same target, and record the two target areas as the same target identifier.The process of the target tracking model performing multi-target tracking processing on each of the detection results can also be as follows: first, performing target prediction on the video frame image corresponding to the detection result, the target prediction is to predict the prediction box of the next frame of the video frame image corresponding to the detection result, that is, predicting the prediction box of the next frame (including confirmed prediction box and unconfirmed prediction box) according to the Kalman filter, and judging the prediction box by the appearance information of the target of the current video frame image to determine whether it is a confirmed prediction box and an unconfirmed prediction box; secondly, observing the current video frame image, that is, observing the time of the current video frame image The detector is used to detect the next video frame image in sequence, and the detection result corresponding to the next video frame image is detected to determine the detection frame (also called the target coordinate area). The IOU matching algorithm is used, and the IOU matching algorithm is an algorithm for calculating the overlap rate of the intersection of two area frames. Each confirmed prediction frame and each detection frame are matched. If the IOU value of the two reaches a preset threshold, the matched confirmed prediction frame and detection frame are associated. If the IOU value of the two does not reach the preset threshold, the IOU matching is performed again. If the preset threshold is still not reached after three times, it means that the matched prediction frame and detection frame are not The motion trajectory of the target is calculated without associating the two. Finally, the error between the prediction box and the detection box is calculated according to the IOU value, and the parameters predicted by the Kalman filter are updated according to the error. Then, the next video frame image is used as the current video frame image, and the Kalman filter is used to continue the prediction, and the execution is continuously looped. After the target tracking model performs multi-target tracking processing on each of the detection results, the detection frames of each target in each video frame image can be associated, so that the detection frames in the detection results associated with each other are tracked. The chain is depicted to form the trajectory of each target, and each target is given a unique target identifier, and the assigned target identifier is marked in the associated video frame image, and the image of the area marked by the target identifier is extracted from the video frame image, and the image of the area is recorded as the target image, so that the target identifier corresponding to each target and the target image containing the target can be obtained, and all target images marked with the same target identifier are recorded as the image set associated with the target identifier, and the image set embodies a collection of target images containing the same target identifier, and the target image is an area image extracted from the video frame image marked with the target identifier and containing the target.

[0029] In one embodiment, if Figure 3 As shown, in step S10, that is, performing target tracking processing on the video to be identified to obtain the target identification of each target and the image set associated with the target identification, it includes:

[0030] S101, performing frame processing on the video to be identified to obtain a plurality of video frame images.

[0031] It can be understood that the video to be identified is a set of images composed of multiple one-frame images played in time sequence, and the frame processing is the operation process of segmenting each frame of the input video or extracting a frame image. By performing the frame processing on the video to be identified, a plurality of video frame images can be obtained, and the video frame image is each frame image in the video to be identified.

[0032] S102, performing multi-target detection processing on each of the video frame images to obtain a detection result of each of the video frame images.

[0033] It can be understood that the process of performing multi-target detection processing on each of the video frame images can be implemented by the trained target detection model, and the video frame image is input into the target detection model, and the video frame image is subjected to data transformation and convolution by the target detection model to extract target features, wherein the target features are relevant features specific to the target to be identified, such as human features (including head, hair, hands, face, torso, clothes, legs, feet, etc. that can reflect the characteristics of a human body), animal features or vehicle features (including front, body, rear, etc. that can reflect the characteristics of a vehicle), etc. The network structure of the target detection model is the network structure of CenterNet, and the processing process of the target detection model is to perform multi-target detection processing on the input video frame image. The image is preprocessed, and the video frame image is scaled to a preset size (for example, 512×512). The preprocessing includes image scaling processing, that is, using image scaling technology, the long side and the short side of the image are scaled to a preset size by scaling the long side and padding the short side with zeros, and then the scaled video frame image is image channel separated to separate the image to be processed containing three channels. The image channel separation is to separate the red, green, and blue channels in the image. The image to be processed is input into the target detection model, and the target features are extracted through the ResNet50 network in the target detection model based on CenterNet. The target detection model detects the target as a point, that is, the center point of the target area is used to represent the target. Target, predict the center point offset (offset) of the target, width and height (size) to obtain the actual area of ​​the object. The target features are relevant features unique to the target to be identified, such as human features (including head, hair, hands, face, torso, clothes, legs, feet, etc. that can reflect the characteristics of a human body), animal features or vehicle features (including front, body, rear, etc. that can reflect the characteristics of a vehicle), etc. Then, the extracted target features are upsampled by the deconvolution module, that is, deconvolution (the inverse process of convolution), and upsampling can be performed three times to obtain the feature map to be predicted. Finally, the feature map to be predicted is predicted by three branch networks, which are the heat map prediction network, the length and width prediction network and the regional center offset prediction network. The target classification is performed on the feature map to be predicted through the heat map prediction network, and the center point of the target area of ​​each target and the radius of the Gaussian circle are predicted. With the center point as the center of the circle, the Gaussian function is decreased outward along the calculated radius, so as to obtain a heat map corresponding to the feature map to be predicted. The length and width area predictions of multiple targets are performed through the length and width prediction network to obtain the length and width map of the target corresponding to the feature map to be predicted. The offset values ​​of multiple targets are predicted through the regional center offset prediction network to obtain the center offset value of the target corresponding to the feature map to be predicted. According to the heat map, the length and width map and the center offset value, the area of ​​each target is determined, that is, the detection result of the video frame image, so as to obtain the detection result of each video frame image. In this way,The target detection model can transform high-dimensional features into low-dimensional feature representations through feature space, and then perform classification and recognition through low-dimensional feature representations to identify the region with target features, thereby identifying the detection result of the coordinate position (region range) of the target in the video frame image. The detection result reflects the coordinate position (region range) with target features in the video frame image, and the detection result can be the result of the coordinate position (region range) of multiple targets corresponding to one video frame image.

[0034] S103, performing multi-target tracking processing on all the detection results to obtain the target identification of each target.

[0035] It is understandable that the process of performing the multi-target tracking process on all the detection results obtained through the multi-target detection process can be realized by the trained target tracking model, and the network structure of the target tracking model is preferably the network structure of DeepSORT. The DeepSORT (Deep Learning Simple Online and Realtime Tracking) algorithm is a method that integrates appearance information (appearance information) and focuses on a multi-target tracking algorithm with simple and efficient algorithms, the process of the target tracking model performing multi-target tracking processing on each of the detection results can also be: first, the video frame image corresponding to the detection result is predicted, and the target prediction is to predict the prediction box of the next frame of the video frame image corresponding to the detection result, that is, the prediction box of the next frame is predicted according to the Kalman filter (including confirmed prediction boxes and unconfirmed prediction boxes), and the prediction box is judged by the appearance information of the target of the current video frame image to determine whether it is a confirmed prediction box and an unconfirmed prediction box, wherein the Kalman filter prediction is to use the Kalman filter to remove noise (Gaussian noise removal), and linearly predict the area of ​​the detection result after the noise is removed, that is, the prediction box is predicted by the parameters predicted by the Kalman filter, such as the prediction method of using a first-order linear function for translation;Secondly, observe the current video frame image, that is, perform detector detection on the next video frame image in the time sequence of the current video frame image, detect the detection result corresponding to the next video frame image, determine the detection frame, and use the IOU matching algorithm, which is an algorithm for calculating the overlap rate of the intersection-over-union ratio of two area frames, to match each confirmed prediction frame with each detection frame, and measure the degree of matching between the two by calculating the output IOU value. If the IOU value of the two reaches the preset threshold, the matched confirmed prediction frame and the detection frame are associated. If the IOU value of the two does not reach the preset threshold, the IOU matching is performed again. If the preset threshold is still not reached after three times, it means that the matched prediction frame and the detection frame are not the motion trajectory of the target, and the two are not matched. Finally, the error between the prediction frame and the detection frame is calculated according to the IOU value, and the parameters predicted by the Kalman filter are updated according to the error, and then the next video frame image is used as the current video frame image, and the Kalman filter is used to continue the prediction, and the execution is continuously cyclical. After the target tracking model performs multi-target tracking processing on each of the detection results, the detection frames of each target in each video frame image can be associated, so that the detection frames in the detection results associated with each other are traced to form the trajectory of each target, and each target is given a unique target identifier, and the assigned target identifier is marked in the associated video frame image, so that the target identifier corresponding to each target and the video frame image containing the target identifier associated with the target identifier can be obtained. ;

[0036] In one embodiment, the step S103, i.e., performing multi-target tracking processing on all the detection results to obtain each target identifier, includes:

[0037] Performing motion tracking analysis on each target coordinate area in the detection results of adjacent video frame images to obtain multiple target trajectories.

[0038] It can be understood that the detection result includes the target coordinate area, and the target coordinate area is the area range of the target in the video frame image. The mobile tracking analysis is an analysis process of predicting the target trajectory of the detection results of two video frame images adjacent in time sequence, as well as noise removal and matching comparison. The process of the mobile tracking analysis is: first, the target is predicted for the video frame image corresponding to the detection result, and the target prediction is to predict the prediction box of the next frame of the video frame image corresponding to the detection result, that is, to predict the prediction box of the next frame according to the Kalman filter (including the confirmed prediction box and the unconfirmed prediction box). Confirmed prediction box), and judge the prediction box through the appearance information of the target of the current video frame image to determine whether it is a confirmed prediction box and an unconfirmed prediction box, wherein the Kalman filter prediction is to use Kalman filtering to remove noise (Gaussian noise removal), and perform linear prediction on the area of ​​the detection result after noise removal, that is, predict the prediction box through the parameters predicted by the Kalman filter, such as the prediction method of using a first-order linear function for translation; secondly, observe the current video frame image, that is, perform a detector detection on the next video frame image in the time sequence of the current video frame image, and detect the next The detection result corresponding to the video frame image is used to determine the target coordinate area, and the IOU matching algorithm is used. The IOU matching algorithm is an algorithm for calculating the overlap rate of the intersection-over-union ratio of two area frames. Each confirmed prediction frame and each target coordinate area are matched, and the matching degree between the two is measured by calculating the output IOU value. If the IOU value of the two reaches the preset threshold, the matched confirmed prediction frame and the target coordinate area are associated. If the IOU value of the two does not reach the preset threshold, the IOU matching is performed again. If the preset threshold is still not reached after three times, it means that the matched prediction frame and the target coordinate area are not the target's motion track. The target coordinate areas of each target in each video frame image are associated with each other, so as to depict the tracking chain of the target coordinate areas in the detection results that are associated with each other, form the trajectory of each target, and obtain the target trajectory of each target.

[0039] The target trajectory reflects the record of the movement trajectory of a target in the video to be identified.

[0040] Each target trajectory is encoded and identified to obtain each target identification.

[0041] It can be understood that each target track is coded and identified as the target, and the coded identification is an identification method of assigning a unique code, so that the target identification corresponding to each target track can be identified.

[0042] The present invention realizes obtaining a plurality of target trajectories by performing mobile tracking analysis on each target coordinate area in the detection results of adjacent video frame images; encoding and identifying each target trajectory to obtain each target identification. In this way, mobile tracking analysis can be performed on the inspection results of the video frame images, and the movement trajectory of the same target can be automatically analyzed, so that the target can be accurately and quickly identified and a target identification can be generated. Therefore, the movement trajectory of the same target in the video to be identified can be accurately found, providing an accurate data basis for subsequent target attribute identification.

[0043] S104, associating the same target identifier and extracting the target from the video frame images to obtain the image sets associated with the target identifiers.

[0044] It can be understood that according to the target trajectory corresponding to each target identifier, the video frame image involved in the same target identifier is determined, so that the video frame image involved in the same target identifier can be associated with the target identifier, and the associated video frame image is subjected to target extraction. The target extraction process is a process of extracting an image containing the target from the video frame image associated with the target identifier according to the target area in the detection result, recording the extracted image as the target image, and recording all the target images extracted from all the video frame images associated with the target identifier as the image set associated with the target identifier, so that each image set associated with each target identifier can be obtained.

[0045] The present invention realizes the steps of performing frame processing on the video to be identified to obtain a plurality of video frame images; performing multi-target detection processing on each of the video frame images to obtain detection results of each of the video frame images; performing multi-target tracking processing on all of the detection results to obtain the target identification of each target; and performing association and target extraction on each of the video frame images with the same target identification to obtain each of the image sets associated with each of the target identifications. In this way, frame processing, multi-target detection processing and multi-target tracking processing can be used to automatically associate the target identification corresponding to each target and the associated image set, accurately identify the image set of multiple targets in the video to be identified, accurately, scientifically and objectively distinguish the video frame images involved in each target, and improve the accuracy and reliability of subsequent target attribute recognition.

[0046] S20, performing target attribute recognition on each target image in each of the image sets to obtain a target recognition result set corresponding one-to-one to each of the image sets, wherein the target recognition result set includes multiple image attribute results, and the image attribute results include branch attribute values ​​of several target attributes and branch confidences corresponding one-to-one to the branch attribute values.

[0047] It is understandable that before performing the target attribute recognition on each of the target images in each of the image sets, image preprocessing can be performed on each of the image sets. The image preprocessing process can be image enhancement processing, and the image enhancement processing method is set according to the needs. For example, image enhancement can include methods for removing noise, contrast enhancement, color enhancement, etc., or a method for completing the image to be detected can be performed before image enhancement processing. The image completion is a method for supplementing the truncated or occluded parts in the image to be detected so that the image to be detected can contain a complete target, such as a human body or a vehicle, etc. The process of performing target attribute recognition on the target image can be implemented by a trained deep learning-based attribute recognition model for identifying target attributes. The attribute recognition model can be a deep learning-based neural network model for multi-task learning, that is, a recognition process of learning the category of target-related attributes through multiple branch tasks, for example At least one branch task of a human attribute is implemented by branches that learn human attribute categories respectively and independently, and can also share a backbone network. Through the backbone network, shared features of the input image can be learned and extracted. The vectors of the extracted shared features can be jointly provided to the branch tasks corresponding to the categories of target-related attributes for identification, thereby reducing the processing load of the processor. There is no need for multiple tasks to extract features in different directions, and the extracted shared feature map is pooled to obtain a pooled shared feature map. The branch network of the target-related attribute is a fully connected layer that receives the pooled shared feature map and identifies a certain attribute category of the target corresponding to the target identifier of the input target image based on the pooled shared feature map. The results output by all the branch networks of the target-related attributes are summarized to obtain a target recognition result set, which embodies an array set of categories of attributes of interest for each target image in the image set associated with the target identifier.

[0048] The target attribute recognition is one human attribute or multiple human attributes recognition.

[0049] In one embodiment, if Figure 4 As shown, in step S20, that is, performing target attribute recognition on each target image in each image set to obtain a target recognition result set corresponding to each image set, including:

[0050] S201, extracting shared features from each target image in the image set through a backbone network in a pre-trained attribute recognition model to obtain a shared feature map of each image.

[0051] It can be understood that the attribute recognition model includes a backbone network, a pooling layer and a fully connected layer. The backbone network is a trained neural network model based on deep learning. The network structure of the backbone network can be a network structure such as the MobileNet series and the ResNet series. Preferably, the network structure of the backbone network is the network structure of ResNet18. The shared features are implicit features learned or obtained by convolution in the shared input image between the target attributes. The shared features are extracted from each channel image in the target image through the backbone network to obtain a shared feature map. For example, the input target image is an image with a size of 256 and 128 and contains 3 channels. After the shared features of the backbone network are extracted, a shared feature map with a shape of (512, 8, 4) is obtained, that is, a feature vector of 8×4 size of 512 channels. The training process of the attribute recognition model is to collect standard An image sample annotated with target-related attributes, the initial parameters of the attribute recognition model are preset, the backbone network in the attribute recognition model is used to extract shared features of the image sample to obtain a shared feature map, and the shared feature map is pooled through the pooling layer in the attribute recognition model to obtain a pooling result, and then the attribute recognition of each branch network is performed on the pooling result to identify the image attribute results that correspond one-to-one with the annotated target-related attributes, and the cross-entropy loss function is used to calculate the attribute loss value between the annotated target-related attributes and the corresponding attribute category in the image attribute result. When it is detected that the attribute loss value does not reach the convergence condition, the initial parameters of the attribute recognition model are iteratively updated, and the step of extracting shared features of the image sample using the backbone network in the attribute recognition model is re-executed until it is detected that the attribute loss value reaches the convergence condition, the training is stopped, and the converged attribute recognition model is recorded as the pre-trained attribute recognition model.

[0052] Among them, the pooling layer is a layer that performs pooling operations on the extracted shared feature map, and the fully connected layer is a layer that receives the pooled shared feature map output by the pooling layer and identifies the attribute category of the target based on the pooled shared feature map.

[0053] S202, performing global average pooling processing on the shared feature map through the pooling layer in the attribute recognition model to obtain a pooled shared feature map.

[0054] Understandably, the global average pooling process is also called Global Average Pooling, which means adding all pixel values ​​in the shared feature map and averaging them to obtain a numerical value, that is, using this numerical value to represent the corresponding feature map. It can structurally regularize the entire network to prevent overfitting, eliminate the features of the black box operation of the fully connected layer, and directly give each channel the category meaning of the actual target attribute, thereby obtaining a pooled shared feature map. For example, a shared feature map with a shape of (512, 8, 4) is subjected to global average pooling of the pooling layer, and the shape of the pooled shared feature map is (512, 1, 1).

[0055] S203, performing attribute recognition on the pooled shared feature map through the fully connected layer in the attribute recognition model to obtain an image attribute result corresponding to the target image.

[0056] Understandably, the attribute recognition process is that each branch network in the attribute recognition model extracts the features of the categories of their respective target attributes from the shared feature graph, and the feature extraction of the categories of the respective target attributes is performed by training with various samples containing different target attributes, such as training with the network structure of VGG16, learning the common features of different target attribute categories, and performing full-connection layer classification based on the learned common features of different target attribute categories, thereby identifying the category of the target attribute of each sample, thereby obtaining the trained branch tasks for identifying various different target attributes, and inputting the shared feature graph into the branch networks of each different attribute recognition respectively, extracting the corresponding features one by one through the branch networks of each different attribute recognition, and classifying them one by one according to the extracted features, thereby According to the classification results, the results of the corresponding categories of different target attributes are identified, for example: the attributes of each human body are identified through a branch network related to human body attributes (such as age, gender, human body direction, hair color, hair length, type of headwear, whether wearing glasses, whether carrying a backpack, whether carrying a shoulder bag, whether stroller, whether holding a child, whether carrying a handbag, whether wearing a coat, whether riding, whether carrying a trolley case, whether answering a call, top color, top texture, top type, bottom color, bottom texture, bottom type, etc. that can reflect the categories of structured attributes of the human body), the image attribute result reflects the results of target-related attributes in the target image, the image attribute result includes the category of at least one target-related attribute, the image attribute result is an array of multiple attribute categories, and one target image corresponds to one image attribute result.

[0057] In one embodiment, the step S203, i.e., performing attribute recognition on the pooled shared feature map through the fully connected layer in the attribute recognition model to obtain an image attribute result corresponding to the target image, includes:

[0058] The attributes of each branch task are identified on the pooled shared feature map through each branch network in the fully connected layer to obtain attribute branch results corresponding to each branch network one by one, and all the attribute branch results are merged into the image attribute result corresponding to the target image.

[0059] It can be understood that the attribute recognition of each branch task is the recognition of branch networks of different target attribute categories. Preferably, the branch task is the category of human body attributes. Different human body attribute branch networks learn different human body attribute categories, and learn their own characteristics according to their respective human body attribute categories, so as to perform the corresponding attribute recognition process, thereby identifying the attribute branch results corresponding to each human body attribute branch network one by one. Because the recognition is performed through a shared feature map, there is a certain correlation between the categories of each target attribute. Through the mutual correlation between each attribute branch result, the learned correlation (linear correlation between parameters) can be used to output the image attribute result. The attribute branch result reflects the category recognized by the human body attribute branch network and includes the confidence of the category. Finally, all the attribute branch results are merged into one result, and the result is recorded as the image attribute result. The image attribute result includes a branch attribute value and a branch confidence.

[0060] S204: Summarize the image attribute results corresponding to all the target images in the image set to obtain the target recognition result set corresponding to the image set.

[0061] It can be understood that the target recognition result set is a set of the image attribute results corresponding to the same target identifier, for example: the image attribute results of obj_1 are (male, 0.8, female, 0.2; wearing glasses, 0.7, not wearing glasses, 0.3), (male, 0.7, female, 0.3; wearing glasses, 0.7, not wearing glasses, 0.3), (male, 0.9, female, 0.1; wearing glasses, 0.8, not wearing glasses, 0.2), (male, 0.4, female, 0.6; wearing glasses, 0.8, not wearing glasses, 0.2), (male, 0.3, female, 0.7; wearing glasses, 0.9, not wearing glasses, 0.1), then the target recognition result set of obj_1 is [(male, 0.8, female, 0.2; wearing glasses, 0.7, not wearing glasses, 0.3); (male, 0.7, female, 0.3; wearing glasses, 0.7, not wearing glasses, 0.3); (male, 0.9, female, 0.1; wearing glasses, 0.8, not wearing glasses, 0.2); (male, 0.4, female, 0.6; wearing glasses, 0.8, not wearing glasses, 0.2); (male, 0.3, female, 0.7; wearing glasses, 0.9, not wearing glasses, 0.1)], so that the target recognition result sets corresponding to all the image sets can be summarized.

[0062] The present invention realizes the extraction of shared features of target images in the image set through a backbone network in an attribute recognition model to obtain a shared feature map of each target image; performs global average pooling processing on the shared feature map through a pooling layer in the attribute recognition model to obtain a pooled shared feature map; performs attribute recognition on the pooled shared feature map through a fully connected layer in the attribute recognition model to obtain an image attribute result corresponding to the target image; summarizes the image attribute results corresponding to all the target images in the image set to obtain the target recognition result set corresponding to the image set, thus realizing the extraction of shared features of target images by sharing a backbone network, automatically identifying a target recognition result set of attribute categories of target concern based on the shared features, and accurately identifying the categories of each target attribute through the correlation between the target attributes in the shared features, providing a data basis for subsequent attribute voting, and providing accurate attribute categories for subsequent screening of training samples, thereby ensuring the accuracy and correctness of screening.

[0063] S30, performing confidence voting on all the branch attribute values ​​and the branch confidences in each target recognition result set to obtain each final target attribute value of each target and a target confidence result corresponding to each final target attribute value.

[0064] Understandably, a comprehensive voting mechanism is used to perform confidence voting on each of the target recognition result sets. The comprehensive voting mechanism is a mechanism for combining different voting results to output confidence. The confidence voting is to use the comprehensive voting mechanism to calculate the results of the attributes that need to be paid attention to in each of the target recognition result sets. The results of preset attributes of concern can be screened out from the target recognition result set. The confidence voting and correction processing process can be to perform a majority voting mechanism on each target recognition result set to obtain the first attribute value of each target attribute corresponding to each target identifier and the first confidence corresponding to the first attribute value; perform a mean voting mechanism on each target recognition result set to obtain the first attribute value of each target attribute corresponding to each target identifier. The second attribute value of each target attribute and the second confidence corresponding to the second attribute value are determined; when it is detected that the first attribute value and the second attribute value of the same target attribute of the same target are the same, the first attribute value or the second attribute value is used as the final target attribute value of the corresponding target attribute of the target recognition result set; the first confidence level is multiplied by the second confidence level to obtain the target confidence results corresponding to each final target attribute value, the target confidence results reflect the confidence that a target attribute recognized by a target image in the target recognition result set is close to the real attribute, and high-quality images can be screened out subsequently through the target confidence results, which is convenient for subsequent model training and improves training efficiency.

[0065] The present invention realizes the method of obtaining a video to be identified and performing target tracking processing on the video to be identified, thereby obtaining a target identifier of each target and an image set associated with the target identifier; performing target attribute recognition on each target image in each of the image sets, thereby obtaining a target recognition result set corresponding to each of the image sets one by one, wherein the target recognition result set includes multiple image attribute results, and the image attribute results include branch attribute values ​​of several target attributes and branch confidences corresponding to the branch attribute values ​​one by one; performing confidence voting on all the branch attribute values ​​and the branch confidences in each of the target recognition result sets, thereby obtaining each final target attribute value of each target and each branch confidence corresponding to the final target attribute. The target confidence result with one-to-one corresponding property value is realized, thus realizing automatic recognition of target identification and image set of each target in the video to be recognized, and using target attribute recognition and confidence voting, automatically and accurately identifying each final target attribute value and corresponding target confidence result of each target in the image set. Therefore, there is no need to manually extract a large number of target images from the video, and it also avoids the situation of human missed recognition or misrecognition. It realizes the target attribute recognition and confidence voting of the image set of each target objectively and scientifically, and can accurately identify the target attributes and target confidence results of each target, which greatly saves labor costs and improves the accuracy and correctness of target attribute recognition.

[0066] In one embodiment, the step S30, i.e., performing confidence voting on all the branch attribute values ​​and the branch confidences in each target recognition result set to obtain each final target attribute value of each target and a target confidence result corresponding to each final target attribute value, includes:

[0067] A majority voting mechanism is performed on all the branch attribute values ​​and the branch confidences of each target attribute in each target recognition result set to obtain a first attribute value of each target attribute of each target and a first confidence corresponding to the first attribute value.

[0068] It can be understood that the majority voting mechanism is to take the branch attribute with the largest number under the same target attribute in the target recognition result set as the first attribute value of the target recognition result set, that is, a mechanism that satisfies the principle of minority obeys majority, counts all branch attribute values ​​under each target attribute, and takes the branch attribute value with the largest number to determine the first attribute value corresponding to the target attribute of this target. The first attribute value is the category of the attribute obtained after the majority voting mechanism. The first confidence corresponds one-to-one to the first attribute value. The calculation method of the first confidence can be the average of all branch confidences with the same first attribute value under the same target attribute of a target, or it can be a weighted average calculation, that is, for each image attribute result in the target recognition result set, if it is the same as the first attribute value, the weight is increased by one, otherwise it is reduced by one, and finally the average is taken.

[0069] A mean voting mechanism is performed on all the branch attribute values ​​and the branch confidences of each target attribute in each target recognition result set to obtain a second attribute value of each target attribute of each target and a second confidence corresponding to the second attribute value.

[0070] It can be understood that the mean voting mechanism is to take the average of the branch confidences corresponding to the same branch attribute values ​​under the same target attribute in the target recognition result set, and use the branch attribute value corresponding to the largest average value under the target attribute as the second attribute value corresponding to the target attribute of this target, and use the largest average value as the second confidence corresponding to the second attribute value.

[0071] When the first attribute value and the second attribute value of the same target attribute of the same target are the same, the first attribute value or the second attribute value is used as the final target attribute value of the target attribute of the target.

[0072] It can be understood that whether the first attribute value and the second attribute value of the same target attribute of the same target are the same is determined. When it is detected that the first attribute value and the second attribute value of the same target attribute of the target in the image set are the same, it means that the recognition accuracy of the target attribute is high, and the first attribute value or the second attribute value is used as the final target attribute value of the target attribute of the target.

[0073] The first confidence and the second confidence corresponding to each final target attribute value are multiplied to obtain the target confidence result corresponding to each final target attribute value.

[0074] It can be understood that after confirming the final target attribute value of a target attribute, the first confidence and the second confidence corresponding to the target attribute are multiplied together to calculate the target confidence result corresponding to the final target attribute value. The target confidence result reflects the confidence or degree to which a target attribute identified by a target image in a target recognition result set is close to the real attribute.

[0075] The present invention realizes obtaining the first attribute value of each target attribute of each target and the first confidence corresponding to the first attribute value by performing a majority voting mechanism on all the branch attribute values ​​and the branch confidence of each target attribute in each target recognition result set; performing a mean voting mechanism on all the branch attribute values ​​and the branch confidence of each target attribute in each target recognition result set to obtain the second attribute value of each target attribute of each target and the second confidence corresponding to the second attribute value; when the first attribute value and the second attribute value of the same target attribute of the same target are the same, the first attribute value or the second attribute value is used as The final target attribute value of the target attribute of the target; multiplying the first confidence and the second confidence corresponding to each of the final target attribute values ​​to obtain the target confidence results corresponding to each final target attribute value, thereby realizing the integration of the majority voting mechanism and the mean voting mechanism, automatically voting and outputting the final target attribute values ​​of each target attribute corresponding to the target identification, and being able to accurately and scientifically vote and output the target confidence results of each target attribute of the target identification, providing a basis for accurate and high-confidence sample data for subsequent model training, and improving the accuracy and correctness of the confidence of target attribute recognition.

[0076] In one embodiment, after obtaining the second attribute value of each target attribute of each target and the second confidence corresponding to the second attribute value, the method further includes:

[0077] When it is detected that the first attribute value and the second attribute value of the same target attribute of the same target are different, the target attribute under the target identification is removed.

[0078] It can be understood that when judging whether the first attribute value and the second attribute value of the same target attribute of the same target are the same, if it is detected that the first attribute value and the second attribute value of a target attribute of a target are not the same, it means that the recognition accuracy of the target attribute is not high, and inconsistent voting occurs. It is necessary to discard all the branch attributes and branch confidences that are the same as the target attribute in the target recognition result set, indicating that there is misrecognition in the recognition of this target attribute, and it is necessary to discard the target attribute of this target, thereby improving the accuracy of target attribute recognition.

[0079] In one embodiment, after obtaining the target confidence results corresponding to the final target attribute values, the method further includes:

[0080] A variance voting mechanism is performed on all the branch confidences corresponding to each of the final target attribute values ​​to obtain a third confidence corresponding to each of the final target attribute values.

[0081] Understandably, the variance voting mechanism includes variance acquisition and confidence adjustment. The variance acquisition is to average all the branch confidences corresponding to each of the final target attribute values ​​to obtain an average confidence corresponding to each of the final target attribute values. The variance of the final target attribute value is calculated based on all the branch confidences corresponding to a final target attribute value and the average confidence, that is, the variance calculation formula is used:

[0082] σ 2 =Σ(X-μ) 2 / N

[0083] Among them, σ 2 is the variance corresponding to a final target attribute value; X is the branch confidence corresponding to the final target attribute value; μ is the average confidence corresponding to the final target attribute value; N is the number of all branch confidences corresponding to the final target attribute value; the variance is used to measure the degree of deviation between all branch confidences and their mean, so as to obtain the processing process of the variance corresponding to each final target attribute value, the confidence adjustment is to add or subtract the corresponding average confidence based on the variance of the final target attribute, and obtain the third confidence corresponding to the final target attribute value, the confidence adjustment can be set according to the demand, for example, the confidence adjustment is based on the proportion of the variance to the average confidence, the range of the proportion is judged to map the corresponding increase or decrease of the supplementary confidence, the average confidence is increased or decreased by the supplementary confidence for calculation, and the third confidence is obtained.

[0084] The target confidence results corresponding to the respective final target attribute values ​​are multiplied by the third confidence to correct the respective target confidence results.

[0085] Understandably, the target confidence result corresponding to the same final target attribute value is multiplied by the third confidence to obtain a multiplied result, and the corresponding target confidence result is revised using the result to obtain a new target confidence result corresponding to the final target attribute value.

[0086] The present invention realizes obtaining the first attribute value of each target attribute of each target and the first confidence corresponding to the first attribute value by performing a majority voting mechanism on all the branch attribute values ​​and the branch confidence of each target attribute in each target recognition result set; performing an average voting mechanism on all the branch attribute values ​​and the branch confidence of each target attribute in each target recognition result set to obtain the second attribute value of each target attribute of each target and the second confidence corresponding to the second attribute value; when the first attribute value and the second attribute value of the same target attribute of the same target are the same, taking the first attribute value or the second attribute value as the final target attribute value of the target attribute of the target; taking the first confidence and the second confidence corresponding to each final target attribute value as the final target attribute value. The confidences of the branches are multiplied to obtain the target confidence results corresponding to the final target attribute values; all the branch confidences corresponding to each of the final target attribute values ​​are processed by the variance voting mechanism to obtain the third confidences corresponding to the final target attribute values; the target confidence results corresponding to the final target attribute values ​​are multiplied by the third confidences to correct the target confidence results. In this way, the target confidence results of each target attribute can be automatically corrected by comprehensive processing through the majority voting mechanism, the mean voting mechanism, and the variance voting mechanism, and the target confidence of the target identification can be voted out more accurately and scientifically, providing an accurate data basis for subsequent target attribute identification and improving the accuracy and correctness of the confidence of target attribute identification.

[0087] In one embodiment, after obtaining each final target attribute value of each target and the target confidence result corresponding to each final target attribute value, the following steps are performed:

[0088] According to the attribute values ​​of the final targets, the corresponding branch attributes in the target recognition result sets are corrected to obtain new target recognition result sets for each target.

[0089] It is understandable that since there may be different branch attribute values ​​of the same target attribute of a target in the target recognition result set, and the video to be recognized belongs to the same shot and the attribute value hardly changes, such different branch attribute values ​​are errors in recognition and need to be corrected. Therefore, it is necessary to perform attribute correction on the corresponding branch attributes in the target recognition result set according to the final target attribute value. The attribute correction is to correct the branch attribute value that is different from the final target attribute value to the same value as the final target attribute value, thereby obtaining a new target recognition result set for the target. In this way, the erroneous branch attribute values ​​in the target recognition result set can be automatically corrected to obtain an accurate new target recognition result set for each target, thereby providing accurate attribute values ​​for subsequent model training.

[0090] In one embodiment, the target attribute recognition method further includes:

[0091] All the target confidence results of each target attribute are threshold screened, and the corresponding image set is labeled according to the target confidence results after the threshold screening, and the labeled image set is recorded as a training sample.

[0092] It can be understood that the threshold screening is performed on all the target confidence results of the same target attribute of different targets. The threshold screening can be performed by screening through a preset screening threshold. The preset screening threshold can be set according to needs. For example, the preset screening threshold is 85%. The target confidence results greater than or equal to the preset screening threshold are screened to obtain the target confidence results after threshold screening. The threshold screening can also be performed by performing sample threshold identification through a threshold recognition model to identify the corresponding sample threshold, and then screening is performed according to the sample threshold. The sample threshold identification is an identification process of identifying the sample threshold in all the target confidence results through a threshold recognition model. The threshold feature is extracted through the threshold recognition model, and the corresponding sample threshold is identified based on the threshold feature. The threshold feature is an adjacent jump feature in all final confidences. Generally, a higher confidence will be in a stable state, and an instantaneous large jump will occur in the process of confidence decreasing. At this time, the adjacent final confidence before the jump can be determined as the sample threshold according to the jump, and the target confidence results greater than or equal to the sample threshold are screened to obtain the target confidence results after threshold screening.

[0093] Among them, the labeling process is the process of associating the branch attribute values ​​in the corrected target recognition result set corresponding to the target confidence result after threshold screening with the target image in the corresponding image set, so as to obtain the training samples for subsequent training. The training samples are image samples that meet the threshold requirements and have been labeled and are required for subsequent training.

[0094] In this way, the present invention realizes accurate and automatic screening of training samples that meet the requirements through sample threshold recognition and threshold screening. Therefore, there is no need to manually extract a large number of images from the video, and manually find the confidence threshold and screen out the training samples, which greatly saves labor costs, removes invalid samples in the target recognition result set, improves the accuracy and correctness of target attribute recognition, and improves the efficiency and model performance for subsequent model training.

[0095] In one embodiment, the step of performing threshold screening on all the target confidence results of each target attribute, labeling the corresponding image set according to the target confidence results after the threshold screening, and recording the labeled image set as a training sample includes:

[0096] All the target confidence results of the same target attribute are sorted to obtain a sorting result.

[0097] Understandably, the method of sorting all the target confidences of the same target attribute can be set according to requirements, such as sorting in descending order or in ascending order, and recording all the sorted target confidence results as the sorting result.

[0098] Threshold features are extracted from the sorting results using a pre-trained threshold recognition model, and sample thresholds of the sorting results are identified based on the extracted threshold features.

[0099] It can be understood that the threshold recognition model is a trained model for identifying the sample threshold in the input sorting result. The threshold recognition model can perform convolution learning by inputting a series of sorting samples marked with thresholds, learn the threshold features in the sorting samples, and continue learning to eventually train the threshold recognition model. The threshold features are adjacent jump features in all final confidence levels. Generally, higher confidence levels will be in a stable state, and there will be instantaneous large jumps in the process of confidence level decreasing. At this time, the adjacent final confidence level before the jump can be determined as the sample threshold based on the jump.

[0100] Among them, the training process of the threshold recognition model is to set the initial model parameters for the threshold recognition model through the sample array containing threshold labels collected historically. The network structure of the threshold recognition model can be the network structure of a convolutional neural network (CNN). The threshold feature extraction is performed on the sample array through the threshold recognition model to extract the features with large instantaneous jump span. A boundary value is output according to the extracted threshold feature convolution. The boundary value reflects the starting jump value in the sample array. The cross entropy loss function is used to calculate the boundary value and the threshold loss value of the threshold label. When it is detected that the threshold loss value does not reach the preset threshold convergence condition, the initial model parameters of the threshold recognition model are iteratively updated, and the threshold feature extraction is performed on the sample array through the threshold recognition model. The threshold loss value is recalculated until it is detected that the threshold loss value reaches the preset threshold convergence condition, and the training is stopped. The converged threshold recognition model is recorded as the trained threshold recognition model.

[0101] The sorting results are subjected to threshold screening to screen out the target confidence results that are greater than or equal to the sample threshold and record them as the final screening results.

[0102] It can be understood that the threshold screening process is a process of screening target confidence results that are greater than or equal to the sample threshold, and the sorting results after the threshold screening are recorded as the final screening results.

[0103] According to the new target recognition result set corresponding to all the final screening results, the corresponding image set is labeled to obtain the training sample.

[0104] It can be understood that the process of labeling the corresponding image set is a process of associating the branch attribute values ​​and / or branch confidences in the new target recognition result set corresponding to the target confidence results after threshold screening with the corresponding target images in the image set one by one, so as to obtain the training samples for subsequent training, and the training samples are high-quality images for training.

[0105] The present invention achieves the following steps: sorting all the target confidence results of the same target attribute to obtain a sorting result; performing threshold feature extraction on the sorting result through a pre-trained threshold recognition model, and identifying the sample threshold of the sorting result according to the extracted threshold feature; performing threshold screening on the sorting result to screen out the target confidence results corresponding to the sample threshold that is greater than or equal to the sample threshold, and recording them as the final screening result; and labeling the corresponding image set according to the new target recognition result set corresponding to all the final screening results to obtain the training sample. In this way, the sample thresholds in all the target confidence results of the same target attribute are automatically identified, and the training samples are automatically screened out through threshold screening and labeling, without manually searching for the sample threshold corresponding to the specified accuracy rate, which greatly reduces the workload of manual recognition, improves the accuracy rate and correctness of target attribute recognition, and improves the efficiency and model performance for subsequent model training.

[0106] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0107] In one embodiment, a target attribute recognition device is provided, and the target attribute recognition device corresponds one-to-one with the target attribute recognition method in the above embodiment. Figure 5 As shown, the target attribute recognition device includes an acquisition module 11, a recognition module 12 and a voting module 13. The detailed description of each functional module is as follows:

[0108] The acquisition module 11 is used to acquire the video to be identified, and perform target tracking processing on the video to be identified to obtain the target identification of each target and the image set associated with the target identification;

[0109] The recognition module 12 is used to perform target attribute recognition on each target image of each image set to obtain a target recognition result set corresponding to each image set, wherein the target recognition result set includes multiple image attribute results, and the image attribute results include branch attribute values ​​of several target attributes and branch confidences corresponding to the branch attribute values ​​one by one;

[0110] The voting module 13 is used to perform confidence voting on all the branch attribute values ​​and the branch confidences in each target recognition result set to obtain each final target attribute value of each target and a target confidence result corresponding to each final target attribute value.

[0111] For the specific definition of the target attribute recognition device, please refer to the definition of the target attribute recognition method above, which will not be repeated here. Each module in the above-mentioned target attribute recognition device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0112] In one embodiment, a computer device is provided. The computer device may be a client or a server. The internal structure diagram thereof may be as follows: Figure 6 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a readable storage medium and an internal memory. The readable storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the readable storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a target attribute recognition method is implemented.

[0113] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the target attribute recognition method in the above embodiment is implemented.

[0114] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the target attribute recognition method in the above embodiment is implemented.

[0115] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0116] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0117] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A target attribute recognition method, characterized in that: include: Acquire a video to be identified, and perform target tracking processing on the video to be identified to obtain a target identifier of each target and an image set associated with the target identifier; Performing target attribute recognition on each target image of each of the image sets to obtain a target recognition result set corresponding to each of the image sets, wherein the target recognition result set includes a plurality of image attribute results, and the image attribute results include branch attribute values ​​of a plurality of target attributes and branch confidences corresponding to the branch attribute values ​​one by one; Performing confidence voting on all the branch attribute values ​​and the branch confidences in each target recognition result set to obtain each final target attribute value of each target and a target confidence result corresponding to each final target attribute value; The confidence voting is performed on all the branch attribute values ​​and the branch confidences in each target recognition result set to obtain each final target attribute value of each target and a target confidence result corresponding to each final target attribute value, including: Performing a majority voting mechanism on all the branch attribute values ​​and the branch confidences of each target attribute in each target recognition result set to obtain a first attribute value of each target attribute of each target and a first confidence corresponding to the first attribute value; Performing a mean voting mechanism on all the branch attribute values ​​and the branch confidences of each target attribute in each target recognition result set to obtain a second attribute value of each target attribute of each target and a second confidence corresponding to the second attribute value; When the first attribute value and the second attribute value of the same target attribute of the same target are the same, the first attribute value or the second attribute value is used as the final target attribute value of the target attribute of the target; The first confidence and the second confidence corresponding to each final target attribute value are multiplied to obtain the target confidence result corresponding to each final target attribute value.

2. The target attribute recognition method according to claim 1, characterized in that: The target tracking process is performed on the video to be identified to obtain the target identification of each target and the image set associated with the target identification, including: Performing frame processing on the video to be identified to obtain multiple video frame images; Performing multi-target detection processing on each of the video frame images to obtain detection results of each of the video frame images; Performing multi-target tracking processing on all the detection results to obtain the target identification of each target; The same target identifier is associated and the target is extracted from each of the video frame images to obtain each of the image sets associated with each of the target identifiers.

3. The target attribute recognition method according to claim 1, characterized in that: The target attribute recognition is performed on each target image in each of the image sets to obtain a target recognition result set corresponding to each of the image sets, including: Extracting shared features from each target image in the image set through a backbone network in a pre-trained attribute recognition model to obtain a shared feature map of each target image; Performing global average pooling processing on the shared feature map through the pooling layer in the attribute recognition model to obtain a pooled shared feature map; Performing attribute recognition on the pooled shared feature map through the fully connected layer in the attribute recognition model to obtain an image attribute result corresponding to the target image; The image attribute results corresponding to all the target images in the image set are aggregated to obtain the target recognition result set corresponding to the image set.

4. The target attribute recognition method according to claim 1, characterized in that: After obtaining the target confidence results corresponding to the final target attribute values, the method further includes: Performing variance voting mechanism processing on all the branch confidences corresponding to each of the final target attribute values ​​to obtain third confidences corresponding to each of the final target attribute values; The target confidence results corresponding to the respective final target attribute values ​​are multiplied by the third confidence to correct the respective target confidence results.

5. The target attribute recognition method according to any one of claims 1 to 4, characterized in that: After obtaining each final target attribute value of each target and the target confidence result corresponding to each final target attribute value, it includes: According to the attribute values ​​of the final targets, the corresponding branch attributes in the target recognition result sets are corrected to obtain new target recognition result sets for each target.

6. The target attribute recognition method according to claim 5, characterized in that: The method further comprises: All the target confidence results of each target attribute are threshold screened, and the corresponding image set is labeled according to the target confidence results after the threshold screening, and the labeled image set is recorded as a training sample.

7. The target attribute recognition method according to claim 6, characterized in that: The step of performing threshold screening on all the target confidence results of each target attribute, labeling the corresponding image set according to the target confidence results after the threshold screening, and recording the labeled image set as a training sample includes: Sorting all the target confidence results of the same target attribute to obtain a sorting result; Extracting threshold features from the sorting results using a pre-trained threshold recognition model, and identifying a sample threshold of the sorting results based on the extracted threshold features; Perform threshold screening on the sorting results, screen out the target confidence results that are greater than or equal to the sample threshold, and record them as the final screening results; According to the new target recognition result set corresponding to all the final screening results, the corresponding image set is labeled to obtain the training sample.

8. A target attribute recognition device, characterized in that: include: An acquisition module is used to acquire a video to be identified, and perform target tracking processing on the video to be identified to obtain a target identifier of each target and an image set associated with the target identifier; an identification module, configured to perform target attribute identification on each target image of each of the image sets, and obtain a target identification result set corresponding to each of the image sets, wherein the target identification result set includes a plurality of image attribute results, and the image attribute results include branch attribute values ​​of a plurality of target attributes and branch confidences corresponding to the branch attribute values ​​one by one; A voting module, used to perform confidence voting on all the branch attribute values ​​and the branch confidences in each target recognition result set, to obtain each final target attribute value of each target and a target confidence result corresponding to each final target attribute value; The confidence voting is performed on all the branch attribute values ​​and the branch confidences in each target recognition result set to obtain each final target attribute value of each target and a target confidence result corresponding to each final target attribute value, including: Performing a majority voting mechanism on all the branch attribute values ​​and the branch confidences of each target attribute in each target recognition result set to obtain a first attribute value of each target attribute of each target and a first confidence corresponding to the first attribute value; Performing a mean voting mechanism on all the branch attribute values ​​and the branch confidences of each target attribute in each target recognition result set to obtain a second attribute value of each target attribute of each target and a second confidence corresponding to the second attribute value; When the first attribute value and the second attribute value of the same target attribute of the same target are the same, the first attribute value or the second attribute value is used as the final target attribute value of the target attribute of the target; The first confidence and the second confidence corresponding to each final target attribute value are multiplied to obtain the target confidence result corresponding to each final target attribute value.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the target attribute recognition method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the target attribute recognition method according to any one of claims 1 to 7 is implemented.

Images