A method and device for identifying pedestrian attributes based on surveillance video

Abstract

The present invention relates to the field of image processing technology, and discloses a method and device for pedestrian attribute recognition based on surveillance video. Pedestrian attributes are divided into fixed attributes and dynamic attributes. A dual-branch attention network of a preset pedestrian attribute recognition network is constructed, and multiple consecutive frames of images of the same pedestrian are used as model input. Pedestrian spatial features are extracted using a unified backbone network, and then the extracted spatial features are input into two branches. A recognition network of preset fixed attribute features is used to recognize fixed attributes, and a recognition network of preset dynamic attribute features is used to recognize dynamic attributes. A time attention module is added to both branches to determine the importance of different pedestrian attributes in different image frames, thereby giving different weights to different image frames. Finally, different pedestrian attributes are classified and output using the extracted time features and spatial features, so as to obtain accurate pedestrian attribute recognition results.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method and device for pedestrian attribute recognition based on surveillance video. Background Art

[0002] At present, by installing high-position video surveillance equipment on the side of urban roads, important traffic light intersections, etc., collecting surveillance video image data, and using visual algorithms to process and analyze the data, the analysis of surveillance videos can be realized, which has important application value in many fields such as intelligent transportation, security, criminal investigation, etc., and can capture evidence and issue warnings for traffic violations such as running red lights and speeding; guide roadside parking and record parking spaces, and update, predict and publish traffic congestion conditions in real time; in application fields such as security and criminal investigation, pedestrian attribute recognition algorithms are used to assist in the capture of criminal suspects.

[0003] At present, for the task of pedestrian attribute recognition, the pedestrian detection algorithm is first used to obtain the pedestrian two-dimensional detection frame, and then the pedestrian attributes in the predefined list are classified. The current pedestrian attributes mainly include gender, age, clothing, accessories and other attributes, and pedestrian attribute recognition is mainly based on single-frame images. However, the pedestrian observed in a single-frame image has a limited viewing angle, the pedestrian's posture is fixed, and the dynamic attributes of the pedestrian, such as the pedestrian's movements, cannot be recognized based on a single-frame image. There is a defect that some pedestrian attributes are invisible and cannot be accurately recognized. Summary of the invention

[0004] In view of this, the present invention provides a method and device for pedestrian attribute recognition based on surveillance video to overcome the problem that some pedestrian attributes are invisible and cannot be accurately recognized due to single-frame images.

[0005] In a first aspect, the present invention provides a method for pedestrian attribute recognition based on surveillance video, the method comprising:

[0006] Acquire a video image dataset of a target area, wherein the video dataset includes at least one short video segment, and each short video segment includes a plurality of continuous frames of images;

[0007] Preprocess the video image dataset;

[0008] By building a preset target pedestrian detection network, two-dimensional target detection is performed on the preprocessed video image data set to obtain pedestrian image data;

[0009] Construct a backbone network of a preset pedestrian attribute recognition network to extract pedestrian image features from pedestrian image data;

[0010] Constructing a dual-branch attention network of a preset pedestrian attribute recognition network, and classifying pedestrian attribute features respectively based on the results of pedestrian image feature extraction, wherein the dual-branch attention network of the preset pedestrian attribute recognition network includes: a recognition network of preset fixed attribute features and a recognition network of preset dynamic attribute features;

[0011] According to the preset loss function, when the classification result meets the preset conditions, the actor's attribute identification is completed.

[0012] In an optional implementation, preprocessing the video image data set includes: target pedestrian detection annotation, pedestrian fixed attribute annotation and pedestrian dynamic attribute annotation, wherein:

[0013] Target pedestrian detection and annotation, used to use the rectangular box tool to mark the target pedestrians in the image;

[0014] Pedestrian fixed attribute labeling, used to label the fixed attributes of the target pedestrian;

[0015] Pedestrian dynamic attribute labeling is used to label the dynamic attributes of the target pedestrian based on the labeling results of fixed attributes.

[0016] In an optional implementation, the preset target pedestrian detection network includes: a feature extraction network and a target detection network, wherein:

[0017] Feature extraction network, used to output feature representation of the image;

[0018] The target detection network is used to output the category and location information of the target pedestrian and extract the target pedestrian image. In an optional implementation, the output of the target detection network is expressed by the following formula:

[0019] O i =[c i ,x i ,y i ,w i ,h i ]

[0020] Among them, O i Denotes the i-th target pedestrian, c i Indicates the pedestrian category of the extracted target, x i ,y i are the center point coordinates of the target pedestrian rectangle, w i ,h i are the length and width of the target pedestrian respectively.

[0021] In an optional implementation, the dual-branch attention network of the preset pedestrian attribute recognition network further includes: a temporal attention module for determining the weights of different pedestrian attributes in different image frames.

[0022] In an optional implementation, the preset loss function includes: a classification and detection box regression loss function for pedestrian target detection, and a classification loss function for pedestrian attribute recognition, wherein:

[0023] The classification and detection box regression loss functions for pedestrian target detection are calculated using the following formula:

[0024] L OD =β1L class +β2L bbox

[0025] L class represents the classification loss function of pedestrian targets, L bbox represents the regression loss function of the pedestrian target detection box, β1 and β2 are both constants;

[0026] The classification loss function for pedestrian attribute recognition adopts the softmax classification loss function.

[0027] In a second aspect, the present invention provides a pedestrian attribute recognition device based on surveillance video, the device comprising:

[0028] A data acquisition module, used to acquire a video image data set of a target area, wherein the video data set includes at least one short video clip, and each short video clip includes multiple frames of continuous images;

[0029] A preprocessing module, used for preprocessing the video image data set;

[0030] The target detection module is used to perform two-dimensional target detection on the preprocessed video image data set by constructing a preset target pedestrian detection network to obtain pedestrian image data;

[0031] A pedestrian image feature extraction module is used to construct a backbone network of a preset pedestrian attribute recognition network and extract pedestrian image features from pedestrian image data;

[0032] A pedestrian attribute feature classification module is used to construct a dual-branch attention network of a preset pedestrian attribute recognition network, and classify pedestrian attribute features based on the results of pedestrian image feature extraction. The dual-branch attention network of the preset pedestrian attribute recognition network includes: a recognition network of preset fixed attribute features and a recognition network of preset dynamic attribute features.

[0033] The pedestrian attribute recognition module is used to complete the attribute recognition of the actor according to the preset loss function when the classification result meets the preset conditions.

[0034] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the pedestrian attribute recognition method based on surveillance video of the above-mentioned first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0035] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the method for pedestrian attribute recognition based on surveillance video of the above-mentioned first aspect or any corresponding embodiment thereof.

[0036] The pedestrian attribute recognition method and device based on surveillance video provided by the present invention divide pedestrian attributes into fixed attributes and dynamic attributes. For fixed attributes, such as the pedestrian's gender, age, tops, bottoms, whether wearing a hat, backpack and other features; for dynamic features of pedestrians, such as pedestrians' movements, postures, etc.; a dual-branch attention network with a preset pedestrian attribute recognition network is constructed, and multiple consecutive frames of images of the same pedestrian are used as model input. The spatial features of the pedestrian are extracted using a unified backbone network, and then the extracted spatial features are input into two branches. The recognition network with preset fixed attribute features is used to identify fixed attributes, and the recognition network with preset dynamic attribute features is used to identify dynamic attributes. A time attention module is added to both branches to determine the importance of different pedestrian attributes in different image frames, thereby giving different weights to different image frames; finally, the extracted time features and spatial features are used to classify and output different pedestrian attributes to obtain accurate pedestrian attribute recognition results. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0038] Figure 1 is a flow chart of a method for pedestrian attribute recognition based on surveillance video according to an embodiment of the present invention;

[0039] Figure 2 is a structural block diagram of a device for identifying pedestrian attributes based on surveillance video according to an embodiment of the present invention;

[0040] Figure 3 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0042] According to an embodiment of the present invention, an embodiment of a method for identifying pedestrian attributes based on surveillance video is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0043] In this embodiment, a method for identifying pedestrian attributes based on surveillance video is provided, which can be used in the above-mentioned mobile terminals, such as mobile phones, tablet computers, etc. Figure 1 is a flow chart of a method for pedestrian attribute recognition based on surveillance video according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0044] Step S101: acquiring a video image dataset of a target area, wherein the video dataset includes at least one short video segment, and each short video segment includes a plurality of continuous frames of images.

[0045] In an embodiment of the present invention, video data of surveillance videos of different areas are collected, such as different cities, different streets, different surveillance videos and image data in different time periods and different seasons, etc. This is only an example and is not limited to this. In actual applications, corresponding surveillance videos are selected according to actual conditions.

[0046] Specifically, the video data set includes at least one short video clip, and each short video clip includes multiple frames of continuous images.

[0047] Step S102: pre-processing the video image data set.

[0048] In an embodiment of the present invention, preprocessing of a video image data set includes: target pedestrian detection and labeling, pedestrian fixed attribute labeling, and pedestrian dynamic attribute labeling. Target pedestrian detection and labeling is used to label the target pedestrian in the image using a rectangular frame tool; pedestrian fixed attribute labeling is used to label the fixed attributes of the target pedestrian; pedestrian dynamic attribute labeling is used to label the dynamic attributes of the target pedestrian based on the labeling results of the fixed attributes. This is only an example and is not intended to be limiting. In actual applications, the corresponding preprocessing means are selected according to the actual situation.

[0049] Specifically, for target pedestrian detection and annotation, the pedestrian category target in the image is annotated using a rectangular box tool, and the rectangular box position P = (X, Y, W, H), where (X, Y) represents the coordinate position of the center point of the pedestrian target, and (W, H) represents the length and width of the pedestrian target rectangular box.

[0050] Specifically, the pedestrian attribute identification and labeling include fixed attributes of pedestrians and dynamic attributes of pedestrians. Among them, the fixed attributes of pedestrians include but are not limited to the pedestrian's gender (male, female), age (divided into different age groups such as children, teenagers, middle-aged, and the elderly), pedestrian's top (long, shorts), pedestrian's top color (red, blue, green, white, etc.), whether the pedestrian wears a hat (yes, no), whether the pedestrian carries a backpack (yes, no), etc. These are just examples and are not limited to this. In actual applications, the corresponding fixed attributes are selected according to the actual situation; for the dynamic features of pedestrians, such as the pedestrian's movements (walking, running, riding a bicycle, standing, etc.), the pedestrian's posture direction (forward, back, sideways, etc.), these are just examples and are not limited to this. In actual applications, the corresponding dynamic features are selected according to the actual situation.

[0051] Step S103, by constructing a preset target pedestrian detection network, two-dimensional target detection is performed on the preprocessed video image data set to obtain pedestrian image data.

[0052] In an embodiment of the present invention, the preset target pedestrian detection network includes: a feature extraction network and a target detection network, wherein, for the feature extraction network part, a commonly used backbone network is adopted, including but not limited to using convolutional neural networks such as ResNet, MobileNet, and VGG, the input is an image, and the output is a feature representation of the image, which is only an example and is not limited to this. In actual applications, the corresponding feature extraction network is selected according to the actual situation. For the target detection network part, it includes but is not limited to using commonly used multi-category detection methods based on YOLO, SSD, CenterNet, etc., and the output is the category and location information of the target.

[0053] The output is defined as:

[0054] O i =[c i ,x i ,y i ,w i ,h i ]

[0055] O i Denoted as the i-th target, c i Indicates the category of the extracted target, x i ,y i are the center point coordinates of the target rectangular box, w i ,h iare the length and width of the target respectively. This is only an example and is not limited to this. In practical applications, the corresponding target detection network is selected according to the actual situation.

[0056] Specifically, according to the target detection network results, the coordinate position of the pedestrian two-dimensional target detection box is obtained, the pedestrian image is extracted from the image, the width and height of the extracted image are calculated, the longer side is taken, the size is fixed to 256, and then the shorter side is reduced in the same proportion. Finally, the size of the shorter side is expanded to 128, and the insufficient part is filled with a pixel value of 0. Finally, the pedestrian image size is unified to 256*128. This is only an example and is not limited to this. In actual applications, the corresponding size is selected according to the actual situation.

[0057] Step S104, constructing a backbone network of a preset pedestrian attribute recognition network, and extracting pedestrian image features from pedestrian image data.

[0058] In the embodiment of the present invention, the backbone network of the preset pedestrian attribute recognition network is used to extract features of pedestrian images, and the input image size is T*256*128*3, where T represents T consecutive frames of images, T=16 can be set, 256*128 represents the height and width of the image respectively, and 3 represents the number of channels, representing an RGB three-channel image;

[0059] Specifically, the backbone network mainly adopts a convolutional combination method such as convolution layer-normalization layer-activation function layer, and performs superposition operations of multiple convolutional combination layers, including but not limited to using different backbone networks such as ResNet50 and MobileNetV3, and the output is T*(256 / R)*(128 / R)*C, where R represents the reduction ratio of the image during the feature extraction process, such as R=16, and C represents the number of channels of high-dimensional features obtained through the feature extraction network, such as C=512 or 1024. The image of the target pedestrian is obtained.

[0060] Step S105, construct a dual-branch attention network of a preset pedestrian attribute recognition network, and classify the pedestrian attribute features of the results of pedestrian image feature extraction, wherein the dual-branch attention network of the preset pedestrian attribute recognition network includes: a recognition network of preset fixed attribute features and a recognition network of preset dynamic attribute features.

[0061] Specifically, the identification of preset fixed attribute features includes: pedestrian's gender, age, top color, etc., which are examples only and are not limited thereto. In actual applications, corresponding fixed attribute features are selected according to actual conditions. The identification of preset dynamic attribute features includes: pedestrian's action, behavior, posture direction, etc., which are examples only and are not limited thereto. In actual applications, corresponding dynamic attribute features are selected according to actual conditions.

[0062] Specifically, for the recognition of fixed attribute features, compared with the recognition of dynamic attribute features, the fixed attribute recognition task is simpler, and the fixed attributes generally do not change in consecutive multiple frames of images, but may be blocked by objects such as vehicles, causing some fixed attributes to be invisible in some image frames. Therefore, for the recognition of fixed attribute features, a lightweight network structure is adopted as a whole, consisting of multiple convolution modules and a pooling module. Each convolution module consists of a convolution layer-normalization layer-activation function layer. In each feature module, the number of feature channels uses a smaller number of channels, such as 64 or 128, and the number of convolution modules can be set to 5. The pooling module can use average pooling operation or maximum pooling operation, and the feature vector after pooling operation is expressed as T*C1; in addition, since some fixed attributes may be blocked by objects such as vehicles, causing them to be invisible in some image frames, a connection time attention module is added after the pooling module of the fixed attribute feature recognition to guide the model to increase the attention weight to the image frames with more pedestrian attribute features in time.

[0063] In an embodiment of the present invention, the dual-branch attention network of the preset pedestrian attribute recognition network also includes: a temporal attention module, which is used to determine the weights of different pedestrian attributes in different image frames.

[0064] Specifically, for the temporal attention module, the input is the output of the pooling module, expressed as T*C1, where C1 is the number of feature channels after the output of the convolution module, and C1=128 can be set. Then, the global pooling operation is used to calculate the feature weight in the time dimension, and the output is T*1. The importance of each frame image in the continuous sequence is obtained, and then the weight coefficient is weighted to the input of the temporal attention by matrix multiplication to achieve re-weighted calculation of the importance of each frame image.

[0065] Specifically, for dynamic attribute feature recognition, the dynamic attribute recognition task is more difficult, and the dynamic attributes may keep changing in multiple consecutive frames of images. Therefore, for the recognition network with preset dynamic attribute features, a deeper network structure is adopted as a whole, which is composed of multiple convolution modules and a pooling module. Each convolution module is composed of a convolution layer-normalization layer-activation function layer. In each feature module, the number of feature channels adopts a larger number of channels, such as 512 or 1024, etc. The number of convolution modules can be set to 8, and the pooling module can use average pooling operation or maximum pooling operation. The feature vector after pooling operation is expressed as T*C2; in addition, since the dynamic attributes of some frames may be invisible due to occlusion by objects such as vehicles, a connection time attention module is added after the pooling module of the recognition network with preset dynamic attribute features to guide the model to increase the attention weight to image frames with more pedestrian attribute features in time, and adopt the same time attention module as the recognition network with preset fixed attribute features.

[0066] Step S106, according to the preset loss function, when the classification result meets the preset conditions, the attribute identification of the actor is completed.

[0067] In the embodiment of the present invention, the loss function includes two aspects, namely, the classification and detection box regression loss function of pedestrian target detection, and the classification loss function of pedestrian attribute recognition. When the two classification functions meet the preset conditions, the attribute recognition of the actor is completed. For example: when the values ​​of the two loss functions are less than the preset values, the attribute recognition of the actor is completed. The preset values ​​are not limited here, and are selected accordingly according to the actual situation. This is only an example and is not limited to this. In actual applications, the corresponding preset conditions are selected according to the actual situation.

[0068] L OD Represents the two-dimensional target detection classification loss function, including target classification and position regression.

[0069] L OD =β1L class +β2L bbox

[0070] L class Represents the classification loss function of the target, including but not limited to using loss functions such as cross entropy, L bbox is a regression loss function of the target detection box, including but not limited to using a smooth L1 loss function;

[0071] Specifically, for the pedestrian attribute classification loss function L CLS , using the standard softmax classification loss function.

[0072] The pedestrian attribute recognition method based on surveillance video provided in this embodiment divides pedestrian attributes into fixed attributes and dynamic attributes. For fixed attributes, such as the pedestrian's gender, age, tops, bottoms, whether wearing a hat, backpack, etc.; for dynamic features of pedestrians, such as pedestrians' movements, postures, etc.; a dual-branch attention network of a preset pedestrian attribute recognition network is constructed, and multiple consecutive frames of images of the same pedestrian are used as model input. The spatial features of the pedestrian are extracted using a unified backbone network, and then the extracted spatial features are input into two branches. The recognition network of the preset fixed attribute features is used to identify the fixed attributes, and the recognition network of the preset dynamic attribute features is used to identify the dynamic attributes. A time attention module is added to both branches to determine the importance of different pedestrian attributes in different image frames, thereby giving different weights to different image frames; finally, the extracted time features and spatial features are used to classify and output different pedestrian attributes to obtain accurate pedestrian attribute recognition results.

[0073] In this embodiment, a pedestrian attribute recognition device based on surveillance video is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0074] This embodiment provides a pedestrian attribute recognition device based on surveillance video, such as Figure 2 As shown, including:

[0075] A data acquisition module 201 is used to acquire a video image dataset of a target area, wherein the video dataset includes at least one short video segment, and each short video segment includes multiple frames of continuous images;

[0076] A preprocessing module 202, used for preprocessing the video image data set;

[0077] The target detection module 203 is used to perform two-dimensional target detection on the preprocessed video image data set by constructing a preset target pedestrian detection network to obtain pedestrian image data;

[0078] The pedestrian image feature extraction module 204 is used to construct a backbone network of a preset pedestrian attribute recognition network and extract pedestrian image features from pedestrian image data.

[0079] Pedestrian attribute feature classification 205 is used to construct a dual-branch attention network of a preset pedestrian attribute recognition network, and classify the pedestrian attribute features of the results of pedestrian image feature extraction, wherein the dual-branch attention network of the preset pedestrian attribute recognition network includes: a recognition network of preset fixed attribute features, a recognition network of preset dynamic attribute features

[0080] Pedestrian attribute recognition 206 is used to complete the attribute recognition of the actor according to a preset loss function when the classification result meets the preset conditions.

[0081] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0082] The pedestrian attribute recognition device based on surveillance video in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0083] The embodiment of the present invention also provides a computer device having the above Figure 3 The pedestrian attribute recognition device based on surveillance video is shown.

[0084] See also Figure 3 , Figure 3 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 3 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the computer device, including instructions stored in or on the memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to an interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 3 A processor 10 is taken as an example.

[0085] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0086] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0087] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage devices. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0088] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0089] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0090] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0091] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for pedestrian attribute recognition based on surveillance video, characterized in that: The method comprises: Acquire a video image dataset of a target area, wherein the video dataset includes at least one short video segment, and each short video segment includes a plurality of continuous frames of images; Preprocess the video image dataset; By building a preset target pedestrian detection network, two-dimensional target detection is performed on the preprocessed video image dataset to obtain pedestrian image data; Construct a backbone network of a preset pedestrian attribute recognition network to extract pedestrian image features from pedestrian image data; Constructing a dual-branch attention network of a preset pedestrian attribute recognition network, and classifying pedestrian attribute features respectively based on the results of pedestrian image feature extraction, wherein the dual-branch attention network of the preset pedestrian attribute recognition network includes: a recognition network of preset fixed attribute features and a recognition network of preset dynamic attribute features; According to the preset loss function, when the classification result meets the preset conditions, the actor's attribute identification is completed.

2. The method according to claim 1, characterized in that The preprocessing of the video image dataset includes: target pedestrian detection annotation, pedestrian fixed attribute annotation and pedestrian dynamic attribute annotation. Target pedestrian detection and annotation, used to use the rectangular box tool to mark the target pedestrians in the image; Pedestrian fixed attribute labeling, used to label the fixed attributes of the target pedestrian; Pedestrian dynamic attribute labeling is used to label the dynamic attributes of the target pedestrian based on the labeling results of fixed attributes.

3. The method according to claim 1, characterized in that The preset target pedestrian detection network includes: feature extraction network and target detection network, where: Feature extraction network, used to output feature representation of the image; The target detection network is used to output the category and location information of the target pedestrian and extract the target pedestrian image.

4. The method according to claim 3, characterized in that The output of the object detection network is expressed by the following formula: O i =[c i ,x i ,y i ,w i ,h i ] Among them, O i Denotes the i-th target pedestrian, c i Indicates the pedestrian category of the extracted target, x i ,y i are the center point coordinates of the target pedestrian rectangle, w i ,h i are the length and width of the target pedestrian respectively.

5. The method according to claim 1, characterized in that: The dual-branch attention network of the preset pedestrian attribute recognition network also includes: a temporal attention module, which is used to determine the weights of different pedestrian attributes in different image frames.

6. The method according to claim 1, characterized in that The preset loss functions include: classification and detection box regression loss functions for pedestrian target detection, and classification loss functions for pedestrian attribute recognition, where: The classification and detection box regression loss functions for pedestrian target detection are calculated using the following formula: L OD =β1L class +β2L bbox L class represents the classification loss function of pedestrian targets, L bbox represents the regression loss function of the pedestrian target detection box, β1 and β2 are both constants; The classification loss function for pedestrian attribute recognition adopts the softmax classification loss function.

7. A pedestrian attribute recognition device based on surveillance video, characterized in that: The device comprises: A data acquisition module, used to acquire a video image data set of a target area, wherein the video data set includes at least one short video clip, and each short video clip includes multiple frames of continuous images; A preprocessing module, used for preprocessing the video image data set; The target detection module is used to perform two-dimensional target detection on the preprocessed video image data set by constructing a preset target pedestrian detection network to obtain pedestrian image data; A pedestrian image feature extraction module is used to construct a backbone network of a preset pedestrian attribute recognition network and extract pedestrian image features from pedestrian image data; A pedestrian attribute feature classification module is used to construct a dual-branch attention network of a preset pedestrian attribute recognition network, and classify pedestrian attribute features based on the results of pedestrian image feature extraction. The dual-branch attention network of the preset pedestrian attribute recognition network includes: a recognition network of preset fixed attribute features and a recognition network of preset dynamic attribute features. The pedestrian attribute recognition module is used to complete the attribute recognition of the actor according to the preset loss function when the classification result meets the preset conditions.

8. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method for pedestrian attribute recognition based on surveillance video as described in any one of claims 1 to 6 by executing the computer instructions.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the pedestrian attribute recognition method based on surveillance video according to any one of claims 1 to 6.

Images