Pedestrian gait recognition model training method, recognition method and device

Abstract

The invention discloses a pedestrian gait recognition model training method, recognition method and device, and the method comprises the steps: obtaining the historical data of a pedestrian gait, marking the historical data, and determining the label information; performing feature extraction on the historical data, and determining a training data set; and training a pedestrian gait recognition model by using the training data set and the label information to generate a target pedestrian gait recognition model. According to the invention, based on a pedestrian attitude detection algorithm, key point coordinates describing pedestrian attitudes are obtained, then a pedestrian attitude conversion module is used to carry out projection transformation and scale alignment operations on the pedestrian attitudes to obtain unified attitude representation, and then a graph convolutional neural network is used to carry out feature learning to obtain a pedestrian attitude feature. The local and global relation of the pedestrian graph structure is established, and finally, the final pedestrian gait feature representation is obtained through pooling operation for recognition.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a pedestrian gait recognition model training method, a recognition method and a device. Background Art

[0002] As the urbanization process continues to accelerate, the number of motor vehicles continues to increase, and many problems such as urban traffic congestion and parking conflicts have also arisen. To this end, relying on artificial intelligence algorithms, cloud service platforms, smart hardware devices, and edge computing devices, an intelligent urban traffic management system has been realized, which is an intelligent management system that can collect, process, and feedback traffic information in real time and accurately.

[0003] At present, for the task of pedestrian re-identification, face recognition and pedestrian appearance features are mainly used for pedestrian re-identification. However, under the monitoring perspective, when the captured face image is blurry, the face is blocked, or the pedestrian changes his appearance features by changing clothes, the use of the above method will lead to inaccurate recognition results. Summary of the invention

[0004] In view of this, the embodiments of the present invention provide a pedestrian gait recognition model training method, a recognition method and a device, which solve the problem of large errors in recognition results in the prior art.

[0005] According to a first aspect, an embodiment of the present invention provides a method for training a pedestrian gait recognition model, comprising:

[0006] Acquire historical data of pedestrian gait, annotate the historical data, and determine label information;

[0007] Extracting features from the historical data to determine a training data set;

[0008] The training data set and the label information are used to train a pedestrian gait recognition model to generate a target pedestrian gait recognition model.

[0009] In combination with the first aspect, in a first implementation of the first aspect, the labeling of the historical data to determine the label information includes: the historical data is image data,

[0010] Performing pedestrian target detection on the image data, marking the pedestrian detection result using a two-dimensional detection frame, and determining a pedestrian target label;

[0011] Performing pedestrian gait detection on the image data, annotating the gait detection results using the annotation method of the COCO human posture dataset, and determining labels of each key point of the pedestrian gait;

[0012] The label information is determined according to the pedestrian target label and each of the key point labels.

[0013] In combination with the first implementation of the first aspect, in the second implementation of the first aspect, after performing pedestrian target detection on the image data, the method further includes:

[0014] Obtaining coordinate information of the two-dimensional detection frame of the pedestrian target;

[0015] Using the coordinate information, extracting the pedestrian target from the image data to determine the extracted image;

[0016] The subtracted images are resized using a preset size to determine a target subtracted image.

[0017] In combination with the second implementation manner of the first aspect, in the third implementation manner of the first aspect, after performing pedestrian gait detection on the image data, the method further includes:

[0018] Determining whether the target subtracted image meets a preset tilt requirement, and when the preset tilt requirement is not met, performing an affine transformation on each key point of the pedestrian's gait to determine a first transformed image;

[0019] When the preset tilt requirement is met, determining the target subtracted image as the first transformed image;

[0020] Performing pedestrian size transformation on the first transformation image according to a preset pedestrian height parameter to determine a second transformation image;

[0021] The coordinates of the key points of the pedestrian are obtained in the second transformed image, and the key point alignment coordinates are determined using the coordinates of the key points of the pedestrian and the preset center point coordinates.

[0022] In combination with the first aspect, in a fourth implementation of the first aspect, extracting features from the historical data to determine a training data set includes:

[0023] Constructing a pedestrian target detection network, wherein the pedestrian target detection network includes a feature extraction network and a target detection network;

[0024] Using the feature extraction network to extract features from the historical data, and outputting feature data of the historical data, including the length and width of the pedestrian target;

[0025] Using the target detection network to perform two-dimensional target detection, and outputting the category and position information of the pedestrian target;

[0026]

[0027] Among them, O iDenoted as the i-th target, c i Indicates the category of the extracted target, x i ,y i are the center point coordinates of the target rectangular box, w i ,h i are the length and width of the target respectively.

[0028] In combination with the fourth implementation manner of the first aspect, in the fifth implementation manner of the first aspect, the extracting features of the historical data to determine the training data set further includes:

[0029] Constructing a pedestrian posture recognition network, wherein the pedestrian posture recognition network includes a backbone network, a feature aggregation network, and a key point prediction network;

[0030] Using the backbone network to extract features from the historical data, and determining images of the same size;

[0031] Aggregating the images of the same size using the feature aggregation network to determine an aggregated feature map;

[0032] Using the key point prediction network to predict the aggregate feature map, determine the classification and position coordinates of each key point;

[0033] The category, location information, classification and location coordinates of each key point of the pedestrian target are determined as a training data set.

[0034] In combination with the fifth implementation of the first aspect, in the sixth implementation of the first aspect, the use of the training data set and the label information to train the pedestrian gait recognition model to generate the target pedestrian gait recognition model includes:

[0035] Build a pedestrian gait recognition model;

[0036] Determine revision parameters according to the training data set and the label information;

[0037] The recognition model is trained using the loss function and the revised parameters to generate a recognition model of the target pedestrian's gait.

[0038] The pedestrian gait recognition model training method provided by the embodiment of the present invention obtains the coordinates of key points describing the pedestrian posture based on the pedestrian posture detection algorithm, and then uses the pedestrian posture conversion module to perform projection transformation and scale alignment operations on the pedestrian posture to obtain a unified posture representation, and then uses a graph convolutional neural network to perform feature learning to achieve the establishment of local and global relationships between the pedestrian graph structure, and finally uses a pooling operation to obtain the final pedestrian gait feature representation for recognition.

[0039] According to a second aspect, an identification method provided by an embodiment of the present invention includes:

[0040] Determine a recognition model of the target pedestrian gait by using the pedestrian gait recognition model training method described in any embodiment of the first aspect of the present invention;

[0041] Acquire a monitoring scene image of an object to be detected;

[0042] The monitoring scene image is recognized using the recognition model of the target pedestrian's gait to determine a recognition result of the pedestrian's gait.

[0043] The recognition method provided by the embodiment of the present invention is based on the pedestrian posture detection algorithm to obtain the coordinates of the key points describing the pedestrian posture, and then uses the pedestrian posture conversion module to perform projection transformation and scale alignment operations on the pedestrian posture to obtain a unified posture representation, and then uses a graph convolutional neural network to perform feature learning to achieve the establishment of the local and global relationship of the pedestrian graph structure, and finally uses the pooling operation to obtain the final pedestrian gait feature representation for recognition.

[0044] According to a third aspect, an embodiment of the present invention provides a pedestrian gait recognition model training device, comprising:

[0045] An acquisition module, used to acquire historical data of pedestrian gait, annotate the historical data, and determine label information;

[0046] A data processing module, used to extract features from the historical data and determine a training data set;

[0047] The training module is used to train the pedestrian gait recognition model using the training data set and the label information to generate a target pedestrian gait recognition model.

[0048] According to a fourth aspect, an edge detection device provided by an embodiment of the present invention includes:

[0049] A first processing module, configured to determine a recognition model of a target pedestrian's gait using the pedestrian gait recognition model training device according to the third aspect of the present invention;

[0050] The second processing module is used to obtain a monitoring scene image of the object to be detected;

[0051] The third processing module is used to use the recognition model of the target pedestrian's gait to recognize the monitored scene image and determine the recognition result of the pedestrian's gait.

[0052] According to the fifth aspect, an embodiment of the present invention provides an electronic device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to execute the pedestrian gait recognition model training method described in the first aspect or any one of the embodiments of the first aspect, or execute the pedestrian gait recognition method described in the second aspect or any one of the embodiments of the second aspect.

[0053] According to the sixth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the pedestrian gait recognition model training method described in the first aspect or any one of the embodiments of the first aspect, or to execute the pedestrian gait recognition method described in the second aspect or any one of the embodiments of the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0055] Figure 1 is a flow chart of a pedestrian gait recognition model training method according to an embodiment of the present invention;

[0056] Figure 2 is a flow chart of a method for identifying pedestrian gait according to an embodiment of the present invention;

[0057] Figure 3 is a schematic diagram of a pedestrian gait recognition model training device according to an embodiment of the present invention;

[0058] Figure 4 is a schematic diagram of a pedestrian gait recognition device according to an embodiment of the present invention;

[0059] Figure 5 It is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0061] In recent years, with the continuous acceleration of urbanization, the number of motor vehicles has continued to increase, and many problems such as urban traffic congestion and parking conflicts have also arisen. To this end, relying on artificial intelligence algorithms, cloud service platforms, intelligent hardware devices, and edge computing devices, an intelligent urban traffic management system has been realized, which can collect, process, and feedback traffic information in real time and accurately.

[0062] At present, the analysis of the monitoring scene is mainly achieved by installing high-position video surveillance equipment on the side of urban roads, important traffic light intersections, etc., collecting video image data of the monitoring scene, and using visual algorithms to process and analyze the data. At present, the data collected by these monitoring devices can not only be used for traffic management, such as capturing evidence and issuing warnings for traffic violations such as running red lights and speeding, guiding roadside parking and recording parking spaces, and updating, predicting and publishing traffic congestion conditions in real time; in addition, more intelligent management of the entire city can be achieved, such as using pedestrian re-identification algorithms to assist in the capture of criminal suspects.

[0063] At present, for the task of pedestrian re-identification, face recognition and pedestrian appearance features are mainly used to re-identify pedestrians. However, under the monitoring perspective, when the facial image is blurred, the face is blocked, and the pedestrian changes his appearance by changing clothes, the above methods may be inaccurate. In the video surveillance scenario, in addition to using face recognition and pedestrian appearance feature judgment methods, using pedestrian gait features to identify pedestrians can improve the accuracy of the pedestrian re-identification task; gait recognition refers to judging the identity of pedestrians by extracting the characteristics of their walking style. The gait characteristics of pedestrians can be obtained by using surveillance cameras at a long distance, while facial features and iris features of the human eye are difficult to capture through surveillance cameras; and gait features are difficult to imitate or hide, while face recognition methods are easily interfered with by fake images or decorations such as wearing masks and hats.

[0064] In this embodiment, a pedestrian gait recognition model training method is provided, which can be used in electronic devices such as computers, mobile phones, tablet computers, etc. Figure 1 is a flow chart of a pedestrian gait recognition model training method according to an embodiment of the present invention. Figure 1As shown, the process includes the following steps:

[0065] S11, obtaining historical data of pedestrian gait, annotating the historical data, and determining label information.

[0066] Among them, video image data of surveillance videos in different areas are collected, such as different cities, different streets, different time periods and different surveillance scene videos and image data in different seasons; historical data are annotated, and the annotation method is pedestrian target detection annotation and pedestrian posture key point annotation.

[0067] S12, extracting features from historical data to determine a training data set. Using a pedestrian target detection network, a pedestrian posture recognition network, and a pedestrian posture key point transformation, extracting features from historical data to determine a training data set.

[0068] S13, using the training data set and label information, training the pedestrian gait recognition model to generate a target pedestrian gait recognition model.

[0069] In practical applications, the pedestrian gait recognition model constructed using existing technology can be an existing identifier or other deep learning network model, as long as it can achieve edge detection, and this embodiment is not limited to this. Then, the pedestrian gait recognition model is trained using the training data set to generate a target pedestrian gait recognition model to achieve accurate pedestrian gait recognition.

[0070] In another embodiment, a method for training a pedestrian gait recognition model is provided, comprising the following steps:

[0071] S21, obtaining historical data of pedestrian gait, annotating the historical data, and determining label information.

[0072] Specifically, the historical data is image data, and the above step S21 includes the following steps:

[0073] S211, perform pedestrian target detection on the image data, use a two-dimensional detection frame to mark the pedestrian detection results, and determine the pedestrian target label; for pedestrian target detection and marking, use a rectangular frame tool to mark the pedestrian category target in the image, and the rectangular frame position P = (X, Y, W, H), where (X, Y) represents the coordinate position of the center point of the pedestrian target, and (W, H) represents the length and width of the pedestrian target rectangular frame.

[0074] S212, perform pedestrian gait detection on the image data, annotate the gait detection results using the annotation method of the COCO human posture data set, and determine the labels of each key point of the pedestrian gait; use the data annotation method of the COCO human posture data set to use 17 key points to describe the human posture, and the 17 key points include: 0 nose tip, 1 left eye, 2 right eye, 3 left ear, 4 right ear, 5 left shoulder joint, 6 right shoulder joint, 7 left elbow joint, 8 right elbow joint, 9 left wrist joint, 10 right wrist joint, 11 left hip joint, 12 right hip joint, 13 left knee joint, 14 right knee joint, 15 left ankle joint, 16 right ankle joint, and the annotation information of each key point includes (v, x, y), where v represents the visibility of the key point, v=0 represents invisible, v=1 represents visible, and (x, y) represents the coordinates of the key point.

[0075] S213, determining label information according to the pedestrian target label and each key point label.

[0076] S22, extract features from historical data and determine a training data set.

[0077] In this embodiment, the above step S22 specifically further includes:

[0078] S221, constructing a pedestrian target detection network, the pedestrian target detection network includes a feature extraction network and a target detection network;

[0079] S222, extracting features from the historical data using a feature extraction network, and outputting feature data of the historical data, including the length and width of the pedestrian target;

[0080] S223, performing two-dimensional target detection using a target detection network, and outputting the category and position information of the pedestrian target;

[0081] In this embodiment, the pedestrian target detection network includes a feature extraction network part and a target detection network part. For the feature extraction part, a commonly used backbone network is used, including but not limited to convolutional neural networks such as ResNet, Mobile Net, and VGG. The input is an image and the output is a feature representation of the image. For the target detection network part, it includes but is not limited to using commonly used multi-category detection methods based on YOLO, SSD, CenterNet, etc., and the output is the category and location information of the target.

[0082] The output is defined as:

[0083]

[0084] Among them, O i Denoted as the i-th target, c i Indicates the category of the extracted target, x i ,yi are the center point coordinates of the target rectangular box, w i ,h i are the length and width of the target respectively.

[0085] Specifically, according to the target detection result, the coordinate position of the pedestrian two-dimensional target detection box is obtained, the pedestrian image is extracted from the image, the width and height of the extracted image are calculated, the longer side is taken, the size is fixed to 256, and then the shorter side is reduced in the same proportion, and finally the size of the shorter side is expanded to 128, and the insufficient part is filled with pixel value 0, and the final pedestrian image size is unified to 256*128; according to the extraction and size transformation of the above-mentioned pedestrian image, the coordinates of the 17 key points correspondingly marked to represent the pedestrian posture are also transformed, and finally the pedestrian key point coordinate information based on the extracted and resized pedestrian image is obtained.

[0086] S224, constructing a pedestrian posture recognition network, the pedestrian posture recognition network including a backbone network, a feature aggregation network, and a key point prediction network;

[0087] S225, extracting features from historical data using a backbone network to determine images of the same size;

[0088] S226, aggregating the images of the same size using a feature aggregation network to determine an aggregated feature map;

[0089] S227, using a key point prediction network to predict the aggregate feature map, and determine the classification and position coordinates of each key point;

[0090] Specifically, the pedestrian posture recognition network is used to identify the pedestrian's action posture, which is then used for subsequent pedestrian gait recognition and judgment; the pedestrian posture recognition network adopts a key point regression method, including a backbone network, a feature aggregation network, and a key point prediction network;

[0091] Specifically, the role of the backbone network is to extract features from pedestrian images. The input image size is 256*128*3, where 256*128 represents the height and width of the image respectively, and 3 represents the number of channels, representing an RGB three-channel image. The backbone network mainly adopts a convolutional combination method such as convolutional layer-normalization layer-activation function layer to perform superposition operations on multiple convolutional combination layers. Specifically, the role of the feature aggregation network is to aggregate the high and low-level features extracted by different layers in the backbone network to provide more feature representations for subsequent key point detection. Specifically, the key point prediction network, with the input being the feature map after the above-mentioned feature aggregation, realizes the classification and positioning of 17 types of key points.

[0092] In the monitoring scenario, due to the different viewing angles and heights of different cameras, the different distances of pedestrians observed by the cameras, and other environmental influences, the obtained pedestrian postures have differences in viewing angles, scales, etc. Therefore, the pedestrian posture key points obtained by the above process are first transformed, including affine transformation, pedestrian body scaling, and key point alignment operations.

[0093] Specifically, for the affine transformation, the pedestrian skeleton tilt problem caused by different camera perspectives is reduced, and the pedestrian posture sequence is defined as:

[0094]

[0095] Among them, T in 、V in , C in Represent the number of input image frames, key points, and coordinates respectively; define P neck is the middle position between the left shoulder and the right shoulder; define P hip represents the middle position between the left hip and the right hip; the pedestrian spine is the position from the neck to the hip, with the spine as the axis and the neck as the center, transforming the inclined skeleton perpendicular to the ground. The rotation angle θ is calculated as:

[0096]

[0097] Among them, c x and c y Represents the coordinates of the x-axis and y-axis;

[0098] The affine transformation is only applicable to sequences with severe tilt problems when the angle θ is larger than a threshold When applied, it is expressed as;

[0099]

[0100] in,

[0101]

[0102] Among them, P a is the output of the affine transformation, M a is the radiation transformation matrix, represents the coordinates of the i-th key point at the k-th position, where k∈{c x , c y}, after the above affine transformation operation, the inclined spine of the pedestrian will be perpendicular to the ground;

[0103] Specifically, in view of the scale change, considering that the scale of the pedestrian skeleton in each frame image is changing, it is necessary to scale the pedestrian body so as to more accurately judge the change of the pedestrian gait. The uniform height of the skeleton is determined by dividing the difference between the maximum and minimum key point values ​​along the vertical axis, which is specifically expressed as:

[0104]

[0105] Among them, h u is the parameter value that controls the height of pedestrians. Through the above-mentioned pedestrian height scale transformation, the height of the same pedestrian in different frames can be unified. The key point alignment operation is to convert the key point coordinates of the pedestrian from the camera coordinate system to the coordinates based on the pedestrian. The specific calculation method is:

[0106] j i =p′ i -p′ neck

[0107] It means that the coordinates of the key points of the pedestrian after the scale transformation in the above steps are subtracted from the coordinates of the center points of the shoulders to obtain the aligned coordinates; according to the above steps, the final coordinates of the key points of the pedestrian posture are obtained for subsequent gait recognition.

[0108] S228, the category, location information, classification and location coordinates of each key point of the pedestrian target are determined as a training data set.

[0109] S23, using the training data set and the label information, training the pedestrian gait recognition model to generate a target pedestrian gait recognition model.

[0110] In this embodiment, the above step S23 specifically further includes:

[0111] S231, constructing a pedestrian gait recognition model;

[0112] Specifically, a residual-based graph convolutional neural network is adopted, which is composed of multiple residual graph convolutional neural network modules, and finally connected to a pooling layer and a fully connected layer; specifically, for each residual graph convolutional neural network module, it is composed of multiple two-dimensional convolutional layers, nonlinear activation layers, graph convolutional layers, and residual connection layers;

[0113] For each residual module, the graph convolutional neural network layer can be expressed as:

[0114]

[0115] Among them, Z represents a two-layer graph convolutional neural network, and X represents the input multi-frame pedestrian posture key points; is the first layer of graph convolutional neural network, W(0) is the learnable parameter in the first layer of graph convolutional neural network, ReLU() represents the nonlinear activation function; W(1) is the learnable parameter in the second layer of graph convolutional neural network, softmax() represents the softmax normalization function;

[0116] The adjacency matrix is ​​represented as:

[0117]

[0118] Specifically, Among them, A represents the relationship matrix of the pedestrian posture key points between two adjacent frames calculated above. In order to increase the relationship between each pedestrian posture key point and itself, a unit matrix is ​​added. Represented as a new adjacency matrix; The degree matrix is ​​represented by the sum of the relationship between each pedestrian posture key point and all other pedestrian posture key points;

[0119] The non-linear activation layer includes but is not limited to using non-linear activation functions such as ReLu; the output is Z, which represents the pedestrian feature representation and can be expressed as:

[0120] Z∈R T×V×C , where C represents the number of channels, T represents the number of frames in the video sequence, and V represents the number of key points; the feature mapping of the output feature in the spatial dimension can be expressed as:

[0121] f vp =ap(Z)+mp(Z)

[0122] Among them, ap represents the average pooling operation, mp represents the maximum pooling operation, and the output f vp ∈R T×1×C , to achieve feature aggregation between different key points in the spatial dimension; then use the pooling operation to achieve feature aggregation in the time dimension, expressed as:

[0123] f tp = sp(f vp )

[0124] Among them, sp represents the maximum pooling operation in the time dimension, and the output f tp ∈R 1×1×C , used for recognition of pedestrian gait characteristics.

[0125] S232, determining revision parameters according to the training data set and label information;

[0126] S233, using the loss function and the revised parameters to train the recognition model, and generate a recognition model of the target pedestrian's gait.

[0127] The loss function includes three aspects: the classification and detection box regression loss function of pedestrian target detection, the pedestrian posture key point loss function, and the loss function of the graph convolutional neural network model;

[0128] L OD Represents the two-dimensional target detection classification loss function, including target classification and position regression.

[0129] L OD =β1L class +β2L bbox

[0130] L class Represents the classification loss function of the target, including but not limited to using loss functions such as cross entropy, L bbox is the regression loss function of the target detection box, including but not limited to the smooth L1 loss function; for the pedestrian posture key point regression loss function L Keypoint , are respectively the regression loss function of the pedestrian center point, and the regression loss function of the offset between the pedestrian center point and the pedestrian posture key point (or the pedestrian 3D envelope key point);

[0131] Specifically, the loss function is:

[0132] L Keypoint =γ1L reg +γ2L offset

[0133] Among them, L reg , L offset They represent the regression loss function of the pedestrian center point, the regression loss function of the offset between the pedestrian center point and the pedestrian posture key point (or the pedestrian 3D envelope key point), γ1 and γ2 are weight coefficients, which are set to 1;

[0134] Specifically, the loss functions of the two parts both use the L1 Loss mean absolute error loss function, which is specifically expressed as:

[0135]

[0136] in, y represents the two vectors that need to be calculated, the ratio is the estimated value and the target value of the model, m represents the total amount of data, and the ultimate goal of the loss function is to minimize the absolute difference between the total estimated value and the target value; for pedestrian posture classification loss function L CLS , using the standard softmax classification loss function;

[0137] Specifically, for the graph convolutional neural network loss function for pedestrian gait recognition, the L2 norm loss function is used for model training, also known as the lowest square error loss function;

[0138]

[0139] Specifically, They respectively represent the pedestrian gait characteristics at a certain moment predicted by the model and the actual pedestrian gait characteristics at a certain moment, and m means there are m samples in total.

[0140] A pedestrian gait recognition model training method provided in this embodiment obtains the coordinates of key points describing the pedestrian's posture based on the pedestrian posture detection algorithm, and then uses the pedestrian posture conversion module to perform projection transformation and scale alignment operations on the pedestrian posture to obtain a unified posture representation, and then uses a graph convolutional neural network to perform feature learning to achieve the establishment of local and global relationships between the pedestrian graph structure, and finally uses a pooling operation to obtain the final pedestrian gait feature representation for recognition.

[0141] In this embodiment, a method for identifying pedestrian gait is also provided. Figure 2 The steps shown include:

[0142] Step S31: using the pedestrian gait recognition model training method in the above embodiment, determine the target pedestrian gait recognition model; see steps S11-S13 for details, which will not be repeated in this embodiment.

[0143] Step S32: Acquire the monitoring scene image of the object to be detected. In this embodiment, the monitoring scene image of the object to be detected is acquired by using the existing monitoring equipment scene.

[0144] Step S33: using the recognition model of the target pedestrian's gait to recognize the monitored scene image, and determining the recognition result of the pedestrian's gait.

[0145] In this embodiment, a pedestrian gait recognition model training device is also provided. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0146] The present invention discloses a pedestrian gait recognition model training device, such as Figure 3 As shown, including:

[0147] The acquisition module 1 is used to obtain the historical data of pedestrian gait, mark the historical data, and determine the label information; the detailed operation process is detailed in the corresponding steps in the above method embodiment.

[0148] The data processing module 2 is used to extract features from historical data and determine a training data set; the detailed operation process is detailed in the corresponding steps in the above method embodiment.

[0149] The training module 3 is used to train the pedestrian gait recognition model using the training data set and label information to generate a target pedestrian gait recognition model; the detailed operation process is detailed in the corresponding steps in the above method embodiment.

[0150] The present invention discloses an edge detection device, such as Figure 4 As shown, including:

[0151] The first processing module 01 is used to determine the recognition model of the target pedestrian gait by using the pedestrian gait recognition model training device as described above; the detailed operation process is detailed in the corresponding steps in the above method embodiment.

[0152] The second processing module 02 is used to obtain a monitoring scene image of the object to be detected; the detailed operation process is detailed in the corresponding steps in the above method embodiment.

[0153] The third processing module 03 is used to use the recognition model of the target pedestrian's gait to recognize the monitored scene image and determine the recognition result of the pedestrian's gait; the detailed operation process is detailed in the corresponding steps in the above method embodiment.

[0154] The present invention also provides an electronic device. Figure 5 , Figure 5 is a schematic diagram of the structure of an electronic device provided by an optional embodiment of the present invention, such as Figure 5 As shown, the electronic device may include: at least one processor 601, such as a CPU (Central Processing Unit), at least one communication interface 603, a memory 604, and at least one communication bus 602. Among them, the communication bus 602 is used to realize the connection and communication between these components. Among them, the communication interface 603 may include a display screen (Display), a keyboard (Keyboard), and the optional communication interface 603 may also include a standard wired interface and a wireless interface. The memory 604 may be a high-speed RAM memory (Random Access Memory, volatile random access memory) or a non-volatile memory (non-volatile memory), such as at least one disk storage. The memory 604 may also be at least one storage device located away from the aforementioned processor 601. The memory 604 stores the application program, and the processor 601 calls the program code stored in the memory 604 to perform any of the above method steps.

[0155] The communication bus 602 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The communication bus 602 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0156] Among them, the memory 604 may include a volatile memory (English: volatile memory), such as a random access memory (English: random-access memory, abbreviated: RAM); the memory may also include a non-volatile memory (English: non-volatile memory), such as a flash memory (English: flash memory), a hard disk drive (English: hard disk drive, abbreviated: HDD) or a solid-state drive (English: solid-state drive, abbreviated: SSD); the memory 604 may also include a combination of the above types of memory.

[0157] The processor 601 may be a central processing unit (CPU), a network processor (NP), or a combination of a CPU and a NP.

[0158] The processor 601 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0159] Optionally, the memory 604 is also used to store program instructions. The processor 601 can call the program instructions to implement the method shown in the embodiment of the present application.

[0160] The embodiment of the present invention further provides a non-transitory computer storage medium, which stores computer executable instructions, and the computer executable instructions can execute the method in any of the above method embodiments. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory (Flash Memory), a hard disk (HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above types of memory.

[0161] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A pedestrian gait recognition model training method, characterized in that: include: Acquire historical data of pedestrian gait, annotate the historical data, and determine label information; Extracting features from the historical data to determine a training data set; The training data set and the label information are used to train a pedestrian gait recognition model to generate a target pedestrian gait recognition model.

2. The method according to claim 1, characterized in that The step of labeling the historical data and determining label information includes: the historical data is image data, Performing pedestrian target detection on the image data, marking the pedestrian detection result using a two-dimensional detection frame, and determining a pedestrian target label; Performing pedestrian gait detection on the image data, annotating the gait detection results using the annotation method of the COCO human posture dataset, and determining labels of each key point of the pedestrian gait; The label information is determined according to the pedestrian target label and each of the key point labels.

3. The method according to claim 2, characterized in that After performing pedestrian target detection on the image data, the method further includes: Obtaining coordinate information of the two-dimensional detection frame of the pedestrian target; Using the coordinate information, extracting the pedestrian target from the image data to determine the extracted image; The subtracted images are resized using a preset size to determine a target subtracted image.

4. The method according to claim 3, characterized in that After performing pedestrian gait detection on the image data, the method further includes: Determining whether the target subtracted image meets a preset tilt requirement, and when the preset tilt requirement is not met, performing an affine transformation on each key point of the pedestrian's gait to determine a first transformed image; When the preset tilt requirement is met, determining the target subtracted image as the first transformed image; Performing pedestrian size transformation on the first transformation image according to a preset pedestrian height parameter to determine a second transformation image; The coordinates of the key points of the pedestrian are obtained in the second transformed image, and the key point alignment coordinates are determined using the coordinates of the key points of the pedestrian and the preset center point coordinates.

5. The method according to claim 1, characterized in that The extracting features of the historical data to determine the training data set includes: Constructing a pedestrian target detection network, wherein the pedestrian target detection network includes a feature extraction network and a target detection network; Using the feature extraction network to extract features from the historical data, and outputting feature data of the historical data, including the length and width of the pedestrian target; Using the target detection network to perform two-dimensional target detection, and outputting the category and position information of the pedestrian target; Among them, O i Denoted as the i-th target, c i Indicates the category of the extracted target, x i ,y i are the center point coordinates of the target rectangular box, w i ,h i are the length and width of the target respectively.

6. The method according to claim 5, characterized in that The extracting features of the historical data to determine the training data set also includes: Constructing a pedestrian posture recognition network, wherein the pedestrian posture recognition network includes a backbone network, a feature aggregation network, and a key point prediction network; Using the backbone network to extract features from the historical data, and determining images of the same size; Aggregating the images of the same size using the feature aggregation network to determine an aggregated feature map; Using the key point prediction network to predict the aggregate feature map, determine the classification and position coordinates of each key point; The category, location information, classification and location coordinates of each key point of the pedestrian target are determined as a training data set.

7. The method according to claim 6, characterized in that The method of training a pedestrian gait recognition model by using the training data set and the label information to generate a target pedestrian gait recognition model includes: Build a pedestrian gait recognition model; Determine revision parameters according to the training data set and the label information; The recognition model is trained using the loss function and the revised parameters to generate a recognition model of the target pedestrian's gait.

8. A method for identifying pedestrian gait, characterized in that: include: Determine a recognition model of a target pedestrian gait by using the pedestrian gait recognition model training method as described in any one of claims 1 to 7; Acquire a monitoring scene image of an object to be detected; The monitoring scene image is recognized using the recognition model of the target pedestrian's gait to determine a recognition result of the pedestrian's gait.

9. A pedestrian gait recognition model training device, characterized in that: include: An acquisition module, used to acquire historical data of pedestrian gait, annotate the historical data, and determine label information; A data processing module, used to extract features from the historical data and determine a training data set; The training module is used to train the pedestrian gait recognition model using the training data set and the label information to generate a target pedestrian gait recognition model.

10. A pedestrian gait recognition device, characterized in that: include: A first processing module, configured to determine a recognition model of a target pedestrian's gait using the pedestrian gait recognition model training device as claimed in claim 9; The second processing module is used to obtain a monitoring scene image of the object to be detected; The third processing module is used to use the recognition model of the target pedestrian's gait to recognize the monitored scene image and determine the recognition result of the pedestrian's gait.

Images