A pedestrian gait recognition model training method, a recognition method and a device

Abstract

The application discloses a pedestrian gait recognition model training method, a recognition method and a device. The pedestrian gait recognition model training method comprises the following steps: obtaining historical data of pedestrian gait, labeling the historical data, and determining label information; performing feature extraction on the historical data to determine a training data set; training a pedestrian gait recognition model by using the training data set and the label information, and generating a target pedestrian gait recognition model. Based on a pedestrian posture detection algorithm, key point coordinates describing the pedestrian posture are obtained, then a pedestrian posture conversion module is used to perform projection transformation and scale alignment operation on the pedestrian posture, a unified posture representation is obtained, then a graph convolutional neural network is used for feature learning, the local and global relationships of a pedestrian graph structure are established, and finally, a pooling operation is used to obtain a final pedestrian gait feature representation for recognition.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to a pedestrian gait recognition model training method and device, and a pedestrian gait recognition method and device. BACKGROUND

[0002] With the continuous acceleration of urbanization and the continuous increase in the number of motor vehicles, many urban traffic congestion and parking conflicts have also arisen. Therefore, relying on artificial intelligence algorithms, cloud service platforms, intelligent hardware devices, and edge computing devices, an intelligent urban traffic management system is implemented, which can collect, process, and feedback traffic information in real time and accurately.

[0003] Currently, for pedestrian re-identification tasks, facial recognition and pedestrian appearance features are mainly used for pedestrian re-identification. However, in the case of a blurred face image, a face being blocked, and a pedestrian changing appearance features by changing clothes under a monitoring angle, the above methods will result in inaccurate recognition results. SUMMARY

[0004] Therefore, the embodiments of the present application provide a pedestrian gait recognition model training method and device, and a pedestrian gait recognition method and device, which solve the problem of large recognition result errors in the prior art.

[0005] According to a first aspect, the embodiments of the present application provide a pedestrian gait recognition model training method, which comprises:

[0006] Obtaining historical data of pedestrian gait, labeling the historical data, and determining label information;

[0007] Performing feature extraction on the historical data to determine a training data set;

[0008] Training a pedestrian gait recognition model using the training data set and the label information to generate a target pedestrian gait recognition model.

[0009] In a first implementation manner of the first aspect, the labeling of the historical data to determine label information comprises: the historical data is image data,

[0010] Performing pedestrian target detection on the image data, labeling pedestrian detection results using a two-dimensional detection frame, and determining pedestrian target labels;

[0011] Performing pedestrian gait detection on the image data, labeling gait detection results using a labeling manner of a COCO human pose data set, and determining labels of each key point of the pedestrian gait;

[0012] According to the pedestrian target label and each key point label, the label information is determined.

[0013] In combination with the first aspect and the first implementation, in a second implementation of the first aspect, after the pedestrian target detection on the image data, the method further includes:

[0014] Obtaining coordinate information of the two-dimensional detection frame of the pedestrian target;

[0015] Cutting out the pedestrian target from the image data by using the coordinate information to determine a cut-out image;

[0016] Uniformly processing the size of the cut-out image by using a preset size to determine a target cut-out image.

[0017] In combination with the second implementation of the first aspect, in a third implementation of the first aspect, after the pedestrian gait detection on the image data, the method further includes:

[0018] Determining whether the target cut-out image meets a preset inclination requirement, and when the target cut-out image does not meet the preset inclination requirement, performing affine transformation on each key point of the pedestrian gait to determine a first transformed image;

[0019] When the target cut-out image meets the preset inclination requirement, the target cut-out image is determined as the first transformed image;

[0020] Performing pedestrian size transformation on the first transformed image according to a preset pedestrian height parameter to determine a second transformed image;

[0021] Obtaining pedestrian key point coordinates in the second transformed image, and determining key point alignment coordinates by using the pedestrian key point coordinates and a preset center point coordinate.

[0022] In combination with the first aspect, in a fourth implementation of the first aspect, the feature extraction on the historical data to determine a training data set includes:

[0023] Constructing a pedestrian target detection network, the pedestrian target detection network including a feature extraction network and a target detection network;

[0024] Performing feature extraction on the historical data by using the feature extraction network to output feature data of the historical data, the feature data including a length and a width of the pedestrian target;

[0025] Performing two-dimensional target detection by using the target detection network to output category and position information of the pedestrian target;

[0026]

[0027] wherein, O irepresents the i-th target, c i represents the category of the extraction target, x i , y i are respectively the center point position coordinates of the target rectangular frame, w i , h i are respectively the length and width of the target.

[0028] In combination with the fourth implementation manner of the first aspect, in a fifth implementation manner of the first aspect, the feature extraction on the historical data and the determination of the training data set further include:

[0029] A pedestrian posture recognition network is constructed, and the pedestrian posture recognition network includes a backbone network, a feature aggregation network and a key point prediction network.

[0030] The historical data are feature-extracted by using the backbone network to determine same-size images.

[0031] Each same-size image is aggregated by using the feature aggregation network to determine an aggregated feature map.

[0032] The aggregated feature map is predicted by using the key point prediction network to determine the classification and position coordinates of each key point.

[0033] The category, position information and classification and position coordinates of each key point of the pedestrian target are determined as the training data set.

[0034] In combination with the fifth implementation manner of the first aspect, in a sixth implementation manner of the first aspect, the training of the recognition model of the pedestrian gait by using the training data set and the label information to generate the recognition model of the target pedestrian gait includes:

[0035] A recognition model of the pedestrian gait is constructed.

[0036] Revision parameters are determined according to the training data set and the label information.

[0037] The recognition model is trained by using a loss function and the revision parameters to generate the recognition model of the target pedestrian gait.

[0038] The pedestrian gait recognition model training method provided by the embodiment of the application is based on a pedestrian posture detection algorithm to obtain key point coordinates describing a pedestrian posture, then a pedestrian posture conversion module is used to perform projection transformation and scale alignment operation on the pedestrian posture to obtain a unified posture representation, then a graph convolutional neural network is used for feature learning to realize the establishment of local and global relationships of a pedestrian graph structure, and finally a pooling operation is used to obtain a final pedestrian gait feature representation for recognition.

[0039] According to a second aspect, embodiments of the present application provide a recognition method, comprising:

[0040] A recognition model of a target pedestrian gait is determined by using the pedestrian gait recognition model training method in any of the embodiments of the first aspect of the present application.

[0041] A monitoring scene image of a to-be-detected object is acquired.

[0042] The monitoring scene image is recognized by using the recognition model of the target pedestrian gait, and a recognition result of the pedestrian gait is determined.

[0043] The recognition method provided by the embodiments of the present application is based on a pedestrian posture detection algorithm, key point coordinates describing a pedestrian posture are obtained, then a pedestrian posture is projected and transformed by using a pedestrian posture conversion module, a scale alignment operation is performed, a unified posture representation is obtained, then a graph convolutional neural network is used for feature learning, a relationship between a local and a global of a pedestrian graph structure is established, and finally a final pedestrian gait feature representation is obtained by using a pooling operation to perform recognition.

[0044] According to a third aspect, embodiments of the present application provide a pedestrian gait recognition model training device, comprising:

[0045] An acquisition module is configured to acquire historical data of a pedestrian gait, label the historical data, and determine label information.

[0046] A data processing module is configured to perform feature extraction on the historical data, and determine a training data set.

[0047] A training module is configured to train a recognition model of a pedestrian gait by using the training data set and the label information, and generate a recognition model of a target pedestrian gait.

[0048] According to a fourth aspect, embodiments of the present application provide an edge detection device, comprising:

[0049] A first processing module is configured to determine a recognition model of a target pedestrian gait by using the pedestrian gait recognition model training device according to the third aspect of the present application.

[0050] A second processing module is configured to acquire a monitoring scene image of a to-be-detected object.

[0051] A third processing module is configured to recognize the monitoring scene image by using the recognition model of the target pedestrian gait, and determine a recognition result of the pedestrian gait.

[0052] According to a fifth aspect, an electronic device is provided and includes a memory and a processor in communication with each other, the memory having stored therein computer instructions that, based on execution by the processor, perform the pedestrian gait recognition model training method of the first aspect or any of the embodiments of the first aspect, or perform the pedestrian gait recognition method of the second aspect or any of the embodiments of the second aspect.

[0053] According to a sixth aspect, a computer readable storage medium is provided and stores computer instructions for causing a computer to perform the pedestrian gait recognition model training method of the first aspect or any of the embodiments of the first aspect, or perform the pedestrian gait recognition method of the second aspect or any of the embodiments of the second aspect. BRIEF DESCRIPTION OF DRAWINGS

[0054] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the drawings needed in the specific embodiments or prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0055] Figure 1 is a flowchart of the pedestrian gait recognition model training method according to an embodiment of the present application;

[0056] Figure 2 is a flowchart of the pedestrian gait recognition method according to an embodiment of the present application;

[0057] Figure 3 is a schematic diagram of the pedestrian gait recognition model training device according to an embodiment of the present application;

[0058] Figure 4 is a schematic diagram of the pedestrian gait recognition device according to an embodiment of the present application;

[0059] Figure 5 is a hardware structure schematic diagram of the electronic device provided by the embodiment of the present application. DETAILED DESCRIPTION

[0060] To make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some, but not all, of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0061] In recent years, with the continuous acceleration of urbanization and the continuous increase in the number of motor vehicles, many urban traffic congestion and parking conflicts have also arisen. Therefore, relying on artificial intelligence algorithms, cloud service platforms, intelligent hardware devices, and edge computing devices, an intelligent urban traffic management system is realized, which can collect, process, and feedback traffic information in real time and accurately.

[0062] Currently, high-position video monitoring devices are mainly installed at urban road sides and important red and green light intersections to collect monitoring scene video image data, and visual algorithms are used to process and analyze the data to realize analysis of the monitoring scene. Currently, the data collected based on these monitoring devices can be used not only for traffic management, such as evidence capture and warning for traffic violations such as red light running and speeding, guidance and parking space recording for roadside parking, real-time updating, prediction, and publishing of traffic congestion conditions, but also for more intelligent management of the entire city, such as assisting in the capture of criminal suspects using pedestrian re-identification algorithms.

[0063] Currently, for pedestrian re-identification tasks, facial recognition and pedestrian appearance features are mainly used for pedestrian re-identification. However, in a monitoring perspective, when the photographed facial image is blurred, the face is obscured, and the pedestrian changes appearance features by changing clothes, the above methods may not be accurate. In addition to the method of using facial recognition and pedestrian appearance features for judgment in a video monitoring scene, pedestrian gait features can be used for pedestrian recognition to improve the accuracy of the pedestrian re-identification task. Gait recognition refers to the judgment of pedestrians by extracting their walking manner features. Pedestrian gait features can be captured by a monitoring camera at a long distance, while facial features and iris features of the eyes are difficult to capture by a monitoring camera. Moreover, gait features are difficult to imitate or hide, while facial recognition is easily disturbed by fake images or wearing masks and hats.

[0064] In the present embodiment, a pedestrian gait recognition model training method is provided, which can be used in electronic devices such as computers, mobile phones, and tablet computers. Figure 1 The flowchart of the pedestrian gait recognition model training method according to the present embodiment is shown in FIG. 1. Figure 1As shown, the flow includes the following steps:

[0065] S11, historical data of pedestrian gait is acquired, the historical data is labeled, and label information is determined.

[0066] The video image data of the monitoring video of different regions is collected, such as different cities, different streets, different time periods, different monitoring scene videos and image data in different seasons, and the like; the historical data is labeled, and the labeling manner is pedestrian target detection labeling and pedestrian posture key point labeling.

[0067] S12, feature extraction is performed on the historical data to determine a training data set. The historical data is extracted by using a pedestrian target detection network, a pedestrian posture recognition network, and a pedestrian posture key point transformation to determine a training data set.

[0068] S13, the training data set and the label information are used to train a pedestrian gait recognition model to generate a target pedestrian gait recognition model.

[0069] In actual application, the existing technology can be used to construct the pedestrian gait recognition model, which can be an existing recognizer or other deep learning network model, as long as edge detection can be realized, and the embodiment is not limited thereto. Then, the training data set is used to train the pedestrian gait recognition model to generate a target pedestrian gait recognition model to realize accurate pedestrian gait recognition.

[0070] In another embodiment, a pedestrian gait recognition model training method is also provided, including the following steps:

[0071] S21, historical data of pedestrian gait is acquired, the historical data is labeled, and label information is determined.

[0072] Specifically, the historical data is image data, and the above step S21 includes the following steps:

[0073] S211, pedestrian target detection is performed on the image data, a two-dimensional detection frame is used to label the pedestrian detection result, and pedestrian target labels are determined; for pedestrian target detection labeling, a rectangular frame tool is used to label the pedestrian target in the image, and the rectangular frame position P=(X, Y, W, H), wherein (X, Y) represents the coordinate position of the center point of the pedestrian target, and (W, H) represents the length and width of the pedestrian target rectangular frame.

[0074] S212, pedestrian gait detection is performed on the image data, the gait detection result is labeled by using a labeling manner of a COCO human pose dataset, and labels of respective key points of the pedestrian gait are determined; the data labeling manner of the COCO human pose dataset is adopted, 17 key points are used to describe the human pose, and the 17 key points include: 0 nose tip, 1 left eye, 2 right eye, 3 left ear, 4 right ear, 5 left shoulder joint, 6 right shoulder joint, 7 left elbow joint, 8 right elbow joint, 9 left wrist joint, 10 right wrist joint, 11 left hip joint, 12 right hip joint, 13 left knee joint, 14 right knee joint, 15 left ankle joint, and 16 right ankle joint; labeling information of each key point includes (v, x, y), wherein v represents visibility of the key point, v = 0 represents invisibility, and v = 1 represents visibility, and (x, y) represents coordinates of the key point.

[0075] S213, label information is determined according to the pedestrian target label and the respective key point label.

[0076] S22, feature extraction is performed on historical data, and a training data set is determined.

[0077] In the embodiment, the step S22 specifically further includes:

[0078] S221, a pedestrian target detection network is constructed, and the pedestrian target detection network includes a feature extraction network and a target detection network;

[0079] S222, feature extraction is performed on the historical data by using the feature extraction network, and feature data of the historical data is output, including length and width of the pedestrian target;

[0080] S223, two-dimensional target detection is performed by using the target detection network, and class information and position information of the pedestrian target are output;

[0081] In the embodiment, the pedestrian target detection network includes a feature extraction network part and a target detection network part, for the feature extraction part, a commonly used backbone network is adopted, including but not limited to a ResNet, a Mobile Net, a VGG, and the like, an input is an image, and an output is a feature representation of the image, for the target detection network part, a commonly used multi-class detection method based on YOLO, SSD, CenterNet, and the like is adopted, and an output is class information and position information of a target;

[0082] The output is defined as:

[0083]

[0084] O i represents the i th target, c i represents the class of the extracted target, x i , and yi respectively the coordinate position of the center point of the target rectangular frame, w i , h i respectively the length and width of the target.

[0085] Specifically, according to the target detection result, the coordinate position of the pedestrian two-dimensional target detection frame is obtained, the pedestrian image is cut out from the image, the width and height of the cut-out image are calculated, the longer side is taken, the size is fixed to 256, then the shorter side is reduced in the same proportion, finally the shorter side size is expanded to 128, and the insufficient part is filled with pixel value 0. The final pedestrian image size is unified to 256*128; according to the above cutting and size transformation of the pedestrian image, the corresponding labeled 17 key point coordinates representing the pedestrian posture are also transformed, and finally the pedestrian key point coordinate information based on the cut-out and re-transformed size of the pedestrian image is obtained.

[0086] S224, a pedestrian posture recognition network is constructed, and the pedestrian posture recognition network includes a backbone network, a feature aggregation network, and a key point prediction network;

[0087] S225, the historical data is feature-extracted by using the backbone network to determine the same size image;

[0088] S226, each same size image is aggregated by using the feature aggregation network to determine the aggregated feature map;

[0089] S227, the aggregated feature map is predicted by using the key point prediction network to determine the classification and position coordinates of each key point;

[0090] Specifically, the pedestrian posture recognition network is used to identify the action posture of the pedestrian, and then is used for subsequent pedestrian gait recognition judgment; the posture recognition network of the pedestrian adopts a key point regression manner, and includes a backbone network, a feature aggregation network, and a key point prediction network.

[0091] Specifically, the backbone network is used for feature extraction of the pedestrian image, and the input image size is 256*128*3, wherein 256*128 respectively represents the height and width of the image, and 3 represents the number of channels, indicating an RGB three-channel image; the backbone network mainly adopts a convolution combination manner of convolution layer-normalization layer-activation function layer, and performs stacking operation of multiple convolution combination layers; specifically, the feature aggregation network is used for aggregating high and low layer features extracted in the backbone network, and provides more feature representation for subsequent key point detection; specifically, the key point prediction network inputs the feature map aggregated above, and realizes classification and position positioning of 17 types of key points.

[0092] In the monitoring scene, due to the different angles and heights of different cameras, the different distances of pedestrians observed by the cameras, and other existing environmental influences, the acquired pedestrian poses have differences in view angle, scale size, etc. Therefore, the pedestrian pose key points obtained by the above process are first transformed, including affine transformation, pedestrian body scaling, and key point alignment operations.

[0093] Specifically, for affine transformation, the problem of pedestrian skeleton tilt caused by different camera angles is solved. The pedestrian pose sequence is defined as:

[0094]

[0095] where T in , V in , C in represent the number of input image frames, key points, and coordinates, respectively; P neck is defined as the middle position of the left shoulder and the right shoulder; P hip represents the middle position of the left hip and the right hip; the pedestrian spine is the position from the neck to the hip, and the tilt skeleton perpendicular to the ground is transformed with the neck as the center and the spine as the axis. The rotation angle θ is calculated as:

[0096]

[0097] where c x and c y represent the x-axis and y-axis coordinates;

[0098] Affine transformation is only applicable to sequences with severe tilt problems. When the angle θ is greater than the threshold , it is applied, which is represented as:

[0099]

[0100] where,

[0101]

[0102] where P a is the output of the affine transformation, M a is the radiation transformation matrix, represents the coordinates of the i-th key point at the k-th position, where k∈{c x , c y}, and after the above affine transformation operation, the tilted pedestrian spine is perpendicular to the ground.

[0103] Specifically, for scale change, considering that the scale of the pedestrian skeleton in each frame of image is transformed, the pedestrian body needs to be scaled, so as to more accurately judge the change of the pedestrian gait, and the uniform height of the skeleton is determined by dividing the difference between the maximum and minimum key point values along the vertical axis, which is specifically expressed as:

[0104]

[0105] Wherein, h u is a parameter value for controlling the height of the pedestrian; through the above pedestrian height scale transformation, the height of the same pedestrian in different frames is unified; the key point alignment operation is to convert the key point coordinates of the pedestrian from the camera coordinate system to the coordinate with the pedestrian as the reference, and the specific calculation method is:

[0106] j i = p' i -p' neck

[0107] indicates that the key point coordinates of the pedestrian after the above step are scaled are subtracted from the coordinates of the center of the shoulders as the aligned coordinates; according to the above step, the final pedestrian posture key point coordinates are obtained, which are used for subsequent gait recognition.

[0108] S228, the category, position information, classification and position coordinates of each key point of the pedestrian target are determined as the training data set.

[0109] S23, using the training data set and the label information, the recognition model of the pedestrian gait is trained to generate the recognition model of the target pedestrian gait.

[0110] In the embodiment, the above step S23 further includes:

[0111] S231, constructing a pedestrian gait recognition model;

[0112] Specifically, a residual-based graph convolutional neural network is adopted, which is composed of multiple residual graph convolutional neural network modules, and finally connected with a pooling layer and a full connection layer; specifically, for each residual graph convolutional neural network module, it is composed of multiple two-dimensional convolutional layers, nonlinear activation layers, graph convolutional layers and residual connection layers;

[0113] For each residual module, the graph convolutional neural network layer therein can be expressed as:

[0114]

[0115] Wherein, Z represents a two-layer graph convolutional neural network, wherein X represents the input multi-frame pedestrian posture key point; For the first layer graph convolutional neural network, W(0) is a learnable parameter in the first layer graph convolutional neural network, and ReLU() represents a nonlinear activation function; W(1) is a learnable parameter in the second layer graph convolutional neural network, and softmax() represents a softmax normalization function;

[0116] The adjacent matrix is represented as:

[0117]

[0118] Specifically, wherein Wherein, A represents the relationship matrix of the pedestrian posture key points between the adjacent two frames calculated above, a unit matrix is added to increase the relationship of each pedestrian posture key point with itself, and A is represented as a new adjacent matrix. L represents a degree matrix, which is represented by the sum of the relationships of each pedestrian posture key point with all other pedestrian posture key points.

[0119] The nonlinear activation layer includes but is not limited to using a nonlinear activation function such as ReLu; the output is Z, which represents a pedestrian feature representation, and can be represented as:

[0120] Z∈R T×V×C Wherein, C represents the number of channels, T represents the number of frames in the video sequence, and V represents the number of key points; the feature mapping of the output feature in the spatial dimension can be represented as:

[0121] f vp =ap(Z)+mp(Z)

[0122] Wherein, ap represents an average pooling operation, and mp represents a maximum pooling operation; the output f vp ∈R T×1×C achieves feature aggregation between different key points in the spatial dimension; then, a pooling operation is used to achieve feature aggregation in the time dimension, which is represented as:

[0123] f tp =sp(f vp )

[0124] Wherein, sp represents a maximum pooling operation in the time dimension, and the output f tp ∈R 1×1×C is used for pedestrian gait feature recognition.

[0125] S232, determining revised parameters according to the training data set and the label information;

[0126] S233, training the recognition model by using the loss function and the revised parameters to generate the recognition model of the target pedestrian gait.

[0127] ​The loss function includes three aspects, which are respectively a pedestrian target detection classification and detection frame regression loss function, a pedestrian posture key point loss function and a graph convolutional neural network model loss function;

[0128] L OD represents a two-dimensional target detection classification loss function, including target classification and position regression two parts,

[0129] L OD = β1L class + β2L bbox

[0130] L class represents a target classification loss function, including but not limited to adopting a cross-entropy loss function and the like, L bbox is a regression loss function of a target detection frame, including but not limited to adopting a smooth L1 loss function; for a pedestrian posture key point regression loss function L Keypoint , which are respectively a regression loss function of a pedestrian center point, a regression loss function of an offset amount of the pedestrian center point and a pedestrian posture key point (or a pedestrian three-dimensional envelope frame key point);

[0131] Specifically, the loss function is:

[0132] L Keypoint = γ1L reg + γ2L offset

[0133] Wherein, L reg , L offset respectively represent a regression loss function of a pedestrian center point, a regression loss function of an offset amount of the pedestrian center point and a pedestrian posture key point (or a pedestrian three-dimensional envelope frame key point), γ1, γ2 are respectively weight coefficients, which are set to 1;

[0134] Specifically, the loss functions of the two parts both adopt L1 Loss average absolute error loss functions, and are specifically represented as:

[0135]

[0136] Wherein, y represents two vectors to be calculated, a ratio is a model estimated value and a target value, m represents a total data amount, and a final target of the loss function is to minimize an absolute difference value of the total estimated value and the target value; for a pedestrian posture classification loss function L CLS , a standard softmax classification loss function is adopted;

[0137] Specifically, for a graph convolutional neural network loss function of pedestrian gait recognition, an L2 norm loss function is used for model training, which is also called a least square error loss function;

[0138]

[0139] Specifically, respectively represent the pedestrian gait features at a certain time predicted by the model and the real pedestrian gait features at a certain time, and m represents a total of m samples.

[0140] The pedestrian gait recognition model training method provided in this embodiment is based on a pedestrian posture detection algorithm to obtain key point coordinates describing the pedestrian posture, then a pedestrian posture conversion module is used to perform projection transformation and scale alignment operation on the pedestrian posture to obtain a unified posture representation, then a graph convolutional neural network is used for feature learning to realize the establishment of local and global relationships of the pedestrian graph structure, and finally a pooling operation is used to obtain a final pedestrian gait feature representation for recognition.

[0141] In this embodiment, a pedestrian gait recognition method is also provided, as shown in Figure 2 The method comprises the following steps:

[0142] Step S31: determining a target pedestrian gait recognition model by using the pedestrian gait recognition model training method in the above embodiment; for details, refer to steps S11-S13, which will not be repeated here.

[0143] Step S32: acquiring a monitoring scene image of a to-be-detected object. In this embodiment, the monitoring scene image of the to-be-detected object is acquired by using an existing monitoring device.

[0144] Step S33: identifying the monitoring scene image by using the target pedestrian gait recognition model to determine a pedestrian gait recognition result.

[0145] In this embodiment, a pedestrian gait recognition model training device is also provided, as used below, the term "module" can be a combination of software and / or hardware that realizes a predetermined function. Although the device described in the following embodiment is preferably realized in software, hardware, or a combination of software and hardware is also possible and is conceived.

[0146] The present application discloses a pedestrian gait recognition model training device, as shown in Figure 3 The device comprises:

[0147] The acquisition module 1 is configured to acquire historical data of pedestrian gait, label the historical data, and determine label information; for details, refer to the corresponding steps in the above method embodiment.

[0148] The data processing module 2 is configured to perform feature extraction on the historical data to determine a training data set; for details, refer to the corresponding steps in the above method embodiment.

[0149] The training module 3 is configured to train the pedestrian gait recognition model by using the training data set and the label information, and generate the target pedestrian gait recognition model.

[0150] The present application discloses an edge detection device, such as Figure 4 as shown, comprising:

[0151] The first processing module 01 is configured to determine the target pedestrian gait recognition model by using the pedestrian gait recognition model training device.

[0152] The second processing module 02 is configured to acquire the monitoring scene image of the to-be-detected object.

[0153] The third processing module 03 is configured to recognize the monitoring scene image by using the target pedestrian gait recognition model, and determine the pedestrian gait recognition result.

[0154] The present application also provides an electronic device, please refer to Figure 5 , Figure 5 is a structural schematic diagram of an electronic device provided by the optional embodiment of the present application, as shown in Figure 5 The electronic device can include at least one processor 601, such as CPU (Central Processing Unit, central processing unit), at least one communication interface 603, memory 604, at least one communication bus 602. Wherein, the communication bus 602 is used to realize the connection communication between these components. Wherein, the communication interface 603 can include display screen (Display), keyboard (Keyboard), and the optional communication interface 603 can also include standard wired interface, wireless interface. The memory 604 can be a high-speed RAM memory (Random Access Memory, volatile random access memory), and can also be a non-volatile memory, such as at least one disk memory. The memory 604 can be optionally at least one storage device located away from the aforementioned processor 601. The memory 604 stores application programs, and the processor 601 calls the program code stored in the memory 604 to execute any method step described above.

[0155] The communication bus 602 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The communication bus 602 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used to represent the communication bus 602 in the figure, but it does not mean that there is only one bus or only one type of bus.

[0156] The memory 604 can include a volatile memory, such as a random-access memory (RAM), and can also include a non-volatile memory, such as a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), or a combination thereof.

[0157] The processor 601 can be a central processing unit (CPU), a network processor (NP), or a combination of the CPU and the NP.

[0158] The processor 601 can further include a hardware chip. The hardware chip can be an application-specific integrated circuit (ASIC), a programmable logic device (PLD) or a combination thereof. The PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0159] Optionally, the memory 604 is further configured to store program instructions. The processor 601 can invoke the program instructions to implement the method as shown in the embodiments of the present application.

[0160] The embodiment of the present application also provides a non-transitory computer storage medium, and the computer storage medium stores computer executable instructions. The computer executable instructions can execute the method in any method embodiment described above. The storage medium can be a magnetic disc, an optical disc, a Read-Only Memory (ROM), a Random Access Memory (RAM), a Flash Memory, a Hard Disk Drive (HDD) or a Solid-State Drive (SSD), etc. The storage medium can also include a combination of the above-mentioned storage mediums.

[0161] Although the embodiments of the present application are described in conjunction with the drawings, various modifications and changes can be made by those skilled in the art without departing from the spirit and scope of the present application, and such modifications and changes are intended to fall within the scope of the appended claims.

Claims

1. A method for training a pedestrian gait recognition model, characterized in that, include: Obtain historical data on pedestrian gait, label the historical data, and determine the label information; Feature extraction is performed on the historical data to determine the training dataset; Using the training dataset and the label information, a pedestrian gait recognition model is trained to generate a target pedestrian gait recognition model. The historical data is labeled to determine tag information, including: the historical data is image data. Pedestrian target detection is performed on the image data, and the pedestrian detection results are labeled using two-dimensional detection boxes to determine pedestrian target labels; The image data is subjected to pedestrian gait detection, and the gait detection results are labeled using the annotation method of the COCO human pose dataset to determine the labels of each key point of the pedestrian gait. The label information is determined based on the pedestrian target label and each of the key point labels; After performing pedestrian target detection on the image data, the method further includes: Obtain the coordinate information of the two-dimensional detection box of the pedestrian target; Using the coordinate information, the pedestrian target is extracted from the image data to determine the extracted image; The target image is determined by performing a size unification operation on the cutout image using a preset size. After performing pedestrian gait detection on the image data, the method further includes: Determine whether the target image meets the preset tilt requirement. If the preset tilt requirement is not met, perform an affine transformation on each key point of the pedestrian's gait to determine the first transformed image. When the preset tilt requirement is met, the target cutout image is determined as the first transformed image; The first transformed image is transformed to determine the pedestrian size based on preset pedestrian height parameters; The coordinates of key pedestrian points are obtained in the second transformed image. The key point alignment coordinates are determined using the key pedestrian point coordinates and the preset center point coordinates.

2. The method according to claim 1, characterized in that, The step of extracting features from the historical data to determine the training dataset includes: A pedestrian target detection network is constructed, which includes a feature extraction network and a target detection network; The feature extraction network is used to extract features from the historical data, and the feature data of the historical data is output, including the length and width of the pedestrian target; The target detection network is used to perform two-dimensional target detection and output the category and location information of the pedestrian target. in, Let i be the i-th target. Indicates the category of the target to be extracted. These are the coordinates of the center point of the target rectangle. These represent the length and width of the target, respectively.

3. The method according to claim 2, characterized in that, The step of extracting features from the historical data to determine the training dataset also includes: A pedestrian pose recognition network is constructed, which includes a backbone network, a feature aggregation network, and a key point prediction network. The backbone network is used to extract features from the historical data to determine images of the same size; The feature aggregation network is used to aggregate the images of the same size to determine the aggregated feature map; The key point prediction network is used to predict the aggregated feature map to determine the classification and location coordinates of each key point; The category, location information, classification of each key point, and location coordinates of the pedestrian target are determined as the training dataset.

4. The method according to claim 3, characterized in that, The step of training a pedestrian gait recognition model using the training dataset and the label information to generate a target pedestrian gait recognition model includes: Construct a pedestrian gait recognition model; The revision parameters are determined based on the training dataset and the label information; The recognition model is trained using the loss function and the revised parameters to generate a recognition model of the target pedestrian's gait.

5. A method for recognizing pedestrian gait, characterized in that, include: Using the pedestrian gait recognition model training method as described in any one of claims 1-4, a recognition model for the target pedestrian gait is determined; Acquire monitoring scene images of the object to be detected; The target pedestrian gait recognition model is used to identify the monitored scene image to determine the pedestrian gait recognition result.

6. A pedestrian gait recognition model training device, characterized in that, include: The acquisition module is used to acquire historical data of pedestrian gait, annotate the historical data, and determine label information; The data processing module is used to extract features from the historical data and determine the training dataset; The training module is used to train the pedestrian gait recognition model using the training dataset and the label information, and generate the target pedestrian gait recognition model. The acquisition module is also used to annotate the historical data and determine tag information, including: the historical data is image data. Pedestrian target detection is performed on the image data, and the pedestrian detection results are labeled using two-dimensional detection boxes to determine pedestrian target labels; The image data is subjected to pedestrian gait detection, and the gait detection results are labeled using the annotation method of the COCO human pose dataset to determine the labels of each key point of the pedestrian gait. The label information is determined based on the pedestrian target label and each of the key point labels; After performing pedestrian target detection on the image data, the method further includes: Obtain the coordinate information of the two-dimensional detection box of the pedestrian target; Using the coordinate information, the pedestrian target is extracted from the image data to determine the extracted image; The target image is determined by performing a size unification operation on the cutout image using a preset size. After performing pedestrian gait detection on the image data, the method further includes: Determine whether the target image meets the preset tilt requirement. If the preset tilt requirement is not met, perform an affine transformation on each key point of the pedestrian's gait to determine the first transformed image. When the preset tilt requirement is met, the target cutout image is determined as the first transformed image; The first transformed image is transformed to determine the pedestrian size based on preset pedestrian height parameters; The coordinates of key pedestrian points are obtained in the second transformed image. The key point alignment coordinates are determined using the key pedestrian point coordinates and the preset center point coordinates.

7. A pedestrian gait recognition device, characterized in that, include: The first processing module is used to determine the recognition model of the target pedestrian's gait using the pedestrian gait recognition model training device as described in claim 6. The second processing module is used to acquire the monitoring scene image of the object to be detected; The third processing module is used to identify the monitored scene image using the target pedestrian gait recognition model and determine the recognition result of the pedestrian gait.

Images