A dynamic small group pedestrian recognition method based on a graph neural network

Abstract

The application discloses a dynamic small group pedestrian recognition method based on a graph neural network, pre-processes pictures in a data set, matches pixel points in the background of the pictures by using a feature matching method, obtains the background features of the pictures by cutting corresponding pictures through the pixel points, dynamically screens the pedestrians, excludes pedestrians not in the same group, and constructs a graph structure; the obtained background features are added to the constructed graph structure to obtain a new graph structure; an optimal model is found by using a multi-head attention graph context information perception transmission method; the problem that features are not robust due to local disturbance in the feature extraction process is solved, and the pedestrian re-identification accuracy is improved; and the dynamic small group pedestrian re-identification method designed based on the algorithm can quickly obtain the pedestrian and the small group pedestrian to be searched.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of computer vision image retrieval, in particular to a dynamic small-group pedestrian recognition method based on a graph neural network. BACKGROUND

[0002] With the rapid development of single-person pedestrian re-identification, more and more attention is paid to its great potential in the real world. The method of single-person pedestrian re-identification has achieved remarkable results. However, for the method of small-group pedestrian re-identification, not only does it have the difficulty of single-person pedestrian re-identification, but also it has the unique problems of the change of the number of group members and the change of the position of group members. Artificially checking the monitoring equipment for confirmation needs to consume a large amount of manpower and time, and is low in efficiency. Therefore, a dynamic small-group pedestrian re-identification method based on a graph neural network is designed.

[0003] The existing related technologies can be divided into two categories: background information matching algorithm and small-group pedestrian re-identification algorithm. (1) In terms of the background information matching algorithm, the existing Chinese patent document discloses DOI: 10.1109 / CVPR42600.2020.00499, a background information matching algorithm based on deep learning. The algorithm designs a neural network which matches two sets of local features by jointly finding the correspondence and rejecting the mismatch points. The distribution is estimated by solving a differentiable optimal transport problem, the cost of which is predicted by a graph neural network. A flexible context aggregation mechanism based on attention is introduced, which can jointly infer the underlying 3D scene and feature distribution. So far, no technology has tried to apply background information matching to small-group pedestrian re-identification. (2) In terms of the small-group pedestrian re-identification algorithm, the existing Chinese patent document discloses DOI: 10.1109 / TPAMI.2020.3032542, a small-group pedestrian re-identification method based on context information. The method proposes a new unified framework based on a graph neural network to simultaneously solve the above two group-based re-id tasks, namely group re-id and group-aware person re-id. Specifically, we construct a context graph with group members as nodes to exploit the dependency between different people. However, the existing method does not consider the change of group members, and the accuracy of small-group pedestrian recognition is low. SUMMARY

[0004] To solve the problems in the background art, the application aims to provide a dynamic small-group pedestrian recognition method based on a graph neural network,

[0005] The application can be achieved by the following technical scheme: a dynamic small-group pedestrian recognition method based on a graph neural network, the method comprising the following steps:

[0006] The pictures in the data set are preprocessed, and the background features in the pictures are obtained by using a background feature matching method;

[0007] The pedestrians are dynamically screened to exclude pedestrians who are not in the same group, and a graph structure is constructed;

[0008] The obtained background features are added to the constructed graph structure to obtain a new graph structure;

[0009] In the new graph structure, a multi-head attention graph context information perception transmission method is used for message passing, which maximizes the update of node information features and improves the accuracy of group recognition.

[0010] Preferably, the process of preprocessing the pictures in the data set and obtaining the background features in the pictures by using the background feature matching method comprises the following steps:

[0011] According to the bounding box of the pedestrian, the key points of the pedestrian are set to 0, and the initial representation of each key point i is (0) xi combines the key point visual appearance and position; a multi-layer perceptron MLP is used enc The key point position is embedded into a high-dimensional vector as follows:

[0012] (0) x i =d i +MLP enc (p i )

[0013] Where d i is the visual descriptor of the key point, p i is the position of the key point, (f) x A i is the element i of image A at the lth layer, m E->i pools all key points, m is the attention weight and, E contains {E self , E cross}, and the remaining message passing update of all i in A is:

[0014]

[0015] Where [||] represents concatenation, and similar updates are performed for all key points in image B; a fixed number of layers L with different parameters are linked, and are alternately aggregated along their own edges and cross edges; therefore, starting from l=1, if l is odd, E=E self , if l is even, E=E cross :

[0016]

[0017] Similarly, B is obtained, and the corresponding abscissa x b and ordinate y b ;

[0018] The obtained f A i and f B j are carried out inner product, and each set is enhanced using dustbin; a = [a T M N] T and b = [1 T N M] T represent the number of matches of each key point in A and B and dustbin; the enhancement allocation has the following constraints:

[0019]

[0020] M and N are the sets of key points of the ith image and the jth image respectively, and the loss function of background matching is as follows:

[0021]

[0022] We obtain a small number of corresponding abscissas x b and ordinates y b by optimizing the matching layer, and finally calculate the point with the minimum absolute value of the difference between the distance from (x 1 b , y 1 b ) to (x M p , y M p ) in the s th image and the distance from (x 2 b , y 2 b ) to (x N p , y N p ) in the jth image, and obtain a 256x128 matrix with the point as coordinates:

[0023]

[0024] Where M and N are the number of pedestrians in the ith and jth images respectively, (x i b , y i b ) and (x j b , y j b(xi, yi) and (xj, yj) are the horizontal and vertical coordinates of the background feature information in the ith picture and the jth picture, respectively. M p (xi, yi) and (xj, yj) are the horizontal and vertical coordinates of the background feature information in the ith picture and the jth picture, respectively. M p (xi, yi) and (xj, yj) are the horizontal and vertical coordinates of the background feature information in the ith picture and the jth picture, respectively. N p (xi, yi) and (xj, yj) are the horizontal and vertical coordinates of the background feature information in the ith picture and the jth picture, respectively. N p (xi, yi) and (xj, yj) are the horizontal and vertical coordinates of the background feature information in the ith picture and the jth picture, respectively.

[0025] Preferably, the dynamic screening of pedestrians is performed to exclude pedestrians not in the same group, and the process of constructing a graph structure comprises the following steps:

[0026] Construct a graph G = {V, E} consisting of N vertices V and a set of edges E. s s s s s s

[0027] Extract the people in the image as the nodes of the image, select the nearest person in each image using the nearest neighbor algorithm, and construct a graph structure.

[0028] Preferably, the edges in the graph structure are divided into strong connected edges, medium connected edges and weak connected edges.

[0029] Preferably, the process of adding the obtained background features to the constructed graph structure to obtain a new graph structure comprises intra-group relationship calculation, inter-group relationship calculation and context information perception.

[0030] The intra-group relationship calculation process comprises: first, calculating between the same parts of intra-group people, and then calculating between the different parts of intra-group people; then, calculating between the same parts of inter-group people, and then calculating between the different parts of inter-group people; and introducing background information.

[0031] Calculate the intra-group correlation: divide each feature into four parts, and select the feature h (t-1) sip s of the s th picture and the background feature h (t-1) sbp s of the s th picture. The message transfer from the feature to the background feature is calculated as follows, and there is only one in the current case:

[0032]

[0033] where φ is a function that measures the correlation between inputs, and W (t-1) e ​​​​​​is the weight matrix that transforms the input features into a higher level representation, the importance weights e sibp are normalized to compute the attention weights:

[0034] a sibp = softmax(e sibp )

[0035] The partial intra-message passed to node i is then computed by multiplying the features of the neighbors by the relevant attention weights:

[0036]

[0037] which continues the computation of the p partials h (t-1) sip of the features of the people in the group with the features of the background other q partials h (t-1) sbq are computed, the importance weights e pq sib are normalized to compute the attention weights, and finally the partial intra-group message passed to node i is computed by multiplying the features of the neighbors by the relevant attention weights:

[0038]

[0039]

[0040]

[0041] The intra-group information is obtained, containing the information m (t) sip between the features of the people and the features of the people in the same partial, the information n (t) sip between the features of the people and the features of the people in different partials, the information o (t) sibp between the features of the background and the features of the people in the same partial, and the information r (t) sib between the features of the background and the features of the people in different partials.

[0042] Preferably, the inter-group relationship computation process is as follows:

[0043] The inter-group relationship only needs to compute the similarity between the nodes of the graph, which not only contains people as feature nodes, but also contains background information as feature nodes. The feature information of the people in the current picture is passed to the feature information of other people in another picture, the p partials h (t-1) si of the features of the people in the s-th picture with the q partials h (t-1) rbThe calculation is performed by using a softmax function on the importance weights z ib The normalization is performed to calculate the attention weights w ib Finally, the partial intra-message passed to node i is obtained by multiplying the features of the neighbors with the relevant attention weights;

[0044]

[0045] w ib = softmax(z ib )

[0046]

[0047] After obtaining the intra-graph and inter-graph messages, the node features are updated by connecting the previous features and all types of messages using a fully connected layer:

[0048]

[0049] Preferably, the context information perception process is as follows: the above-mentioned feature updating step using the intra-graph and inter-graph attention mechanism is repeated T rounds, and then the model is designed to learn the group and individual correspondence respectively; first, a graph-level representation is constructed through a readout operation, and self-attention is applied on the graph nodes, and the final graph representation hs is the weighted sum of the node-level features:

[0050]

[0051]

[0052]

[0053] where h si is the i-th individual in the s-th picture, W (T) u is a weight matrix that converts the input features into a higher-level representation. Similarly, another graph h r can be obtained. In order to learn the group correspondence, the circleloss loss function is used to pull the features of the same group closer and push the features of different groups far away:

[0054]

[0055] where a j s and a i r are non-negative weighting factors, and γ is a proportionality factor. The pair-wiseloss loss function is used to obtain For individual and background information level correspondence learning, the pair-wiseloss loss function is still used to obtain

[0056]

[0057]

[0058] where y pp is the label of the pair, m is the margin, h ip is the p-th personal feature of the i-th image, h jp is the q-th personal feature of the j-th image, h bi,p is the background information feature of the i-th image. When the pair shares the same person ID, y pp = 1; when the pair consists of different persons, y pp = -1. The value S gt is calculated by cross-entropy between the predicted matrix S and the true matrix.

[0059]

[0060] where S gt ∈ R ns×nr is a binary matrix, S gt i，j＝1 If the i-th person in G s and the j-th person in G r belong to the same identity, the final loss function is a linear combination of the loss functions of all functions:

[0061]

[0062] Preferably, the strongly connected edge refers to a connected graph formed by node 1 and node 2 selected from each other; node 2 selects node 3 as the nearest neighbor of node 2, and node 1 selects node 2 as the nearest neighbor of node 1, in which case the edge connecting node 2 and node 3 is called a medium connected edge; the weakly connected edge refers to node 1 selecting node 2 as the neighbor of node 1, and node 1 is not selected by other nodes as the neighbor of other nodes.

[0063] An apparatus comprising:

[0064] one or more processors;

[0065] a memory for storing one or more programs;

[0066] When one or more of the programs are executed by the one or more processors, the one or more processors implement a dynamic small group pedestrian recognition method based on a graph neural network as described above.

[0067] A storage medium comprising computer executable instructions, wherein the computer executable instructions, when executed by a computer processor, are used to perform a dynamic small group pedestrian recognition method based on a graph neural network as described above.

[0068] Advantages of the present application:

[0069] A local alignment pedestrian re-identification framework based on a video sequence is embedded as a core algorithm in trajectory monitoring. The framework realizes the alignment of regions between different videos by using posture information as a reference, and then realizes the alignment of regions between different frames in the same video by using a correlation attention module inside the video sequence. The framework solves the problem of feature instability caused by local disturbance in the feature extraction process, and improves the accuracy of pedestrian re-identification. The dynamic small group pedestrian re-identification method designed based on the algorithm can quickly obtain the pedestrian and small group of pedestrians to be searched. BRIEF DESCRIPTION OF DRAWINGS

[0070] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, for those skilled in the art, other drawings can also be obtained based on these drawings without creative labor;

[0071] Figure 1 is a flowchart of the dynamic small group pedestrian re-identification method based on the graph neural network provided by the present application;

[0072] Figure 2 is a network structure diagram of the dynamic small group pedestrian re-identification framework based on the graph neural network proposed by the present application;

[0073] Figure 3 is a decomposition module schematic diagram of the preprocessing module (dynamic screening pedestrian) in the dynamic small group pedestrian re-identification framework based on the graph neural network;

[0074] Figure 4 is a schematic diagram of the decomposition module calculated after adding background features in the dynamic small group pedestrian re-identification framework based on the graph neural network;

[0075] Figure 5 is a training flowchart of the neural network model provided by the present application;

[0076] Figure 6 is an application scenario diagram of the dynamic small group pedestrian re-identification method based on the graph neural network provided by the present application. DETAILED DESCRIPTION

[0077] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0078] As shown in the figure, a small group pedestrian re-identification algorithm based on pedestrian dynamic screening. Specifically as follows: Figure 1

[0079] Step1: Preprocess the pictures in the data set, and use the background feature matching method to obtain the background features in the pictures;

[0080] Step2: Dynamic screening of pedestrians, excluding pedestrians not in the same group, while constructing a graph structure;

[0081] Step3: Add the obtained background features to the constructed graph structure to obtain a new graph structure;

[0082] Step4: In the new graph structure, use the multi-head attention graph context information perception transmission method for message passing, maximize the update of node information features, and improve the accuracy of group identification.

[0083] Specifically, the Step1 of the small group pedestrian re-identification algorithm based on pedestrian dynamic screening:

[0084] The initial representation of each key point i (0) x i Combining the visual appearance and position of the key points; using a multi-layer perceptron MLP enc Embedding the key point position into a high-dimensional vector, as follows:

[0085] (0) x i =d i +MLP enc (p i )

[0086] Where d i is the visual descriptor of the key point, p i is the position of the key point, (f) x A i is the element i of image A at the lth layer, m E->i pools all key points, m is the attention weight and, E contains {E self , E cross}, and the remaining message passing of all i in A is updated as:

[0087] ​

[0088] where [||] denotes concatenation, while similar updates are performed for all key points in image B; a fixed number of layers L with different parameters are chained and alternately aggregate along self-edges and cross-edges; thus, starting from l = 1, if l is odd, E = E self , if l is even, E = E cross :

[0089]

[0090] Similarly, B is obtained, as well as the corresponding horizontal coordinates x b and vertical coordinates y b ;

[0091] The resulting f A i and f B j are taken to perform inner products, and each set is augmented with a dustbin; let a = [a T M N] T and b = [1 T N M] T denote the number of matches for each key point in A and B with the dustbin; the augmentations are assigned with the following constraints:

[0092]

[0093] M and N are the sets of key points in the i-th and j-th images, respectively, and the loss function for background matches is as follows:

[0094]

[0095] We obtain a small number of corresponding horizontal coordinates x b and vertical coordinates y b by optimizing the matching layers, and finally calculate the distance from (x i b , y i b ) to (x M p , y M p ) in the s-th image and from (x j b , y j b ) to (x N p , y N pFind the point with the smallest absolute value of the difference between the distances to () and (), and obtain a 20×20 matrix with this point as its coordinates:

[0096]

[0097] Where M and N are the number of pedestrians in the i-th and j-th images, respectively, (x i b y i b ) and (x j b y j b ) represent the x and y coordinates of the background feature information in the i-th and j-th images, respectively. M p y M p ) and (x N p y N p ) are the horizontal and vertical coordinates of all pedestrians in the i-th and j-th images, respectively.

[0098] Specifically, Step 2 of the method for re-identifying small groups of pedestrians based on dynamic pedestrian screening:

[0099] We construct a system consisting of N s Vertex V s and a set of edges E s The image G is composed of s ={V s E s We extract people from the images as nodes, use the nearest neighbor algorithm to select the nearest person in each image, and construct a graph structure. We then categorize the edges in the graph structure into strongly connected, moderately connected, and weakly connected edges. For example... Figure 3 As shown, a strongly connected edge is a connected graph formed by nodes 1 and 2 choosing each other. A moderately connected edge means that node 2 chooses node 3 as its nearest neighbor, and node 1 chooses node 2 as its nearest neighbor. In this case, the edge connecting node 2 and node 3 is called a moderately connected edge. A weakly connected edge means that node 1 chooses node 2 as its neighbor, but node 1 is not chosen as a neighbor by any other node. In this case, the edge between node 1 and node 2 is called a weakly connected edge. We use the following formula to determine whether a group consisting of more than one member is the same group. If they are determined to be the same group, the graph structure is constructed by combining the background feature information obtained in Step 1. If they cannot be identified as the same group, nodes with longer edges and weakly connected edges are pruned. After pruning, the determination of whether the groups are the same continues, and so on.

[0100] Then, we determine whether a plurality of members of a group is the same group, as shown in the following formula. If there are many people in a graph, there can be multiple subgraph structures under a graph. Let's select a subgraph structure for calculation. When it is determined that a group does not belong to the same group, the node corresponding to the longer weak connection edge is selected for pruning. After pruning, it is determined whether the group is the same group, until the group is determined to be the same group, and then a new graph structure is constructed. We first determine whether the group is a connected graph, and we find that according to the nearest neighbor algorithm, nodes 1, 2, 3, 4, and 5 are a connected graph, and nodes 6, 7, and 8 are another connected graph. Let's take one of the connected graphs as an example, taking the first five nodes of the connected graph as an example, at this time, the length of the weak connection edge connecting node 1 and node 5 is determined to be longer, because the corresponding edge of node 5 is longer, and then the pruning operation is performed to obtain a graph of four nodes; at this time, the group is determined to be different, because there is only one weak connection edge at this time, and then node 1 and its edges are pruned to obtain a graph of three nodes; at this time, the group is determined to be the same group

[0101]

[0102] We divide the pedestrian group into three-person groups, four-person groups, and five-person groups, as Figure 3 If the five-person group is determined to be a different group, the weak connection edge is pruned until it is determined to be the same group.

[0103] Specifically, the method of the small group pedestrian re-identification algorithm based on pedestrian dynamics screening Step3:

[0104] We add the background information node to the pruned graph structure, and finally obtain a new graph structure. We capture the context information according to the multi-level attention mechanism in the graph structure. The final graph-level representation is learned from the node-level features in the form of sub-attention. In addition, we use the circle loss loss function when performing group feature matching.

[0105] Step3.1: Intra-group relationship

[0106] First, calculate the same part between the different parts of the group, and then calculate the same part between the different parts of the group. Then, the background information is introduced.

[0107] First, calculate the background information related calculation in the group: for the calculation of the correlation in the group, we divide each feature into four parts, and the same part of the feature is selected (t-1) sip , and the background feature h(t-1) sbp The message from the person features to the background features is computed as follows, in the current case there is and only one message:

[0108]

[0109] where φ is a function that measures the correlation between the inputs, W (t-1) e is a weight matrix that transforms the input features into a higher level representation, the attention weights e sibp are computed by normalizing the importance weights using a softmax function:

[0110] a sibp = softmax(e sibp )

[0111] The partial intra-message passed to node i is then computed by combining the features of the neighbors with the relevant attention weights:

[0112]

[0113] which continues the computation of the p partials of the features of the people in the group h (t-1) sip with the q partials of the features of the background h (t-1) sbq are computed by normalizing the importance weights using a softmax function, the attention weights e pq sib are computed, and the partial intra-message passed to node i is then computed by combining the features of the neighbors with the relevant attention weights:

[0114]

[0115]

[0116]

[0117] The group information is obtained, containing the information of the same parts between the person features and the person features m (t) sip , the information of the different parts between the person features and the person features n (t) sip , the information of the same parts between the background features and the person features o (t) sibp , the information of the different parts between the background features and the person features r (t) sib .

[0118] Step 3.2: Inter-group relationships

[0119] The inter-group relationship only needs to calculate the similarity between the nodes of the graph, and the graph nodes not only contain people as feature nodes, but also background information as feature nodes. To calculate the transmission of the feature information of the current picture to the feature information of other people in another picture, the feature information of the current person also needs to be calculated to the feature information of another background information, and the feature part h (t-1) si The rth background feature part h (t-1) rb The importance weight z ib is calculated by using the softmax function, and the attention weight w ib is calculated by normalizing the importance weight z ib . Finally, the part of the intra-group message transmitted to node i is calculated by the neighbor's feature and the related attention weight:

[0120]

[0121] w ib =softmax(z ib )

[0122]

[0123] By connecting the previous features and all types of messages, the node features are updated using a fully connected layer:

[0124]

[0125] Step 3.3: Contextual information perception

[0126] The above feature update step using intra-graph and inter-graph attention mechanism is repeated for T rounds, and then the model is designed to learn group and individual correspondence respectively; first, a graph-level representation is constructed by reading operation, and self-attention is applied on the graph nodes, and the final graph representation h s is the weighted sum of node-level features:

[0127]

[0128]

[0129]

[0130] where h si is the ith person in the s-th picture, W (T) u is the weight matrix for converting input features to higher-level representations. Similarly, another graph h r can be obtained. In order to learn the group correspondence, the circle loss function is used to pull the features of the same group closer and push the features of different groups far away:

[0131]

[0132] where a j s and a i r are non-negative weighted factors, and γ is a scale factor. The pair-wise loss function is used to get For the personal and background information level correspondence learning, the pair-wise loss function is still used to get

[0133]

[0134]

[0135] where y pp is the label of the pair, m is the margin, h ip is the p-th personal feature of the i-th image, h jp is the q-th personal feature of the j-th image, and h bi,p is the background information feature of the i-th image. When the pair shares the same person ID, y pp = 1; when the pair consists of different persons, y pp = -1. The value S gt is calculated by the cross-entropy between the predicted matrix S and the true value matrix.

[0136]

[0137] where S gt ∈ R ns×nr is a binary matrix, S gt i，j＝1 If the i-th person in G s and the j-th person in G r belong to the same identity, the final loss function is a linear combination of the loss functions of all functions:

[0138]

[0139] Specifically, the Step 4 of the method of the small group pedestrian re-identification algorithm based on pedestrian dynamic screening:

[0140] We evaluate the proposed group re-identification method on three public datasets: (1) RoadGroup dataset, where 162 pairs of group images are captured by two cameras. (2) CUHK-SYSUGroup dataset contains 3839 images of 1558 extracted groups. The groups in RoadGroup dataset are more compact, but the image quality is worse with larger illumination variation. (3) DukeMTMCGroup dataset contains more than 1000 images. DukeMTMCGroup and CUHK-SYSUGroup datasets have larger groups, thus experience more group layout and member variation. In RoadGroup and CUHK-SYSUGroup datasets, people always walk in the same scene, they always go into other scenes. However, in DukeMTMC Group dataset, the same group of people always walks into another scene, while different groups and noise are in the same scene.

[0141] We randomly split each dataset into training and testing sets, and use the average cumulative matching characteristic (CMC) scores as evaluation metrics. We use ResNet50 pre-trained on ImageNet as the backbone. The person images are resized to 256x128 as input. The initial learning rate is set to 0.0003, reduced by 10 times at the 100th and 200th epoch, and the training stage ends at the 300th epoch. Since the size of the group is different, we construct a graph with the same number of nodes to facilitate implementation, and add virtual nodes to the group with limited members. We only perform person correspondence learning on positive group pairs, because there is no correspondence relationship for negative group pairs. We use two layers (i.e., T = 2) GNN in the framework. We train our model on one GPU, and it takes about 60 hours to converge the model on the CUHK-SYSUGroup dataset.

[0142] Figure 5 The training flowchart of the neural network model provided by the application is shown, including the following steps: we first preprocess the small group of pedestrian pictures, through data enhancement, crop to 512*512 pixels, at the same time, horizontal and vertical flip, finally all images are converted into tensor data form, using channel mean and standard deviation for standardization, then the tensor is normalized to (0, 1); after starting training, we perform forward propagation in the neural network to obtain features and predicted labels; then calculate the loss function and perform back propagation; finally, the accuracy in the test set is calculated; if the accuracy is high, the model is saved; complete a training. After reaching the training times, end the training.

[0143] Figure 6The application scenario of the method of dynamic small group pedestrian re-identification based on a graph neural network is summarized. Our scenario is applied to various scenarios, such as railway stations, airports, hospitals, shopping malls, gas stations, and the like. In the application scenario, a large number of cameras are distributed, real-time pictures are taken and uploaded to the cloud server. The system uses the cloud server for calculation and storage. After matching by our algorithm, the small group of pedestrians to be queried is obtained, and the pedestrian pictures are uploaded to the system terminal operation platform.

[0144] Based on the same inventive concept, the application further provides a computer device, which comprises one or more processors and a memory for storing one or more computer programs; the program comprises program instructions, and the processor is configured to execute the program instructions stored in the memory. The processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., which are the computing core and control core of the terminal, and are configured to implement one or more instructions, and are specifically configured to load and execute one or more instructions in the computer storage medium to implement the above method.

[0145] It needs to be further explained that based on the same inventive concept, the application further provides a computer storage medium, and the storage medium stores a computer program, and the computer program is executed by the processor to execute the above method. The storage medium can adopt any combination of one or more computer readable media. The computer readable medium can be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium can be, but is not limited to, an electrical, magnetic, optical, electrical, magnetic, infrared, or semiconductor system, device or component, or any combination of the above. More specific examples (non-exhaustive list) of the computer readable storage medium include: an electrical connection with one or more conductive wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or component.

[0146] It needs to be further explained that in the specific implementation process:

[0147] The performance on the three mainstream pedestrian re-identification data sets of RoadGroup, DukeMTMCGroup and CUHK-SYSUGroup reaches an advanced level, and most of the performance indicators reach the current highest level.

[0148] Table 1 Accuracy comparison of the algorithm and other pedestrian re-identification algorithms

[0149]

[0150] (2) Social: The present application can be applied in crowded areas such as shopping malls, airports and stations, combined with cameras and other instruments, small groups of pedestrians needing to be detected are quickly identified and uploaded to the cloud; the images of the target small group of pedestrians can be obtained in a short time.

[0151] (3) Economic: The present application uses an automated way to identify small groups of pedestrians, reducing the workload of manual visits and surveys, and saving a lot of labor costs and time costs.

[0152] In the description of the present specification, the description of the reference terms "one embodiment", "example", "specific example" and the like means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are contained in at least one embodiment or example of the present disclosure. In the present specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the described specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0153] The basic principles, main features and advantages of the present disclosure are shown and described above. Those skilled in the art should understand that the present disclosure is not limited by the above embodiments, and the above embodiments and descriptions in the specification are only to illustrate the principles of the present disclosure. Without departing from the spirit and scope of the present disclosure, various changes and improvements of the present disclosure fall within the scope of the claimed present disclosure.

Claims

1. A dynamic small group pedestrian recognition method based on a graph neural network, characterized in that, The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: Compute group-wise correlation: Divide each feature into four parts, the same part of the feature h under the selection of the s-th picture (t-1) sip , and the background feature h of the s-th picture (t-1) sbp The message from the character feature to the background feature is calculated as follows, and there is only one in the current case: where φ is a function that measures the correlation between the inputs, W (t-1) e is a weight matrix that transforms the input features into a higher-level representation, the importance weights e sibp are normalized by using a softmax function to compute the attention weights: a sibp = soft max( esibp ) The method comprises the following steps: It continues to compute p parts h of the features of the people in the group (t-1) sip With the background other q parts h (t-1) sbq The computation is done by using a softmax function on the importance weights e pq sib The attention weights are computed by normalizing the importance weights, and the final passed-in part of the intra-group message to node i is computed by the features of the neighbors and the relevant attention weights: obtained within the group, containing information m of the same parts between the human features and the human features (t) sip , information n of the different parts between the human features and the human features (t) sip , information o of the same parts between the background features and the human features (t) sibp , information r of the different parts between the background features and the human features (t) sib ; The method comprises the following steps:

2. The dynamic small group pedestrian recognition method based on a graph neural network according to claim 1, characterized in that, The method comprises the following steps: Set the key points of the pedestrian to 0 according to the bounding box of the pedestrian, the initial representation of each key point i (0) xi combines the key point visual appearance and position; uses a multi-layer perceptron, MLP enc Embed the key point position into a high-dimensional vector as follows: (0) x i = d i + MLP enc (p i ) where d i is the visual descriptor of the key point, p i is the position of the key point, (f) x A i is the element i, m E->i of image A at layer l, m is the attention weight and E contains {E self , E cross}, the remaining message passing update for all i in A is: where [||] denotes concatenation, while similar updates are performed for all key points in image B; a fixed number of layers L with different parameters are chained and alternately aggregate along their own edges and cross edges; thus, starting from l = 1, if l is odd, E = E self , if l is even, E = E cross ​ B is obtained in the same way, and the corresponding abscissa x b and ordinate y b ; The resulting f A i and f B j Perform inner product, augment each set with dustbin; let a = [a T M N] T and b = [1 T N M] T denote the number of matches for each keypoint in A and B with the dustbin; augment assignment with the constraint that and The method comprises the following steps: By optimizing the matching layer to get a small amount of corresponding horizontal coordinate x b and vertical coordinate y b , finally calculate the distance between (x 1 b , y 1 b ) in the s-th picture and (x M p , y M p ) in the j-th picture, and the absolute value of the difference between the distance of (x 2 b , y 2 b ) in the s-th picture and (x N p , y N p ) in the j-th picture, and get the 256x128 matrix with this point as the coordinate: where M and N are the number of pedestrians in the ith and jth images, respectively, (x i b , y i b ) and (x j b , y j b ) are the horizontal and vertical coordinates of the background feature information in the ith and jth images, respectively, and (x M p , y M p ) and (x N p , y N p ) are the horizontal and vertical coordinates of all pedestrians in the ith and jth images, respectively.

3. The dynamic small group pedestrian recognition method based on a graph neural network according to claim 1, characterized in that, The method comprises the following steps: Construct a graph G = {V, E} consisting of N s vertices V s and a set of edges E s s {V s , E s}​ The method comprises the following steps:

4. The dynamic small group pedestrian recognition method based on a graph neural network according to claim 3, characterized in that, The method comprises the following steps:

5. The dynamic small group pedestrian recognition method based on a graph neural network according to claim 1, characterized in that, The method comprises the following steps: The inter-group relationship only needs to calculate the similarity between the nodes of the graph, and the graph nodes not only include people as feature nodes, but also background information as feature nodes. The feature information of the people in the current picture is transmitted to the feature information of other people in another picture. The feature part h (t-1) si The background feature part h (t-1) rb The importance weight z ib The attention weight w is calculated by normalizing the importance weight z ib Finally, the intra-node message transmitted to node i is obtained by combining the features of the neighbors and the related attention weight; w ib = soft max(z ib ) The method comprises the following steps:

6. The dynamic small group pedestrian recognition method based on a graph neural network according to claim 1, characterized in that, The method comprises the following steps: where h si is the i-th person in the s-th image, W (T) u is the weight matrix that converts the input features into a higher-level representation, and similarly, we can get another image h r To learn the group correspondence, we use the circle loss function to pull the features of the same group closer and push the features of different groups far away: where a j s and a i r are non-negative weighting factors, γ is a scaling factor, and the pair-wise loss function is used For the personal-to-background information level correspondence learning, the pair-wise loss function is used again where y pp is the label of the pair, m is the margin, h ip is the p-th personal feature of the i-th image, h jp is the q-th personal feature of the j-th image, h bi,p is the background information feature of the i-th image, y pp = 1 when the pair shares the same person ID; y pp = -1 when the pair consists of different persons, the value of the predicted matrix S is calculated by cross-entropy with the ground truth matrix Sgt; where S gt ∈ R ns×nr is a binary matrix, S gt i，j＝1 If the i-th person in G s belongs to the same equivalence class as the j-th person in G r , then the final loss function is a linear combination of the loss functions of all functions:

7. The dynamic small group pedestrian recognition method based on a graph neural network according to claim 4, characterized in that, The method comprises the following steps:

8. An apparatus, comprising: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps:

9. A storage medium containing computer-executable instructions, wherein: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the following steps: The method comprises the

Images