Pedestrian re-identification method, device, electronic device and readable storage medium

Abstract

The present invention discloses a re-identification method, device, electronic device and readable storage medium, the method comprising: obtaining a pair of portrait images to be identified; obtaining the first-order features of each of the portrait images, the first-order features including the local features of each key point in the portrait image and the global features of the portrait image; obtaining the relationship information of the portrait images based on the first-order features of the portrait images; obtaining the human body topology information of each of the portrait images based on the first-order features and the relationship information; obtaining the pedestrian re-identification result based on the relationship information and the human body topology information of each of the portrait images, the pedestrian re-identification result including whether the pair of portrait images contain the same portrait. The method solves the technical problem that the existing pedestrian re-identification scheme is not robust and affects the accuracy of the recognition result. The beneficial effect of improving the accuracy of the pedestrian re-identification result is achieved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a pedestrian re-identification method, device, electronic device and readable storage medium. Background Art

[0002] The goal of the person re-identification (ReID) task is to match images of the same person captured by different cameras. It is widely used in video analysis, smart cities and other fields. Although many methods for ReID have been proposed recently, most of them focus on the full body image of the person, ignoring the more challenging pedestrian occlusion problem that often occurs in practical applications.

[0003] Compared with matching the entire person's information, ReID in the case of occlusion is more challenging for the following reasons: in the occluded area, the image contains less discriminative information, making it easier to be matched to the wrong person; although matching based on feature information between body parts is effective, it requires strict body alignment in advance, so the effect is not good when occlusion is severe.

[0004] In recent years, many ReID methods targeting occlusion or specific parts have been proposed. However, most of them only consider the first-order information of feature learning and alignment, which is not robust and affects the accuracy of the recognition results. Summary of the invention

[0005] The present invention provides a pedestrian re-identification method, device, electronic device and readable storage medium to partially or completely solve the above-mentioned problems related to the pedestrian re-identification process in the prior art.

[0006] According to a first aspect of the present invention, a pedestrian re-identification method is provided, comprising:

[0007] Get a pair of portrait images to be identified;

[0008] Acquire the first-order features of each of the portrait images, wherein the first-order features include the local features of each key point in the portrait image and the global features of the portrait image;

[0009] Based on the first-order features of the portrait image, acquiring relationship information of the portrait image;

[0010] Based on the first-order features and the relationship information of each of the portrait images, obtaining the human body topology information of each of the portrait images;

[0011] Based on the relationship information and the human body topology information of each of the portrait images, a person re-identification result is obtained, where the person re-identification result includes whether the pair of portrait images contain the same portrait.

[0012] Optionally, the step of acquiring the relationship information of the portrait image based on the first-order features of the portrait image includes:

[0013] For each of the portrait pictures, based on the first-order features of the portrait pictures, obtaining the relationship information of the portrait pictures through a direction-adaptive graph convolution layer;

[0014] Wherein, the direction-adaptive graph convolution layer is configured as follows:

[0015] Based on the difference between the global feature and the local feature of the portrait picture, obtaining an adaptive neighboring order matrix of the portrait picture;

[0016] Relation information of the portrait picture is obtained according to the adaptive neighboring order matrix, the local features and the global features.

[0017] Optionally, the step of acquiring an adaptive neighboring order matrix of the portrait picture based on a difference between the global feature and the local feature of the portrait picture comprises:

[0018] Constructing a K×K dimensional global feature matrix based on the global features, where the elements in the global feature matrix are global features;

[0019] Constructing a local feature matrix of K×K dimensions based on all the local features, wherein the elements in the global feature matrix are local features;

[0020] Performing element-wise subtraction operation on the global feature matrix and the local feature matrix to obtain a first matrix;

[0021] Performing absolute value processing and batch normalization processing on the elements in the first matrix, and using the processed matrix as the input of the first fully connected layer, obtaining the output of the first fully connected layer as the adaptive neighbor-order matrix;

[0022] Wherein, K is the number of key points.

[0023] Optionally, the step of acquiring the relationship information of the portrait picture according to the adaptive neighboring order matrix, the local features and the total features includes:

[0024] Performing a matrix multiplication operation on the adaptive neighboring matrix and the local features, and inputting the operation result into a second fully connected layer, and simultaneously inputting the local features into a third fully connected layer;

[0025] An element-wise addition operation is performed on the output data of the second fully connected layer and the output data of the third fully connected layer, and relationship information of the portrait image is obtained based on a result of the element-wise addition operation and the global feature.

[0026] Optionally, the step of acquiring the relationship information of each portrait picture based on the first-order features of the portrait picture through a direction-adaptive graph convolution layer includes:

[0027] For each of the portrait pictures, based on the first-order features of the portrait picture and the original neighbor-order matrix, the relationship information of the portrait picture is obtained through the direction-adaptive graph convolution layer;

[0028] The step of acquiring the relationship information of the portrait picture according to the adaptive neighboring order matrix, the local features and the global features comprises:

[0029] Performing element-wise multiplication operation on the adaptive neighboring matrix and the original neighboring matrix to obtain a first neighboring matrix;

[0030] Performing a matrix multiplication operation on the first neighboring matrix and the local feature, and inputting the operation result into a second fully connected layer, and inputting the local feature into a third fully connected layer;

[0031] An element-wise addition operation is performed on the output data of the second fully connected layer and the output data of the third fully connected layer, and relationship information of the portrait image is obtained based on a result of the element-wise addition operation and the global feature.

[0032] Optionally, the step of acquiring the human body topology information of each portrait image based on the first-order features and the relationship information of each portrait image includes:

[0033] Based on the first-order features and the relationship information of each of the portrait images, the human body topology information of each of the portrait images is acquired through a cross-graph embedding alignment layer;

[0034] Wherein, the cross-graph embedding alignment layer is configured as follows:

[0035] According to the first-order features and relationship information of the portrait images, a third-neighborhood-order matrix representing alignment information of key points in the two portrait images is obtained through a graph matching strategy;

[0036] According to the third neighboring-order matrix and the first-order features and relationship information of the portrait pictures, each portrait picture is used to enhance another portrait picture, so as to obtain the human body topology information of each portrait picture.

[0037] Optionally, the step of obtaining a third neighboring-order matrix representing alignment information of key points in two portrait images through a graph matching strategy according to the first-order features and relationship information of the portrait images comprises:

[0038] Inputting the first-order features and relationship information of each of the portrait images into a hidden space to obtain hidden features of each of the portrait images;

[0039] Obtaining a correlation matrix between the hidden features of the two portrait images through a graph matching strategy, which is the third neighbor-order matrix;

[0040] The latent space includes at least one network structure layer, and the network structure layer is a cascade of a fully connected layer and a linear rectification function layer, or is a fully connected layer.

[0041] Optionally, the step of enhancing another portrait picture by each portrait picture according to the third neighboring-order matrix and the first-order features and relationship information of the portrait pictures to obtain the human body topology information of each portrait picture includes:

[0042] According to the third neighboring matrix and the hidden features of each of the portrait images, the human body topology information of each of the portrait images is obtained.

[0043] Optionally, the step of obtaining a person re-identification result based on the relationship information and the human body topology information of each of the portrait images, wherein the person re-identification result includes whether the pair of portrait images contain the same portrait, comprises:

[0044] Based on the relationship information of each of the portrait pictures, obtaining a first similarity between two of the portrait pictures;

[0045] Based on the human body topology information of each of the portrait pictures, obtaining a second similarity between the two portrait pictures;

[0046] Acquire a final similarity between the two portrait images according to the first similarity and the second similarity;

[0047] In response to the similarity being higher than a preset similarity threshold, it is confirmed that the pair of portrait images include the same portrait.

[0048] According to a second aspect of the present invention, there is provided a pedestrian re-identification device, comprising:

[0049] A portrait image acquisition module is used to acquire a pair of portrait images to be identified;

[0050] A first-order semantic module, used to obtain the first-order features of each of the portrait images, wherein the first-order features include the local features of each key point in the portrait image and the global features of the portrait image;

[0051] A high-order relationship module, used to obtain relationship information of the portrait image based on the first-order features of the portrait image;

[0052] A high-order human body topology module, used for acquiring human body topology information of each of the portrait images based on the first-order features and the relationship information of each of the portrait images;

[0053] The portrait detection module is used to obtain a person re-identification result based on the relationship information and the human body topology information of each portrait image, wherein the person re-identification result includes whether the pair of portrait images contain the same portrait.

[0054] Optionally, the high-order relationship module is specifically configured to obtain, for each of the portrait pictures, relationship information of the portrait pictures based on the first-order features of the portrait pictures through a direction-adaptive graph convolution layer;

[0055] Wherein, the direction-adaptive graph convolution layer is configured as follows:

[0056] Based on the difference between the global feature and the local feature of the portrait picture, obtaining an adaptive neighboring order matrix of the portrait picture;

[0057] Relation information of the portrait picture is obtained according to the adaptive neighboring order matrix, the local features and the global features.

[0058] Optionally, the direction-adaptive graph convolution layer is specifically configured as follows:

[0059] Constructing a K×K dimensional global feature matrix based on the global features, where the elements in the global feature matrix are global features;

[0060] Constructing a local feature matrix of K×K dimensions based on all the local features, wherein the elements in the global feature matrix are local features;

[0061] Performing element-wise subtraction operation on the global feature matrix and the local feature matrix to obtain a first matrix;

[0062] Performing absolute value processing and batch normalization processing on the elements in the first matrix, and using the processed matrix as the input of the first fully connected layer, obtaining the output of the first fully connected layer as the adaptive neighbor-order matrix;

[0063] Wherein, K is the number of key points.

[0064] Optionally, the direction-adaptive graph convolution layer is further configured as follows:

[0065] Performing a matrix multiplication operation on the adaptive neighboring matrix and the local features, and inputting the operation result into a second fully connected layer, and simultaneously inputting the local features into a third fully connected layer;

[0066] An element-wise addition operation is performed on the output data of the second fully connected layer and the output data of the third fully connected layer, and relationship information of the portrait image is obtained based on a result of the element-wise addition operation and the global feature.

[0067] Optionally, the high-order relationship module is further used to obtain, for each of the portrait pictures, the relationship information of the portrait pictures based on the first-order features of the portrait pictures and the original neighbor-order matrix through the direction-adaptive graph convolution layer;

[0068] The direction-adaptive graph convolution layer is further configured as follows:

[0069] Performing element-wise multiplication operation on the adaptive neighboring matrix and the original neighboring matrix to obtain a first neighboring matrix;

[0070] Performing a matrix multiplication operation on the first neighboring matrix and the local feature, and inputting the operation result into a second fully connected layer, and inputting the local feature into a third fully connected layer;

[0071] An element-wise addition operation is performed on the output data of the second fully connected layer and the output data of the third fully connected layer, and relationship information of the portrait image is obtained based on a result of the element-wise addition operation and the global feature.

[0072] Optionally, the high-order human topology module is specifically used to obtain the human topology information of each of the portrait images through a cross-graph embedding alignment layer based on the first-order features and the relationship information of each of the portrait images;

[0073] Wherein, the cross-graph embedding alignment layer is configured as follows:

[0074] According to the first-order features and relationship information of the portrait images, a third-neighborhood-order matrix representing alignment information of key points in the two portrait images is obtained through a graph matching strategy;

[0075] According to the third neighboring-order matrix and the first-order features and relationship information of the portrait pictures, each portrait picture is used to enhance another portrait picture, so as to obtain the human body topology information of each portrait picture.

[0076] Optionally, the cross-graph embedding alignment layer is specifically configured as follows:

[0077] Inputting the first-order features and relationship information of each of the portrait images into a hidden space to obtain hidden features of each of the portrait images;

[0078] Obtaining a correlation matrix between the hidden features of the two portrait images through a graph matching strategy, which is the third neighbor-order matrix;

[0079] The latent space includes at least one network structure layer, and the network structure layer is a cascade of a fully connected layer and a linear rectification function layer, or is a fully connected layer.

[0080] Optionally, the cross-graph embedding alignment layer is further configured:

[0081] According to the third neighboring matrix and the hidden features of each of the portrait images, the human body topology information of each of the portrait images is obtained.

[0082] Optionally, the portrait detection module includes:

[0083] A first similarity acquisition submodule, configured to acquire a first similarity between two portrait pictures based on the relationship information of each portrait picture;

[0084] A second similarity acquisition submodule, configured to acquire a second similarity between two portrait images based on the human body topology information of each portrait image;

[0085] a third similarity acquisition submodule, configured to acquire a final similarity between the two portrait images according to the first similarity and the second similarity;

[0086] The same portrait confirmation submodule is used to confirm that the pair of portrait images contain the same portrait in response to the similarity being higher than a preset similarity threshold.

[0087] According to a third aspect of the present invention, there is provided an electronic device, comprising:

[0088] A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any one of the pedestrian re-identification methods described in the first aspect is implemented.

[0089] According to a fourth aspect of the present invention, a readable storage medium is provided. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute any one of the pedestrian re-identification methods described in the first aspect.

[0090] According to the pedestrian re-identification method of the present invention, pedestrian re-identification is performed by learning high-order relationships with discriminative features and human topological information. This solves the technical problem that the existing pedestrian re-identification scheme is not robust and affects the accuracy of the recognition result. The beneficial effect of improving the accuracy of the pedestrian re-identification result is achieved.

[0091] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0092] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0093] Figure 1 It shows one of the flow charts of steps of a pedestrian re-identification method according to an embodiment of the present invention;

[0094] Figure 2 A schematic diagram of a conventional pedestrian re-identification process according to an embodiment of the present invention is shown;

[0095] Figure 3 A second flowchart of a method for pedestrian re-identification according to an embodiment of the present invention is shown;

[0096] Figure 4 A schematic diagram of an ADGC layer according to an embodiment of the present invention is shown;

[0097] Figure 5 A schematic diagram of a CGEA layer according to an embodiment of the present invention is shown;

[0098] Figure 6 A schematic structural diagram of a framework for pedestrian re-identification according to an embodiment of the present invention is shown; and

[0099] Figure 7 A schematic structural diagram of a pedestrian re-identification device according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0100] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0101] Reference Figure 1 , shows a flowchart of the steps of a pedestrian re-identification method in an embodiment of the present invention.

[0102] Step 110, obtaining a pair of portrait images to be identified.

[0103] Step 120, obtaining the first-order features of each of the portrait images, wherein the first-order features include the local features of each key point in the portrait image and the global features of the portrait image.

[0104] Step 130: Acquire relationship information of the portrait image based on the first-order features of the portrait image.

[0105] Step 140: Acquire the human body topology information of each of the portrait images based on the first-order features and the relationship information of each of the portrait images.

[0106] Step 150: Based on the relationship information and the human body topology information of each of the portrait images, a person re-identification result is obtained, where the person re-identification result includes whether the pair of portrait images contain the same portrait.

[0107] The portrait image may be any image containing a portrait, such as any video frame in a surveillance video of a camera, a surveillance image, an image obtained by any other available method, etc. The portrait image may be an image including at least a portrait, or may be an image containing only a portrait after processing, which is not limited in the embodiment of the present invention.

[0108] like Figure 2 As shown in the figure, people can be easily blocked by obstacles (such as luggage, counters, crowds, cars, trees), or by part of their body walking out of the camera's shooting area. Therefore, it is necessary to accurately match pedestrian images that are only partially observable, which is the so-called occluded pedestrian re-identification problem.

[0109] Compared with matching the whole person, ReID in occluded areas is more challenging for the following reasons: in occluded areas, the image contains less discriminative information, making it easier to match to the wrong person; matching based on feature information between body parts is effective, but it requires strict body alignment in advance, so it is not effective when occlusion is severe. Figure 2 In (a), we can see that key point information is affected by occlusion (key points 1, 2) and outliers (key point 3). For example, key points 1 and 2 are occluded, resulting in meaningless features; key point 3 is an outlier, resulting in deviation. For example, Figure 2 (b) shows a solution for person re-identification under common occlusion conditions. It extracts local features in the key point area and assumes that all key points are accurate and the local features are well aligned. At this time, the three stages of feature extraction (Feature Learning), alignment (Alignment Learning), and matching (Matching) all rely on the first-order information of each key point, and are not very robust.

[0110] In an embodiment of the present invention, in order to better solve the occlusion ReID problem. In addition to considering the first-order information of feature learning and alignment, high-order information is also considered. Specifically, in order to solve the occlusion problem in the pedestrian re-identification scenario, a new method is proposed in an embodiment of the present application to jointly model high-order relationships and human topology information. The high-order information may include but is not limited to relationship information.

[0111] Therefore, in an embodiment of the present invention, in the feature learning stage, a set of local features of an image is regarded as a node of a graph, that is, a key point, to learn relational information. By transmitting information in the graph, the meaningless feature problem caused by the occlusion of the key point can be improved by its adjacent meaningful features. Specifically, the first-order features of each of the portrait images are obtained, and the first-order features include the local features of each key point in the portrait image and the global features of the portrait image. Moreover, in an embodiment of the present invention, the key points of the portrait image can be identified by any available method, and the local features of each key point and the global features of the portrait image can be obtained, which is not limited by the embodiment of the present invention. For example, the first-order semantic features of the key point area can be extracted as its local features. This is done because many methods have proved that feature representation based on local parts of the body is effective for pedestrian re-identification; secondly, accurate alignment of local features is also necessary for ReID for occlusion or specific parts. For example, a convolutional neural network can be used to extract local features of different key points.

[0112] For example, given a portrait image, its feature map can be learned through a convolutional neural network (CNN), its key points can be learned through a pose estimator, and the feature vectors of all its key points can be learned through element-product operations and global average pooling (GP / g(.)) operations, that is, the local features of each key point.

[0113] It should be noted that although the prediction of human key points can achieve a very high accuracy, the performance of feature extraction is still poor due to occlusion and only part of the body in the picture, resulting in inaccurate position of the key points and their confidence. That is, although the first-order semantic information of different key point areas can be obtained by semantic feature extraction, this still cannot cope with the occluded ReID problem caused by incomplete pedestrian pictures. Therefore, in order to obtain more discriminative features, in an embodiment of the present invention, further modeling can be performed to obtain high-order relationship information. Specifically, the relationship information of the portrait image can be obtained based on the first-order features of the portrait image. In an embodiment of the present invention, the relationship information of the portrait image can be obtained by any available method, and this embodiment of the present invention is not limited. For example, high-order relationship information can be modeled by a graph convolutional network (GCN) method. In GCN, the semantic features of different key point areas can be regarded as nodes. By transferring information between nodes, both first-order semantic information (node ​​features) and high-order features (edge ​​features / relationship information) can be taken care of. Among them, the relationship information can be understood as the edge features that characterize the edges connecting any two key points in the portrait image.

[0114] After obtaining the first-order features and relationship information in each portrait picture, the human body topology information of each portrait picture can be further obtained based on the first-order features and the relationship information of each portrait picture. In an embodiment of the present invention, the human body topology information of the portrait picture can be obtained by any available method, and the embodiment of the present invention is not limited to this. Moreover, when constructing the human body topology information of each portrait picture, the alignment information between the two portrait pictures can also be referred to, and the useful information in one of the portrait pictures can be used to repair and enhance the other portrait picture, thereby improving the accuracy of the obtained human body topology information. Moreover, in an embodiment of the present invention, the alignment information between the two portrait pictures can be obtained by any available method, and the embodiment of the present invention is not limited to this. Among them, the alignment information between the two portrait pictures can be understood as the matching degree of the local features of each key point in the two pictures, the matching degree of the global features, and so on.

[0115] For example, in the alignment stage, any available algorithm such as graph matching can be used to learn robust alignment capabilities. In addition to aligning with point-to-point correspondences, this method can also model edge-to-edge correspondences. Then, by constructing a cross-image graph, the alignment information can be embedded into the features. Moreover, the information can be transferred by treating the learned alignment information as an adjacency matrix. In this way, the useful information in one portrait image can be transferred to enhance another portrait image, and the alignment information can be embedded in the features. Because of this, the features of abnormal key points can be repaired by their corresponding features in another image. The alignment information can include the key point-to-key point correspondence, edge-to-edge correspondence, and so on in the two portrait images.

[0116] For the two corresponding portrait images, the human body topology information of each portrait image can be obtained based on each enhanced portrait image, and then based on the human body topology information of each portrait image, it is detected whether the two portrait images contain the same portrait. Thus, the portrait images can be classified according to the portraits contained in the portrait images.

[0117] For example, for two mutually enhanced portrait images, the similarity of the human body topology information of the two images can be obtained by any available method. If the similarity of the human body topology information of the two images reaches a preset threshold, it can be determined that the two images contain the same portrait.

[0118] Moreover, in the embodiment of the present invention, the similarity between the human topology information of two portrait images can be obtained by any available method, which is not limited by the embodiment of the present invention. For example, under the supervision of verification loss, any available method such as a fully connected layer network can be used to learn the similarity between the human topology information of two portrait images that have been restored and enhanced to each other, so as to know whether each portrait image contains the same portrait.

[0119] Reference Figure 3 In the embodiment of the present invention, the relationship information of the portrait image can be obtained in the following manner:

[0120] Step S131, for each of the portrait pictures, based on the first-order features of the portrait picture, obtaining the relationship information of the portrait picture through a direction-adaptive graph convolution layer;

[0121] Wherein, the direction-adaptive graph convolution layer is configured as follows:

[0122] S1. Obtain an adaptive neighboring order matrix of the portrait picture based on a difference between the global feature and the local feature of the portrait picture.

[0123] S2, acquiring relationship information of the portrait image according to the adaptive neighboring order matrix, the local features and the global features.

[0124] As mentioned above, in order to obtain more discriminative features, high-order relational information can be modeled through the GCN network. Nevertheless, there is still a problem with occluded ReID, that is, the features of the occluded area are often meaningless or even noisy. When these features are transmitted in the graph, it may even bring more noise, which has side effects on the occluded ReID. Therefore, in an embodiment of the present invention, an adaptive graph convolution layer (Adaptive Directed Graph Convolutional, ADGC) is proposed to learn and transmit relational information. The ADGC layer can dynamically learn the direction and degree of information transmission. Therefore, the ADGC layer can promote the message transmission of semantic features and suppress meaningless and noisy messages. Finally, the output node contains both semantic information (that is, first-order features) and related information (that is, relational information).

[0125] The input of the ADGC layer includes a global feature V of the portrait image. g and K local features V l . You can use the local feature V l and the global feature V g The difference between them is used to dynamically predict the weight of each edge, that is, the adaptive adjacency matrix A adp In addition, in order to stabilize the training, the global feature V g and / or K local features V l It can also be integrated into the output of the ADGC layer. The output of the ADGC layer is the relationship information of the corresponding portrait image. The value of K is the number of key points in the corresponding portrait image. K is a positive integer, and the specific value of K can be customized according to the needs, which is not limited in the embodiment of the present invention. For example, K can be 13, 10, etc.

[0126] For example, the ADGC layer may include three fully connected layers, wherein the output data of each fully connected layer is concatenated to obtain the output of the ADGC layer. Moreover, the inputs of each fully connected layer are the global features V g , local features V l and the adaptive adjacency matrix A adp .

[0127] Optionally, in the embodiment of the present invention, step S1 may further include:

[0128] S11, constructing a K×K dimensional global feature matrix based on the global features, where the elements in the global feature matrix are global features; wherein K is the number of the key points.

[0129] S12, constructing a local feature matrix of K×K dimensions based on all the local features, where the elements in the global feature matrix are local features;

[0130] S13, performing element subtraction operation on the global feature matrix and the local feature matrix to obtain a first matrix;

[0131] S14, performing absolute value processing and batch normalization processing on the elements in the first matrix, and using the processed matrix as the input of the first fully connected layer, obtaining the output of the first fully connected layer as the adaptive neighbor-order matrix.

[0132] like Figure 4 The following is a schematic diagram of an ADGC layer. l in is a local feature, V g in is a global feature. Through any available method such as the repeat function, a K×K-dimensional global feature matrix can be constructed based on the global feature, and a K×K-dimensional local feature matrix can be constructed based on all the local features, where K is the number of key points, the elements in the global feature matrix are local features, and the elements in the global feature matrix are global features. Abs means absolute value operation, bn means batch normalization operation, fc means fully connected layer, represents element-wise subtraction, represents element-wise add, represents element-wise multiplication, Represents matrix multiplication, which can also be understood as outer product. adp It is the adaptive neighboring matrix.

[0133] like Figure 4As shown, after obtaining the global feature matrix and the local feature matrix, element subtraction operation can be further performed on the global feature matrix and the local feature matrix to obtain a first matrix; then, the elements in the first matrix are processed with absolute values ​​and batch normalization in turn, and the processed matrix is ​​used as the input of the first fully connected layer, and the output of the first fully connected layer is obtained as the adaptive neighbor-order matrix; wherein, K is the number of the key points.

[0134] Optionally, in the embodiment of the present invention, S2 may further include:

[0135] Step S21, performing a matrix multiplication operation on the adaptive neighboring matrix and the local features, and inputting the operation result into a second fully connected layer, and inputting the local features into a third fully connected layer;

[0136] Step S22: performing an element-wise addition operation on the output data of the second fully connected layer and the output data of the third fully connected layer, and obtaining the relationship information of the portrait image based on the operation result of the element-wise addition operation and the global features.

[0137] Correspondingly, after obtaining the current adaptive neighboring matrix, a matrix multiplication operation can be performed on the adaptive neighboring matrix and the local features, that is, the above-mentioned matrix multiplication operation, and the operation result is input into the second fully connected layer, and the local features are input into the third fully connected layer; then, an element addition operation is performed on the output data of the second fully connected layer and the output data of the third fully connected layer, and based on the operation result of the element addition operation and the global features, the relationship information of the portrait image is obtained. For example, the operation result of the element addition operation on the output data of the second fully connected layer and the output data of the third fully connected layer can be concatenated with the global features to obtain k+1-dimensional relationship information.

[0138] Optionally, in an embodiment of the present invention, the step S131 may further include: for each of the portrait pictures, based on the first-order features of the portrait picture and the original neighboring-order matrix, obtaining the relationship information of the portrait picture through the direction-adaptive graph convolution layer.

[0139] In addition, in the embodiment of the present invention, in order to improve the performance of the ADGC layer and further improve the accuracy of the re-identification result, an original neighboring order matrix is ​​also set to adjust the performance of the ADGC layer. The specific value of the original neighboring order matrix can be customized according to the needs, and the embodiment of the present invention is not limited to this.

[0140] Then at this time, step S2 may further include:

[0141] Step S23, performing element-wise multiplication operation on the adaptive neighboring matrix and the original neighboring matrix to obtain a first neighboring matrix.

[0142] Step S24, performing a matrix multiplication operation on the first neighboring matrix and the local feature, and inputting the operation result into the second fully connected layer, and inputting the local feature into the third fully connected layer.

[0143] Step S25, performing an element-wise addition operation on the output data of the second fully connected layer and the output data of the third fully connected layer, and obtaining the relationship information of the portrait image based on the operation result of the element-wise addition operation and the global features.

[0144] At this time, the original adjacency matrix exists in the ADGC layer as follows: Figure 4 As shown. Then at this time, when obtaining the relationship information, the adaptive neighboring matrix and the original neighboring matrix can be element-wise multiplied to obtain a first neighboring matrix. Then, a matrix multiplication operation is performed on the first neighboring matrix and the local features, and the operation result is input into the second fully connected layer, and the local features are input into the third fully connected layer. The output data of the second fully connected layer and the output data of the third fully connected layer are further element-wise added, and based on the operation result of the element-wise addition operation and the global features, the relationship information of the portrait image is obtained.

[0145] For example, given a portrait image x, a convolutional neural network (CNN) can be used to learn the feature map m of x. cnn (Feature Map), learn the key point heat map m of x through a pose estimator (Pose Estimator) kp , the feature vectors of all key points of x are learned through outer product operation and global average pooling operation. That is, a set of local features of key point areas (i.e. first-order semantic features) can be obtained. and a global feature

[0146]

[0147] like Figure 4 The ADGC layer shown in the figure takes as input a global feature V of the portrait image x. g and K local features V l , and the predefined original neighbor matrix A. The local feature V l and the global feature V g The difference between them is used to dynamically predict the weight of the edge corresponding to each key point in the portrait image x, that is, the adaptive adjacency matrix Aadp In order to stabilize the training, the input local feature V l It will be fused into the output of the ADGC layer. Then the local feature V l and the adaptive adjacency matrix A adp The multiplication operation between constructs a simple graph convolution ( Figure 4 At this time, the operation process of the ADGC layer is shown in the following formula:

[0148]

[0149] Among them, f1 and f2 are two non-shared fully connected layers.

[0150] At this point, given a portrait image, the above formula can be used Obtain its semantic features, and then obtain its relationship features, that is, the relationship information can be obtained through the following formula:

[0151] V R =fR(V S ) (3)

[0152] Among them, f R It can be understood as the operation process of the ADGC layer.

[0153] Moreover, assuming that two pedestrian images (x1, x2) are given, their similarity based on relationship information can be calculated by the following formula. K is the number of key points, β is the confidence of the corresponding human key points, and v is the feature of the corresponding key points after high-order relationship learning optimization, that is, the relationship information.

[0154]

[0155] Reference Figure 3 In the embodiment of the present invention, the human body topology information of each portrait image can be obtained by:

[0156] Step 141, based on the first-order features and the relationship information of each of the portrait images, the human body topology information of each of the portrait images is obtained through a cross-graph embedding alignment layer.

[0157] Wherein, the cross-graph embedding alignment layer is configured as follows:

[0158] T1, according to the first-order features and relationship information of the portrait images, obtaining a third-neighborhood matrix representing the alignment information of key points in the two portrait images through a graph matching strategy;

[0159] T2, according to the third neighboring-order matrix and the first-order features and relationship information of the portrait pictures, each portrait picture is used to enhance another portrait picture to obtain the human body topology information of each portrait picture.

[0160] Body part-based feature representation has been shown to be effective for person re-identification. A simple alignment strategy is to directly match features between the same key points. However, this first-order alignment strategy cannot handle outliers, especially when the person in the picture is occluded by a large area.

[0161] On the contrary, the graph matching (GM) strategy can naturally take into account the high-order information of human topology. However, it can only learn a one-to-one correspondence, which makes this hard alignment strategy still very sensitive to outliers and the performance is easily disturbed. To this end, a cross-graph embedding alignment (CGEA) layer is proposed in an embodiment of the present invention, which can not only make full use of the human topology information learned by the graph matching algorithm, but also avoid interference from the one-to-one alignment.

[0162] For the CGEA layer, it takes two portrait images containing their respective first-order features and relationship information as input, uses a graph matching strategy to learn the correspondence between the key points on the two portrait images, that is, the alignment information of the key points in the two portrait images, and transmits information by treating the learned alignment information as a third adjacency matrix. In this way, the useful information in one portrait image can be transferred to enhance another portrait image, and the alignment information can be embedded in the features of the portrait image, thereby improving the accuracy and comprehensiveness of the human topological structure of each portrait image finally obtained. Moreover, the CGEA layer can output embedded features, which may specifically include but are not limited to the semantic characteristics of each portrait image and the human topological information guided by the alignment information.

[0163] Optionally, in this embodiment of the present invention, step T1 may further include:

[0164] T11, inputting the first-order features and relationship information of each portrait image into a hidden space to obtain the hidden features of each portrait image;

[0165] T12, obtaining the correlation matrix between the hidden features of the two portrait images through a graph matching strategy, that is, the third-order matrix; wherein the hidden space contains at least one network structure layer, and the network structure layer is a cascade of a fully connected layer and a linear rectification function layer, or is a fully connected layer.

[0166] The step T2 may further include: obtaining the human body topology information of each of the portrait images according to the third neighboring matrix and the hidden features of each of the portrait images.

[0167] For example, the structure of the CGEA layer can be as follows Figure 5 First, given two portrait images x1 and x2, the first-order features and relationship information of x1 and x2 are used to and The first-order features and relationship information of x1 and the first-order features and relationship information of x2 can be input into a hidden space respectively to obtain two sets of hidden features: and Then, you can By formula U * = argmaxU T MU,st||U||=1 pair and Perform graph matching (GM) and get and The affinity matrix U k×k , as the third-order matrix representing the alignment information of key points in two portrait images x1 and x2. Among them, U(i, j) represents and The hidden space may contain at least one network structure layer, which is a cascade of a fully connected layer and a ReLU (Rectified Linear Unit) layer, or a fully connected layer. That is, ReLU is optional in the network structure layer. For a hidden space composed of N cascaded network structure layers, the number of ReLU layers is <= N, for example Figure 5 The hidden space in the CGEA layer shown includes a network structure layer, and the network structure layer is a cascade of a fully connected layer and a ReLU layer. Furthermore, the output of the CGEA layer can be obtained according to the third neighboring matrix and the hidden features of each portrait image, that is, the human body topology information of each portrait image. For example, Figure 5 The output of the CGEA layer shown can be obtained by referring to the following formula:

[0168]

[0169] Here, [·,·] represents the cascade operation according to the matrix dimension, and f represents the fully connected layer.

[0170] In addition, in the embodiment of the present invention, the output of the CGEA layer can also be two portrait images, wherein each portrait image contains its own human body topology information. Figure 5 In , U represents the association matrix obtained through training.

[0171] In the embodiment of the present invention, in order to quickly detect whether two portrait images contain the same portrait, a high-order human-topology module can be constructed, which includes a cascade of CGEA layers F T and a similarity prediction layer F P Given a pair of portrait images (x1, x2), their relationship features can be obtained through the above formula (3) Then their topological characteristics can be obtained by the following formula (6): That is, the human topological information mentioned above. Furthermore, the similarity between the topological features of two portrait images can be calculated by the following formula (7),

[0172]

[0173]

[0174] Among them, |·| represents the absolute value operation at the element level, f s is a fully connected layer, and σ is the sigmoid activation function.

[0175] Specifically, in order to solve the occlusion problem in pedestrian re-identification scenarios, a new framework is proposed to jointly model high-order relationship information and human topology information. Figure 6 As shown, the framework for performing person re-identification in the embodiment of the present invention may include three modules, a first-order semantic module (One-Order Semantic Module, S), a high-order relation module (High-Order Relation Module, R) and a high-order human topology module (T).

[0176] Among them, S can take the semantic features of the key point area of ​​the human body. In S, first, the CNN backbone can be used to learn the feature map, and the key point estimation model can be used to learn the key points. Then, the semantic information of the corresponding key points can be extracted, which is the first-order feature mentioned above; (2) R can model the relationship information between different semantic local features. In R, people regard the learned image semantic features as the key points of the graph, and then propose an adaptive-direction graph convolutional layer (ADGC) to learn and transmit edge feature information. The ADGC layer can automatically determine the direction and degree of each edge. Thereby promoting the information transmission of semantic features and suppressing the transmission of meaningless and noisy features. Finally, the learned nodes contain semantic and relational information. (3) T can learn robust alignment capabilities and predict the similarity between two images. In T, a cross-graph embedded-alignment (CGEA) layer is proposed. It takes two portrait images (graph) as input, uses a graph matching strategy to learn the corresponding relationship between the nodes, and then regards the learned corresponding relationship as an adjacency matrix to transmit information. Because of this, the associated features can be enhanced and the alignment information can be embedded in the features. Finally, in order to avoid the situation of forced one-to-one alignment, the similarity of the two portrait images can be predicted by mapping them to a logit model (rating model) and supervising them with a validation loss.

[0177] Reference Figure 3 In the embodiment of the present invention, step 150 may further include:

[0178] Step 151, based on the relationship information of each of the portrait pictures, obtaining a first similarity between two of the portrait pictures;

[0179] Step 152, based on the human body topology information of each of the portrait pictures, obtaining a second similarity between the two portrait pictures;

[0180] Step 153, obtaining a final similarity between the two portrait images according to the first similarity and the second similarity;

[0181] Step 154 ​​: in response to the similarity being higher than a preset similarity threshold, confirming that the pair of portrait images include the same portrait.

[0182] In an embodiment of the present invention, in order to improve the accuracy of the recognition result, the similarity between the two portrait images can be obtained by referring to the relationship information and the human topology information at the same time. Moreover, in an embodiment of the present invention, the first similarity and the second similarity can be obtained by any available method, which is not limited by the embodiment of the present invention. Moreover, the corresponding relationship between the first similarity, the second similarity and the final similarity can also be customized according to the needs, which is not limited by the embodiment of the present invention.

[0183] For example, for similarity calculation, given a pair of portrait images (x1, x2), the similarity can be calculated by formula (4): That is, the first similarity mentioned above, in order to obtain the relationship information between them, the similarity obtained by formula (7) That is, the second similarity mentioned above is used to obtain the human body topology information. When calculating the final similarity, these two similarities can be combined and calculated according to the following formula:

[0184]

[0185] Here, γ may be a preset weight, and the specific value may be customized according to the requirements, which is not limited in the embodiment of the present invention.

[0186] like Figure 6 As shown, during the training process of the above framework, a classification loss function, a triplet loss function, and a verification loss function can be used to supervise the training. For example, during the training process, the framework can be trained end-to-end by minimizing the loss function of the framework.

[0187] For example, for the first-order semantic module used to extract first-order features in the framework, its loss function can be set to include the classification loss L cls and triplet loss L tri , at this time, the loss function Ls of the first-order semantic module can be obtained by the following formula (9).

[0188]

[0189] Among them, β k =max(m kp [k])∈[0,1] is the confidence of the kth key point, and the confidence of the global feature is 1. It is a first-order feature The probability of the true feature value that matches the classifier's prediction, α is a redundant parameter, Represents a positive pair from the same portrait image, that is, a positive pair The distance between It is a pair of images from different portraits, where the classifiers of different local features are not shared.

[0190] For the high-order relation module in the framework used to extract high-order relation information, its loss function can also be set to include the classification loss L cls and triplet loss L tri , at this time the loss function L of the high-order relationship module R It can be obtained by the following formula (10).

[0191]

[0192] For the high-order human topology module in the framework used to extract human topology information, its loss function can be set as shown in the following formula:

[0193]

[0194] Among them, y represents the true value. If the portrait pictures x1 and x2 are portrait pictures of the same identity, that is, the portrait pictures x1 and x2 contain portraits of the same identity, then the value of y is 1, otherwise the value of y is 0.

[0195] In the inference phase, given a portrait image x_q to be queried, we can first obtain its similarity x with all candidate portrait images in the library by any available means. R , get the n pictures with the highest similarity, and then use formula (8) to calculate the final similarity s to correct the selection of these n portrait pictures.

[0196] Of course, it is also possible to directly select the target portrait image whose similarity satisfies the preset similarity threshold based on the final similarity obtained by this solution, or to obtain the N portrait images with the highest similarity to the target portrait image as the portrait images containing the same portrait as the target portrait image, and so on.

[0197] This paper proposes a new framework to learn high-order relations with discriminative features and human topological information. To learn relational information, local features of the image are represented as nodes of a graph, and an adaptive directional graph convolution (ADGC) layer is proposed to promote the information transfer of semantic features and suppress the information transfer of meaningless and noisy features.

[0198] For learning topological information, a cross-graph embedding alignment layer (CGEA) is proposed, which takes two graphs as input, uses a graph matching strategy to learn the correspondence between the nodes, and then regards the learned correspondence as an adjacency matrix to transfer information. It can avoid sensitive hard one-to-one alignment and perform robust soft alignment. It can effectively improve the accuracy of pedestrian re-identification results.

[0199] For the method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0200] Reference Figure 7 , shows a schematic structural diagram of a pedestrian re-identification device in an embodiment of the present invention.

[0201] A portrait image acquisition module 210 is used to acquire a pair of portrait images to be identified;

[0202] A first-order semantic module 220, configured to obtain first-order features of each of the portrait images, wherein the first-order features include local features of each key point in the portrait image and global features of the portrait image;

[0203] A high-order relationship module 230, configured to obtain relationship information of the portrait image based on the first-order features of the portrait image;

[0204] A high-order human body topology module 240 is used to obtain human body topology information of each of the portrait images based on the first-order features and the relationship information of each of the portrait images;

[0205] The portrait detection module 250 is used to obtain a person re-identification result based on the relationship information and the human body topology information of each portrait image, and the person re-identification result includes whether the pair of portrait images contain the same portrait.

[0206] Optionally, the high-order relationship module 230 is specifically configured to obtain, for each of the portrait pictures, relationship information of the portrait pictures based on the first-order features of the portrait pictures through a direction-adaptive graph convolution layer;

[0207] Wherein, the direction-adaptive graph convolution layer is configured as follows:

[0208] Based on the difference between the global feature and the local feature of the portrait picture, obtaining an adaptive neighboring order matrix of the portrait picture;

[0209] Relation information of the portrait picture is obtained according to the adaptive neighboring order matrix, the local features and the global features.

[0210] Optionally, the direction-adaptive graph convolution layer is specifically configured as follows:

[0211] Constructing a K×K dimensional global feature matrix based on the global features, where the elements in the global feature matrix are global features;

[0212] Constructing a local feature matrix of K×K dimensions based on all the local features, wherein the elements in the global feature matrix are local features;

[0213] Performing element-wise subtraction operation on the global feature matrix and the local feature matrix to obtain a first matrix;

[0214] Performing absolute value processing and batch normalization processing on the elements in the first matrix, and using the processed matrix as the input of the first fully connected layer, obtaining the output of the first fully connected layer as the adaptive neighbor-order matrix;

[0215] Wherein, K is the number of key points.

[0216] Optionally, the direction-adaptive graph convolution layer is further configured as follows:

[0217] Performing a matrix multiplication operation on the adaptive neighboring matrix and the local features, and inputting the operation result into a second fully connected layer, and simultaneously inputting the local features into a third fully connected layer;

[0218] An element-wise addition operation is performed on the output data of the second fully connected layer and the output data of the third fully connected layer, and relationship information of the portrait image is obtained based on a result of the element-wise addition operation and the global feature.

[0219] Optionally, the high-order relationship module 230 is further used to obtain the relationship information of each portrait picture through the direction-adaptive graph convolution layer based on the first-order features of the portrait picture and the original neighbor-order matrix;

[0220] Accordingly, the direction-adaptive graph convolution layer is further configured as follows:

[0221] Performing element-wise multiplication operation on the adaptive neighboring matrix and the original neighboring matrix to obtain a first neighboring matrix;

[0222] Performing a matrix multiplication operation on the first neighboring matrix and the local feature, and inputting the operation result into a second fully connected layer, and inputting the local feature into a third fully connected layer;

[0223] An element-wise addition operation is performed on the output data of the second fully connected layer and the output data of the third fully connected layer, and relationship information of the portrait image is obtained based on a result of the element-wise addition operation and the global feature.

[0224] Optionally, the high-order human topology module 240 is specifically used to obtain the human topology information of each of the portrait images through a cross-graph embedding alignment layer based on the first-order features and the relationship information of each of the portrait images;

[0225] Wherein, the cross-graph embedding alignment layer is configured as follows:

[0226] According to the first-order features and relationship information of the portrait images, a third-neighborhood-order matrix representing alignment information of key points in the two portrait images is obtained through a graph matching strategy;

[0227] According to the third neighboring-order matrix and the first-order features and relationship information of the portrait pictures, each portrait picture is used to enhance another portrait picture, so as to obtain the human body topology information of each portrait picture.

[0228] Optionally, the cross-graph embedding alignment layer is specifically configured as follows:

[0229] Inputting the first-order features and relationship information of each of the portrait images into a hidden space to obtain hidden features of each of the portrait images;

[0230] Obtaining a correlation matrix between the hidden features of the two portrait images through a graph matching strategy, which is the third neighbor-order matrix;

[0231] The latent space includes at least one network structure layer, and the network structure layer is a cascade of a fully connected layer and a linear rectification function layer, or is a fully connected layer.

[0232] Optionally, the cross-graph embedding alignment layer is further configured:

[0233] According to the third neighboring matrix and the hidden features of each of the portrait images, the human body topology information of each of the portrait images is obtained.

[0234] Optionally, the portrait detection module 250 includes:

[0235] A first similarity acquisition submodule, configured to acquire a first similarity between two portrait pictures based on the relationship information of each portrait picture;

[0236] A second similarity acquisition submodule, configured to acquire a second similarity between two portrait images based on the human body topology information of each portrait image;

[0237] a third similarity acquisition submodule, configured to acquire a final similarity between the two portrait images according to the first similarity and the second similarity;

[0238] The same portrait confirmation submodule is used to confirm that the pair of portrait images contain the same portrait in response to the similarity being higher than a preset similarity threshold.

[0239] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0240] An embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any one of the aforementioned pedestrian re-identification methods when executing the computer program.

[0241] In an embodiment of the present invention, a computer-readable storage medium is further provided, on which a computer program is stored, characterized in that when the program is executed by a processor, the steps of any one of the aforementioned pedestrian re-identification methods are implemented.

[0242] The algorithm and display provided herein are not inherently related to any particular computer, virtual system or other device. Various general purpose systems can also be used together with the teachings based on this. According to the above description, it is obvious that the structure required for constructing such systems. In addition, the present invention is not directed to any specific programming language either. It should be understood that various programming languages ​​can be utilized to realize the content of the present invention described herein, and the description of the above specific languages ​​is for disclosing the best mode of the present invention.

[0243] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0244] Similarly, it should be understood that in order to streamline the present disclosure and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting the intention that the claimed invention requires more features than those explicitly recited in each claim. More specifically, as reflected in the claims below, inventive aspects lie in less than all the features of the individual embodiments disclosed above. Therefore, the claims that follow the specific embodiment are hereby expressly incorporated into the specific embodiment, with each claim itself serving as a separate embodiment of the present invention.

[0245] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition they may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0246] In addition, those skilled in the art will appreciate that, although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present invention and form different embodiments. For example, in the claims below, any one of the claimed embodiments may be used in any combination.

[0247] The various component embodiments of the present invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that a microprocessor or digital signal processor (DSP) may be used in practice to implement some or all of the functions of some or all of the components of the device for pedestrian re-identification according to an embodiment of the present invention. The present invention may also be implemented as a device or apparatus program (e.g., a computer program and a computer program product) for executing part or all of the methods described herein. Such a program implementing the present invention may be stored on a computer-readable medium, or may be in the form of one or more signals. Such a signal may be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0248] It should be noted that the above embodiments illustrate the present invention rather than limit it, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be construed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising a number of different elements and by means of a suitably programmed computer. In a unit claim enumerating a number of devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc., does not indicate any order. These words may be interpreted as names.

Claims

1. A pedestrian re-identification method, characterized in that: include: Get a pair of portrait images to be identified; Acquire the first-order features of each of the portrait images, wherein the first-order features include the local features of each key point in the portrait image and the global features of the portrait image; Based on the first-order features of the portrait image, acquiring relationship information of the portrait image; wherein the relationship information is an edge feature of an edge connected between any two key points in the portrait image; Based on the first-order features and the relationship information of each of the portrait images, obtaining the human body topology information of each of the portrait images; Based on the relationship information and the human body topology information of each of the portrait images, a person re-identification result is obtained, where the person re-identification result includes whether the pair of portrait images contain the same portrait.

2. The method according to claim 1, characterized in that The step of acquiring the relationship information of the portrait picture based on the first-order features of the portrait picture comprises: For each of the portrait pictures, based on the first-order features of the portrait pictures, obtaining the relationship information of the portrait pictures through a direction-adaptive graph convolution layer; Wherein, the direction-adaptive graph convolution layer is configured as follows: Based on the difference between the global feature and the local feature of the portrait picture, obtaining an adaptive neighboring order matrix of the portrait picture; Relation information of the portrait picture is obtained according to the adaptive neighboring order matrix, the local features and the global features.

3. The method according to claim 2, characterized in that The step of obtaining an adaptive neighboring order matrix of the portrait picture based on the difference between the global feature and the local feature of the portrait picture comprises: Constructing a K×K dimensional global feature matrix based on the global features, where the elements in the global feature matrix are global features; Constructing a local feature matrix of K×K dimensions based on all the local features, wherein the elements in the global feature matrix are local features; Performing element-wise subtraction operation on the global feature matrix and the local feature matrix to obtain a first matrix; Performing absolute value processing and batch normalization processing on the elements in the first matrix, and using the processed matrix as the input of the first fully connected layer, obtaining the output of the first fully connected layer as the adaptive neighbor-order matrix; Wherein, K is the number of key points.

4. The method according to claim 3, characterized in that The step of acquiring the relationship information of the portrait picture according to the adaptive neighboring order matrix, the local features and the total features comprises: Performing a matrix multiplication operation on the adaptive neighboring matrix and the local features, and inputting the operation result into a second fully connected layer, and simultaneously inputting the local features into a third fully connected layer; An element-wise addition operation is performed on the output data of the second fully connected layer and the output data of the third fully connected layer, and relationship information of the portrait image is obtained based on a result of the element-wise addition operation and the global feature.

5. The method according to claim 2, characterized in that: The step of obtaining the relationship information of each portrait picture based on the first-order features of the portrait picture through a direction-adaptive graph convolution layer comprises: For each of the portrait pictures, based on the first-order features of the portrait picture and the original neighbor-order matrix, the relationship information of the portrait picture is obtained through the direction-adaptive graph convolution layer; The step of acquiring the relationship information of the portrait picture according to the adaptive neighboring order matrix, the local features and the global features comprises: Performing element-wise multiplication operation on the adaptive neighboring matrix and the original neighboring matrix to obtain a first neighboring matrix; Performing a matrix multiplication operation on the first neighboring matrix and the local feature, and inputting the operation result into a second fully connected layer, and inputting the local feature into a third fully connected layer; An element-wise addition operation is performed on the output data of the second fully connected layer and the output data of the third fully connected layer, and relationship information of the portrait image is obtained based on a result of the element-wise addition operation and the global feature.

6. The method according to any one of claims 1 to 5, characterized in that The step of acquiring the human body topology information of each portrait image based on the first-order features and the relationship information of each portrait image comprises: Based on the first-order features and the relationship information of each of the portrait images, the human body topology information of each of the portrait images is acquired through a cross-graph embedding alignment layer; Wherein, the cross-graph embedding alignment layer is configured as follows: According to the first-order features and relationship information of the portrait images, a third-neighborhood-order matrix representing alignment information of key points in the two portrait images is obtained through a graph matching strategy; According to the third neighboring-order matrix and the first-order features and relationship information of the portrait pictures, each portrait picture is used to enhance another portrait picture, so as to obtain the human body topology information of each portrait picture.

7. The method according to claim 6, characterized in that The step of obtaining a third neighboring-order matrix representing alignment information of key points in two portrait images through a graph matching strategy according to the first-order features and relationship information of the portrait images comprises: Inputting the first-order features and relationship information of each of the portrait images into a hidden space to obtain hidden features of each of the portrait images; Obtaining a correlation matrix between the hidden features of the two portrait images through a graph matching strategy as the third neighboring order matrix; The latent space includes at least one network structure layer, and the network structure layer is a cascade of a fully connected layer and a linear rectification function layer, or is a fully connected layer.

8. The method according to claim 7, characterized in that The step of enhancing another portrait picture by each portrait picture according to the third neighboring-order matrix and the first-order features and relationship information of the portrait pictures to obtain the human body topology information of each portrait picture comprises: According to the third neighboring matrix and the hidden features of each of the portrait images, the human body topology information of each of the portrait images is obtained.

9. The method according to claim 1, characterized in that: The step of obtaining a person re-identification result based on the relationship information and the human body topology information of each of the portrait images comprises: Based on the relationship information of each of the portrait pictures, obtaining a first similarity between two of the portrait pictures; Based on the human body topology information of each of the portrait pictures, obtaining a second similarity between the two portrait pictures; Acquire a final similarity between the two portrait images according to the first similarity and the second similarity; In response to the similarity being higher than a preset similarity threshold, it is confirmed that the pair of portrait images include the same portrait.

10. A pedestrian re-identification device, characterized in that: include: A portrait image acquisition module is used to acquire a pair of portrait images to be identified; A first-order semantic module, used to obtain the first-order features of each of the portrait images, wherein the first-order features include the local features of each key point in the portrait image and the global features of the portrait image; A high-order relationship module, used to obtain relationship information of the portrait image based on the first-order features of the portrait image; wherein the relationship information is an edge feature of an edge connected between any two key points in the portrait image; A high-order human body topology module, used for acquiring human body topology information of each of the portrait images based on the first-order features and the relationship information of each of the portrait images; The portrait detection module is used to obtain a person re-identification result based on the relationship information and the human body topology information of each portrait image, wherein the person re-identification result includes whether the pair of portrait images contain the same portrait.

11. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the pedestrian re-identification method according to any one of claims 1 to 9 when executing the computer program.

12. A readable storage medium, characterized in that: When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the pedestrian re-identification method as described in any one of claims 1 to 9.

Images