Pedestrian re-identification method and device, electronic equipment and storage medium

Abstract

The application provides a pedestrian re-identification method and device, electronic equipment and a storage medium. The method comprises: determining a to-be-identified image; performing spatial relationship identification on the head-shoulder feature of the to-be-identified image to obtain an identification feature, the identification feature being used to represent head-shoulder information and the spatial relationship of the head-shoulder feature; and determining a pedestrian re-identification result based on the identification feature. The method, device, electronic equipment and storage medium provided by the application perform spatial relationship identification on the head-shoulder feature of the to-be-identified image to obtain an identification feature used to represent head-shoulder information and the spatial relationship thereof. Since the head-shoulder region of a pedestrian is not easily occluded and the head-shoulder feature can better represent a pedestrian, the accuracy of pedestrian re-identification is improved. The identification feature not only covers the head-shoulder feature but also covers the spatial relationship of the head-shoulder feature, thereby improving the representation ability of the identification feature for the head-shoulder part, making the identification feature more discriminative, and further improving the accuracy of pedestrian re-identification.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a pedestrian re-identification method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the rapid development of artificial intelligence, the application of person re-identification is becoming increasingly widespread. Person re-identification, also known as person re-identification, is a technique that uses computer vision technology to determine whether a specific pedestrian exists in an image or video sequence.

[0003] Currently, pedestrian re-identification tasks are mostly performed using pedestrian re-identification models. However, in some special scenarios, such as pedestrians' torsos being partially obscured due to crowding, pedestrians changing clothes, or different pedestrians wearing similar clothing, existing pedestrian re-identification models cannot accurately identify them. Summary of the Invention

[0004] This invention provides a pedestrian re-identification method, apparatus, electronic device, and storage medium to address the shortcomings of low accuracy in existing pedestrian re-identification technologies.

[0005] This invention provides a pedestrian re-identification method, comprising:

[0006] Identify the image to be recognized;

[0007] Spatial relationship recognition is performed on the head and shoulder features of the image to be recognized to obtain recognition features, which are used to characterize the spatial relationship between head and shoulder information and the head and shoulder features;

[0008] Based on the identified features, the pedestrian re-identification result is determined.

[0009] According to a pedestrian re-identification method provided by the present invention, the step of performing spatial relationship identification on the head and shoulder features of the image to be identified to obtain identification features includes:

[0010] The head and shoulder features are divided into multiple local features;

[0011] Based on the similarity between every two local features among the plurality of local features, the spatial relationship between every two local features is determined;

[0012] Based on the spatial relationship between each pair of local features, graph convolution is performed on each local feature to obtain the recognition features.

[0013] According to a pedestrian re-identification method provided by the present invention, the head and shoulder features are determined based on the following steps:

[0014] The head and shoulder positions are obtained by detecting the global features of the image to be identified.

[0015] Based on the head and shoulder position, the head and shoulder features in the global features are determined.

[0016] According to a pedestrian re-identification method provided by the present invention, the method further includes determining the head and shoulder features in the global features based on the head and shoulder position, and then further includes:

[0017] Determine the two-dimensional positional offset of the head and shoulder features;

[0018] The head and shoulder features are updated based on the two-dimensional positional offset.

[0019] According to a pedestrian re-identification method provided by the present invention, determining the head and shoulder features in the global features based on the head and shoulder position includes:

[0020] Based on the head and shoulder position, determine the region of interest;

[0021] Identify the features of interest that belong to the region of interest from the global features;

[0022] The features of interest are pooled to obtain the head and shoulder features.

[0023] According to a pedestrian re-identification method provided by the present invention, the head and shoulder features are determined based on the global features of the image to be identified, and the global features are determined based on the following steps:

[0024] Global feature extraction is performed on the image to be identified to obtain global image features;

[0025] Determine the positional relationship between every two locations in the global features of the image;

[0026] The positional relationships are fused with the features of each position in the global features of the image to obtain the global features.

[0027] According to the pedestrian re-identification method provided by the present invention, the identification features are obtained by identifying the image to be identified based on an identification model, and the identification model is trained based on the following steps:

[0028] Acquire a sample recognition image, a positive sample image corresponding to the sample recognition image, and a negative sample image corresponding to the sample recognition image;

[0029] The sample recognition image is input into the first model to be trained to obtain the first sample recognition feature output by the first model to be trained.

[0030] The positive sample image is input into the second model to be trained to obtain the second sample recognition feature output by the second model to be trained.

[0031] The negative sample image is input into the third model to be trained to obtain the third sample recognition features output by the third model to be trained;

[0032] The loss function is determined based on the similarity between the first sample identification feature and the second sample identification feature, and the similarity between the first sample identification feature and the third sample identification feature.

[0033] Based on the loss function, the first model to be trained, the second model to be trained, and the third model to be trained are trained, and the trained first model to be trained is determined as the recognition model.

[0034] The present invention also provides a pedestrian re-identification device, comprising:

[0035] Image determination module, used to determine the image to be recognized;

[0036] The relationship recognition module is used to perform spatial relationship recognition on the head and shoulder features of the image to be recognized, and obtain recognition features, which are used to characterize the spatial relationship between the head and shoulder information and the head and shoulder features;

[0037] The result determination module is used to determine the pedestrian re-identification result based on the identified features.

[0038] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the pedestrian re-identification method as described above.

[0039] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the pedestrian re-identification method as described above.

[0040] The pedestrian re-identification method, apparatus, electronic device, and storage medium provided by this invention perform spatial relationship recognition on the head and shoulder features of the image to be identified, obtaining recognition features used to characterize head and shoulder information and their spatial relationships. Since the head and shoulder area of ​​a pedestrian is not easily occluded, and head and shoulder features can better characterize a pedestrian, the accuracy of pedestrian re-identification is improved. Furthermore, the recognition features not only cover head and shoulder features but also their spatial relationships, thereby enhancing the characterization ability of the recognition features for the head and shoulder area, making the recognition features more discriminative, and further improving the accuracy of pedestrian re-identification. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0042] Figure 1 This is one of the flowcharts illustrating the pedestrian re-identification method provided by the present invention;

[0043] Figure 2 The second flowchart illustrates the pedestrian re-identification method provided by this invention.

[0044] Figure 3 The third flowchart illustrating the pedestrian re-identification method provided by the present invention;

[0045] Figure 4 The fourth flowchart illustrating the pedestrian re-identification method provided by this invention;

[0046] Figure 5 Fifth flowchart of the pedestrian re-identification method provided by the present invention;

[0047] Figure 6 A schematic diagram illustrating the model training process provided by this invention;

[0048] Figure 7 This is a schematic diagram of the pedestrian re-identification device provided by the present invention;

[0049] Figure 8 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0051] With the rapid development of artificial intelligence, the application of person re-identification is becoming increasingly widespread. Person re-identification, also known as person re-identification, is a technique that uses computer vision technology to determine whether a specific pedestrian exists in an image or video sequence.

[0052] Currently, most pedestrian re-identification tasks rely on pedestrian re-identification models, which achieve good performance on some publicly available pedestrian recognition datasets with relatively ideal environments. However, in some special scenarios, such as pedestrians' torsos being obscured due to crowding, pedestrians changing clothes, or different pedestrians wearing similar clothing, existing pedestrian re-identification models cannot accurately identify them.

[0053] Based on the above, considering that the head and shoulders region of a pedestrian is less likely to be obscured compared to other parts of the pedestrian's body, and that the appearance information contained in the head and shoulders, such as the pedestrian's hairstyle and head-to-shoulder ratio, can effectively characterize the pedestrian. However, simply locating and cropping the head and shoulders region and directly extracting its features from the cropped image does not result in high accuracy for pedestrian re-identification.

[0054] To address the above problems, the present invention proposes the following embodiments. Figure 1 This is one of the flowcharts illustrating the pedestrian re-identification method provided by the present invention, such as... Figure 1 As shown, the pedestrian re-identification method includes:

[0055] Step 110: Determine the image to be recognized.

[0056] Here, the image to be identified is the image for which pedestrian re-identification is required. This image includes pedestrians; the number of pedestrians can be one or more, and the torso of the pedestrians may be occluded. Alternatively, the image may not include pedestrians. The image to be identified can be an image from a set of images to be identified, or it can be an image frame from a video.

[0057] In one specific embodiment, there are multiple images to be identified, so that the pedestrian re-identification result can be determined based on the recognition features of the multiple images to be identified.

[0058] Step 120: Spatial relationship recognition is performed on the head and shoulder features of the image to be recognized to obtain recognition features, which are used to characterize the spatial relationship between the head and shoulder information and the head and shoulder features.

[0059] Here, head and shoulder features are used to characterize the head and shoulder information of pedestrians, that is, to characterize the head and shoulder information of pedestrians.

[0060] Here, the identification features encompass head and shoulder features, as well as the spatial relationships between these features, thereby enhancing the ability of the identification features to represent the head and shoulder area and thus improving the accuracy of pedestrian re-identification. These spatial relationships contain rich contextual information about the head and shoulders, such as the proportions of the head and shoulders.

[0061] Specifically, spatial relationship recognition is performed on the head and shoulder features of the image to be recognized to obtain the spatial relationship of the head and shoulder features. Based on the spatial relationship, the head and shoulder features are aggregated so that the features at each position in the head and shoulder features cover the spatial relationship, thereby improving the representational ability of the recognition features, making them more discriminative, and thus improving the accuracy of pedestrian re-identification.

[0062] In one embodiment, the head and shoulder features are segmented to obtain multiple local features; the spatial relationship between every two local features is determined; and based on the spatial relationship between every two local features, each local feature is aggregated to obtain the recognition features.

[0063] Step 130: Based on the identified features, determine the pedestrian re-identification result.

[0064] Specifically, the pedestrian re-identification result is determined based on the similarity between multiple recognition features. These multiple recognition features can be features corresponding to multiple images to be identified, or they can include a first target recognition feature corresponding to a single image to be identified, and a second target recognition feature corresponding to a known pedestrian image in a known image database. The known pedestrian image is an image whose target pedestrian is known to belong to.

[0065] In some embodiments, the similarity between a first target recognition feature and each second target recognition feature is calculated, and a pedestrian re-identification result is determined based on each similarity. In one embodiment, the maximum similarity among the various similarities is determined. If the maximum similarity is greater than a preset threshold, the known pedestrian image corresponding to the maximum similarity is used as the recognition result of the image to be identified, thereby determining which person the known pedestrian image belongs to, and thus determining the pedestrian re-identification result. In another embodiment, the maximum similarity among the various similarities is determined, and the known pedestrian image corresponding to the maximum similarity is used as the recognition result of the image to be identified, thereby determining which person the known pedestrian image belongs to, and thus determining the pedestrian re-identification result. In yet another embodiment, a target similarity among the various similarities that is greater than a preset threshold is determined, and the known pedestrian image corresponding to the target similarity is used as the recognition result of the image to be identified, thereby determining which person the known pedestrian image belongs to, and thus determining the pedestrian re-identification result. The preset threshold can be set according to actual needs, and this embodiment of the invention does not specifically limit it. The pedestrian re-identification method provided in this invention performs spatial relationship recognition on the head and shoulder features of the image to be identified, obtaining recognition features that characterize head and shoulder information and their spatial relationships. Since the head and shoulder region of a pedestrian is not easily occluded, and head and shoulder features can better represent a pedestrian, the accuracy of pedestrian re-identification is improved. Furthermore, the recognition features not only cover head and shoulder features but also their spatial relationships, thereby enhancing the representational ability of the recognition features for the head and shoulder area, making the recognition features more discriminative, and further improving the accuracy of pedestrian re-identification.

[0066] Based on the above embodiments, Figure 2 This is a second flowchart illustrating the pedestrian re-identification method provided by the present invention, as shown below. Figure 2 As shown, step 120 above includes:

[0067] Step 121: Divide the head and shoulder features to obtain multiple local features.

[0068] Here, the local feature refers to the features of a portion of the head and shoulder features. The number of these local features can be set according to actual needs, and this embodiment of the invention does not impose a specific limitation on this.

[0069] In one specific embodiment, the head and shoulder features are divided into multiple local features based on a preset number of divisions and a preset division ratio. The preset number of divisions determines the number of local features; for example, if the preset number of divisions is N, then the head and shoulder features are divided into N local features. The preset division ratio determines the size of each local feature; for example, it can be an average division, but this embodiment of the invention does not specifically limit this.

[0070] Step 122: Determine the spatial relationship between each pair of local features based on the similarity between each pair of local features.

[0071] Here, the similarity calculation method can be set as needed, such as the first norm calculation method (Manhattan distance), the second norm calculation method (Euclidean distance), the Minkowski distance calculation method, the Chebyshev distance calculation method, etc.

[0072] Specifically, the similarity between any two local features can be directly used to determine the spatial relationship between them, or further data processing can be performed on the similarity between any two local features to obtain the spatial relationship. Specifically, a spatial relationship can be determined based on the similarity between any two local features.

[0073] In one embodiment, the formula for calculating the spatial relationship between every two local features is as follows:

[0074]

[0075] In the formula, A i,j Let represent the spatial relationship between the i-th local feature and the j-th local feature, where e is the natural constant and x is the x-th local feature. i Let x represent the i-th local feature. j Let x represent the j-th local feature. i -x j ||2 represents the i-th local feature x i With the j-th local feature x j Euclidean distance.

[0076] Step 123: Based on the spatial relationship between every two local features, perform graph convolution processing on each local feature to obtain the recognition features.

[0077] Here, the identified features are determined based on local features obtained after multiple graph convolution processes.

[0078] Specifically, a graph network of head and shoulder features is constructed, where the nodes of the graph network are local features and the edges of the graph network represent the spatial relationship between two local features. Based on the graph convolutional network layer, graph convolution processing is performed on the graph network to obtain the recognition features.

[0079] The graph convolutional network layer can aggregate each local feature, that is, aggregate each local feature with the local features of its neighbors. Understandably, the graph convolutional network layer can dynamically embed information from all nodes of the graph network into each node. Based on this, the spatial relationship between each local feature and all other local features can be adaptively embedded, thereby improving the representational ability of the recognition features for the head and shoulder region, making the recognition features more discriminative, and further improving the accuracy of pedestrian re-identification.

[0080] The number of layers in the convolutional network shown in this figure can be set according to actual needs, and the embodiments of the present invention do not impose specific limitations on this.

[0081] In one embodiment, the dynamic embedding formula for each convolutional layer of the graph convolutional network is as follows:

[0082]

[0083] In the formula, Represents the i-th local feature x i In the output of layer l, α represents the activation function. Represents the i-th local feature x i The output of the (l-1)th layer, where N is the number of local features, A i,j Represents the i-th local feature x i With the j-th local feature x j Spatial relationships, F l This represents the network parameters of layer l. Represents the j-th local feature x j The output at layer (l-1). If l = 1, then For the i-th local feature x i , Represents the j-th local feature x j .

[0084] The pedestrian re-identification method provided in this invention divides the head and shoulder features of the image to be identified into multiple local features. Then, based on the similarity between every two local features, the spatial relationship between each pair of local features is determined, providing support for determining the spatial relationship of the head and shoulder features. Furthermore, based on the spatial relationship between every pair of local features, graph convolution processing is performed on each local feature. This allows for the adaptive embedding of its spatial relationship with all local features within each local feature, thereby improving the ability of the identification features to represent the head and shoulder area, making the identification features more discriminative, and further improving the accuracy of pedestrian re-identification.

[0085] Based on any of the above embodiments Figure 3 This is the third flowchart illustrating the pedestrian re-identification method provided by the present invention, as shown below. Figure 3 As shown, the head and shoulder features are determined based on the following steps:

[0086] Step 310: Detect the head and shoulder positions by performing head and shoulder position detection on the global features of the image to be identified.

[0087] Here, the head and shoulder position is used to characterize the location of the head and shoulder portion in the image to be identified, and this head and shoulder position can be represented by coordinate values.

[0088] In one specific embodiment, based on the head and shoulder position detection model, the global features of the image to be recognized are used to detect the head and shoulder position to obtain the head and shoulder position; that is, the global features are input into the head and shoulder position detection model to obtain the head and shoulder position output by the head and shoulder position detection model.

[0089] The head and shoulder position detection model can be constructed based on a convolutional neural network to regress the head and shoulder positions. The head and shoulder position detection model is trained based on a first loss function. This first loss function can be set according to actual needs, such as the L2 loss function; however, this embodiment of the invention does not impose a specific limitation on it.

[0090] Step 320: Based on the head and shoulder position, determine the head and shoulder features in the global features.

[0091] Specifically, based on the head and shoulder position, the head and shoulder features corresponding to that position are determined in the global features, that is, the head and shoulder features are located from the global features.

[0092] The pedestrian re-identification method provided in this embodiment of the invention performs head and shoulder position detection on the global features of the image to be identified, and then determines the head and shoulder features in the global features based on the head and shoulder position, so as to pay more attention to the head and shoulder features. Since the head and shoulder area of ​​pedestrians is not easily occluded, and the head and shoulder features can better represent pedestrians, the accuracy of pedestrian re-identification is improved.

[0093] Based on any of the above embodiments, considering that head and shoulder position detection may be inaccurate, or that the images to be identified for similarity comparison may be misaligned, resulting in the inability to accurately determine the similarity between the images to be identified, and thus the inability to accurately obtain pedestrian re-identification results, this method further includes the following after step 320:

[0094] Determine the two-dimensional positional offset of the head and shoulder features;

[0095] The head and shoulder features are updated based on the two-dimensional positional offset.

[0096] Here, the two-dimensional positional offset is used to characterize the offset of each feature value in the head-shoulder feature along the horizontal axis and the offset of each feature value in the head-shoulder feature along the vertical axis. For example, the two-dimensional positional offset is Δp. n Δp n =[p x p y ]; where p x p represents the offset of the eigenvalue along the horizontal axis, i.e., the offset in the horizontal direction. y This represents the offset of the eigenvalue along the ordinate direction, i.e., the offset in the vertical direction.

[0097] In some embodiments, the two-dimensional positional offset of the head and shoulder features is determined based on the head and shoulder features.

[0098] In one specific embodiment, a two-dimensional positional offset detection model is used to detect the two-dimensional positional offset of the head and shoulder features, thereby obtaining the two-dimensional positional offset of the head and shoulder features. That is, the head and shoulder features are input into the two-dimensional positional offset detection model to obtain the two-dimensional positional offset output by the model.

[0099] The two-dimensional positional offset detection model is trained based on a second loss function. This second loss function can be the loss function of the recognition model. Further, this second loss function is a triplet loss function. Specifically, the two-dimensional positional offset detection model is trained based on the following steps: acquiring a sample recognition image, a corresponding positive sample image, and a corresponding negative sample image; inputting the sample recognition image into a first model to be trained to obtain the first sample recognition feature output by the first model to be trained; inputting the positive sample image into a second model to be trained to obtain the second sample recognition feature output by the second model to be trained; inputting the negative sample image into a third model to be trained to obtain the third sample recognition feature output by the third model to be trained; determining the second loss function based on the similarity between the first and second sample recognition features, and the similarity between the first and third sample recognition features; and training the first, second, and third models to be trained based on the second loss function. Each of the first, second, and third models to be trained includes a two-dimensional positional offset detection model.

[0100] The structure of this two-dimensional positional offset detection model can be customized according to actual needs. For example, the model includes convolutional layers and a detection layer. Head and shoulder features are input into the convolutional layer to obtain a feature vector. This feature vector is then input into the detection layer to obtain the two-dimensional positional offset output by the detection layer. The kernel size of the convolutional layer can be set to 1×1, and the stride can be set to 1. In other words, the convolutional layer learns each feature value from the head and shoulder features, and then the detection layer detects each learned feature value to obtain the two-dimensional positional offset.

[0101] In other embodiments, the two-dimensional position offset can be preset.

[0102] Here, the position of each feature value in the updated head and shoulder features is the updated position. The updated position can be the sum of the original position and the two-dimensional position offset.

[0103] In one embodiment, the updated target position of any feature value is determined based on the following steps: adding the original position of any feature value to the two-dimensional position offset of any feature value to obtain the target position of any feature value. For example, the two-dimensional position offset is Δp. n The original position is p n The target position is p′ n Then p′ n =p n +Δp n .

[0104] Furthermore, considering that the two-dimensional positional offset may have small values, the feature values ​​of the updated positions are determined using bilinear interpolation. Specifically, based on the two-dimensional positional offset, the position of each feature value in the head and shoulder features is updated. Based on each feature value (original feature) in the head and shoulder features, bilinear interpolation is used to determine the new feature value of each updated position, thereby obtaining the new head and shoulder features.

[0105] It should be noted that each feature value of the original head and shoulder features corresponds to a fixed size and position in the original image (the image to be identified). After spatial transformation, the feature at each position is obtained by interpolating multiple different original features. Therefore, the original image position corresponding to it is of arbitrary size and shape, which makes head and shoulder localization more flexible and further improves the accuracy of pedestrian re-identification.

[0106] The pedestrian re-identification method provided in this invention updates head and shoulder features based on two-dimensional positional offset, thereby enabling spatial alignment of head and shoulder features in different images. This makes the determination of similarity between different images more accurate, leading to more accurate determination of subsequent pedestrian re-identification results and ultimately improving the accuracy of pedestrian re-identification.

[0107] Based on any of the above embodiments, in this method, step 320 includes:

[0108] Based on the head and shoulder position, determine the region of interest;

[0109] Identify the features of interest that belong to the region of interest from the global features;

[0110] The features of interest are pooled to obtain the head and shoulder features.

[0111] Specifically, based on the head and shoulder position, the region of interest (ROI) corresponding to that head and shoulder position is determined in the global features; the ROI is projected onto the global features to obtain the ROI features; the ROI features are then subjected to max pooling or average pooling to obtain the head and shoulder features.

[0112] Furthermore, the features of interest are divided into multiple local features, and pooling is performed on each of these local features to obtain head and shoulder features.

[0113] The pedestrian re-identification method provided in this invention determines the region of interest (ROI) based on head and shoulder positions, and then performs pooling processing on ROI-related features within the global features, thereby improving the efficiency of model training. Simultaneously, it supports the determination of head and shoulder features within the global features, giving greater attention to these features. Since the head and shoulder region of a pedestrian is less prone to occlusion and these features can better represent a pedestrian, the accuracy of pedestrian re-identification is improved.

[0114] Based on any of the above embodiments Figure 4 This is the fourth flowchart illustrating the pedestrian re-identification method provided by the present invention, as shown below. Figure 4 As shown, the head and shoulder features are determined based on the global features of the image to be identified, and the global features are determined based on the following steps:

[0115] Step 410: Perform global feature extraction on the image to be identified to obtain global image features.

[0116] Specifically, based on the feature extraction layer, global features are extracted from the image to be recognized to obtain global image features. This feature extraction layer can be configured according to actual needs, for example, using a ResNet50 network.

[0117] Step 420: Determine the positional relationship between every two positions in the global features of the image.

[0118] Specifically, based on the feature relation model, the positional relationship between every two positions in the global features of the image is extracted. That is, the global features of the image are input into the feature relation model, and the positional relationship between every two positions is obtained from the output of the feature relation model.

[0119] The feature relationship model is trained based on a second loss function. This second loss function can be the loss function of the recognition model. Further, the second loss function is a triplet loss function. Specifically, the feature relationship model is trained based on the following steps: acquiring a sample recognition image, a corresponding positive sample image, and a corresponding negative sample image; inputting the sample recognition image into a first training model to obtain the first sample recognition feature output by the first training model; inputting the positive sample image into a second training model to obtain the second sample recognition feature output by the second training model; inputting the negative sample image into a third training model to obtain the third sample recognition feature output by the third training model; determining the second loss function based on the similarity between the first and second sample recognition features, and the similarity between the first and third sample recognition features; and training the first, second, and third training models based on the second loss function. Each of the first, second, and third training models includes a feature relationship model.

[0120] Step 430: The positional relationship is fused with the features of each position in the global features of the image to obtain the global features.

[0121] Specifically, the positional relationship between each pair of positions (the global positional relationship) is embedded into each position in the global features of the image.

[0122] The pedestrian re-identification method provided in this invention fuses the positional relationship between every two locations in the global image features with the features of each location in the global features. This allows the resulting global features to represent not only global image information but also the spatial relationships of global image features, thereby improving the discriminative power and representational ability of the global features, and ultimately enhancing the accuracy of pedestrian re-identification. Simultaneously, since the features at each location are fused with global positional relationships, global information can be obtained at each location, thus improving the global representation of the features and further enhancing the accuracy of pedestrian re-identification.

[0123] Based on any of the above embodiments Figure 5 This is the fifth flowchart illustrating the pedestrian re-identification method provided by the present invention, as shown below. Figure 5 As shown, the recognition features are obtained by recognizing the image to be recognized based on a recognition model, which is trained based on the following steps:

[0124] Step 510: Obtain the sample recognition image, the positive sample image corresponding to the sample recognition image, and the negative sample image corresponding to the sample recognition image.

[0125] Here, the sample recognition image is the training sample corresponding to the image to be recognized. A positive sample image is an image belonging to the same category as the sample recognition image; that is, the person in the positive sample image is the same person as the person in the sample recognition image. A negative sample image is an image not belonging to the same category as the sample recognition image; that is, the person in the negative sample image is not the same person as the person in the sample recognition image. In other words, a tripartite image set is obtained, which includes the sample recognition image, the positive sample image, and the negative sample image.

[0126] The sample recognition image, positive sample image, and negative sample image form a training dataset, which can be divided into a training set and a test set for model training and model testing, respectively.

[0127] In one embodiment, the sample recognition image is an anchor point image, so that spatial relationships can be better identified and spatial alignment can be better performed subsequently.

[0128] Step 520: Input the sample recognition image into the first model to be trained to obtain the first sample recognition feature output by the first model to be trained.

[0129] Step 530: Input the positive sample image into the second training model to obtain the second sample recognition feature output by the second training model.

[0130] Step 540: Input the negative sample image into the third model to be trained to obtain the third sample recognition features output by the third model to be trained.

[0131] Here, the first, second, and third models to be trained have basically the same model structure, and their initial model parameters can be the same or different. The specific structures of the first, second, and third models to be trained can be referred to in the above embodiments, and will not be repeated here.

[0132] In one specific embodiment, the first model to be trained includes a feature extraction layer, a head and shoulder localization network layer, and a graph convolutional network layer. Specifically, the sample recognition image is input to the feature extraction layer of the first model to be trained to obtain the global features output by the feature extraction layer. The global features are then input to the head and shoulder localization network layer of the first model to be trained to obtain the head and shoulder features output by the head and shoulder localization network layer. The head and shoulder features are then input to the graph convolutional network layer of the first model to be trained to obtain the first sample recognition features output by the graph convolutional network layer. The second and third models to be trained have essentially the same model structure as the first model to be trained, and will not be described in detail here.

[0133] Step 550: Determine the loss function based on the similarity between the first sample identification feature and the second sample identification feature, and the similarity between the first sample identification feature and the third sample identification feature.

[0134] Specifically, the first similarity between the identification features of the first sample and the identification features of the second sample is determined, and the second similarity between the identification features of the first sample and the identification features of the third sample is determined; based on the difference between the first similarity and the second similarity, the loss function is determined.

[0135] In one embodiment, the loss function is as follows:

[0136] L tri =max(||d a -d p ||-‖d a -d n +margin, 0);

[0137] In the formula, L tri Denotes the loss function, d a d represents the first sample identification feature. p d represents the identification feature of the second sample. n This represents the third sample identification feature, and margin represents a constant greater than 0.

[0138] Step 560: Based on the loss function, train the first model to be trained, the second model to be trained, and the third model to be trained, and determine the trained first model to be trained as the recognition model.

[0139] Specifically, the loss function is calculated, and the model parameters of each model to be trained are updated through backpropagation. It should be noted that the model parameters of each model to be trained may be different after the final training. Based on this, the first model to be trained after training is selected as the recognition model.

[0140] It is understandable that training optimizes each model to make the features of the sample recognition image and the positive sample image more similar, and the features of the sample recognition image and the negative sample image more dissimilar.

[0141] Here, the recognition model is used to identify pedestrian images and obtain recognition features. In some embodiments, the image to be recognized is input into the recognition model to obtain the first target recognition feature output by the recognition model; each known pedestrian image in the known image library is input into the recognition model to obtain each second target recognition feature output by the recognition model; the similarity between the first target recognition feature and each second target recognition feature is calculated, and the pedestrian re-recognition result is determined based on each similarity.

[0142] In one embodiment, the maximum similarity among the various similarities is determined. If the maximum similarity is greater than a preset threshold, the known pedestrian image corresponding to the maximum similarity is used as the recognition result of the image to be recognized, thereby determining which person the known pedestrian image belongs to, so as to determine the pedestrian re-identification result.

[0143] In another embodiment, the maximum similarity among the various similarities is determined, and the known pedestrian image corresponding to the maximum similarity is used as the recognition result of the image to be recognized, thereby determining which person the known pedestrian image belongs to, so as to determine the pedestrian re-recognition result.

[0144] In another embodiment, a target similarity greater than a preset threshold is determined among the various similarities, and the known pedestrian image corresponding to the target similarity is used as the recognition result of the image to be recognized, thereby determining which person the known pedestrian image belongs to, so as to determine the pedestrian re-recognition result.

[0145] The preset threshold can be set according to actual needs, and the embodiments of the present invention do not impose specific limitations on it.

[0146] The pedestrian re-identification method provided in this embodiment of the invention divides the training samples into sample recognition images, positive sample images, and negative sample images, and inputs the three images into the corresponding models to be trained. Based on the similarity between the first sample recognition feature and the second sample recognition feature, and the similarity between the first sample recognition feature and the third sample recognition feature, a loss function is calculated. Thus, through the above-mentioned triplet images and triplet loss function, the model can be trained better, and the robustness and accuracy of the recognition model can be improved.

[0147] To facilitate understanding of the above embodiments, the present invention will be described using a specific embodiment as an example. For example...Figure 6 As shown, during the training process of the recognition model, the sample recognition image is input into the first feature extraction layer of the first model to be trained to obtain the first global feature output by the first feature extraction layer. The first global feature is then input into the first head and shoulder localization network layer of the first model to be trained to obtain the first head and shoulder feature output by the first head and shoulder localization network layer. The first head and shoulder feature is then input into the first graph convolutional network layer of the first model to be trained to obtain the first sample recognition feature output by the first graph convolutional network layer. Similarly, the positive sample image is input into the second feature extraction layer of the second model to be trained to obtain the second global feature output by the second feature extraction layer. The second global feature is then input into the second head and shoulder localization network layer of the second model to be trained to obtain the first sample recognition feature output by the second head and shoulder localization network layer. The second head and shoulder feature is input into the second graph convolutional network layer of the second model to be trained, resulting in the second sample recognition feature output by the second graph convolutional network layer. The negative sample image is input into the third feature extraction layer of the third model to be trained, resulting in the third global feature output by the third feature extraction layer. The third global feature is input into the third head and shoulder localization network layer of the third model to be trained, resulting in the third head and shoulder feature output by the third head and shoulder localization network layer. The third head and shoulder feature is input into the third graph convolutional network layer of the third model to be trained, resulting in the third sample recognition feature output by the third graph convolutional network layer. Finally, the first sample recognition feature, the second sample recognition feature, and the third sample recognition feature are input into the loss function calculation layer to complete the model training.

[0148] The pedestrian re-identification device provided by the present invention is described below. The pedestrian re-identification device described below can be referred to in correspondence with the pedestrian re-identification method described above.

[0149] Figure 7 This is a schematic diagram of the pedestrian re-identification device provided by the present invention, as shown below. Figure 7 As shown, the pedestrian re-identification device includes:

[0150] Image determination module 710 is used to determine the image to be recognized;

[0151] The relationship recognition module 720 is used to perform spatial relationship recognition on the head and shoulder features of the image to be recognized to obtain recognition features, which are used to characterize the spatial relationship between the head and shoulder information and the head and shoulder features;

[0152] The result determination module 730 is used to determine the pedestrian re-identification result based on the identification features.

[0153] The pedestrian re-identification device provided in this invention performs spatial relationship recognition on the head and shoulder features of the image to be identified, obtaining recognition features that characterize head and shoulder information and their spatial relationships. Since the head and shoulder area of ​​a pedestrian is not easily occluded, and head and shoulder features can better characterize a pedestrian, the accuracy of pedestrian re-identification is improved. Furthermore, the recognition features not only cover head and shoulder features but also their spatial relationships, thereby enhancing the characterization ability of the recognition features for the head and shoulder area, making the recognition features more discriminative, and further improving the accuracy of pedestrian re-identification.

[0154] Based on any of the above embodiments, the relationship identification module 720 includes:

[0155] The feature segmentation unit is used to segment the head and shoulder features to obtain multiple local features;

[0156] A relationship determination unit is used to determine the spatial relationship between each pair of local features based on the similarity between each pair of local features among the plurality of local features;

[0157] The graph convolution processing unit is used to perform graph convolution processing on each of the local features based on the spatial relationship between each pair of local features to obtain recognition features.

[0158] Based on any of the above embodiments, the relationship recognition module 720 further includes a first feature determination unit, which is further configured to:

[0159] The head and shoulder positions are obtained by detecting the global features of the image to be identified.

[0160] Based on the head and shoulder position, the head and shoulder features in the global features are determined.

[0161] Based on any of the above embodiments, the relationship identification module 720 further includes:

[0162] An offset determination unit is used to determine the two-dimensional positional offset of the head and shoulder features;

[0163] The feature update unit is used to update the head and shoulder features based on the two-dimensional position offset.

[0164] Based on any of the above embodiments, the first feature determining unit is further configured to:

[0165] Based on the head and shoulder position, determine the region of interest;

[0166] Identify the features of interest that belong to the region of interest from the global features;

[0167] The features of interest are pooled to obtain the head and shoulder features.

[0168] Based on any of the above embodiments, the head and shoulder features are determined based on the global features of the image to be identified. The relationship recognition module 720 further includes a second feature determination unit, which is used for:

[0169] Global feature extraction is performed on the image to be identified to obtain global image features;

[0170] Determine the positional relationship between every two locations in the global features of the image;

[0171] The positional relationships are fused with the features of each position in the global features of the image to obtain the global features.

[0172] Based on any of the above embodiments, the recognition feature is obtained by recognizing the image to be recognized based on a recognition model. The device further includes a model training module, which is used for:

[0173] Acquire a sample recognition image, a positive sample image corresponding to the sample recognition image, and a negative sample image corresponding to the sample recognition image;

[0174] The sample recognition image is input into the first model to be trained to obtain the first sample recognition feature output by the first model to be trained.

[0175] The positive sample image is input into the second model to be trained to obtain the second sample recognition feature output by the second model to be trained.

[0176] The negative sample image is input into the third model to be trained to obtain the third sample recognition features output by the third model to be trained;

[0177] The loss function is determined based on the similarity between the first sample identification feature and the second sample identification feature, and the similarity between the first sample identification feature and the third sample identification feature.

[0178] Based on the loss function, the first model to be trained, the second model to be trained, and the third model to be trained are trained, and the trained first model to be trained is determined as the recognition model.

[0179] Figure 8 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 8As shown, the electronic device may include a processor 810, a communications interface 820, a memory 830, and a communication bus 840, wherein the processor 810, the communications interface 820, and the memory 830 communicate with each other via the communication bus 840. The processor 810 can call logical instructions in the memory 830 to execute a pedestrian re-identification method, which includes: determining an image to be identified; performing spatial relationship identification on the head and shoulder features of the image to be identified to obtain identification features, wherein the identification features are used to characterize the spatial relationship between the head and shoulder information and the head and shoulder features; and determining a pedestrian re-identification result based on the identification features.

[0180] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0181] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the pedestrian re-identification method provided by the above methods. The method includes: determining an image to be identified; performing spatial relationship identification on the head and shoulder features of the image to be identified to obtain identification features, wherein the identification features are used to characterize the spatial relationship between the head and shoulder information and the head and shoulder features; and determining a pedestrian re-identification result based on the identification features.

[0182] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When executed by a processor, the computer program is implemented to perform the pedestrian re-identification method provided by the above methods. The method includes: determining an image to be identified; performing spatial relationship identification on the head and shoulder features of the image to be identified to obtain identification features, the identification features being used to characterize the spatial relationship between the head and shoulder information and the head and shoulder features; and determining a pedestrian re-identification result based on the identification features.

[0183] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0184] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0185] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A pedestrian re-identification method, characterized in that, include: Identify the image to be recognized; Spatial relationship recognition is performed on the head and shoulder features of the image to be recognized to obtain recognition features, which are used to characterize the spatial relationship between head and shoulder information and the head and shoulder features; Based on the identified features, the pedestrian re-identification result is determined; The step of performing spatial relationship recognition on the head and shoulder features of the image to be recognized to obtain recognition features includes: The head and shoulder features are divided into multiple local features; the local features are features of a portion of the head and shoulder features. Based on the distance similarity between every two local features, the spatial relationship between each two local features is determined. Construct a graph network for the head and shoulder features, where the nodes of the graph network are local features and the edges of the graph network represent the spatial relationship between two local features; Based on the graph convolutional network layer, graph convolution processing is performed on the graph network to obtain recognition features; The head and shoulder features were determined based on the following steps: The head and shoulder positions are obtained by detecting the global features of the image to be identified. Based on the head and shoulder position, the head and shoulder features in the global features are determined.

2. The pedestrian re-identification method according to claim 1, characterized in that, The step of determining the head and shoulder features in the global features based on the head and shoulder position further includes: Determine the two-dimensional positional offset of the head and shoulder features; The head and shoulder features are updated based on the two-dimensional positional offset.

3. The pedestrian re-identification method according to claim 1, characterized in that, The step of determining the head and shoulder features in the global features based on the head and shoulder position includes: Based on the head and shoulder position, determine the region of interest; Identify the features of interest that belong to the region of interest from the global features; The features of interest are pooled to obtain the head and shoulder features.

4. The pedestrian re-identification method according to claim 1, characterized in that, The head and shoulder features are determined based on the global features of the image to be identified, which are determined based on the following steps: Global feature extraction is performed on the image to be identified to obtain global image features; Determine the positional relationship between every two locations in the global features of the image; The positional relationships are fused with the features of each position in the global features of the image to obtain the global features.

5. The pedestrian re-identification method according to any one of claims 1 to 4, characterized in that, The recognition features are obtained by recognizing the image to be recognized based on the recognition model, which is trained based on the following steps: Acquire a sample recognition image, a positive sample image corresponding to the sample recognition image, and a negative sample image corresponding to the sample recognition image; The sample recognition image is input into the first model to be trained to obtain the first sample recognition feature output by the first model to be trained. The positive sample image is input into the second model to be trained to obtain the second sample recognition feature output by the second model to be trained. The negative sample image is input into the third model to be trained to obtain the third sample recognition features output by the third model to be trained; The loss function is determined based on the similarity between the first sample identification feature and the second sample identification feature, and the similarity between the first sample identification feature and the third sample identification feature. Based on the loss function, the first model to be trained, the second model to be trained, and the third model to be trained are trained, and the trained first model to be trained is determined as the recognition model.

6. A pedestrian re-identification device, characterized in that, include: Image determination module, used to determine the image to be recognized; The relationship recognition module is used to perform spatial relationship recognition on the head and shoulder features of the image to be recognized, and obtain recognition features, which are used to characterize the spatial relationship between the head and shoulder information and the head and shoulder features; The result determination module is used to determine the pedestrian re-identification result based on the identified features; The relationship identification module is specifically used for: The head and shoulder features are divided into multiple local features; the local features are features of a portion of the head and shoulder features. Based on the distance similarity between every two local features, the spatial relationship between each two local features is determined. Based on the spatial relationship between every two local features, graph convolution is performed on each local feature to obtain the recognition features; The head and shoulder features were determined based on the following steps: The head and shoulder positions are obtained by detecting the global features of the image to be identified. Based on the head and shoulder position, the head and shoulder features in the global features are determined.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the pedestrian re-identification method as described in any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the pedestrian re-identification method as described in any one of claims 1 to 5.

Images