Pedestrian matching method, device, electronic device, and computer-readable storage medium

Abstract

A pedestrian matching method, device, electronic device, and computer-readable storage medium include the steps of: acquiring a first image and a second image; performing feature extraction on each of the first and second images to obtain first local features, second local features, first high-level features, and second high-level features, where the first local features and second local features include local feature vectors corresponding to important points on the human body; performing feature alignment on the first local features and second local features to obtain first fusion features and second fusion features; performing feature alignment on the first high-level features and second high-level features to obtain first high-level fusion features and second high-level fusion features; and determining whether the first and second images contain the same pedestrian based on the first fusion features, second fusion features, first high-level fusion features, and second high-level fusion features. This method can reduce the impact of abnormal points in the images on pedestrian matching, resulting in more accurate matching results.

Description

[Technical Field]

[0001] The present disclosure relates to the field of image processing technology, and in particular to a pedestrian matching method, device, electronic device, and computer-readable storage medium. [Background technology]

[0002] Image matching refers to a method of finding similar image targets by analyzing similarity and consistency based on the correspondence between image content, features, structure, relationship, texture, grayscale, etc. The image matching method may be used for pedestrian matching, which can compare whether pedestrians appearing in different images are the same pedestrian.

[0003] Most conventional pedestrian matching methods directly use global feature matching of the human body or are based on local feature matching of the human body. However, in actual situations, most images usually have different qualities, such as the human body being occluded in the image, the human body posture being different, and the image brightness being different, that is, there will be some abnormal points or outliers. When performing pedestrian matching, the feature alignment will easily not match, which will result in the problem that the pedestrian matching results are not accurate enough. Summary of the Invention [Problem to be solved by the invention]

[0004] In view of this, the embodiments of the present disclosure provide a pedestrian matching method, device, electronic device and computer-readable storage medium to solve the problem that in the prior art, pedestrian matching results are inaccurate due to the presence of some abnormal points or outliers in the image. [Means for solving the problem]

[0005] The first aspect of the disclosed embodiment is acquiring a first image and a second image; performing feature extraction on each of the first image and the second image to obtain first local features, second local features, first high-level features, and second high-level features, wherein the first local features and the second local features each include a local feature vector corresponding to each important point of a human body; performing feature alignment on the first local features and the second local features to obtain first fusion features and second fusion features; performing feature alignment on the first high-level features and the second high-level features to obtain first high-level fusion features and second high-level fusion features; and determining whether the first image and the second image contain the same pedestrian based on the first fusion feature, the second fusion feature, the first high-order fusion feature, and the second high-order fusion feature.

[0006] A second aspect of the disclosed embodiment is an acquisition module for acquiring a first image and a second image; a feature extraction module for performing feature extraction on the first image and the second image, respectively, to obtain first local features, second local features, first high-level features, and second high-level features, wherein the first local features and the second local features each include a local feature vector corresponding to each important point on a human body; a first feature alignment module for performing feature alignment on the first local features and the second local features to obtain first fusion features and second fusion features; a second feature alignment module for performing feature alignment on the first high-level features and the second high-level features to obtain first high-level fusion features and second high-level fusion features; and a matching module for determining whether the first image and the second image contain the same pedestrian based on the first fusion feature, the second fusion feature, the first high-order fusion feature, and the second high-order fusion feature.

[0007] A third aspect of an embodiment of the present disclosure provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable by the processor, the electronic device implementing the steps of the above method when the processor executes the computer program.

[0008] A fourth aspect of an embodiment of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, the computer program implementing the steps of the above method when executed by a processor. [Effects of the Invention]

[0009] The present disclosure provides the following advantageous effects over the prior art: by performing feature extraction on the first and second captured images, respectively, local features and high-level features corresponding to important points on the human body in the two images are obtained, which are designated as the first local feature, the second local feature, the first high-level feature, and the second high-level feature, respectively; performing feature alignment on the first and second local features to obtain the first fusion feature and the second fusion feature; performing feature alignment on the first and second high-level features to obtain the first high-level fusion feature and the second high-level fusion feature; and determining whether the same pedestrian is included in the two images based on the first fusion feature, the second fusion feature, the first high-level fusion feature, and the second high-level fusion feature; extracting local features and high-level features of the human body in the images, and performing feature alignment on the local features and the high-level features respectively using different feature alignment strategies, incorporating the relationships between the features and integrating the information between the features, obtaining more effective features, reducing the impact of abnormal or outlying points in the images on pedestrian matching, and obtaining more accurate matching results. [Brief explanation of the drawings]

[0010] In order to more clearly explain the technical solutions in the embodiments of the present disclosure, the following briefly introduces drawings necessary for describing the embodiments or prior art. Obviously, the drawings described below are only some embodiments of the present disclosure, and those skilled in the art can obtain other drawings based on these drawings without the need for creative work.

[0011] [Figure 1] 1 is a flowchart of a pedestrian matching method provided in an embodiment of the present disclosure. [Figure 2] 1 is a flowchart provided in an embodiment of the present disclosure for determining whether a first image and a second image contain the same pedestrian based on a first fusion feature, a second fusion feature, a first high-order fusion feature, and a second high-order fusion feature. [Figure 3] 1 is a flowchart provided in an embodiment of the present disclosure for performing feature aggregation on local features to obtain higher-level features. [Figure 4] 1 is a schematic diagram of the important points number of the human body provided in the examples of the present disclosure. [Figure 5] 10 is a flowchart provided in an embodiment of the present disclosure for performing feature alignment on a first local feature and a second local feature to obtain a first fusion feature and a second fusion feature. [Figure 6] 10 is a flowchart for performing feature alignment on a first local feature and a second local feature, respectively, provided in an embodiment of the present disclosure. [Figure 7] 10 is a flowchart provided in an embodiment of the present disclosure for performing feature alignment on a first high-level feature and a second high-level feature to obtain a first high-level fusion feature and a second high-level fusion feature. [Figure 8] 10 is a flowchart for performing feature alignment on a first high-level feature and a second high-level feature, respectively, provided in an embodiment of the present disclosure. [Figure 9] 1 is a flowchart of another pedestrian matching method provided in an embodiment of the present disclosure. [Figure 10] FIG. 1 is a structural schematic diagram of a pedestrian matching device provided in an embodiment of the present disclosure. [Figure 11]FIG. 1 is a structural schematic diagram of another pedestrian matching device provided in an embodiment of the present disclosure. [Figure 12] 1 is a structural schematic diagram of an electronic device provided in an embodiment of the present disclosure; DETAILED DESCRIPTION OF THE INVENTION

[0012] In the following description, for purposes of explanation, not limitation, specific details, such as particular system structures and techniques, are provided to thoroughly understand the embodiments of the present disclosure. However, those skilled in the art should understand that the present disclosure can be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present disclosure with unnecessary details.

[0013] Hereinafter, a pedestrian matching method and device according to an embodiment of the present disclosure will be described in detail with reference to the drawings.

[0014] 1 is a flowchart of a pedestrian matching method provided in an embodiment of the present disclosure. As shown in FIG. 1, the pedestrian matching method includes steps S101 to S105.

[0015] In S101, a first image and a second image are acquired.

[0016] The first image and the second image are images for matching pedestrians, and in this embodiment, by comparing the first image and the second image, it is determined whether the same pedestrian is included in them.

[0017] Specifically, the first and second images may be obtained from a database or extracted in real time from images collected by a monitoring device. In one embodiment, when the pedestrian matching method is used for a scene where people are staying, both the first and second images may be extracted from images collected by a monitoring device. Two images are extracted from the images collected by the monitoring device at predetermined intervals and compared to determine whether they contain the same pedestrian and whether people are staying in the same location for a long time. Furthermore, in one embodiment, when people are detected in the images collected by the monitoring device, the pedestrian matching method is triggered, and two images are extracted at predetermined intervals to determine whether people are staying in the same location for a long time through pedestrian matching. In other embodiments, the pedestrian matching method may be applied to other scenes.

[0018] In the examples disclosed in this application, the concepts of "first", "second", etc. mentioned are merely used to distinguish between different names and do not indicate any actual meaning.

[0019] In S102, feature extraction is performed on each of the first image and the second image to obtain a first local feature, a second local feature, a first high-level feature, and a second high-level feature.

[0020] To enable a computer to "understand" an image, we typically extract useful data or information from the image to obtain "non-image" representations or descriptions of the image, such as numbers, vectors, and codes. This process is called feature extraction, and these extracted "non-image" representations or descriptions are called features. Images have their own characteristics that distinguish them from other types of images. These include intuitive, natural features such as brightness, edges, texture, and color, as well as features that require transformations or processing to obtain, such as moments, histograms, and principal components.

[0021] In some embodiments, feature extraction for an image can be achieved using a neural network. In this embodiment, since the purpose is to perform pedestrian matching, it is necessary to extract the features of the human body contained in the image during feature extraction, so feature extraction can be performed using a human body detection module and a human body key point estimation model.

[0022] The local features include local features of local feature vectors corresponding to each important point on the human body. In this embodiment, for the purpose of distinction, the features extracted from the first image are referred to as the first local features and the first high-level features, and the features extracted from the second image are referred to as the second local features and the second high-level features.

[0023] In some embodiments, the step of performing feature extraction on the first image and the second image, respectively, to obtain first local features, second local features, first high-level features, and second high-level features includes the steps of performing human body local feature extraction on the first image and the second image, respectively, to obtain first local features and second local features, and performing feature aggregation on the first local features to obtain first high-level features, and performing feature aggregation on the second local features to obtain second high-level features.

[0024] In a specific embodiment, the step of extracting features from the first image to obtain first local features includes the steps of inputting the first image into a human body detection model, performing human body detection on the first image, and obtaining a human body detection frame in the first image, and inputting the human body detection frame into a human body key point estimation model to obtain first local features that are characteristics of each human body key point corresponding to the human body detection frame. Naturally, the step of extracting features from the second image to obtain second local features is similar to the local feature extraction from the first image, and will not be described again here. Here, any one type of human body detection model or human body key point estimation model can be used to extract features from an image.

[0025] Furthermore, in a specific embodiment, the human body detection model extracts features from an image to obtain a human body detection frame, and then inputs the human body detection frame into the human body key point estimation model. The specific process by which the human body key point estimation model processes the input human body detection frame is to first jointly convert each feature map to 1024 by 1*1 convolution, and then jointly pool the length and width of the feature map to [16,8] by ROIAlign (calibration), thereby obtaining each human body feature map G1 with dimensions [1024,16,8]. G1 is then convolved through three blocks (conv(3,3), BN, relu) to sequentially obtain feature maps [1024,16,8], [512,16,8], and [256,16,8]. Finally, conv(1,1), BN, and relu are used to obtain a heat map of all the body's important points. If there are 14 important points on the body, the heat map can be expressed as [14,16,8]. Here, BN (Batch Normalization) normalizes the input of each layer. relu (Rectified Linear Unit), also known as the modified linear unit, is an activation function commonly used in artificial neural networks. It refers to a nonlinear function usually represented by a ramp function or its variant.

[0026] The heatmap dimension of each key point is [1,16,8], then iteratively stacks to change its dimension to [1024,16,8], multiplies it by the human body feature map G1 [1024,16,8] to obtain the feature map of the key point [2048,16,8], and finally performs global average pooling to obtain the feature vector of the key point

[1024] . Similarly, according to the above method, the feature vector of each key point of the human body can be obtained, where each key point of the human body in the first image is the first local feature, and each key point of the human body in the second image is the second local feature.

[0027] In some embodiments, after acquiring local features, higher-level features can be acquired by performing feature aggregation on each local feature. Similarly, in these embodiments, a higher-level feature acquired by aggregating first local features will be referred to as a first higher-level feature, and a higher-level feature acquired by performing feature aggregation on second local features will be referred to as a second higher-level feature. Specifically, the specific process of acquiring higher-level features by performing feature aggregation on local features will be described in detail in later embodiments, and will not be described again here.

[0028] In step S103, feature alignment is performed on the first local feature and the second local feature to obtain a first fusion feature and a second fusion feature.

[0029] Pedestrians appear in different images at different times in different poses, and when performing pedestrian matching on different images, image features are aligned to find identical points in the two images and determine whether they contain the same pedestrian. In this embodiment, feature alignment is performed on the human body local features extracted from the two images, and the aligned features are referred to as the first fusion feature and the second fusion feature. The specific steps for aligning the first and second local features to obtain the first and second fusion features will be described in detail in a later embodiment.

[0030] In step S104, feature alignment is performed on the first and second high-level features to obtain first and second high-level fusion features.

[0031] In this embodiment, when extracting features, low-level human body local features are obtained, and high-level image features are obtained using the low-level human body local features. Therefore, when aligning features, low-level feature alignment is performed for the low-level local features, and high-level feature alignment strategy is used for the high-level features, making full use of the collected image features to improve the extracted features. Furthermore, in this embodiment, after aligning the high-level features, the obtained features are referred to as the first high-level fusion feature and the second high-level fusion feature.

[0032] In S105, it is determined whether the same pedestrian is included in the first image and the second image based on the first fusion feature, the second fusion feature, the first high-order fusion feature, and the second high-order fusion feature.

[0033] In one embodiment, as shown in FIG. 2 , the step of determining whether the first image and the second image contain the same pedestrian based on the first fusion feature, the second fusion feature, the first high-order fusion feature, and the second high-order fusion feature includes steps S201 to S203.

[0034] In S201, a first fusion feature and a first higher-order fusion feature are spliced ​​to obtain a first spliced ​​feature.

[0035] In S202, splicing is performed on the second fusion feature and the second higher-order fusion feature to obtain a second spliced ​​feature.

[0036] In some embodiments, feature splicing can be completed by a concat function, which is used to connect two or more arrays. In this embodiment, a feature obtained by splicing a first fusion feature or a first higher-order fusion feature is referred to as a first spliced ​​feature, and a feature obtained by splicing a second fusion feature or a second higher-order fusion feature is referred to as a second spliced ​​feature. In other embodiments, feature splicing can be achieved in other ways.

[0037] In S203, a comparative loss value between the first splice feature and the second splice feature is calculated, and whether the same pedestrian is included in the first image and the second image is determined based on the comparative loss value.

[0038] In some embodiments, a comparative loss function is used to calculate a comparative loss value between the first splice feature and the second splice feature, and the comparative loss function can be effectively expressed as a matching degree for a sample. In this embodiment, the comparative loss function is used to calculate a comparative loss value between the first splice feature and the second splice feature, which can be used to describe the matching degree between the first image and the second image. Specifically, in this embodiment, any comparative loss function can be used to calculate a comparative loss value between two splice features.

[0039] After obtaining the comparison loss value, it can be determined whether the first image and the second image contain the same pedestrian based on the comparison loss value. In one embodiment, determining whether the first image and the second image contain the same pedestrian based on the comparison loss value can be performed by setting a predetermined condition, and if the comparison loss value meets the predetermined condition, it can be determined that the first image and the second image contain the same pedestrian. The predetermined condition can be set according to actual circumstances, and the higher the degree of matching between the first image and the second image determined based on the comparison loss value, the higher the possibility that the first image and the second image contain the same pedestrian.

[0040] In another embodiment, after obtaining the first splice feature and the second splice feature, it is possible to determine whether the same pedestrian is contained in the first image and the second image in another manner based on the first splice feature and the second splice feature. For example, the similarity between the first splice feature and the second splice feature is calculated, and it is determined whether the same pedestrian is contained in the first image and the second image based on the similarity.

[0041] According to the technical solution provided in the disclosed embodiments, feature extraction is performed on the captured first and second images, respectively, to obtain local features and high-level features corresponding to important points of the human body in the two images, which are respectively referred to as the first local feature, the second local feature, the first high-level feature, and the second high-level feature, feature alignment is performed on the first local feature and the second local feature to obtain first fusion features and second fusion features, feature alignment is performed on the first high-level features and the second high-level features to obtain first high-level fusion features and second high-level fusion features, and whether the same pedestrian is included in the two images is determined based on the first fusion features, the second fusion features, the first high-level fusion features, and the second high-level fusion features. The local features and high-level features of the human body in the images are extracted, and feature alignment is performed on the local features and the high-level features respectively using different feature alignment strategies, incorporating the relationships between the features and integrating the information between the features to obtain more effective features, reducing the impact of abnormal or outlying points in the images on pedestrian matching, and obtaining more accurate matching results.

[0042] In some embodiments, the key points of the human body include two or more. In this embodiment, as shown in FIG. 3 , the step of performing feature aggregation on the local features to obtain higher-level features includes steps S301 to S306, where the local features include a first local feature and a second local feature, and the higher-level features include a first high-level feature and a second high-level feature.

[0043] In S301, a first parameter, a second parameter, and a third parameter corresponding to the local feature are learned and identified based on the local feature vector corresponding to each important point of the human body.

[0044] A human body usually includes two or more important points, and when extracting the human body local features, a local feature vector corresponding to each important point of the human body can be obtained, and the number of local feature vectors included in the local features corresponds to the number of important points of the human body. In this embodiment, the local features include a first local feature and a second local feature extracted from the first image and the second image, respectively.

[0045] Specifically, learning the first parameter, the second parameter, and the third parameter corresponding to the local feature based on the local feature vector corresponding to each important point of the human body may be realized in any manner, for example, by a neural network, and in a specific embodiment, by a Transformer network (submitted by Ashish Vaswani). In some embodiments, the ranges of the first parameter, the second parameter, and the third parameter corresponding to the local vector are all R 1024×256 is.

[0046] S302: generating a query, a key and a value corresponding to each local feature vector based on the first parameter, the second parameter and the third parameter.

[0047] Query, key, and value are concepts in Transformer networks. Based on the first parameter, the second parameter, and the third parameter, each local feature vector generates a corresponding query, key, and value, which may be expressed as follows:

[0048] query:Q i =f i *W q and W q represents the first parameter, key:K i =f i *W k and W k represents the second parameter, value:V i =f i *W v and W v represents the third parameter.

[0049] In S303, a weighting factor between each pair of local feature vectors is determined based on the query and the key.

[0050] In some embodiments, the step of determining a weight coefficient between each pair of local feature vectors based on a query and a key includes selecting two local feature vectors as a first target local feature vector and a second target local feature vector, and calculating a weight coefficient between the first target local feature vector and the second target local feature vector using the query of the first target local feature vector and the key of the second target local feature vector.

[0051] Furthermore, in a specific embodiment, the weighting coefficient of the first target local feature vector and the second target local feature vector may be calculated based on the query of the first target local feature vector and the key of the second target local feature vector, and may be expressed as follows: JPEG0007749132000001.jpg37167

[0052] a ij represents the weight coefficient between the local feature vectors of number i and number j, where i and j respectively represent the numbers corresponding to the important points of the human body. The important points of the human body can be numbered in advance. Q i represents the query corresponding to the first target local feature vector, and K j represents the key of the second target local feature vector, and K j T is K j represents the transposition of Q, and 256 represents the i , K. j The dimension of, softmax() represents the logistic regression algorithm, and a ij represents the weighting coefficient between the first target local feature vector and the second target local feature vector. Here, the transpose is defined as: when all elements of A are mirror-reflected around a 45-degree ray from the bottom right starting from the element in the first row and first column, the transpose of A is obtained.

[0053] According to the above method, it is possible to calculate the weighting coefficient between the local feature vectors corresponding to each pair of important points on the human body.

[0054] In step S304, for any one of the local feature vectors, an adjustment factor is determined based on a relationship between the local feature vector and each of the other local feature vectors.

[0055] The relationship between a local feature vector and other local feature vectors can be determined by pre-assigned numbers for key points on the human body. In a specific embodiment, a number is assigned to each key point on the human body, as shown in Fig. 4: head = 1, neck = 2, left shoulder = right shoulder = 3, left elbow = right elbow = 4, left hand = right hand = 5, left waist = right waist = 6, left knee = right knee = 7, and left foot = right foot = 8. When matching pedestrians, the closer the positions are, the greater the influence between key points on the human body. Therefore, in this embodiment, an adjustment coefficient is set to aggregate key points according to the degree of influence between them to obtain high-level features.

[0056] In a specific embodiment, the adjustment coefficient may be determined based on the relationship between the local feature vector and other local feature vectors, and may be expressed as follows: JPEG0007749132000002.jpg31148

[0057] p represents the adjustment coefficient, r represents a hyperparameter, the value of which can be defined according to the actual situation, and i and j represent the numbers of the important points, respectively.

[0058] S305: Obtaining aggregate features to which the local features correspond based on the weight coefficients, the adjustment coefficients and the values.

[0059] In some embodiments, the step of obtaining the aggregate feature to which the local feature corresponds based on the weighting factor, the adjustment factor, and the value may be expressed as follows: JPEG0007749132000003.jpg26154

[0060] f i at represents the aggregate feature corresponding to the local feature vector of number i, and a ijrepresents the weighting coefficient between the local feature vectors of number i and number j, p represents the adjustment coefficient, and V j represents the value to which the local feature vector with number j corresponds, i, j respectively represent the numbers of the important points, and 14 represents the number of local feature vectors (important points on the human body).

[0061] S306, wherein each local feature is calculated based on a corresponding aggregate feature. Signs be identified.

[0062] In step S305, a corresponding aggregate feature can be obtained for each key point (local feature vector) of the human body, and the aggregate features are combined to obtain a corresponding high-level feature. In this embodiment, the first high-level feature is obtained by aggregating the first local feature, and the second high-level feature is obtained by aggregating the second local feature.

[0063] Furthermore, the process of acquiring high-level features by aggregating based on the local features described above may be realized by a transformer model.

[0064] According to the technical solution provided in the disclosed embodiments, a transformer model learns corresponding first, second, and third parameters for the input first and second local features, respectively, calculates the query, key, and value of each local feature vector based on the first, second, and third parameters, calculates a weighting factor between each pair of local feature vectors based on the query and key corresponding to each local feature vector, and determines an adjustment factor based on the relationship between each pair of local feature vectors. Finally, new local features are aggregated based on the weighting factor, the corresponding values ​​of the local feature vectors, and the adjustment factor, and finally, a fusion feature is obtained by aggregating the local features. In this embodiment, the adjustment factor is designed based on the relationship between the local feature vectors, i.e., the a priori topological structure relationship of important points on the human body, and the adjustment factor is used to add the influence degree between the local feature vectors to the feature aggregation, thereby improving the high-level feature obtained by aggregation.

[0065] In some embodiments, as shown in FIG. 5, performing feature alignment on the first local features and the second local features to obtain the first fusion feature and the second fusion feature includes steps S501 to S504.

[0066] In S501, a local feature similarity between a first local feature and a second local feature is calculated.

[0067] The calculation of the similarity between the local features may be realized by any one of methods. In one embodiment, the first local feature and the second local feature are mapped to the same feature space, and the cosine distance between the two local features is calculated as the feature similarity between the two local features. Furthermore, the step of mapping the first local feature and the second local feature to the same feature space may be realized by normalizing the distribution of the first local feature and the second local feature. In one embodiment, the step of mapping the first local feature and the second local feature to the same feature space and calculating the cosine distance between the two local features as the feature similarity between the two local features may be expressed as follows: M1=normalize(F p )*[normalize(F g )] T

[0068] Fp represents the first local feature, Fg represents the second local feature, normalize() represents normalizing the local feature, and M1 represents the similarity between the first local feature Fp and the second local feature Fg.

[0069] In other embodiments, the calculation of the local feature similarity between the first local feature and the second local feature may be achieved in other ways.

[0070] S502: Identifying a first matching feature in the first local feature of the second local feature based on the local feature similarity.

[0071] In some embodiments, the step of identifying a first matching feature in the first local feature of the second local feature based on the second local feature and the local feature similarity includes: calculating a first product of the local feature similarity and the second local feature; and identifying a sum value of the first product and the first local feature as the first matching feature.

[0072] A first product of the local feature similarity and the second local feature is calculated to represent points included in the first local feature that match with the second local feature, and based on the matching points, it is possible to identify which points in the first local feature should be used as priority points and eliminate outlying points or occlusion points.

[0073] A sum value of the first product and the first local feature is calculated, and the original feature including the first local feature also includes a relationship with the second local feature. In this embodiment, the sum value is referred to as a first matching feature, and based on the first matching feature, it is possible to identify what points in the first image need to be matched with the second image.

[0074] In a specific embodiment, calculating a first product of the local feature similarity and the second local feature; F g 1 =M1*F g It may also be expressed as:

[0075] F g 1 represents the first product, M1 represents the local feature similarity, and Fg represents the second local feature.

[0076] Identifying a sum value of the first product and the first local feature as a first matching feature; F p 2 =F p +F g 1 It is expressed as:

[0077] F p 2denotes the first matching feature, and Fp denotes the first local feature.

[0078] S503: Identifying a second matching feature in the second local feature of the first local feature based on the first local feature and local feature similarity.

[0079] Similar to identifying the first matching feature, the second matching feature can be identified based on the first local feature and the local feature similarity, except that in this embodiment, the local feature similarity is first transposed, and then the transposed local feature similarity is used to identify the second matching feature.

[0080] In some embodiments, the step of identifying a second matching feature in the second local feature of the first local feature based on the local feature similarity includes transposing the local feature similarity to obtain a transposed local feature similarity. do a step of calculating a second product of the transposed local feature similarity and the first local feature; and a step of identifying a sum value of the second product and the second local feature as a second matching feature.

[0081] A second product of the transposed local feature similarity and the first local feature is calculated to represent points in the second local feature that match with the first local feature. Based on the matching points, it is possible to identify which points in the second local feature should be used as priority points and eliminate outlying points or occlusion points.

[0082] A sum value of the second product and the second local feature is calculated, and the original feature including the first local feature also includes a relationship with the second local feature. In this embodiment, the sum value is referred to as a second matching feature. Based on the second matching feature, it is possible to identify what points in the second image need to be matched with the first image and what points do not need to be matched.

[0083] In a specific embodiment, a second product of the transposed local feature similarity and the first local feature is calculated, and F p1 =M1 T *F p It may also be expressed as:

[0084] F p 1 represents the second product, and M1 T represents the local feature similarity after transposition, and F p represents the first local feature.

[0085] Identifying a sum value of the second product and the second local feature as a second matching feature; F g 2 =F g +F p 1 It is expressed as:

[0086] F g 2 represents the second matching feature, and F g represents the second local feature.

[0087] In S504, feature extraction is performed on the first matching feature to obtain a first fusion feature, and feature extraction is performed on the second matching feature to obtain a second fusion feature.

[0088] After obtaining the first matching feature and the second matching feature, feature extraction is performed, and new features are obtained by integrating the low-level features after alignment, namely the first fusion feature and the second fusion feature.

[0089] In some embodiments, feature extraction is performed for each input matching feature, which can be realized by fc+relu. The fc layer is a fully connected layer, and each node of the fully connected layer is connected to all nodes of the previous layer and is used to combine previously extracted features. The Relu layer is usually composed of activation functions. The matching features include a first matching feature and a second matching feature.

[0090] In a specific embodiment, as shown in FIG. 6, a step flowchart for performing feature alignment on the first local feature and the second local feature is shown. F p is the first local feature, F g is the second local feature, F p t is the first fusion feature, F g t represents the second fusion feature.

[0091] According to the technical solution provided in the disclosed embodiments, local feature similarities between local features are calculated, first matching features and second matching features are calculated based on the local feature similarities, and finally feature extraction is performed on the first matching features and the second matching features, respectively, to obtain new features aligned with low-level features, i.e., first fusion features and second fusion features. Then, pedestrian matching is performed based on the aligned local features, thereby achieving more accurate matching results.

[0092] In some other embodiments, as shown in FIG. 7, the step of performing feature alignment on the first high-level features and the second high-level features and obtaining the first high-level fusion features and the second high-level fusion features includes steps S701 to S704.

[0093] In S701, a high-level feature similarity between a first high-level feature and a second high-level feature is calculated.

[0094] The calculation of the similarity between the high-level features may be realized in any manner. In one embodiment, the similarity between the high-level features may be learned and determined by a network, which may be expressed as follows: JPEG0007749132000004.jpg25170

[0095] JPEG0007749132000005.jpg25170A represents the network parameter matrix. The dimension of network parameter matrix A is [1024, 1024].

[0096] In other embodiments, the calculation of the similarity between the first high-level feature and the second high-level feature may be realized by any other method.

[0097] S702: Identifying a first high-level matching feature in the first high-level feature of the second high-level feature based on the second high-level feature and the high-level feature similarity.

[0098] In some embodiments, the step of identifying a first high-order matching feature in the first high-order feature of the second high-order feature based on the second high-order feature and the high-order feature similarity includes the step of calculating a third product of the high-order feature similarity and the second high-order feature, and the step of identifying the sum value of the third product and the first high-order feature as the first high-order matching feature.

[0099] In a specific embodiment, a third product of the high-level feature similarity and the second high-level feature is calculated; It may also be represented as JPEG0007749132000006.jpg22123.

[0100] JPEG0007749132000007.jpg21170

[0101] Identifying a sum value of the third product and the first high-order feature as a first high-order matching feature; Represented as JPEG0007749132000008.jpg37155.

[0102] JPEG0007749132000009.jpg16170

[0103] In S703, a second high-level matching feature in the second high-level feature of the first high-level feature is identified based on the first high-level feature and the high-level feature similarity.

[0104] Similar to identifying the first high-level matching feature, the second high-level matching feature can be identified based on the first high-level feature and the high-level feature similarity. The difference is that in this embodiment, the high-level feature similarity is first transposed, and then the transposed high-level feature similarity is used to identify the second high-level matching feature.

[0105] In some embodiments, the step of identifying a second higher-order matching feature in the second higher-order feature of the first high-order feature based on the first high-order feature and the high-order feature similarity includes the steps of transposing the high-order feature similarity to obtain a transposed high-order feature similarity, calculating a fourth product of the transposed high-order feature similarity and the first high-order feature, and identifying a sum value of the fourth product and the second high-order feature as the second high-order matching feature.

[0106] In a specific embodiment, a fourth product of the transposed high-level feature similarity and the first high-level feature is calculated; It may also be represented as JPEG0007749132000010.jpg24117.

[0107] JPEG0007749132000011.jpg20170

[0108] Identifying a sum value of the fourth product and the second high-order feature as a second high-order matching feature; JPEG0007749132000012.jpg27159

[0109] JPEG0007749132000013.jpg16170

[0110] In S704, feature extraction is performed on the first high-order matching feature to obtain a first high-order fusion feature, and feature extraction is performed on the second high-order matching feature to obtain a second high-order fusion feature.

[0111] After obtaining the first high-level matching feature and the second high-level matching feature, feature extraction is performed respectively, and new features are obtained by integrating the high-level features after alignment, namely the first high-level fusion feature and the second high-level fusion feature.

[0112] In some embodiments, feature extraction is performed for each input high-level matching feature, which can be realized by fc+relu. The fc layer is a fully connected layer, and each node of the fully connected layer is connected to all nodes of the previous layer and is used to combine previously extracted features. The Relu layer is usually composed of activation functions. The high-level matching features include a first high-level matching feature and a second high-level matching feature.

[0113] In a specific embodiment, as shown in FIG. 8, the first high-level feature and the second high-level feature are respectively subjected to feature alignment. JPEG0007749132000014.jpg21170

[0114] According to the technical solution provided in the disclosed embodiments, the high-level feature similarity between the high-level features is calculated, the first high-level matching feature and the second high-level matching feature are calculated based on the high-level feature similarity, and finally, feature extraction is performed on the first high-level matching feature and the second high-level matching feature, respectively, to obtain new features aligned with the high-level features, i.e., the first high-level fusion feature and the second high-level fusion feature. Then, pedestrian matching is performed based on the aligned high-level features, thereby achieving more accurate matching results.

[0115] In a specific embodiment, as shown in FIG. 9, a specific flowchart of a pedestrian matching method in a specific embodiment is shown.

[0116] (1) Input two photos of the probe and gallery: Each time, through the human body detection model, the human body detection frame bbox is obtained and sent to the human body key point estimation module.

[0117] (2) Input the human body detection frame obtained in step (1): First, the feature map corresponding to each human body detection frame is converted to 1024 by 1*1 convolution, and then the length and width of the feature map are pooled to [16, 8] by ROIAlign, and the dimensions of each human body feature map G1 are obtained as [1024, 16, 8].

[0118] G1 is processed through three convolutional blocks (conv(3,3), BN, relu) to obtain feature maps [1024,16,8], [512,16,8], and [256,16,8], respectively. Finally, conv(1,1), BN, relu are used to obtain a heat map of all important points [14,16,8]. (Here, there are 14 important points on the human body: head, neck, left hand, left elbow, left shoulder, left hip, left knee, left foot, right hand, right elbow, right shoulder, right hip, right knee, and right foot.)

[0119] The heat map dimension of each important point is [1,16,8]. Iteratively convert the heat map of each important point to [1024,16,8], and multiply it by the human body feature map G1 [1024,16,8] to obtain the feature map of the important point [2048,16,8]. Finally, global average pooling is performed to obtain the feature vector of the important point

[1024] . Similarly, the feature vector of each important point can be obtained. That is, the first local feature F p and the second local feature F g Get F p and F g The dimensions of both are [14,1024].

[0120] (3)F p and F g to the transformer coding module, JPEG0007749132000015.jpg17170 For details on how to aggregate and obtain high-level features, please refer to the description of the embodiment shown in FIG.

[0121] (4)F p and F gto the low-level feature alignment strategy module, JPEG0007749132000016.jpg17132, all of which have dimensions [14, 1024]. For a specific example of how to perform low-level feature alignment, please refer to the description of the embodiment shown in FIG.

[0122] (5) According to step (3), JPEG0007749132000017.jpg18129, whose dimensions are all [14,1024], are sent to the high-dimensional feature matching strategy module. JPEG0007749132000018.jpg24155, all of which have dimensions [14, 1024]. Here, for a specific example of how to perform high-dimensional feature alignment, please refer to the description of the embodiment shown in FIG.

[0123] JPEG0007749132000019.jpg68170JPEG0007749132000020.jpg46170Finally, based on the comparison loss value, it is determined whether the same pedestrian is contained in the photo probe and gallery.

[0124] In the pedestrian matching method of the disclosed embodiment, a feature alignment module is designed to incorporate the relationships between features through low-level feature alignment and high-level feature alignment strategies, and to integrate and utilize the information between features to improve the extracted features. Here, the low-level feature alignment strategy matches based on the similarity of local features of key points on the human body. Whether or not two points are matched is determined not only by the similarity between the two points, but also by considering the impact of the two points matching on other matches. Therefore, a match is selected to maximize the overall energy. Therefore, a high-level feature matching strategy (high-level features can be understood as edge feature information) is also added to the pedestrian matching method of the disclosed embodiment. Specifically, the transformer coding module aggregates the local features of key points on the human body previously extracted into high-level feature information, and then the network learns how to match these high-level features, resulting in higher robustness. In addition, when aggregating high-level features using transformer coding, an adjustment coefficient p is calculated based on the a priori topological structure relationship of important points on the human body, and high-level semantic information is aggregated based on the degree of influence between important points.

[0125] The present application further provides application scenarios in which the above-mentioned pedestrian matching method is applied, specifically, the application of the pedestrian matching method in the application scenarios is as follows:

[0126] In this embodiment, the pedestrian matching method is applied to an application scenario of detecting people staying in a location during video surveillance. In this application scenario, it is necessary to detect and report suspicious behaviors caused by people staying or wandering for a long time. By comparing the human features detected in multiple consecutive frames, it is possible to determine whether the person is the same person and whether the person has been staying in the same place for a long time.

[0127] In this application scenario, when a person is detected in a surveillance image, a pedestrian matching method is triggered. Two images are acquired from the monitoring device at predetermined intervals, and pedestrian matching detection is performed on the two images. Furthermore, when the same pedestrian is detected multiple times in succession in two images acquired at predetermined intervals, this indicates that the person in the surveillance image has been staying there for a long time, and this situation can be reported. Here, the predetermined time may be set according to the actual situation, for example, 1 minute, 3 minutes, 5 minutes, etc. Depending on the actual situation, the system may be configured to trigger a report when the number of consecutive detections of the same pedestrian in the surveillance image reaches a predetermined number.

[0128] All the above-mentioned optional technical solutions may be arbitrarily combined to form optional embodiments of the present application, and will not be described one by one here.

[0129] The following are disclosed apparatus embodiments for carrying out the disclosed method embodiments. For details not disclosed in the disclosed apparatus embodiments, please refer to the disclosed method embodiments.

[0130] 10 is a schematic diagram of a pedestrian matching device provided in an embodiment of the present disclosure. As shown in FIG. 10, the pedestrian matching device includes: an acquisition module 1001 for acquiring a first image and a second image; a feature extraction module 1002 for extracting features from the first image and the second image, respectively, to obtain first local features, second local features, first high-level features, and second high-level features, wherein the first local features and the second local features each include a local feature vector corresponding to each important point on the human body; a first feature alignment module 1003 for performing feature alignment on the first local feature and the second local feature to obtain a first fusion feature and a second fusion feature; a second feature alignment module 1004 for performing feature alignment on the first high-level features and the second high-level features to obtain first high-level fusion features and second high-level fusion features; and a matching module 1005 for determining whether the first image and the second image contain the same pedestrian based on the first fusion feature, the second fusion feature, the first high-order fusion feature, and the second high-order fusion feature.

[0131] According to the technical solution provided in the disclosed embodiments, feature extraction is performed on the captured first and second images, respectively, to obtain local features and high-level features corresponding to important points of the human body in the two images, which are respectively referred to as the first local feature, the second local feature, the first high-level feature, and the second high-level feature, feature alignment is performed on the first local feature and the second local feature to obtain first fusion features and second fusion features, feature alignment is performed on the first high-level features and the second high-level features to obtain first high-level fusion features and second high-level fusion features, and whether the same pedestrian is included in the two images is determined based on the first fusion features, the second fusion features, the first high-level fusion features, and the second high-level fusion features. The local features and high-level features of the human body in the images are extracted, and feature alignment is performed on the local features and the high-level features respectively using different feature alignment strategies, incorporating the relationships between the features and integrating the information between the features to obtain more effective features, reducing the impact of abnormal or outlying points in the images on pedestrian matching, and obtaining more accurate matching results.

[0132] In some embodiments, the feature extraction module of the pedestrian matching device comprises: a local feature extraction submodule for extracting human body local features from the first image and the second image, respectively, to obtain first local features and second local features; and an aggregation sub-module for performing feature aggregation on the first local features to obtain first higher-level features and for performing feature aggregation on the second local features to obtain second higher-level features.

[0133] In some embodiments, the human body has two or more important points, as shown in FIG. 11 , in this embodiment, the feature extraction module of the device comprises: a parameter learning submodule 1101 for learning and identifying a first parameter, a second parameter and a third parameter corresponding to the local features according to the local feature vector corresponding to each important point of the human body; a parameter identification sub-module 1102 for generating a query, a key and a value corresponding to each local feature vector according to the first parameter, the second parameter and the third parameter; a weight coefficient determining sub-module 1103 for determining a weight coefficient between each pair of local feature vectors according to a query and a key; an adjustment coefficient determination sub-module 1104 for determining, for any local feature vector, an adjustment coefficient based on a relationship between the local feature vector and each of the other local feature vectors; an aggregation sub-module 1105 for obtaining aggregate features to which the local features correspond according to the weight coefficients, the adjustment coefficients and the values; Each local feature is assigned a corresponding aggregate feature to generate a higher-level feature. Signs and a high-level feature identification sub-module 1106 for identifying the local features, the high-level features including a first local feature and a second local feature, and the high-level features including a first high-level feature and a second high-level feature.

[0134] In some embodiments, with continued reference to FIG. 11 , the first feature alignment module of the apparatus comprises: a local feature calculation sub-module 1107 for calculating a local feature similarity between the first local feature and the second local feature; a first matching feature identification sub-module 1108 for identifying a first matching feature in the first local feature of the second local feature based on the second local feature and local feature similarity; a second matching feature identification sub-module 1109 for identifying a second matching feature in the second local feature of the first local feature based on the first local feature and local feature similarity; a feature extraction sub-module 1110 for performing feature extraction on the first matching feature to obtain a first fusion feature, and for performing feature extraction on the second matching feature to obtain a second fusion feature.

[0135] In some embodiments, the first matching feature identification sub-module of the above device comprises a multiplication unit for calculating a first product of the local feature similarity and the second local feature, and an addition unit for identifying a sum value of the first product and the first local feature as the first matching feature.

[0136] In some embodiments, the second matching feature identification sub-module of the above apparatus comprises: a transposition unit for transposing the local feature similarity to obtain a transposed local feature similarity; a multiplication unit for calculating a second product of the transposed local feature similarity and the first local feature; and an addition unit for identifying a sum value of the second product and the second local feature as the second matching feature.

[0137] In some embodiments, with continued reference to FIG. 11 , the second feature alignment module of the apparatus comprises: a high-level feature similarity calculation submodule 1111 for calculating high-level feature similarity between the first high-level feature and the second high-level feature; a first high-level matching identification sub-module 1112 for identifying a first high-level matching feature in the first high-level feature of the second high-level feature based on the second high-level feature and the high-level feature similarity; a second high-level matching identification submodule 1113 for identifying a second high-level matching feature in the second high-level feature of the first high-level feature based on the first high-level feature and the high-level feature similarity; and a feature extraction sub-module 1114 for performing feature extraction on the first high-order matching feature to obtain a first high-order fusion feature, and for performing feature extraction on the second high-order matching feature to obtain a second high-order fusion feature.

[0138] In some embodiments, with continued reference to FIG. 11 , the matching module of the device may include: a first splicing sub-module 1115 for splicing the first fused feature, the first higher-order fused feature, to obtain a first spliced ​​feature; a second splice sub-module 1116 for splicing the second fused feature, the second higher-order fused feature, to obtain a second spliced ​​feature; and a comparison loss calculation sub-module 1117 for calculating a comparison loss value between the first splice feature and the second splice feature, and determining whether the first image and the second image contain the same pedestrian based on the comparison loss value.

[0139] It should be understood that the magnitude of the numbers of each step in the above embodiments does not mean the order of execution, and the execution order of each process should be determined by its function and inherent logic, and does not arbitrarily limit the implementation process of the embodiments disclosed herein.

[0140] 12 is a schematic diagram of an electronic device 12 provided in an embodiment of the present disclosure. As shown in FIG. 12, the electronic device 12 of the embodiment includes a processor 1201, a memory 1202, and a computer program 1203 stored in the memory 1202 and executable by the processor 1201. When the processor 1201 executes the computer program 1203, it realizes the steps in each of the above method embodiments. Alternatively, when the processor 1201 executes the computer program 1203, it realizes the functions of each module / unit in each of the above device embodiments.

[0141] For example, the computer program 1203 may be divided into one or more modules / units, and the one or more modules / units may be stored in the memory 1202 and executed by the processor 1201 to accomplish the present disclosure. The one or more modules / units may be a series of computer program command sections capable of performing a specific function, and the command sections are intended to explain the process of the computer program 1203 being executed in the electronic device 12.

[0142] The electronic device 12 may be an electronic device such as a desktop computer, a laptop computer, a palmtop computer, or a cloud server. The electronic device 12 may include, but is not limited to, a processor 1201 and a memory 1202. As will be appreciated by those skilled in the art, FIG. 12 is merely an example of the electronic device 12 and is not intended to limit the electronic device 12, which may include more or fewer components than those shown, or may combine certain components or different components; for example, the electronic device may include input / output devices, network access devices, buses, etc.

[0143] Processor 1201 may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor, any common processor, etc.

[0144] The memory 1202 may be an internal storage unit of the electronic device 12, such as a hard disk or RAM of the electronic device 12. The memory 1202 may also be an external storage device of the electronic device 12, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc., that is attached to the electronic device 12. Furthermore, the memory 1202 may include not only the internal storage unit of the electronic device 12 but also an external storage device. The memory 1202 is for storing computer programs and other programs and data required by the electronic device. The memory 1202 may also be used to temporarily store data that has been output or that is to be output.

[0145] Those skilled in the art will understand that, for convenience and brevity, only the division of the above functional units and modules has been used as an example. However, in actual applications, the above functions can be assigned to different functional units or modules as needed, i.e., all or part of the above-described functions can be achieved by dividing the internal structure of the device into different functional units or modules. The functional units and modules in the embodiments may be integrated into a single processing unit, each unit may exist physically independently, or two or more units may be integrated into a single unit. The integrated unit may be implemented in the form of hardware or software functional units. The specific names of the functional units and modules are provided solely for the purpose of easy distinction from one another and do not limit the scope of protection of the present application. For the specific operating processes of the units and modules in the above system, reference may be made to the corresponding processes in the above-described method embodiments, and further description will be omitted here.

[0146] In the above embodiments, the description of each embodiment has its own emphasis, and for the details or parts not described in an embodiment, reference can be made to the relevant descriptions of other embodiments.

[0147] Those skilled in the art can recognize that the units and algorithm steps of each example described in the embodiments disclosed herein can be realized by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed by hardware or software is determined by the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to realize the described functions for each specific application, but such realization should not be considered beyond the scope of this disclosure.

[0148] In the embodiments provided in this disclosure, it should be understood that the disclosed devices / electronic devices and methods can be realized in other ways. For example, the device / electronic device embodiments described above are merely illustrative, and the division into modules or units is merely a logical division of functions. In actual implementation, other division methods may be used. For example, multiple units or components may be combined or integrated into other systems, or some features may be omitted or not implemented. Furthermore, the couplings or direct couplings or communication connections shown or discussed may be indirect couplings or communication connections through several interfaces, devices, or units, and may be electrical, mechanical, or other types of couplings.

[0149] Units described as separate components may or may not be physically separated, and components shown as units may or may not be physical units, i.e., located in one location or distributed across multiple network units, some or all of which may be selected according to actual needs to achieve the objectives of the solutions of this embodiment.

[0150] Note that the functional units in this disclosure may be integrated into one processing unit, each unit may exist physically independently, or two or more units may be integrated into one unit. The integrated unit may be realized in the form of hardware or in the form of a software functional unit.

[0151] The integrated module / unit may be realized in the form of a software functional unit and stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the present disclosure provides that the realization of all or part of the processes in the above-described method embodiments can be accomplished by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the above-described method embodiments can be realized. The computer program may include computer program code, which may be in source code format, object code format, an executable file, or some intermediate format. The computer-readable storage medium may include any entity or device capable of carrying computer program code, such as a recording medium, a U-disk, a removable hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier wave signal, an electrical communication signal, and a software distribution medium. Furthermore, the content contained on a computer-readable storage medium may be increased or decreased as required by the legislation and patent practice of a jurisdiction. For example, in some jurisdictions, the legislation and patent practice may require that a computer-readable storage medium not include electrical carrier signals and telecommunications signals.

[0152] The above-mentioned embodiments are only for illustrating the technical solutions of the present disclosure, and are not intended to limit the same. Although the present disclosure has been described in detail with reference to the above-mentioned embodiments, those skilled in the art may still amend the technical solutions described in the above-mentioned embodiments or equivalently replace some technical features thereof, and it should be understood that such amendments or replacements will not deviate from the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and all of them should be included in the protection scope of the present disclosure.

Claims

1. A pedestrian matching method, comprising: acquiring a first image and a second image; performing feature extraction on each of the first image and the second image to obtain first local features, second local features, first high-level features, and second high-level features, wherein the first local features and the second local features each include a local feature vector corresponding to each important point on a human body; performing feature alignment on the first local features and the second local features to obtain first fusion features and second fusion features; performing feature alignment on the first high-level features and the second high-level features to obtain first high-level fusion features and second high-level fusion features; determining whether the first image and the second image contain the same pedestrian based on the first fusion feature, the second fusion feature, the first high-order fusion feature, and the second high-order fusion feature. A pedestrian matching method characterized by:

2. The step of extracting features from the first image and the second image to obtain first local features, second local features, first high-level features, and second high-level features includes: extracting human body local features from the first image and the second image, respectively, to obtain the first local features and the second local features; performing feature aggregation on the first local features to obtain first high-level features, and performing feature aggregation on the second local features to obtain second high-level features.

2. The method of claim 1 .

3. The human body has two or more important points; The step of performing feature aggregation on the first local features to obtain a first high-level feature and performing feature aggregation on the second local features to obtain a second high-level feature includes: learning and identifying first, second and third parameters corresponding to the local features based on the local feature vectors corresponding to each of the human body important points; generating a query, a key, and a value corresponding to each of the local feature vectors based on the first parameter, the second parameter, and the third parameter; determining a weighting factor between each pair of the local feature vectors based on the query and the key; for any one of the local feature vectors, identifying an adjustment factor based on a relationship between the local feature vector and each of the other local feature vectors; obtaining an aggregate feature corresponding to the local feature based on the weight coefficient, the adjustment coefficient, and the value; and wherein the higher-level features are identified by aggregate features to which each of the local features corresponds.

4. The step of performing feature alignment on the first local features and the second local features to obtain a first fusion feature and a second fusion feature includes: calculating a local feature similarity between the first local feature and the second local feature; identifying a first matching feature in the first local feature of the second local feature based on the local feature similarity; identifying a second matching feature in the second local feature of the first local feature based on the first local feature and the local feature similarity; performing feature extraction on the first matching feature to obtain the first fusion feature; and performing feature extraction on the second matching feature to obtain the second fusion feature.

2. The method of claim 1 .

5. The step of identifying a first matching feature in the first local feature of the second local feature based on the local feature similarity includes: calculating a first product of the local feature similarity and the second local feature; and identifying a sum of the first product and the first local feature as the first matching feature; The step of identifying a second matching feature in the second local feature of the first local feature based on the local feature similarity includes: transposing the local feature similarity to obtain a transposed local feature similarity; calculating a second product of the transposed local feature similarity and the first local feature; and identifying a sum value of the second product and the second local feature as the second matching feature.

5. The method of claim 4.

6. performing feature alignment on the first high-level features and the second high-level features to obtain a first high-level fusion feature and a second high-level fusion feature; calculating a high-level feature similarity between the first high-level feature and the second high-level feature; Identifying a first high-level feature matching the second high-level feature in the first high-level feature based on the second high-level feature and the high-level feature similarity; Identifying a second high-level feature in the second high-level feature that matches the first high-level feature based on the first high-level feature and the high-level feature similarity; performing feature extraction on the first high-order matching feature to obtain the first high-order fusion feature; and performing feature extraction on the second high-order matching feature to obtain the second high-order fusion feature.

2. The method of claim 1 .

7. determining whether the first image and the second image include the same pedestrian based on the first fusion feature, the second fusion feature, the first high-level fusion feature, and the second high-level fusion feature, splicing the first fusion feature and the first higher-order fusion feature to obtain a first spliced ​​feature; splicing the second fusion feature and the second higher-order fusion feature to obtain a second spliced ​​feature; calculating a comparative loss value between the first splice feature and the second splice feature, and determining whether the first image and the second image contain the same pedestrian based on the comparative loss value.

2. The method of claim 1 .

8. A pedestrian matching device, an acquisition module for acquiring a first image and a second image; a feature extraction module for performing feature extraction on each of the first image and the second image to obtain first local features, second local features, first high-level features, and second high-level features, a feature extraction module, in which the first local feature and the second local feature respectively include local feature vectors corresponding to each important point of a human body; a first feature alignment module for performing feature alignment on the first local features and the second local features to obtain a first fusion feature and a second fusion feature; and a feature alignment module for performing feature alignment on the first high-level features and the second high-level features to obtain a first high-level fusion feature and a second high-level fusion feature. and a second feature alignment module for obtaining a second higher-order fusion feature. a matching module for determining whether the first image and the second image include the same pedestrian based on the first fusion feature, the second fusion feature, the first high-order fusion feature, and the second high-order fusion feature. A pedestrian matching device characterized by:

9. 1. An electronic device comprising: a memory; a processor; and a computer program stored in the memory and executable by the processor, The processor, when executing the computer program, performs the steps of the method of claim 1. An electronic device characterized by:

10. A computer-readable storage medium on which a computer program is stored, The computer program, when executed by a processor, implements the steps of the method of claim 1. A computer-readable storage medium comprising:

Images