Pedestrian matching method, device, electronic device, and computer-readable storage medium

The pedestrian matching method improves accuracy by extracting and aligning local and higher-order features in images to account for image quality variations and outliers, ensuring precise identification of matching pedestrians.

JP2024542639AActive Publication Date: 2024-11-15SHENZHEN XUMI YUNTU SPACE TECH CO LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
JP2024532210
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2021-12-01
Filing Date
2021-12-31
Publication Date
2024-11-15
Estimated Expiration
2041-12-31

AI Technical Summary

Technical Problem

Traditional pedestrian matching methods are inaccurate due to abnormal points or outliers in images, such as varying quality, obstruction, and different body postures, leading to insufficient matching results.

Method used

A pedestrian matching method that involves feature extraction to obtain local and higher-order features, followed by feature alignment to fuse these features, and identification based on fusion features to determine if the same pedestrian is present in two images, using different alignment strategies for local and high-order features to reduce the influence of outliers.

Benefits of technology

The method enhances the accuracy of pedestrian matching by effectively incorporating relationships between features and integrating information, reducing the impact of abnormal points and resulting in more accurate matching results.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024542639000001_ABST
    Figure 2024542639000001_ABST
Patent Text Reader

Abstract

A pedestrian matching method, device, electronic device, and computer-readable storage medium include the steps of: acquiring a first image and a second image; performing feature extraction on each of the first and second images to obtain first local features, second local features, first high-level features, and second high-level features, where the first local features and second local features include local feature vectors corresponding to important points on the human body; performing feature alignment on the first local features and second local features to obtain first fusion features and second fusion features; performing feature alignment on the first high-level features and second high-level features to obtain first high-level fusion features and second high-level fusion features; and determining whether the first and second images contain the same pedestrian based on the first fusion features, second fusion features, first high-level fusion features, and second high-level fusion features. This method can reduce the impact of abnormal points in the images on pedestrian matching, resulting in more accurate matching results.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure relates to the field of image processing technology, and in particular to a pedestrian matching method, device, electronic device, and computer-readable storage medium. [Background technology]

[0002] Image matching refers to a method of finding similar image targets by analyzing similarity and consistency according to the correspondence of image contents, features, structures, relationships, textures, grayscales, etc. The image matching method may be used for pedestrian matching, which can compare whether pedestrians appearing in different images are the same pedestrian.

[0003] Most of the conventional pedestrian matching methods directly use global feature matching of the human body or are based on local feature matching of the human body. However, in practical situations, most images usually have different qualities, for example, the human body is occluded in the image, the human body posture is different, and the image brightness is different, that is, there will be some abnormal points or outliers. When performing pedestrian matching, it is easy for the feature alignment to not match, which leads to the problem that the pedestrian matching result is not accurate enough. Summary of the Invention [Problem to be solved by the invention]

[0004] In view of this, the embodiments of the present disclosure provide a pedestrian matching method, apparatus, electronic device and computer-readable storage medium to solve the problem in the prior art that pedestrian matching results are inaccurate due to the existence of some abnormal points or outliers in the image. [Means for solving the problem]

[0005] The first aspect of the disclosed embodiment is acquiring a first image and a second image; performing feature extraction on the first image and the second image, respectively, to obtain a first local feature, a second local feature, a first high level feature, and a second high level feature, wherein the first local feature and the second local feature each include a local feature vector corresponding to each important point of a human body; performing feature alignment on the first local feature and the second local feature to obtain a first fusion feature and a second fusion feature; performing feature alignment on the first high-level feature and the second high-level feature to obtain a first high-level fusion feature and a second high-level fusion feature; and determining whether the first image and the second image contain the same pedestrian based on the first fusion feature, the second fusion feature, the first high-order fusion feature, and the second high-order fusion feature.

[0006] A second aspect of the disclosed embodiment is an acquisition module for acquiring a first image and a second image; a feature extraction module for performing feature extraction on the first image and the second image, respectively, to obtain a first local feature, a second local feature, a first high-level feature, and a second high-level feature, wherein the first local feature and the second local feature respectively include local feature vectors corresponding to each important point of a human body; a first feature alignment module for performing feature alignment on the first local feature and the second local feature to obtain a first fusion feature and a second fusion feature; a second feature alignment module for performing feature alignment on the first high-level feature and the second high-level feature to obtain a first high-level fusion feature and a second high-level fusion feature; A pedestrian matching device is provided, comprising: a matching module for determining whether the first image and the second image contain the same pedestrian based on the first fusion feature, the second fusion feature, the first high-order fusion feature, and the second high-order fusion feature.

[0007] A third aspect of an embodiment of the present disclosure provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable by the processor, the electronic device implementing the steps of the above method when the processor executes the computer program.

[0008] In a fourth aspect of an embodiment of the present disclosure, there is provided a computer-readable storage medium having a computer program stored thereon, the computer program implementing the steps of the above method when executed by a processor. Effect of the Invention

[0009] Beneficial effects of the disclosed embodiment compared with the prior art: by performing feature extraction on the first and second images respectively obtained, obtaining local features and high-level features corresponding to important points of the human body in the two images, which are respectively denoted as the first local feature, the second local feature, the first high-level feature and the second high-level feature, performing feature alignment on the first local feature and the second local feature, obtaining the first fusion feature and the second fusion feature, performing feature alignment on the first high-level feature and the second high-level feature, obtaining the first high-level fusion feature and the second high-level fusion feature, and determining whether the same pedestrian is included in the two images according to the first fusion feature, the second fusion feature, the first high-level fusion feature and the second high-level fusion feature. Extracting the human body local features and high-level features in the images, performing feature alignment on the local features and the high-level features respectively using different feature alignment strategies, incorporating the relationship between the features, and integrating the information between the features, obtaining more effective features, reducing the impact of abnormal points or outliers in the images on pedestrian matching, and obtaining more accurate matching results. [Brief description of the drawings]

[0010] In order to more clearly describe the technical solutions in the embodiments of the present disclosure, the following briefly introduces drawings necessary for describing the embodiments or prior art. Obviously, the drawings described below are only some embodiments of the present disclosure, and those skilled in the art can obtain other drawings based on these drawings without the need for creative work.

[0011] [Figure 1] 1 is a flow chart of a pedestrian matching method provided in an embodiment of the present disclosure. [Diagram 2] 1 is a flowchart provided in an embodiment of the present disclosure for determining whether a first image and a second image contain the same pedestrian based on a first fusion feature, a second fusion feature, a first higher-order fusion feature, and a second higher-order fusion feature. [Diagram 3] 1 is a flowchart provided in an embodiment of the present disclosure for performing feature aggregation on local features to obtain higher-level features. [Figure 4] FIG. 2 is a schematic diagram of important points numbers of the human body provided in the examples of the present disclosure. [Diagram 5] 1 is a flowchart provided in an embodiment of the present disclosure for performing feature alignment on a first local feature and a second local feature to obtain a first fusion feature and a second fusion feature. [Figure 6] 10 is a flowchart for performing feature alignment on a first local feature and a second local feature, respectively, provided in an embodiment of the present disclosure. [Figure 7] 1 is a flowchart provided in an embodiment of the present disclosure for performing feature alignment on a first high-level feature and a second high-level feature to obtain a first high-level fusion feature and a second high-level fusion feature. [Figure 8] 1 is a flowchart for performing feature alignment on a first high-level feature and a second high-level feature provided in an embodiment of the present disclosure. [Figure 9] 1 is a flow chart of another pedestrian matching method provided in an embodiment of the present disclosure. [Figure 10] FIG. 2 is a structural schematic diagram of a pedestrian matching device provided in an embodiment of the present disclosure. [Figure 11]FIG. 2 is a structural schematic diagram of another pedestrian matching device provided in an embodiment of the present disclosure. [Figure 12] FIG. 2 is a structural schematic diagram of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] In the following description, for the purposes of explanation, not limitation, specific details such as specific system structures, techniques, etc. are provided to thoroughly understand the embodiments of the present disclosure. However, those skilled in the art should understand that the present disclosure can be realized in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present disclosure with unnecessary details.

[0013] Hereinafter, a pedestrian matching method and device according to an embodiment of the present disclosure will be described in detail with reference to the drawings.

[0014] 1 is a flow chart of a pedestrian matching method provided in an embodiment of the present disclosure. As shown in FIG. 1, the pedestrian matching method includes steps S101 to S105.

[0015] In S101, a first image and a second image are acquired.

[0016] The first image and the second image are images for matching pedestrians, and in this embodiment, the first image and the second image are compared to determine whether they contain the same pedestrian.

[0017] Specifically, the first image and the second image may be obtained from a database or may be extracted in real time from an image collected by a monitoring device. In one embodiment, when the pedestrian matching method is used for a scene where people stay, the first image and the second image may both be extracted from an image collected by a monitoring device, and two images are extracted from the images collected by the monitoring device at predetermined times and compared to determine whether the same pedestrians are included, and further determine whether there are people staying at the same location for a long time. Furthermore, in one embodiment, when people are detected from the images collected by the monitoring device, the pedestrian matching method can be triggered, and two images are extracted at predetermined times, and whether people stay at the same location for a long time can be determined by pedestrian matching. In other embodiments, the pedestrian matching method can also be applied to other scenes.

[0018] In addition, in the embodiments disclosed in this application, the concepts of "first", "second", etc. mentioned are merely for distinguishing different names and do not indicate actual meaning.

[0019] In S102, feature extraction is performed on each of the first image and the second image to obtain a first local feature, a second local feature, a first high level feature, and a second high level feature.

[0020] To allow a computer to "understand" an image, we typically extract useful data or information from the image to obtain "non-image" representations or descriptions of the image, such as numbers, vectors, codes, etc. This process is called feature extraction, and these extracted "non-image" representations or descriptions are features. For images, each image has its own characteristics that distinguish it from other types of images, such as natural features that are intuitively felt, such as brightness, edges, texture, color, etc., and other features that require transformations or processing to obtain, such as moments, histograms, principal components, etc.

[0021] In some embodiments, feature extraction for an image can be realized by using a neural network. In this embodiment, since the purpose is to perform pedestrian matching, it is necessary to extract the features of the human body contained in the image during feature extraction, so that the feature extraction can be performed by using a human body detection module and a human body key point estimation model.

[0022] The local features include local features of local feature vectors corresponding to each important point of the human body, and in this embodiment, for the purpose of distinction, the features extracted from the first image are denoted as first local features and first higher level features, and the features extracted from the second image are denoted as second local features and second higher level features.

[0023] In some embodiments, the step of performing feature extraction on the first image and the second image, respectively, to obtain a first local feature, a second local feature, a first high-level feature, and a second high-level feature includes a step of performing human body local feature extraction on the first image and the second image, respectively, to obtain a first local feature and a second local feature, and a step of performing feature aggregation on the first local features to obtain a first high-level feature, and performing feature aggregation on the second local features to obtain a second high-level feature.

[0024] In a specific embodiment, the step of performing feature extraction on the first image to obtain the first local feature includes the steps of inputting the first image into a human body detection model, performing human body detection on the first image, and obtaining a human body detection frame in the first image, and inputting the human body detection frame into a human body important point estimation model, and obtaining the first local feature, which is a feature of each human body important point corresponding to the human body detection frame. Of course, the step of performing feature extraction on the second image to obtain the second local feature is similar to the local feature extraction on the first image, and will not be described here repeatedly. Here, any one kind of human body detection model and human body important point estimation model can be used to perform feature extraction on the image.

[0025] Furthermore, in a specific embodiment, after the human body detection model performs feature extraction on the image to obtain a human body detection frame, the human body detection frame is input to the human body key point estimation model, where the specific process of the human body key point estimation model processing the input human body detection frame is to first convert each feature map into 1024 by 1*1 convolution in an integrated manner, and then pool the length and width of the feature map into [16,8] by ROIAlign (calibration), and obtain the dimension of each human body feature map G1 as [1024,16,8]. G1 is convolved through three blocks (conv(3,3), BN, relu) to obtain feature maps [1024,16,8], [512,16,8], and [256,16,8] in order. Finally, we obtain the heat map of all the important points of the body by conv(1,1), BN, and relu, and if there are 14 important points of the body, the heat map may be expressed as [14,16,8]. Here, BN (Batch Normalization) normalizes the input of each layer. relu (Rectified Linear Unit), also known as the modified real unit, is an activation function often used in artificial neural networks, and refers to a nonlinear function usually represented by a ramp function and its variant.

[0026] The heatmap dimension of each important point is [1,16,8], then iteratively stack to change its dimension to [1024,16,8], multiply it with the feature map G1 of the human body [1024,16,8] to obtain the feature map of the important point [2048,16,8], and finally perform global average pooling to obtain the feature vector of the important point

[1024] . Similarly, according to the above method, the feature vector of each important point of the human body can be obtained, and in the first image, each important point of the human body is the first local feature, and in the second image, each important point of the human body is the second local feature.

[0027] In some embodiments, after obtaining local features, higher-level features can be obtained by performing feature aggregation on each local feature, and similarly, in the present embodiments, the higher-level feature obtained by aggregating the first local feature is referred to as the first higher-level feature, and the higher-level feature obtained by performing feature aggregation on the second local feature is referred to as the second higher-level feature. Specifically, the specific process of performing feature aggregation on the local features to obtain the higher-level feature will be described in detail in a later embodiment, and will not be described again here.

[0028] In step S103, feature alignment is performed on the first local feature and the second local feature to obtain a first fusion feature and a second fusion feature.

[0029] Since the postures of pedestrians appearing in different images at different times are not the same, when pedestrian matching is performed for different images, image features are aligned to search for identical points in the two images and to determine whether the same pedestrian is included therein. In this embodiment, feature alignment is performed for the human body local features extracted from the two images, respectively, and the features obtained by the alignment are referred to as the first fusion feature and the second fusion feature. Specifically, the specific steps of aligning the first local feature and the second local feature to obtain the first fusion feature and the second fusion feature will be described in detail in the following embodiment.

[0030] In step S104, feature alignment is performed on the first high-level feature and the second high-level feature to obtain a first high-level fusion feature and a second high-level fusion feature.

[0031] In this embodiment, when extracting features, low-level human body local features are obtained, and the high-level features of the image are obtained by the low-level human body local features, so that when aligning features, low-level feature alignment is performed for the low-level local features, and high-level feature alignment strategy is used for the high-level features, so as to fully utilize the collected image features and improve the extracted features.Furthermore, in this embodiment, after aligning the high-level features, the obtained features are referred to as the first high-level fusion feature and the second high-level fusion feature.

[0032] In S105, it is determined whether the same pedestrian is included in the first image and the second image based on the first fusion feature, the second fusion feature, the first high-order fusion feature, and the second high-order fusion feature.

[0033] In one embodiment, as shown in FIG. 2, the step of determining whether the first image and the second image contain the same pedestrian based on the first fusion feature, the second fusion feature, the first high-order fusion feature and the second high-order fusion feature includes steps S201 to S203.

[0034] In S201, a first fusion feature and a first higher-order fusion feature are spliced ​​to obtain a first spliced ​​feature.

[0035] In S202, splicing is performed on the second fusion feature and the second higher-order fusion feature to obtain a second spliced ​​feature.

[0036] In some embodiments, feature splicing can be completed by a concat function, which is for connecting two or more arrays. In this embodiment, the feature obtained by splicing the first fusion feature or the first higher order fusion feature is referred to as the first splice feature, and the feature obtained by splicing the second fusion feature or the second higher order fusion feature is referred to as the second splice feature. In other embodiments, feature splicing can be realized in other ways.

[0037] In S203, a comparative loss value between the first splice feature and the second splice feature is calculated, and it is determined whether the same pedestrian is included in the first image and the second image based on the comparative loss value.

[0038] In some embodiments, a comparative loss value between the first splice feature and the second splice feature is calculated by a comparative loss function, which can be conveniently expressed as a matching degree for a sample, and in this embodiment, the comparative loss function is used to calculate a comparative loss value between the first splice feature and the second splice feature, which can be used to describe the matching degree between the first image and the second image. Specifically, in this embodiment, any comparative loss function can be used to calculate a comparative loss value between two splice features.

[0039] After obtaining the comparison loss value, it can be determined whether the first image and the second image contain the same pedestrian based on the comparison loss value. In one embodiment, determining whether the first image and the second image contain the same pedestrian based on the comparison loss value can be performed by setting a predetermined condition, and if the comparison loss value meets the predetermined condition, it can be determined that the first image and the second image contain the same pedestrian, the predetermined condition can be set according to an actual situation, and the higher the matching degree between the first image and the second image is determined based on the comparison loss value, the higher the possibility that the first image and the second image contain the same pedestrian.

[0040] In another embodiment, after obtaining the first splice feature and the second splice feature, it is possible to determine whether the first image and the second image contain the same pedestrian in another manner based on the first splice feature and the second splice feature, for example, by calculating the similarity between the first splice feature and the second splice feature, and determining whether the first image and the second image contain the same pedestrian based on the similarity.

[0041] According to the technical solution provided in the disclosed embodiment, by performing feature extraction on the first and second images respectively, obtain local features and high-level features corresponding to important points of the human body in the two images, which are respectively denoted as a first local feature, a second local feature, a first high-level feature and a second high-level feature, perform feature alignment on the first local feature and the second local feature, obtain a first fusion feature and a second fusion feature, perform feature alignment on the first high-level feature and the second high-level feature, obtain a first high-level fusion feature and a second high-level fusion feature, and determine whether the same pedestrian is included in the two images according to the first fusion feature, the second fusion feature, the first high-level fusion feature and the second high-level fusion feature. Extract the human body local features and high-level features in the images, perform feature alignment on the local features and the high-level features respectively using different feature alignment strategies, incorporate the relationship between the features, and use the information between the features in an integrated manner to obtain more effective features, reduce the impact of abnormal points or outliers in the images on pedestrian matching, and obtain a more accurate matching result.

[0042] In some embodiments, the important points of the human body include two or more, and in this embodiment, as shown in FIG. 3 , the step of performing feature aggregation on the local features to obtain higher-level features includes steps S301 to S306, where the local features include a first local feature and a second local feature, and the higher-level features include a first higher-level feature and a second higher-level feature.

[0043] In S301, a first parameter, a second parameter and a third parameter corresponding to the local features are learned and identified based on the local feature vectors corresponding to the important points of each human body.

[0044] A human body generally includes two or more important points, and when extracting the human body local features, a corresponding local feature vector can be obtained for each important point of the human body, and the number of the local feature vectors included in the local features corresponds to the number of important points of the human body. The local features in this embodiment include a first local feature and a second local feature extracted from the first image and the second image respectively.

[0045] Specifically, learning the first parameter, the second parameter, and the third parameter corresponding to the local feature based on the local feature vector corresponding to each important point of the human body may be realized in any manner, for example, by a neural network, and in a specific embodiment, by a Transformer network (submitted by Ashish Vaswani). In some embodiments, the ranges of the first parameter, the second parameter, and the third parameter corresponding to the local vector are all within R 1024×256 It is.

[0046] S302: generating a query, a key and a value corresponding to each local feature vector based on the first parameter, the second parameter and the third parameter.

[0047] Query, key and value are concepts in the Transformer network. Based on the first parameter, the second parameter and the third parameter, each local feature vector generates a corresponding query, key and value, which may be expressed as follows:

[0048] query:Q i =f i *W q And W q represents the first parameter, Key:K i =f i *W k And W k represents the second parameter, value:V i =f i *W v And W v represents the third parameter.

[0049] S303: determining a weighting factor between each pair of local feature vectors based on the query and the key.

[0050] In some embodiments, the step of determining a weight coefficient between each pair of local feature vectors based on a query and a key includes selecting two local feature vectors as a first target local feature vector and a second target local feature vector, and calculating a weight coefficient between the first target local feature vector and the second target local feature vector using the query of the first target local feature vector and the key of the second target local feature vector.

[0051] Furthermore, in a specific embodiment, a weighting coefficient between the first target local feature vector and the second target local feature vector may be calculated according to the query of the first target local feature vector and the key of the second target local feature vector, and may be expressed as follows: JPEG2024542639000002.jpg37167

[0052] a ij represents the weight coefficient between the local feature vectors of number i and number j, where i and j respectively represent the numbers corresponding to the important points of the human body. The important points of the human body can be numbered in advance. Q i represents the query that the first target local feature vector corresponds to, and K j represents the key of the second target local feature vector, and K j T is K j , and 256 represents Q i , K j dimensionality, softmax() represents the logistic regression algorithm, and a ij represents the weighting coefficient between the first target local feature vector and the second target local feature vector. Here, the transpose is defined as: when all elements of A are mirror-inverted around a 45-degree ray from the lower right starting from the element in the first row and first column, the transpose of A is obtained.

[0053] According to the above method, the weight coefficient between the local feature vectors corresponding to each pair of important points of the human body can be calculated.

[0054] In step S304, for any one of the local feature vectors, an adjustment factor is determined based on a relationship between the local feature vector and each of the other local feature vectors.

[0055] The relationship between the local feature vector and other local feature vectors can be determined from the preset numbers for the important points of the human body. In a specific embodiment, a number is set for each important point of the human body, and as shown in Fig. 4, the head is 1, the neck is 2, the left shoulder is 3, the left elbow is 4, the right elbow is 5, the left hip is 6, the left knee is 7, and the left foot is 8. When performing pedestrian matching, the closer the positions are, the greater the influence between the important points of the human body is. Therefore, in this embodiment, an adjustment coefficient is set to aggregate the important points according to the degree of influence between them to obtain high-level features.

[0056] In a specific embodiment, the adjustment coefficient is determined based on the relationship between the local feature vector and other local feature vectors, and may be expressed as follows: JPEG2024542639000003.jpg31148

[0057] p is an adjustment coefficient, r is a hyperparameter, the value of which can be defined according to the actual situation, and i and j are the numbers of the important points, respectively.

[0058] S305: Obtaining aggregate features to which the local features correspond based on the weight coefficients, the adjustment coefficients and the values.

[0059] In some embodiments, the step of obtaining aggregate features to which local features correspond based on the weighting coefficients, the adjustment coefficients, and the values ​​may be expressed as follows: JPEG2024542639000004.jpg26154

[0060] f i at represents the aggregate feature corresponding to the local feature vector of number i, and a ijrepresents the weighting coefficient between the local feature vectors of number i and number j, p represents the adjustment coefficient, and V j represents the value to which the local feature vector with number j corresponds, i, j respectively represent the numbers of the important points, and 14 represents the number of local feature vectors (important points of the human body).

[0061] In S306, a higher level feature is identified according to the aggregate feature to which each local feature corresponds.

[0062] In step S305, each of the important points (local feature vectors) of the human body can obtain a corresponding aggregate feature, and each aggregate feature is combined together to obtain a corresponding high-level feature. In this embodiment, the first local feature is aggregated to obtain a first high-level feature, and the second local feature is aggregated to obtain a second high-level feature.

[0063] Furthermore, the process of aggregating local features to obtain high-level features may be realized by a transformer model.

[0064] According to the technical solution provided in the disclosed embodiment, the transformer model learns the corresponding first parameter, second parameter, and third parameter for the input first local feature and second local feature, respectively, calculates the query, key, and value of each local feature vector according to the first parameter, second parameter, and third parameter, and then calculates the weight coefficient between each pair of local feature vectors according to the query and key corresponding to each local feature vector, and determines the adjustment coefficient according to the relationship between each pair of local feature vectors, and finally obtains a new local feature by aggregation according to the weight coefficient, the value corresponding to the local feature vector, and the adjustment coefficient, and finally obtains a fusion feature by aggregation of local features. This embodiment designs the adjustment coefficient according to the relationship between local feature vectors, i.e., the a priori topological structure relationship of important points of the human body, and adds the influence degree between the local feature vectors to the feature aggregation through the adjustment coefficient, thereby improving the high-level feature obtained by aggregation.

[0065] In some embodiments, as shown in FIG. 5, performing feature alignment on the first local feature and the second local feature to obtain the first fusion feature and the second fusion feature includes steps S501 to S504.

[0066] In S501, a local feature similarity between a first local feature and a second local feature is calculated.

[0067] The calculation of the similarity between the local features may be realized by any one of methods. In one embodiment, the first local feature and the second local feature are mapped into the same feature space, and the cosine distance between the two local features is calculated as the feature similarity between the two local features. Furthermore, the step of mapping the first local feature and the second local feature into the same feature space may be realized by normalizing the distribution of the first local feature and the second local feature. In one embodiment, the step of mapping the first local feature and the second local feature into the same feature space and calculating the cosine distance between the two local features as the feature similarity between the two local features may be expressed as follows: M1 = normalize(F p )*[normalize(F g )] T

[0068] Fp represents the first local feature, Fg represents the second local feature, normalize() represents normalization processing of the local feature, and M1 represents the similarity between the first local feature Fp and the second local feature Fg.

[0069] In other embodiments, the calculation of the local feature similarity between the first local feature and the second local feature may be realized in other manners.

[0070] S502: Identifying a first matching feature in the first local feature of the second local feature based on the second local feature and local feature similarity.

[0071] In some embodiments, the step of identifying a first matching feature in the first local feature of the second local feature based on the second local feature and the local feature similarity includes the steps of calculating a first product of the local feature similarity and the second local feature, and identifying a sum value of the first product and the first local feature as the first matching feature.

[0072] A first product of the local feature similarity and the second local feature is calculated to represent points in the first local feature that match the second local feature; and based on the matching points, it is possible to identify which points in the first local feature should be used as key points and eliminate outlying or occluded points.

[0073] A sum value of the first product and the first local feature is calculated, and the original feature including the first local feature also includes a relationship with the second local feature. In this embodiment, the sum value is referred to as a first matching feature, and based on the first matching feature, it is possible to identify what points in the first image need to be matched with the second image.

[0074] In a specific embodiment, a first product of the local feature similarity and a second local feature is calculated; F g 1 =M1*F g It may also be expressed as:

[0075] F g 1 represents the first product, M1 represents the local feature similarity, and Fg represents the second local feature.

[0076] A sum value of the first product and the first local feature is identified as a first matching feature; F p 2 =F p +F g 1 It is expressed as:

[0077] F p 2Let Fp denote the first matching feature and Fp the first local feature.

[0078] S503: identifying a second matching feature in the second local feature of the first local feature based on the first local feature and the local feature similarity.

[0079] Similar to identifying the first matching feature, the second matching feature can be identified based on the first local feature and the local feature similarity, except that in this embodiment, the local feature similarity is first transposed, and the transposed local feature similarity is used to identify the second matching feature.

[0080] In some embodiments, the step of identifying a second matching feature in the second local feature of the first local feature based on the first local feature and the local feature similarity includes the steps of transposing the local feature similarity to obtain a transposed local feature similarity, calculating a second product of the transposed local feature similarity and the first local feature, and identifying a sum value of the second product and the second local feature as the second matching feature.

[0081] A second product of the post-transposition local feature similarity and the first local feature is calculated to represent points in the second local feature that match with the first local feature; and based on the matching points, it is possible to identify which points in the second local feature should be used as key points and eliminate outlying or occluded points.

[0082] A sum value of the second product and the second local feature is calculated, and the original feature including the first local feature also includes a relationship with the second local feature. In this embodiment, the sum value is referred to as a second matching feature. Based on the second matching feature, it is possible to identify what points in the second image need to be matched with the first image and what points do not match.

[0083] In a specific embodiment, a second product of the local feature similarity after transposition and the first local feature is calculated, and F p 1 =M 1T *F p It may also be expressed as:

[0084] F p 1 represents the second product, and M 1 T represents the local feature similarity after transposition, and F p represents the first local feature.

[0085] A sum value of the second product and the second local feature is identified as a second matching feature; F g 2 =F g +F p 1 It is expressed as:

[0086] F g 2 represents the second matching feature, F g represents the second local feature.

[0087] In S504, feature extraction is performed on the first matching feature to obtain a first fusion feature, and feature extraction is performed on the second matching feature to obtain a second fusion feature.

[0088] After the first matching feature and the second matching feature are obtained, feature extraction is performed respectively, and new features are obtained by integrating the low-level features after alignment, namely the first fusion feature and the second fusion feature.

[0089] In some embodiments, feature extraction is performed for each input matching feature, which can be realized by fc+relu. The fc layer is a fully connected layer, and each node of the fully connected layer is connected to all nodes of the previous layer and is used to combine previously extracted features. The Relu layer is usually composed of activation functions. The matching features include a first matching feature and a second matching feature.

[0090] In a specific embodiment, as shown in FIG. 6, a step flow chart of performing feature alignment on the first local feature and the second local feature respectively, p is the first local feature, F g is the second local feature, F p t is the first fusion feature, F g t represents the second fusion feature.

[0091] According to the technical solution provided in the disclosed embodiment, the local feature similarity between local features is calculated, a first matching feature and a second matching feature are calculated according to the local feature similarity, and finally, feature extraction is performed on the first matching feature and the second matching feature respectively to obtain new features aligned with low-level features, i.e., a first fusion feature and a second fusion feature. Then, pedestrian matching is performed based on the aligned local features, so that a more accurate matching result can be obtained.

[0092] In some other embodiments, as shown in FIG. 7, the step of performing feature alignment on the first high-level feature and the second high-level feature and obtaining the first high-level fusion feature and the second high-level fusion feature includes steps S701 to S704.

[0093] In S701, a high-level feature similarity between a first high-level feature and a second high-level feature is calculated.

[0094] The calculation of the similarity between the high-level features may be implemented in any manner. In one embodiment, the similarity between the high-level features may be learned and determined by a network, which may be expressed as follows: JPEG2024542639000005.jpg25170

[0095] JPEG2024542639000006.jpg25170A represents a network parameter matrix. The dimension of network parameter matrix A is [1024, 1024].

[0096] In other embodiments, the calculation of the similarity between the first high-level feature and the second high-level feature may be realized in any other manner.

[0097] In S702, a first high-level matching feature in the first high-level feature of the second high-level feature is identified based on the second high-level feature and the high-level feature similarity.

[0098] In some embodiments, the step of identifying a first higher-order matching feature in the first higher-order feature of the second higher-order feature based on the second higher-order feature and the high-order feature similarity includes a step of calculating a third product of the high-order feature similarity and the second higher-order feature, and a step of identifying a sum value of the third product and the first higher-order feature as the first higher-order matching feature.

[0099] In a specific embodiment, a third product of the high-level feature similarity and the second high-level feature is calculated; It may also be represented as JPEG2024542639000007.jpg22123.

[0100] JPEG2024542639000008.jpg21170

[0101] A sum value of the third product and the first high-order feature is identified as a first high-order matching feature; It is represented as JPEG2024542639000009.jpg37155.

[0102] JPEG2024542639000010.jpg16170

[0103] In S703, a second high-level matching feature in the second high-level feature of the first high-level feature is identified based on the first high-level feature and the high-level feature similarity.

[0104] Similar to identifying the first higher-order matching feature, the second higher-order matching feature can be identified based on the first higher-order feature and the higher-order feature similarity. The difference is that in this embodiment, the higher-order feature similarity is first transposed, and the transposed higher-order feature similarity is used to identify the second higher-order matching feature.

[0105] In some embodiments, the step of identifying a second higher-order matching feature in the second higher-order feature of the first higher-order feature based on the first higher-order feature and the high-order feature similarity includes the steps of transposing the high-order feature similarity to obtain a transposed high-order feature similarity, calculating a fourth product of the transposed high-order feature similarity and the first higher-order feature, and identifying a sum value of the fourth product and the second higher-order feature as the second higher-order matching feature.

[0106] In a specific embodiment, a fourth product of the transposed high-level feature similarity and the first high-level feature is calculated; It may also be represented as JPEG2024542639000011.jpg24117.

[0107] JPEG2024542639000012.jpg20170

[0108] A sum value of the fourth product and the second high-level feature is identified as a second high-level matching feature; JPEG2024542639000013.jpg27159

[0109] JPEG2024542639000014.jpg16170

[0110] In S704, feature extraction is performed on the first high-order matching feature to obtain a first high-order fusion feature, and feature extraction is performed on the second high-order matching feature to obtain a second high-order fusion feature.

[0111] After the first high-level matching feature and the second high-level matching feature are obtained, feature extraction is performed respectively, and new features are obtained by integrating the high-level features after alignment, namely the first high-level fusion feature and the second high-level fusion feature.

[0112] In some embodiments, feature extraction is performed on each input high-level matching feature, which can be realized by fc+relu. The fc layer is a fully connected layer, and each node of the fully connected layer is connected to all nodes of the previous layer and is used to combine previously extracted features. The Relu layer is usually composed of activation functions. The high-level matching features include a first high-level matching feature and a second high-level matching feature.

[0113] In a specific embodiment, as shown in FIG. 8, a step flow chart of performing feature alignment on a first high level feature and a second high level feature respectively is shown, JPEG2024542639000015.jpg21170

[0114] According to the technical solution provided in the disclosed embodiment, the high-level feature similarity between the high-level features is calculated, a first high-level matching feature and a second high-level matching feature are calculated according to the high-level feature similarity, and finally, feature extraction is performed on the first high-level matching feature and the second high-level matching feature respectively to obtain new features aligned with the high-level features, i.e., a first high-level fusion feature and a second high-level fusion feature. Then, pedestrian matching is performed based on the aligned high-level features, so that a more accurate matching result can be obtained.

[0115] In a specific embodiment, as shown in FIG. 9, a specific flow chart of a pedestrian matching method in a specific embodiment is shown.

[0116] (1) Input two photos of probe and gallery: Through the human body detection model respectively, the human body detection frame bbox is obtained and sent to the human body key point estimation module.

[0117] (2) Input the human body detection frame obtained in step (1): First, the feature map corresponding to each human body detection frame is jointly converted to 1024 by 1*1 convolution, and then the length and width of the feature map are jointly pooled to [16, 8] by ROIAlign, and the dimensions of each human body feature map G1 are obtained as [1024, 16, 8].

[0118] G1 is processed by three convolution blocks (conv(3,3), BN, relu) to obtain feature maps [1024,16,8], [512,16,8], [256,16,8] in order. Finally, conv(1,1), BN, relu are used to obtain a heatmap [14,16,8] of all important points. (Here, there are 14 important points on the human body, including the head, neck, left hand, left elbow, left shoulder, left hip, left knee, left foot, right hand, right elbow, right shoulder, right hip, right knee, and right foot.)

[0119] The heat map dimension of each important point is [1,16,8], and the heat map of each important point is repeatedly changed to its dimension [1024,16,8], and multiplied with the human body feature map G1 [1024,16,8] to obtain the feature map of the important point [2048,16,8], and finally, the global average pooling is performed to obtain the feature vector of the important point

[1024] . Similarly, the feature vector of each important point can be obtained. That is, the first local feature F p and the second local feature F g Get F p and F g The dimensions of both are [14,1024].

[0120] (3)F p and F g to the transformer coding module, JPEG2024542639000016.jpg17170 For details on how to aggregate and obtain high-level features, please refer to the embodiment shown in FIG.

[0121] (4)F p and F gsend it to the low-level feature alignment strategy module, JPEG2024542639000017.jpg17132, both of which have dimensions [14, 1024]. Here, for a specific example of how to perform low-level feature alignment, please refer to the description of the embodiment shown in FIG.

[0122] (5) According to step (3), JPEG2024542639000018.jpg18129, whose dimensions are [14,1024], are sent to the high-level feature matching strategy module, JPEG2024542639000019.jpg24155, all of which have dimensions [14, 1024]. For a specific example of how to perform high-dimensional feature alignment, please refer to the embodiment shown in FIG.

[0123] JPEG2024542639000020.jpg68170JPEG2024542639000021.jpg46170Finally, based on the comparison loss value, it is determined whether the same pedestrian is contained in the photos probe and gallery.

[0124] According to the pedestrian matching method of the disclosed embodiment, a feature alignment module is designed, and the relationship between features is incorporated through a low-level feature alignment strategy and a high-level feature alignment strategy, and the information between features is integrated and used to improve the extracted features. Here, the low-level feature alignment strategy matches according to the similarity of local features of important points of the human body, but whether or not to match between two points is not only determined by the similarity between the two points, but also needs to consider the impact on other matching when two points are matched, so that one matching is selected to maximize the overall energy. Therefore, a high-level feature matching strategy (high-level features can be understood as edge feature information) is also added to the pedestrian matching method of the disclosed embodiment, specifically, the local features of important points of the human body previously extracted are aggregated into high-level feature information by a transformer coding module, and then the network is made to learn how to match these high-level features, so that the robustness is higher. In addition, when aggregating high-level features using transformer coding, an adjustment coefficient p is calculated based on the a priori topological structure relationship of important points on the human body, and high-level semantic information is aggregated based on the degree of influence between important points.

[0125] The present application further provides application scenarios in which the above-mentioned pedestrian matching method is applied. Specifically, the application of the pedestrian matching method in the application scenarios is as follows:

[0126] In this embodiment, the pedestrian matching method is applied to an application scenario of detecting people staying in a place during video surveillance, where it is necessary to detect and report suspicious phenomena caused by people staying or wandering for a long time. By comparing the human body features detected in multiple consecutive frames, it can be determined whether they are the same person and whether a person has been staying in the same place for a long time.

[0127] In this application scenario, when a person is detected to appear in a monitoring image, a pedestrian matching method is triggered. Two images are acquired from the monitoring device at predetermined time intervals, and pedestrian matching detection is performed on the two images. Furthermore, when the same pedestrian is detected to appear in two images at predetermined time intervals multiple times in succession, it indicates that the person appearing in the monitoring image has been staying there for a long time, and this situation can be notified. Here, the predetermined time may be set according to the actual situation, and may be set as, for example, 1 minute, 3 minutes, 5 minutes, etc. Depending on the actual situation, it can be set so that a notification is triggered when the number of times the same pedestrian is detected to appear in the monitoring image continuously reaches a predetermined number.

[0128] All the above-mentioned optional technical solutions may be combined in any way to form optional embodiments of the present application, and will not be described one by one here.

[0129] Below are apparatus embodiments of the present disclosure for carrying out the method embodiments of the present disclosure. For details not disclosed in the apparatus embodiments of the present disclosure, please refer to the method embodiments of the present disclosure.

[0130] 10 is a schematic diagram of a pedestrian matching device provided in an embodiment of the present disclosure. As shown in FIG. 10, the pedestrian matching device includes: an acquisition module 1001 for acquiring a first image and a second image; a feature extraction module 1002 for extracting features from the first image and the second image to obtain a first local feature, a second local feature, a first high-level feature, and a second high-level feature, in which the first local feature and the second local feature each include a local feature vector corresponding to each important point of the human body; a first feature alignment module 1003 for performing feature alignment on the first local feature and the second local feature to obtain a first fusion feature and a second fusion feature; a second feature alignment module 1004 for performing feature alignment on the first high-level feature and the second high-level feature to obtain a first high-level fusion feature and a second high-level fusion feature; and a matching module 1005 for determining whether the first image and the second image contain the same pedestrian based on the first fusion feature, the second fusion feature, the first high-order fusion feature and the second high-order fusion feature.

[0131] According to the technical solution provided in the disclosed embodiment, by performing feature extraction on the first and second images respectively, obtain local features and high-level features corresponding to important points of the human body in the two images, which are respectively denoted as a first local feature, a second local feature, a first high-level feature and a second high-level feature, perform feature alignment on the first local feature and the second local feature, obtain a first fusion feature and a second fusion feature, perform feature alignment on the first high-level feature and the second high-level feature, obtain a first high-level fusion feature and a second high-level fusion feature, and determine whether the same pedestrian is included in the two images according to the first fusion feature, the second fusion feature, the first high-level fusion feature and the second high-level fusion feature. Extract the human body local features and high-level features in the images, perform feature alignment on the local features and the high-level features respectively using different feature alignment strategies, incorporate the relationship between the features, and use the information between the features in an integrated manner to obtain more effective features, reduce the impact of abnormal points or outliers in the images on pedestrian matching, and obtain a more accurate matching result.

[0132] In some embodiments, the feature extraction module of the pedestrian matching device further comprises: a local feature extraction submodule for extracting human body local features from the first image and the second image, respectively, to obtain first local features and second local features; and an aggregation sub-module for performing feature aggregation on the first local features to obtain a first higher-level feature and for performing feature aggregation on the second local features to obtain a second higher-level feature.

[0133] In some embodiments, the human body has two or more important points. As shown in FIG. 11, in this embodiment, the feature extraction module of the device includes: a parameter learning submodule 1101 for learning and identifying a first parameter, a second parameter and a third parameter corresponding to the local features according to the local feature vector corresponding to each important point of the human body; a parameter identification sub-module 1102 for generating a query, a key and a value corresponding to each local feature vector according to the first parameter, the second parameter and the third parameter; a weight coefficient determining sub-module 1103 for determining a weight coefficient between every two local feature vectors according to the query and the key; an adjustment factor determining sub-module 1104 for determining, for any local feature vector, an adjustment factor based on a relationship between the local feature vector and each of the other local feature vectors; an aggregation submodule 1105 for obtaining aggregate features to which the local features correspond according to the weight coefficients, the adjustment coefficients and the values; and a high-level feature identification sub-module 1106 for identifying higher-level features by aggregate features corresponding to each local feature, where the local features include a first local feature and a second local feature, and the higher-level features include a first high-level feature and a second high-level feature.

[0134] In some embodiments, with continued reference to FIG. 11, the first feature alignment module of the apparatus comprises: a local feature calculation submodule 1107 for calculating a local feature similarity between the first local feature and the second local feature; a first matching feature identification submodule 1108 for identifying a first matching feature in the first local feature of the second local feature based on the second local feature and local feature similarity; a second matching feature identification submodule 1109 for identifying a second matching feature in the second local feature of the first local feature based on the first local feature and local feature similarity; A feature extraction sub-module 1110 for performing feature extraction on the first matching feature to obtain a first fusion feature, and for performing feature extraction on the second matching feature to obtain a second fusion feature.

[0135] In some embodiments, the first matching feature identification sub-module of the above apparatus includes a multiplication unit for calculating a first product of the local feature similarity and the second local feature, and an addition unit for identifying a sum value of the first product and the first local feature as the first matching feature.

[0136] In some embodiments, the second matching feature identification sub-module of the above apparatus includes a transposition unit for transposing the local feature similarity to obtain a transposed local feature similarity, a multiplication unit for calculating a second product of the transposed local feature similarity and the first local feature, and an addition unit for identifying a sum value of the second product and the second local feature as the second matching feature.

[0137] In some embodiments, with continued reference to FIG. 11, the second feature alignment module of the apparatus further comprises: a high-level feature similarity calculation submodule 1111 for calculating high-level feature similarity between a first high-level feature and a second high-level feature; a first high-level matching identification submodule 1112 for identifying a first high-level matching feature in the first high-level feature of the second high-level feature based on the second high-level feature and the high-level feature similarity; a second high-level matching identification submodule 1113 for identifying a second high-level matching feature in the second high-level feature of the first high-level feature based on the first high-level feature and the high-level feature similarity; and a feature extraction sub-module 1114 for performing feature extraction on the first high-order matching feature to obtain a first high-order fusion feature, and for performing feature extraction on the second high-order matching feature to obtain a second high-order fusion feature.

[0138] In some embodiments, with continued reference to FIG. 11, the matching module of the apparatus may further include: a first splice submodule 1115 for splicing the first fused feature and the first higher-order fused feature to obtain a first splice feature; a second splice submodule 1116 for splicing the second fused feature, the second higher-order fused feature, to obtain a second splice feature; and a comparison loss calculation sub-module 1117 for calculating a comparison loss value between the first splice feature and the second splice feature, and for determining whether the first image and the second image contain the same pedestrian based on the comparison loss value.

[0139] It should be understood that the size of the numbers of each step in the above embodiments does not mean the order of execution, and the execution order of each process should be determined by its function and inherent logic, and does not arbitrarily limit the implementation process of the embodiments disclosed herein.

[0140] Fig. 12 is a schematic diagram of an electronic device 12 provided in an embodiment of the present disclosure. As shown in Fig. 12, the electronic device 12 of the embodiment includes a processor 1201, a memory 1202, and a computer program 1203 stored in the memory 1202 and executable by the processor 1201. When the processor 1201 executes the computer program 1203, it realizes steps in each of the above method embodiments. Or, when the processor 1201 executes the computer program 1203, it realizes the functions of each module / unit in each of the above device embodiments.

[0141] For illustrative purposes, the computer program 1203 may be divided into one or more modules / units, and one or more modules / units may be stored in the memory 1202 and executed by the processor 1201 to accomplish the present disclosure. One or more modules / units may be a series of computer program command sections capable of performing a specific function, and the command sections are for explaining the process of the computer program 1203 being executed in the electronic device 12.

[0142] The electronic device 12 may be an electronic device such as a desktop computer, a laptop, a palmtop computer, and a cloud server. The electronic device 12 may include, but is not limited to, a processor 1201 and a memory 1202. As will be appreciated by those skilled in the art, FIG. 12 is merely an example of the electronic device 12, and is not intended to limit the electronic device 12, which may include more or fewer components than shown, or may combine certain components or different components, for example, the electronic device may include input / output devices, network access devices, buses, etc.

[0143] The processor 1201 may be a Central Processing Unit (CPU) or other general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc. The general purpose processor may be a microprocessor, any common processor, etc.

[0144] The memory 1202 may be an internal storage unit of the electronic device 12, such as a hard disk or RAM of the electronic device 12. The memory 1202 may be an external storage device of the electronic device 12, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc., that is provided in the electronic device 12. Furthermore, the memory 1202 may include not only the internal storage unit of the electronic device 12 but also an external storage device. The memory 1202 is for storing computer programs and other programs and data required by the electronic device. The memory 1202 may also be used to temporarily store data that has been output or is to be output.

[0145] Those skilled in the art can clearly understand that, for convenience and simplicity of explanation, only the division of each of the above functional units and modules is taken as an example for explanation, but in actual application, the above functions can be assigned to different functional units and modules to complete them as necessary, that is, all or part of the above-described functions can be completed by dividing the internal structure of the device into different functional units or modules. Each functional unit and module in the embodiment may be integrated into one processing unit, each unit may exist physically alone, or two or more units may be integrated into one unit, and the integrated unit may be realized in the form of hardware or software functional unit. Note that the specific names of each functional unit and module are provided for easy distinction from each other, and do not limit the scope of protection of the present application. For the specific operation processes of the units and modules in the above system, reference may be made to the corresponding processes in the embodiments of the above method, and the description will be omitted here.

[0146] In the above embodiments, the description of each embodiment has its own emphasis, and for the details or parts not described in an embodiment, reference may be made to the relevant descriptions of other embodiments.

[0147] Those skilled in the art can be aware that the units and algorithm steps of each example described in the embodiments disclosed herein can be realized in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software is determined by the specific application and design constraints of the technical solution. Those skilled in the art may implement the described functions using different methods for each specific application, but such implementation should not be considered to go beyond the scope of this disclosure.

[0148] In the embodiments provided in this disclosure, it should be understood that the disclosed devices / electronic devices and methods can be realized in other ways. For example, the above-described device / electronic device embodiments are merely illustrative, and the division into modules or units is merely a logical functional division, and other division methods may be applied in actual implementation, for example, multiple units or components may be combined or integrated into other systems, or some features may be ignored or not implemented. In addition, the couplings or direct couplings or communication connections between the displayed or discussed devices may be indirect couplings or communication connections through some interfaces, devices or units, and may be electrical, mechanical, or other forms.

[0149] Units described as separate components may or may not be physically separated, and components shown as units may or may not be physical units, i.e., located in one place or distributed among multiple network units, some or all of which may be selected according to actual needs to achieve the objectives of the solutions of the present embodiment.

[0150] In addition, each functional unit in this disclosure may be integrated into one processing unit, each unit may exist physically alone, or two or more units may be integrated into one unit. The integrated unit may be realized in the form of hardware or a software functional unit.

[0151] The integrated module / unit may be realized in the form of a software functional unit and stored in a computer-readable storage medium when sold or used as an independent product. With this understanding, the present disclosure can realize all or part of the processes in the method of the above embodiments by commanding relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of each of the above method embodiments can be realized. The computer program may include a computer program code, and the computer program code may be in source code form, object code form, executable file, or some intermediate form, etc. The computer-readable storage medium may include any entity or device capable of carrying a computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer memory, a Read-Only Memory (ROM), a Random Access Memory (RAM), an electric carrier wave signal, an electric communication signal, and a software distribution medium, etc. Furthermore, the content contained on a computer-readable storage medium may be increased or decreased as required by the legislation and patent practice of a jurisdiction. For example, in some jurisdictions, based on legislation and patent practice, a computer-readable storage medium does not include electrical carrier signals and telecommunications signals.

[0152] The above-mentioned embodiments are only for illustrating the technical solutions of the present disclosure, but are not intended to limit the same. Although the present disclosure has been described in detail with reference to the above embodiments, those skilled in the art may still amend the technical solutions described in the above embodiments, or equivalently replace some of the technical features therein, but it should be understood that such amendments or replacements do not deviate from the substance of the corresponding technical solutions from the idea and scope of the technical solutions of the embodiments of the present disclosure, and all of them should be included in the protection scope of the present disclosure.

Claims

1. A pedestrian matching method, comprising: acquiring a first image and a second image; performing feature extraction on each of the first image and the second image to obtain a first local feature, a second local feature, a first high level feature, and a second high level feature, wherein the first local feature and the second local feature each include a local feature vector corresponding to each important point of a human body; performing feature alignment on the first local feature and the second local feature to obtain a first fusion feature and a second fusion feature; performing feature alignment on the first high-level feature and the second high-level feature to obtain a first high-level fusion feature and a second high-level fusion feature; determining whether the first image and the second image include a same pedestrian based on the first fusion feature, the second fusion feature, the first high-level fusion feature, and the second high-level fusion feature. A pedestrian matching method comprising:

2. The step of extracting features from the first image and the second image to obtain a first local feature, a second local feature, a first high level feature, and a second high level feature includes: performing human body local feature extraction on the first image and the second image, respectively, to obtain the first local feature and the second local feature; performing feature aggregation on the first local features to obtain a first high-level feature, and performing feature aggregation on the second local features to obtain the second high-level feature.

2. The method of claim 1 .

3. The human body includes two or more important points; The step of acquiring high-level features by performing feature aggregation on the local features includes: learning and identifying a first parameter, a second parameter and a third parameter corresponding to each of the local features based on a local feature vector corresponding to each of the important points of the human body; generating a query, a key and a value corresponding to each of the local feature vectors based on the first parameter, the second parameter and the third parameter; determining a weighting factor between each pair of the local feature vectors based on the query and the key; determining, for any one of the local feature vectors, an adjustment factor based on a relationship between the local feature vector and each of the other local feature vectors; obtaining an aggregate feature to which the local feature corresponds based on the weighting factor, the adjustment factor and the value; the higher level features are identified by aggregate features corresponding to each of the local features; The local features include the first local feature and the second local feature, and the high-level features include the first high-level feature and the second high-level feature.

3. The method of claim 2 .

4. The step of performing feature alignment on the first local feature and the second local feature to obtain a first fusion feature and a second fusion feature includes: calculating a local feature similarity between the first local feature and the second local feature; identifying a first matching feature in the first local feature of the second local feature based on the second local feature and the local feature similarity; identifying a second matching feature in the second local feature of the first local feature based on the first local feature and the local feature similarity; performing feature extraction on the first matching feature to obtain the first fusion feature; and performing feature extraction on the second matching feature to obtain the second fusion feature.

2. The method of claim 1 .

5. Identifying a first matching feature in the first local feature of the second local feature based on the local feature similarity, the first matching feature being based on the local feature similarity, the first matching feature being based on the local feature similarity, the second local ... calculating a first product of the local feature similarity and the second local feature; and identifying a sum of the first product and the first local feature as the first matching feature; Identifying a second matching feature in the second local feature of the first local feature based on the first local feature and the local feature similarity includes: transposing the local feature similarity to obtain a transposed local feature similarity; calculating a second product of the transposed local feature similarity and the first local feature; identifying a sum value of the second product and the second local feature as the second matching feature.

5. The method of claim 4.

6. The step of performing feature alignment on the first high-level feature and the second high-level feature to obtain a first high-level fusion feature and a second high-level fusion feature includes: Calculating a high-level feature similarity between the first high-level feature and the second high-level feature; identifying a first high level feature in the first high level feature that matches the second high level feature based on the second high level feature and the high level feature similarity; identifying a second high-level matching feature in the second high-level feature of the first high-level feature based on the first high-level feature and the high-level feature similarity; performing feature extraction on the first high-order matching feature to obtain the first high-order fusion feature; and performing feature extraction on the second high-order matching feature to obtain the second high-order fusion feature.

2. The method of claim 1 .

7. determining whether the first image and the second image include a same pedestrian based on the first fusion feature, the second fusion feature, the first high-level fusion feature, and the second high-level fusion feature, performing splicing on the first fusion feature and the first higher-order fusion feature to obtain a first splice feature; splicing the second fusion feature and the second higher-order fusion feature to obtain a second splice feature; calculating a comparative loss value between the first splice feature and the second splice feature, and determining whether the first image and the second image contain the same pedestrian based on the comparative loss value.

2. The method of claim 1 .

8. A pedestrian matching device, comprising: an acquisition module for acquiring a first image and a second image; a feature extraction module for performing feature extraction on the first image and the second image, respectively, to obtain a first local feature, a second local feature, a first high level feature, and a second high level feature, wherein the first local feature and the second local feature each include a local feature vector corresponding to each important point of a human body; a first feature alignment module for performing feature alignment on the first local feature and the second local feature to obtain a first fusion feature and a second fusion feature; a second feature alignment module for performing feature alignment on the first high-level feature and the second high-level feature to obtain a first high-level fusion feature and a second high-level fusion feature; a matching module for determining whether the first image and the second image include a same pedestrian based on the first fusion feature, the second fusion feature, the first high-level fusion feature, and the second high-level fusion feature. A pedestrian matching device comprising:

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable by the processor, The processor, when executing the computer program, performs the steps of the method of claim 1.

1. An electronic device comprising:

10. A computer-readable storage medium on which a computer program is stored, The computer program, when executed by a processor, performs the steps of the method of claim 1. A computer-readable storage medium comprising:

Citation Information

Patent Citations

  • Pedestrian re-identification method and device, electronic equipment and readable storage medium

    CN112200020A

  • Pedestrian re-recognition system and method based on spatial sequence feature learning

    CN113239784A

  • Pedestrian re-identification method and system based on key point optimization and multi-hop attention graph convolution

    CN113361334A