Pedestrian re-identification method, apparatus and device, and medium

Abstract

The invention discloses a pedestrian re-identification method and device, equipment and a medium, and the method comprises the steps: obtaining a to-be-identified pedestrian image through an embedded equipment end, and carrying out the preprocessing of the to-be-identified pedestrian image, and obtaining a preprocessed image; detecting a region image conforming to at least one body region category from the pre-processed image; respectively extracting region feature information from each region image; and combining and comparing the feature information of each region with the historical feature information to generate a re-identification result. Therefore, resources consumed by feature extraction are reduced, and efficient and accurate pedestrian re-identification can still be realized under the condition that the resources are limited.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a pedestrian re-identification method, device, equipment and medium. Background Art

[0002] Person Re-identification (ReID) is a technology that uses computer vision technology to retrieve whether a specific pedestrian exists in an image or video sequence. It can be combined with pedestrian detection, pedestrian tracking and other technologies, and has a wide range of applications in many fields such as intelligent security and video surveillance.

[0003] In the process of pedestrian re-identification, the quality of feature extraction directly affects the pedestrian tracking effect. How to extract better feature representation is of great significance in the pedestrian tracking algorithm. At the same time, the computing resource limitations of the embedded device must also be considered. The traditional extraction method based on color histogram or local binary can no longer meet the needs of personnel re-identification.

[0004] Therefore, the existing technology proposes to extract ReID features through deep learning models. For example, the convolutional neural network model is trained on a large-scale pedestrian image dataset to learn ReID features. However, due to the large model capacity of the convolutional network model, the resource consumption in the feature extraction process is high and the optimization is difficult, making it difficult to achieve efficient and accurate pedestrian re-identification. Summary of the invention

[0005] The present invention provides a pedestrian re-identification method, device, equipment and medium, which solves the technical problem that it is difficult to achieve efficient and accurate pedestrian re-identification due to the large model capacity of the convolutional network model, the high resource consumption and great optimization difficulty in the feature extraction process.

[0006] A first aspect of the present invention provides a pedestrian re-identification method, which is applied to an embedded device, and the method includes:

[0007] Acquire the pedestrian image to be identified and preprocess it to obtain a preprocessed image;

[0008] Locating a region position that conforms to at least one body region category from the preprocessed image;

[0009] intercepting a regional image from the image of the pedestrian to be identified according to the regional position;

[0010] Extracting regional feature information from each of the regional images respectively;

[0011] The regional feature information and the historical feature information are combined and compared to generate a re-identification result.

[0012] Optionally, the acquiring and preprocessing the image of the pedestrian to be identified to obtain the preprocessed image includes:

[0013] Obtain an image of a pedestrian to be identified;

[0014] Detecting the coordinate position of the pedestrian from the image of the pedestrian to be identified, and capturing a position frame diagram;

[0015] The position frame images are normalized to the same size to obtain a preprocessed image.

[0016] Optionally, locating a region position that conforms to at least one body region category from the preprocessed image comprises:

[0017] If the computing power of the embedded device reaches the minimum computing power threshold, the segmentation network is called to perform category judgment on each pixel in the preprocessed image to obtain pixel categories respectively;

[0018] taking a pixel region whose pixel category matches a body region category as a region position;

[0019] If the computing power of the embedded device does not reach the minimum computing power threshold, the detection network is called to detect a bounding box that meets the body region category from the pre-processed image as the region position.

[0020] Optionally, the extracting a regional image from the image of the pedestrian to be identified according to the regional position includes:

[0021] Capturing an initial image of the area position from the image of the pedestrian to be identified;

[0022] Each of the initial images is adjusted according to a fixed size and normalized to obtain a regional image.

[0023] Optionally, extracting the regional feature information from each of the regional images includes:

[0024] Convolving the regional image according to a preset convolution size to obtain input features;

[0025] Extracting a plurality of convolution features from each of the input features using a plurality of convolution kernels of different sizes;

[0026] Concatenating the convolutional features to obtain a concatenated feature;

[0027] Performing convolution operation, global average pooling operation and normalization mapping operation on the spliced ​​features in sequence to generate channel weights;

[0028] The channel weights are used to perform weighted fusion on the splicing features to obtain regional feature information.

[0029] Optionally, the historical feature information includes at least one feature sub-information that matches the body region category; and the combining and comparing each of the region feature information with the historical feature information to generate a re-identification result includes:

[0030] If the number of the regional feature information is equal to the number of the feature sub-information, respectively calculating the first similarity between each regional feature information and each feature sub-information;

[0031] Performing weighted accumulation of the first similarities according to the area weights corresponding to the body area categories to obtain a pedestrian similarity;

[0032] If the amount of the regional feature information is not equal to the amount of the feature sub-information, calculating the second similarity between the regional feature information and the feature sub-information of the same body region category;

[0033] Adjusting the region weight according to the body region category to which the region feature information belongs to obtain an updated weight;

[0034] Performing weighted accumulation of each of the second similarities according to the updated weight to obtain a pedestrian similarity;

[0035] A re-identification result is generated according to a comparison result of the pedestrian similarity and a preset similarity threshold.

[0036] Optionally, generating a re-identification result according to a comparison result of the pedestrian similarity and a preset similarity threshold includes:

[0037] comparing the pedestrian similarity with a preset similarity threshold;

[0038] If the pedestrian similarity is greater than or equal to a preset similarity threshold, it is determined that the pedestrian to which the regional feature information belongs is the same as the pedestrian to which the historical feature information belongs, and a re-identification result is generated;

[0039] If the pedestrian similarity is less than a preset similarity threshold, it is determined that the pedestrian to which the regional feature information belongs is different from the pedestrian to which the historical feature information belongs, and a re-identification result is generated.

[0040] A second aspect of the present invention provides a pedestrian re-identification device, which is applied to an embedded device, and the device includes:

[0041] An image preprocessing module is used to obtain and preprocess the image of the pedestrian to be identified to obtain a preprocessed image;

[0042] A region positioning module, used for locating a region position conforming to at least one body region category from the pre-processed image;

[0043] A regional image capture module, used for capturing a regional image from the image of the pedestrian to be identified according to the regional position;

[0044] A feature extraction module, used to extract regional feature information from each of the regional images;

[0045] The feature combination comparison module is used to combine and compare the regional feature information and the historical feature information to generate a re-identification result.

[0046] A third aspect of the present invention provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the pedestrian re-identification method as described in any one of the first aspect of the present invention.

[0047] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed, implements the pedestrian re-identification method as described in any one of the first aspects of the present invention.

[0048] It can be seen from the above technical solutions that the present invention has the following advantages:

[0049] The present invention obtains the image of the pedestrian to be identified through the embedded device end and pre-processes it to obtain the pre-processed image; detects the regional image that meets at least one body region category from the pre-processed image; extracts the regional feature information from each regional image respectively; combines and compares the regional feature information with the historical feature information to generate the re-identification result. Thus, the resources spent on feature extraction are reduced, and efficient and accurate pedestrian re-identification can still be achieved under the condition of limited resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0051] Figure 1 A flowchart of a method for pedestrian re-identification provided by an embodiment of the present invention;

[0052] Figure 2 A flowchart of the detailed steps of a pedestrian re-identification method provided by an embodiment of the present invention;

[0053] Figure 3 A model architecture diagram of a feature extraction network provided by an embodiment of the present invention;

[0054] Figure 4A structural block diagram of a pedestrian re-identification device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0055] Embodiments of the present invention provide a pedestrian re-identification method, apparatus, device and medium for solving the technical problem that efficient and accurate pedestrian re-identification is difficult to achieve due to the large model capacity of the convolutional network model, high resource consumption and high optimization difficulty in the feature extraction process.

[0056] In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described below are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0057] See also Figure 1 , Figure 1 A flowchart of a method for pedestrian re-identification provided by an embodiment of the present invention.

[0058] The present invention provides a pedestrian re-identification method, which is applied to an embedded device, and the method includes:

[0059] Step 101, obtaining an image of a pedestrian to be identified and preprocessing it to obtain a preprocessed image;

[0060] The pedestrian image to be identified refers to the image data captured or collected with pedestrians as the subject, which is usually a frame in a continuous image or video data. Its source can be surveillance cameras, mobile devices or public data sets.

[0061] Embedded devices refer to hardware platforms that embed computer systems into specific devices to enable them to have specific functions. They are usually not general-purpose computers, but are designed and manufactured to meet specific application scenarios. They have certain limitations in resources such as computing power, storage capacity, and power consumption. Examples include security cameras, transportation terminal equipment, or smart home devices.

[0062] In this embodiment, a single frame of pedestrian image to be identified is obtained from a camera module or a public data set through an embedded device. Since there may be more than one pedestrian in the image, the pedestrian image to be identified is preprocessed, such as intercepting the pedestrian position image to obtain the coordinate position of each pedestrian, and then further normalized to the same size to obtain a preprocessed image.

[0063] Step 102, locating a region position that meets at least one body region category from the preprocessed image;

[0064] Body region categories refer to single or multiple body regions that describe basic forms such as human posture, movement mode, and overall appearance. For the convenience of detection, body region categories can be preset as head region, torso region, and lower limb region. Region position refers to the position coordinates of a body region that meets any of the above body region categories on the preprocessed image.

[0065] After obtaining the preprocessed image, in order to reduce the impact of image background on pedestrian re-identification, the preprocessed image can be further divided according to body region categories, and image detection can be performed on each divided part to obtain the region position that meets the body region category as the data basis for subsequent feature extraction.

[0066] For a single pedestrian, there may be a situation where it is blocked, and one or two body region categories may not be detected. Therefore, when performing region detection on the preprocessed image, the detected region position only needs to meet at least one body region category. The region position detection can be achieved through a detection network or a segmentation network, which can be selected based on the computing power of the embedded device and the model effect.

[0067] Step 103, capturing a regional image from the image of the pedestrian to be identified according to the regional position;

[0068] In this embodiment, after the regional position is determined, in order to further improve the image accuracy, an image matching the regional position may be captured from the image of the pedestrian to be identified and normalized to obtain a regional image.

[0069] Step 104, extracting regional feature information from each regional image;

[0070] Regional feature information refers to a series of identifiable characteristic descriptions of a certain body region category that can reflect the unique properties and status of the region. It can be identified in a vector form to quantify and represent the data structure of specific region characteristics.

[0071] After obtaining a regional image of at least one body region type, a feature extraction model or a feature extraction module can be called to extract regional features of different spatial sizes from each regional image and weightedly fuse them into regional feature information to obtain regional feature information that characterizes the characteristics of the body region.

[0072] Step 105: Combine and compare the feature information of each region with the historical feature information to generate a re-identification result.

[0073] The historical feature information refers to the region feature information in which the image source and the image of the pedestrian to be identified belong to the same image stream or video stream and the extraction time is before the image of the pedestrian to be identified.

[0074] After extracting the regional feature information corresponding to each regional image, the regional feature information and historical feature information belonging to the same body region category are compared to determine the similarity of each body region category. Then, the similarities are weighted and accumulated according to the weight of the body region category to generate the pedestrian similarity. According to the comparison result of the pedestrian similarity and the similarity threshold, it is determined whether the pedestrian in the pedestrian image to be identified is the same person as the pedestrian in the historical feature information, and the determination result is used as the re-identification result.

[0075] In the embodiment of the present application, the image of the pedestrian to be identified is obtained by the embedded device end and preprocessed to obtain a preprocessed image; the regional image that meets at least one body region category is detected from the preprocessed image; the regional feature information is extracted from each regional image respectively; the regional feature information is combined and compared with the historical feature information to generate a re-identification result. In this way, the resources spent on feature extraction are reduced, and efficient and accurate pedestrian re-identification can still be achieved under resource-constrained conditions.

[0076] See also Figure 2 , Figure 2 This is a flowchart of the detailed steps of a pedestrian re-identification method in an embodiment of the present invention, wherein step 104 is further refined.

[0077] The embodiment of the present invention provides a pedestrian re-identification method, which is applied to an embedded device, and the method includes:

[0078] Step 201, obtaining a pedestrian image to be identified and preprocessing it to obtain a preprocessed image;

[0079] In an example of the present application, step 201 may include the following sub-steps:

[0080] Obtain an image of a pedestrian to be identified;

[0081] Detect the coordinate position of the pedestrian from the image of the pedestrian to be identified and capture the position frame diagram;

[0082] The position frame map is normalized to the same size to obtain the preprocessed image.

[0083] In the embodiments of the present application, the embedded device end is usually installed in a place that needs to be supervised or used in a scene where a person is searched. For example, a video stream of pedestrians in a certain area is obtained through the embedded device end, and a frame image required by the user is extracted as an image of a pedestrian to be identified. The coordinate position of the pedestrian is detected from the image of the pedestrian to be identified by the embedded device end, and a position frame diagram is intercepted according to the coordinate position of the pedestrian to select the pedestrian in the image. If there are multiple pedestrians in the image of the pedestrian to be identified at the same time, the position frame diagram of each pedestrian is intercepted separately, so as to obtain multiple position frame diagrams.

[0084] Since the pedestrians in the pedestrian image to be identified are at different distances, the intercepted position frame maps will also be of different sizes. In order to facilitate subsequent area detection and feature extraction, the intercepted position frame maps are normalized to the same size, such as by calculating the mean and variance of each position frame map in each channel, subtracting the mean from each pixel of the position frame map and dividing by the variance to obtain a preprocessed image.

[0085] Among them, the detection of pedestrian coordinate positions and the selection of position boxes can be achieved through target detection algorithms such as the YOLOv series algorithms.

[0086] Step 202, locating a region position that meets at least one body region category from the preprocessed image;

[0087] In an example of the present application, step 202 may include the following sub-steps:

[0088] If the computing power of the embedded device reaches the minimum computing power threshold, the segmentation network is called to judge the category of each pixel in the preprocessed image and obtain the pixel category respectively;

[0089] The pixel region whose pixel category matches the body region category is taken as the region position;

[0090] If the computing power of the embedded device does not reach the minimum computing power threshold, the detection network is called to detect a bounding box that meets the body region category from the preprocessed image as the region location.

[0091] Device computing power refers to the amount of data that a device can process or the number of computing operations it can perform per unit time.

[0092] The segmentation network refers to the image segmentation network, which is used to make pixel-level category judgments on images.

[0093] In an embodiment of the present application, after obtaining the preprocessed image, due to the resource limitations and computing power limitations on the embedded device side, before dividing the regional position, the device computing power on the embedded device side can be detected, and the segmentation network or detection network can be called according to whether the device computing power reaches the minimum computing power threshold of the segmentation network. If the device computing power reaches the minimum computing power threshold of the segmentation network, the segmentation network is called to perform category judgment on each pixel in the preprocessed image, and the objects that meet the characteristics of pedestrians in the preprocessed image are separated from the background or other objects to obtain the pixel category corresponding to each pixel. Further, multiple body area categories are matched according to the pixel category, such as matching the head area, torso area and lower limb area, to obtain pixels that meet the categories of each body area, and the corresponding regional positions are generated in combination, thereby effectively removing the background area and reducing the impact of irrelevant features on the final pedestrian Reid features. Among them, the image segmentation network can be a fully convolutional network (FCN) model or a U-Net model, etc.

[0094] For example, for a grayscale image, if it is known that the pixel values ​​of the target object are generally higher than the background pixel values, a suitable threshold can be selected to divide the pixels with values ​​greater than the threshold into the target area and those with values ​​less than the threshold into the background area.

[0095] If the computing power of the device does not reach the minimum computing threshold of the segmentation network, the detection network can be called to further detect bounding box images that meet the body area category from the preprocessed image, and determine the category to which each bounding box image belongs to generate the area location, which can be achieved through the YOLOv series detection model.

[0096] It should be noted that a single pedestrian may be occluded. In this case, one or two regions may not be detected in either the segmentation network or the detection network. In this case, the output can be set to at least one relatively simple and obvious region, and up to three regions, to minimize the time spent on regional detection. At the same time, there is no need to extract features from the undetectable regions in the future, so as to reduce the impact of irrelevant features of the occluded regions on the final pedestrian representation features to a certain extent, and improve the accuracy of pedestrian feature representation.

[0097] Step 203, capturing a regional image from the image of the pedestrian to be identified according to the regional position;

[0098] In an example of the present application, step 203 may include the following sub-steps:

[0099] An initial image of the region position is captured from the image of the pedestrian to be identified;

[0100] Each initial image is adjusted according to a fixed size and normalized to obtain a regional image.

[0101] In this embodiment, after completing the detection of the area position, the initial image of the corresponding position can be captured and saved from the image of the pedestrian to be identified according to the coordinates of the area position and the body area category to which it belongs, so as to facilitate comparison with the area image of the next frame.

[0102] The initial image is then unified to a fixed size and normalized according to the process of step 201 above, thereby obtaining a regional image of each body region category of each pedestrian.

[0103] Step 204, convolving the regional image according to a preset convolution size to obtain input features;

[0104] In this embodiment, since the regional images belong to multiple different body region categories and the image sizes vary greatly, in order to accurately extract the feature information of each regional image, the various regional images can be convolved according to a preset convolution size to obtain corresponding input features, respectively. The feature dimension of the input feature is b*c*h*w, where b is the Batchsize (i.e., the number of features processed at a single time), c is the number of channels, h is the length of the input feature, and w is the width of the input feature.

[0105] Step 205, extracting multiple convolution features from each input feature using multiple convolution kernels of different sizes;

[0106] After obtaining the input features corresponding to each regional image, each input feature is convolved using multiple convolution kernels of different sizes to extract convolution features of multiple different spatial sizes.

[0107] Specifically, the sizes of the convolution kernels can include 1*1, 3*3, 5*5, and avepool average pooling.

[0108] Step 206, concatenating the convolutional features to obtain a concatenated feature;

[0109] After obtaining the convolution features of various sizes, the convolution features are spliced ​​according to the channel dimension to obtain the spliced ​​features, so that the convolution features extracted by different convolution kernels can make comprehensive use of the diverse convolution features in subsequent features.

[0110] Step 207, performing convolution operation, global average pooling operation and normalization mapping operation on the spliced ​​features in sequence to generate channel weights;

[0111] In this embodiment, after the splicing feature is generated, a convolution operation is performed on the splicing feature using a convolution kernel of size 1×1 to adjust the number of channels of the splicing feature to the required number, thereby reducing the feature dimension and linearly combining the information between channels to further fuse the features.

[0112] Furthermore, a global average pooling operation is performed on the convolved splicing features, and the average value of the entire splicing features is calculated for each channel to compress the splicing features into a one-dimensional vector in two dimensions, extract the global feature information of each channel, and reduce the computational complexity of subsequent operations.

[0113] After completing the global average pooling operation, the Sigmoid function is called to perform normalization mapping on each element in the vector to normalize the vector value in each channel to the range of 0 to 1, providing channel weights for subsequent weighted fusion, so that adaptive weighting can be performed according to the importance of different channel features.

[0114] Step 208: Use channel weights to perform weighted fusion on the splicing features to obtain regional feature information.

[0115] After determining the channel weight of each channel of the convolution feature, the concatenated features are weighted and summed according to the channel weight to highlight important features and suppress unimportant features, thereby obtaining more discriminative regional feature information.

[0116] In a specific implementation, the process of steps 204-208 can be implemented through a feature extraction network. At the same time, the above process can be cyclically extracted multiple times, that is, the regional feature information is used as a new input feature to repeat steps 205-208 to further improve the accuracy of the regional feature information.

[0117] The model architecture of its feature extraction network is as follows Figure 3 As shown, in this embodiment, convolution features of different spatial sizes are extracted from the input features through the CNN of the multi-scale network, and then the splicing features are obtained by combining them according to the channel dimension through the splicing module Concate. The weights of each channel are determined by 1*1 convolution, global pooling and sigmoid normalization, and then the splicing features are weightedly fused according to the weights of each channel to obtain the regional feature information.

[0118] It should be noted that the feature extraction network can be optimized through triplet loss and classification loss. The same feature extraction network is used for each body region category and the network weights are shared to extract the regional feature information of the corresponding region.

[0119] Step 209: Combine and compare the feature information of each region with the historical feature information to generate a re-identification result.

[0120] In an example of the present application, the historical feature information includes at least one feature sub-information that conforms to a body region category; step 209 may include the following sub-steps S11-S16:

[0121] S11, if the number of regional feature information is equal to the number of feature sub-information, respectively calculating the first similarity between each regional feature information and each feature sub-information;

[0122] S12, performing weighted accumulation of the first similarities according to the area weights corresponding to the body area categories to obtain the pedestrian similarity;

[0123] In the embodiment of the present application, the historical feature information is the sum of the regional feature information of the image to be identified in the historical frame, and the historical frame can be any frame before the current frame. If the number of regional feature information and the number of feature sub-information are equal, it indicates that the pedestrian in the image to be identified in the current frame is not blocked, and all its body region categories are complete, and the first similarity between each regional feature information and each feature sub-information can be calculated according to the body region category.

[0124] Then, the first similarities are weighted and accumulated according to the regional weights corresponding to the body region categories, so as to obtain the pedestrian similarity between the current frame image to be identified and the historical frame image to be identified. The regional weights can take into account the effectiveness of the three body region categories of pedestrians. The area with the most obvious pedestrian feature information is the torso (clothing color), followed by the lower limbs (pants color), and finally the head area (hair length, whether wearing a hat, etc.). For example, the weighted weight of the head area feature similarity is 0.2, the weighted weight of the upper limb area pedestrian similarity is 0.5, and the weighted weight of the lower limb area pedestrian similarity is 0.4. The calculation formula for weighted accumulation can be as follows:

[0125]

[0126] in, is the pedestrian similarity, is the regional weight of the ith body region category, and its cumulative value sum is 1. is the regional feature information of the i-th body region category, the minimum value of i is 1, and the maximum value may be determined according to the number of body region categories, which is 3 in this embodiment.

[0127] The first similarity can be calculated by Euclidean distance or cosine similarity.

[0128] S13, if the amount of the regional feature information is not equal to the amount of the feature sub-information, calculating the second similarity between the regional feature information and the feature sub-information of the same body region category;

[0129] S14, adjusting the regional weight according to the body region category to which the regional feature information belongs, to obtain an updated weight;

[0130] S15, performing weighted accumulation of the second similarities according to the updated weights to obtain pedestrian similarity;

[0131] In this embodiment, there are special cases, sometimes a certain area of ​​the pedestrian is blocked. At this time, it is only necessary to compare the similarity of the pedestrian's unblocked area, and the regional weight can be adjusted accordingly. If the number of regional feature information is not equal to the number of feature sub-information, it indicates that the pedestrian may be blocked in a certain part of the body area in the current frame to be identified. At this time, the second similarity between the regional feature information and the feature sub-information of the same body area category can be calculated, and then the regional weight is adjusted to obtain the updated weight. For example, only the upper limb and head areas, according to the original head is 0.2, the upper limb area is 0.5, and the head area weight is transformed into 0.28=0.2 / (0.2+0.5), and the upper limb area weight is 0.71=0.5 / (0.2+0.5). The calculation process of the pedestrian similarity can refer to the calculation formula of step S12.

[0132] S16. Generate a re-identification result according to the comparison result between the pedestrian similarity and the preset similarity threshold.

[0133] In another example of the present application, S16 may include the following sub-steps:

[0134] Compare pedestrian similarity with a preset similarity threshold;

[0135] If the pedestrian similarity is greater than or equal to the preset similarity threshold, it is determined that the pedestrian belonging to the regional feature information is the same as the pedestrian belonging to the historical feature information, and a re-identification result is generated;

[0136] If the pedestrian similarity is less than a preset similarity threshold, it is determined that the pedestrian to which the regional feature information belongs is different from the pedestrian to which the historical feature information belongs, and a re-identification result is generated.

[0137] In this embodiment, after completing the calculation of the pedestrian similarity, the pedestrian similarity is compared with the preset similarity threshold. If it exceeds the similarity threshold, it means that the pedestrian to whom the feature information of the surface area belongs is the same as the pedestrian to whom the historical feature information belongs. At this time, a re-identification result can be generated for the same pedestrian. Otherwise, a re-identification result for different pedestrians is generated.

[0138] In an embodiment of the present application, an image of a pedestrian to be identified is obtained through an embedded device and preprocessed to obtain a preprocessed image; a regional image that meets at least one body region category is detected from the preprocessed image; regional feature information is extracted from each regional image; and the feature information of each region is combined and compared with the historical feature information to generate a re-identification result. Extracting features only from the corresponding region can reduce the features of model learning, and to a certain extent reduce the model capacity, that is, the model reasoning time. The reduction in model capacity also reduces the difficulty of model optimization. After dividing the region, the influence of the background can be reduced, and the effectiveness of the re-ID model in extracting the features of the region can be improved. At the same time, the pedestrian feature similarity is improved by using a method of focusing on weighted representative features to represent pedestrian features. In the case of occlusion, the weighted weights in the occlusion are classified and processed, thereby improving the final similarity value between pedestrians, thereby reducing the resources spent on feature extraction, and achieving efficient and accurate pedestrian re-identification under resource-constrained conditions.

[0139] See also Figure 4 , Figure 4 A structural block diagram of a pedestrian re-identification device provided in an embodiment of the present invention.

[0140] The embodiment of the present invention provides a pedestrian re-identification device, which is applied to an embedded device, and the device includes:

[0141] The image preprocessing module 401 is used to obtain and preprocess the image of the pedestrian to be identified to obtain a preprocessed image;

[0142] A region positioning module 402, configured to locate a region position that conforms to at least one body region category from the pre-processed image;

[0143] The regional image interception module 403 is used to intercept the regional image from the image of the pedestrian to be identified according to the regional position;

[0144] The feature extraction module 404 is used to extract the regional feature information from each regional image;

[0145] The feature combination comparison module 405 is used to combine and compare the feature information of each region with the historical feature information to generate a re-identification result.

[0146] Optionally, the image preprocessing module 401 is specifically used for:

[0147] Obtain an image of a pedestrian to be identified;

[0148] Detect the coordinate position of the pedestrian from the image of the pedestrian to be identified and capture the position frame diagram;

[0149] The position frame map is normalized to the same size to obtain the preprocessed image.

[0150] Optionally, the area positioning module 402 is specifically used for:

[0151] If the computing power of the embedded device reaches the minimum computing power threshold, the segmentation network is called to judge the category of each pixel in the preprocessed image and obtain the pixel category respectively;

[0152] The pixel region whose pixel category matches the body region category is taken as the region position;

[0153] If the computing power of the embedded device does not reach the minimum computing power threshold, the detection network is called to detect a bounding box that meets the body region category from the preprocessed image as the region location.

[0154] Optionally, the regional image interception module 403 is specifically used for:

[0155] An initial image of the region position is captured from the image of the pedestrian to be identified;

[0156] Each initial image is adjusted according to a fixed size and normalized to obtain a regional image.

[0157] Optionally, the feature extraction module 404 is specifically used for:

[0158] Convolve the regional image according to the preset convolution size to obtain input features;

[0159] Multiple convolution features are extracted from each input feature through multiple convolution kernels of different sizes;

[0160] Concatenate the convolutional features to obtain concatenated features;

[0161] The concatenated features are sequentially subjected to convolution, global average pooling, and normalized mapping operations to generate channel weights.

[0162] The channel weights are used to perform weighted fusion on the splicing features to obtain regional feature information.

[0163] Optionally, the historical feature information includes at least one feature sub-information that conforms to the body region category; the feature combination comparison module 405 includes:

[0164] A first similarity calculation submodule, configured to respectively calculate a first similarity between each piece of regional feature information and each piece of feature sub-information if the amount of regional feature information is equal to the amount of feature sub-information;

[0165] A first weighted accumulation submodule, used for performing weighted accumulation of each first similarity according to the area weight corresponding to each body area category, to obtain a pedestrian similarity;

[0166] A second similarity calculation submodule, configured to calculate a second similarity between the regional feature information and the feature sub-information of the same body region category if the amount of the regional feature information is not equal to the amount of the feature sub-information;

[0167] A weight adjustment submodule, used to adjust the regional weight according to the body region category to which the regional feature information belongs, to obtain an updated weight;

[0168] A second weighted accumulation submodule, used for weighted accumulation of each second similarity according to the updated weight to obtain the pedestrian similarity;

[0169] The re-identification comparison submodule is used to generate a re-identification result according to the comparison result between the pedestrian similarity and the preset similarity threshold.

[0170] Optionally, the re-identification comparison submodule is specifically used for:

[0171] Compare pedestrian similarity with a preset similarity threshold;

[0172] If the pedestrian similarity is greater than or equal to the preset similarity threshold, it is determined that the pedestrian belonging to the regional feature information is the same as the pedestrian belonging to the historical feature information, and a re-identification result is generated;

[0173] If the pedestrian similarity is less than a preset similarity threshold, it is determined that the pedestrian to which the regional feature information belongs is different from the pedestrian to which the historical feature information belongs, and a re-identification result is generated.

[0174] An embodiment of the present invention provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the pedestrian re-identification method as described in any embodiment of the present invention.

[0175] An embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed, the method for pedestrian re-identification as described in any embodiment of the present invention is implemented.

[0176] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0177] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0178] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0179] In addition, each functional module in each embodiment of the present invention may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of software functional modules.

[0180] If the integrated module is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for enabling an electronic device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0181] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A pedestrian re-identification method, characterized in that: Applied to an embedded device, the method includes: Acquire the pedestrian image to be identified and preprocess it to obtain a preprocessed image; locating a region position that conforms to at least one body region category from the preprocessed image; intercepting a regional image from the image of the pedestrian to be identified according to the regional position; Extracting regional feature information from each of the regional images respectively; The regional feature information and the historical feature information are combined and compared to generate a re-identification result.

2. The method according to claim 1, characterized in that The step of obtaining the image of the pedestrian to be identified and preprocessing it to obtain the preprocessed image includes: Obtain an image of a pedestrian to be identified; Detecting the coordinate position of the pedestrian from the image of the pedestrian to be identified, and capturing a position frame diagram; The position frame images are normalized to the same size to obtain a preprocessed image.

3. The method according to claim 1, characterized in that The step of locating a region position that conforms to at least one body region category from the pre-processed image comprises: If the computing power of the embedded device reaches the minimum computing power threshold, the segmentation network is called to perform category judgment on each pixel in the preprocessed image to obtain pixel categories respectively; taking a pixel region whose pixel category matches a body region category as a region position; If the computing power of the embedded device does not reach the minimum computing power threshold, the detection network is called to detect a bounding box that meets the body region category from the pre-processed image as the region position.

4. The method according to claim 1, characterized in that: The step of intercepting a regional image from the image of the pedestrian to be identified according to the regional position includes: Capturing an initial image of the area position from the image of the pedestrian to be identified; Each of the initial images is adjusted according to a fixed size and normalized to obtain a regional image.

5. The method according to claim 1, characterized in that The extracting the regional feature information from each of the regional images respectively includes: Convolving the regional image according to a preset convolution size to obtain input features; Extracting a plurality of convolution features from each of the input features using a plurality of convolution kernels of different sizes; Concatenating the convolutional features to obtain a concatenated feature; Performing convolution operation, global average pooling operation and normalization mapping operation on the spliced ​​features in sequence to generate channel weights; The channel weights are used to perform weighted fusion on the splicing features to obtain regional feature information.

6. The method according to claim 1, characterized in that The historical feature information includes at least one feature sub-information that matches the body region category; the combination and comparison of each of the region feature information and the historical feature information to generate a re-identification result includes: If the number of the regional feature information is equal to the number of the feature sub-information, respectively calculating the first similarity between each regional feature information and each feature sub-information; Performing weighted accumulation of the first similarities according to the area weights corresponding to the body area categories to obtain a pedestrian similarity; If the amount of the regional feature information is not equal to the amount of the feature sub-information, calculating the second similarity between the regional feature information and the feature sub-information of the same body region category; Adjusting the region weight according to the body region category to which the region feature information belongs to obtain an updated weight; Performing weighted accumulation of each of the second similarities according to the updated weight to obtain a pedestrian similarity; A re-identification result is generated according to a comparison result of the pedestrian similarity and a preset similarity threshold.

7. The method according to claim 6, characterized in that The generating a re-identification result according to the comparison result of the pedestrian similarity and the preset similarity threshold comprises: comparing the pedestrian similarity with a preset similarity threshold; If the pedestrian similarity is greater than or equal to a preset similarity threshold, it is determined that the pedestrian to which the regional feature information belongs is the same as the pedestrian to which the historical feature information belongs, and a re-identification result is generated; If the pedestrian similarity is less than a preset similarity threshold, it is determined that the pedestrian to which the regional feature information belongs is different from the pedestrian to which the historical feature information belongs, and a re-identification result is generated.

8. A pedestrian re-identification device, characterized in that: Applied to an embedded device, the device comprises: An image preprocessing module is used to obtain and preprocess the image of the pedestrian to be identified to obtain a preprocessed image; A region positioning module, used for locating a region position conforming to at least one body region category from the pre-processed image; A regional image capture module, used for capturing a regional image from the image of the pedestrian to be identified according to the regional position; A feature extraction module, used to extract regional feature information from each of the regional images; The feature combination comparison module is used to combine and compare the regional feature information and the historical feature information to generate a re-identification result.

9. An electronic device, characterized in that: It comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the pedestrian re-identification method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed, the pedestrian re-identification method according to any one of claims 1 to 7 is implemented.

Images