A method for re-identification of occluded pedestrians based on subject target discrimination

Abstract

The present invention relates to a method for re-identifying occluded pedestrians based on subject target discrimination, and belongs to the field of computer vision technology. The present invention first obtains a pedestrian snapshot slice, performs human instance segmentation on the pedestrian snapshot slice, and obtains a segmentation mask set; then scores each mask in the set, and determines the subject target segmentation mask of the current pedestrian snapshot slice according to the mask score; finally, the occluded pedestrian is identified based on the subject target segmentation mask. The present invention can accurately discriminate the subject target, and perform image-level interference information filtering on the occluded pedestrian slice based on the subject target, thereby effectively alleviating the interference of the background and non-target pedestrians in the picture on the subject target feature learning, and improving the association accuracy of the pedestrian re-identification model under occlusion.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and more particularly to a method for re-identifying occluded pedestrians based on subject-target discrimination. Background Art

[0002] Pedestrian re-identification technology aims to identify captured images of the same pedestrian from multi-camera scenes, and has important application value in fields such as security monitoring, intelligent transportation, and human-computer interaction.

[0003] In recent years, deep learning technology has made great progress in the field of computer vision. Models based on convolutional neural networks (CNNs) and Transformer architectures have been widely used for feature extraction in pedestrian images. These models can automatically extract useful features from images by learning from large amounts of data.

[0004] However, in real-world multi-camera scenarios, pedestrian images often contain occlusions or interfering people, which poses a challenge to accurately extracting pedestrian features. First, these occlusions and interfering people affect the integrity of the target pedestrian's image information, making the target pedestrian features acquired by the feature learning model semantically incomplete. Second, the image information of occlusions and interfering people introduces noise interference to the target pedestrian's feature learning process, resulting in inaccurate extracted target pedestrian features. To address these issues, occluded pedestrian re-identification technology has emerged.

[0005] For example, patent CN202310940241 proposes a multi-feature fusion method that enhances pedestrian snapshot slices in the training set, enabling the model to learn pedestrian features unrelated to obstacles. Another patent, CN201810269305, combines centralized learning with deep network learning to optimize the deep network model and reduce the interference of obstacles on pedestrian feature learning.

[0006] Although the above methods have alleviated the occlusion problem to a certain extent, they still fail to fully solve the noise interference caused by non-target people on the feature learning of the main target pedestrian in complex scenarios with multiple occlusions. As a result, the model may mistakenly include the feature information of the interfering people, thereby affecting the accuracy of pedestrian re-identification.

[0007] Therefore, how to provide a method for re-identifying occluded pedestrians that can accurately extract the main target features in complex scenes with multiple occlusions is an urgent problem that technicians in this field need to solve. Summary of the Invention

[0008] In view of this, in order to at least partially solve the above technical problems, the present invention provides a method for re-identifying occluded pedestrians based on subject target discrimination.

[0009] In order to achieve the above object, the present invention adopts the following technical solutions:

[0010] A method for re-identifying occluded pedestrians based on subject target discrimination, comprising:

[0011] Obtaining pedestrian snapshot slices, performing human instance segmentation on the pedestrian snapshot slices, and obtaining a segmentation mask set;

[0012] Score each mask in the set and determine the main target segmentation mask of the current pedestrian capture slice based on the mask score;

[0013] Recognize occluded pedestrians based on subject segmentation mask.

[0014] Preferably, the mask with the largest score is used as the main body of the current pedestrian capture slice.

[0015] Preferably, the pedestrian snapshot slice is obtained by acquiring a pedestrian snapshot picture and performing human target detection on the pedestrian snapshot picture.

[0016] Preferably, each mask in the set is scored according to the following formula:

[0017]

[0018] Where, represents the mask score, i represents the i-th pedestrian capture slice, j represents the j-th mask, represents the normalized distance from the jth mask to the i-th pedestrian capture slice border, Indicates the proximity of the jth mask to the center of the i-th pedestrian capture slice in the horizontal distribution; represents the pixel coverage of the j-th mask in the i-th pedestrian capture slice, and is the regularization coefficient.

[0019] Preferably,

[0020] Where, is the weight in the d direction, , represents the i-th pedestrian capture slice, represents the jth mask in the i-th pedestrian capture slice;

[0021]

[0022] Where, represents the absolute value function, represents the normalized horizontal coordinate mean of the j-th mask in the i-th pedestrian capture slice;

[0023]

[0024] Where, Representation mask The number of pixels covered, H i and W i are the height and width of the i-th pedestrian capture slice respectively.

[0025] Preferably, the step of identifying the obscured pedestrian based on the subject target segmentation mask includes:

[0026] Use the subject segmentation mask to filter the background of the pedestrian snapshot slice, and refill the background with the unmanned street scene background image;

[0027] Feature extraction is performed based on the captured slices of pedestrians with refilled background to achieve re-identification of occluded pedestrians.

[0028] Preferably, background filtering is performed on the captured pedestrian slices using the subject target segmentation mask as follows:

[0029]

[0030] Where, represents the pedestrian capture slice after background filtering, represents the subject target segmentation mask, Indicates that the coordinates on the subject target segmentation mask are The pixel value of Indicates that the coordinates of the i-th pedestrian capture slice are Pixel value.

[0031] Preferably, when the aspect ratio of the pedestrian snapshot slice after background filtering is less than a preset threshold, the aspect ratio filling is performed simultaneously as follows:

[0032]

[0033] Where, represents the pedestrian capture slice filled with aspect ratio, Background filtered image representing the main target At coordinates The value of the pixel, Indicates the height of the pedestrian capture slice after background filtering, Indicates the height of the pedestrian capture after filling, .

[0034] Preferably, the step of obtaining the unmanned street view image includes:

[0035] Obtain street view images for target detection and obtain a set of pedestrian detection frames;

[0036] The left and right unmanned areas are cropped based on the pedestrian detection frame with the largest height to obtain an unmanned street view background image with the same height as the street view image.

[0037] Preferably, the step of refilling the background with the unmanned street view background image includes:

[0038] The unmanned street scene background image is horizontally cropped to keep the length unchanged and the aspect ratio consistent with the aspect ratio of the pedestrian snapshot slice before refilling;

[0039] Scale the cropped unmanned street view background image to have the same height and width as the pedestrian snapshot slices before refilling;

[0040] Refill the background with a zoomed, empty street view background image.

[0041] It can be seen from the above technical solution that compared with the existing technology, the present invention discloses a method for re-identifying occluded pedestrians based on subject target discrimination. By adopting the subject target discrimination algorithm based on instance segmentation designed in this application, the instance segmentation model and subject target scoring method can be used to effectively evaluate the score of each human body belonging to the subject target, thereby achieving accurate subject target discrimination.

[0042] Further combined with the proposed image-level background interference information filtering and filling, it can effectively alleviate the interference of background and non-target pedestrians in the image on the main target feature learning, eliminate the noise caused by interfering people, so that the feature learning model can obtain more accurate main target features, thereby effectively improving the pedestrian re-identification effect in scenes with occlusions or interfering people, and improving the association accuracy of the pedestrian re-identification model in occlusion conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0044] Figure 1 Flowchart of the occluded pedestrian re-identification method based on subject target discrimination provided by the present invention;

[0045] Figure 2 This is a flow chart of an example of the occluded pedestrian re-identification method based on subject target discrimination provided by the present invention. DETAILED DESCRIPTION

[0046] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0047] The embodiment of the present invention discloses a method for re-identifying occluded pedestrians based on subject target discrimination, which effectively alleviates the interference problem caused by non-target people on the extraction of subject target features in occluded scenes without introducing additional training overhead, and is conducive to improving the recognition accuracy of pedestrian re-identification models in real complex scenes.

[0048] In the present invention, Figure 1 ,The occluded pedestrian re-identification method based on subject target discrimination,,including the following steps:

[0049] Obtaining pedestrian snapshot slices, performing human instance segmentation on the pedestrian snapshot slices, and obtaining a segmentation mask set;

[0050] Score each mask in the set, and determine the main target segmentation mask of the current pedestrian capture slice according to the mask score; preferably, the mask with the largest score is used as the main body of the current pedestrian capture slice.

[0051] Recognize occluded pedestrians based on subject segmentation mask.

[0052] Furthermore, in an exemplary embodiment, referring to Figure 2 ,The process of the occluded pedestrian re-identification method based on subject target discrimination is as follows:

[0053] S1. Obtain pedestrian snapshot slices;

[0054] In this embodiment, based on the videos collected by multiple surveillance cameras, pedestrian snapshot images are obtained, and human target detection is performed on the pedestrian snapshot images to obtain pedestrian snapshot slices. In one embodiment, the steps include:

[0055] S11: Sample and collect original images captured in the video captured by the surveillance camera to obtain a set of original images captured by the camera ,in The number of original images captured by the camera;

[0056] S12: Obtain the parameters of the pre-trained YOLOv8s target detection deep learning model and build the model;

[0057] S13: Use the built YOLOv8s model to capture the original image collection of the camera Perform target detection and sample P identity categories from the pedestrian snapshot slices obtained by target detection. Sample K snapshot slices x for each identity category. i , get the pedestrian capture slice set .

[0058] S2. Performing human instance segmentation on the captured pedestrian slices to obtain a set of segmentation masks. The segmentation process includes:

[0059] S21: Get the pre-trained YOLOv8x-seg instance segmentation deep learning model parameters and build the model;

[0060] S22: Use the built YOLOv8x-seg model to capture a collection of pedestrian slices Perform human instance segmentation. For each pedestrian snapshot, slice x i , get the human instance segmentation mask set , where N i Capture and slice pedestrians x i The number of pedestrians obtained by instance segmentation, H i and W i They are respectively captured slices of pedestrians x i height and width.

[0061] In particular, if the pedestrian is captured and sliced ​​x i If the human instance segmentation does not obtain any human segmentation mask, no more snapshot slice x will be taken for the pedestrian. i Perform the processing from S3 to S9, and directly .

[0062] S3, score each mask in the set;

[0063] The scoring method is:

[0064]

[0065] Where, represents the mask score, i represents the i-th pedestrian capture slice, j represents the j-th mask, represents the normalized distance from the jth mask to the i-th pedestrian capture slice border, Indicates the proximity of the jth mask to the center of the i-th pedestrian capture slice in the horizontal distribution; represents the pixel coverage of the j-th mask in the i-th pedestrian capture slice, and is the regularization coefficient.

[0066] in,

[0067]

[0068] Where, is the weight in the d direction, , represents the i-th pedestrian capture slice, represents the jth mask in the i-th pedestrian capture slice;

[0069]

[0070] Where, represents the absolute value function, represents the normalized horizontal coordinate mean of the j-th mask in the i-th pedestrian capture slice;

[0071]

[0072] Where, Representation mask The number of pixels covered, H i and W i are the height and width of the i-th pedestrian capture slice respectively.

[0073] Finally, for each pedestrian snapshot slice x i A set of segmentation mask subject target scores can be obtained ,in .

[0074] S4, determine the main target segmentation mask of the current pedestrian capture slice according to the score of the mask; in this embodiment, for each pedestrian capture slice x i , according to the segmentation mask subject target score , the human instance segmentation mask set corresponding to the pedestrian snapshot slice The segmentation mask corresponding to the discriminant subject target ;

[0075] As a preference, for each pedestrian snapshot slice x i The corresponding segmentation mask subject object score set , respectively select the largest Corresponding human instance segmentation mask , as the main target segmentation mask of the pedestrian capture slice , the superscript of the subject object segmentation mask The formula is defined as:

[0076]

[0077] Then, the occluded pedestrian is identified based on the subject object segmentation mask. In another embodiment, the steps include:

[0078] S5. Use the subject target segmentation mask to perform background filtering on the captured pedestrian slices;

[0079] For each pedestrian snapshot slice x i , segmentation mask based on the subject target Original snapshot slice x i Perform background filtering, the specific formula is:

[0080]

[0081] Where, represents the pedestrian capture slice after background filtering, Background filtered image representing the main target At coordinates The value of the pixel, represents the subject target segmentation mask, Indicates that the coordinates on the subject target segmentation mask are The pixel value of Indicates that the coordinates of the i-th pedestrian capture slice are Pixel value, where and .

[0082] In an optional embodiment, when the aspect ratio of the background-filtered pedestrian capture slice is less than a preset threshold r, the aspect ratio is simultaneously filled as follows to restore the half-body pedestrian capture slice to the full pedestrian height. The specific formula is:

[0083]

[0084] Where, represents the pedestrian capture slice filled with aspect ratio, Background filtered image representing the main target At coordinates The value of the pixel, Indicates the height of the pedestrian capture slice after background filtering, Indicates the height of the pedestrian capture after filling, .

[0085] S6. Obtain an unmanned street scene background image. The acquisition steps are as follows:

[0086] Obtain street view images for target detection and obtain a set of pedestrian detection frames;

[0087] The left and right unmanned areas are cropped based on the pedestrian detection frame with the largest height to obtain an unmanned street view background image with the same height as the street view image.

[0088] In this embodiment, obtaining an unmanned street scene background image from the Cityscapes street scene image dataset is used as an example for description; the specific steps include:

[0089] S61. Obtain the CityScapes street view image dataset;

[0090] S62, for each street view picture in the Cityscapes dataset Perform target detection and obtain a set of pedestrian detection frames ,in express The number of pedestrian detection boxes in and Street View images The height and width, and Pedestrian detection box height and width;

[0091] S63: In every street view picture Pedestrian detection box set Select the pedestrian detection frame with the largest height , superscript The formula is defined as:

[0092]

[0093] S64: For each street view image , based on the pedestrian detection frame with the largest height The position of the left and right unmanned areas are cropped to obtain Background image of an unmanned street scene at the same height ; Perform the above processing on all street view images to obtain a set of unmanned street view background images ,in The number of unmanned street view background images.

[0094] S7. Refilling the background using the unmanned street scene background image. The filling step includes:

[0095] The unmanned street scene background image is horizontally cropped to keep the length unchanged and the aspect ratio consistent with the aspect ratio of the pedestrian snapshot slice before refilling;

[0096] Scale the cropped unmanned street view background image to have the same height and width as the pedestrian snapshot slices before refilling;

[0097] Refill the background with a zoomed, empty street view background image.

[0098] In this embodiment, for each aspect ratio filled subject target background filter picture , using the unmanned street view background image collection Fill the background with the picture in the image to get the main target background filling picture with aspect ratio filling , where H i and W i is the length and width of the original captured image;

[0099] S71: Filter the subject target background image after aspect ratio filling , randomly select a street view picture from the collection of unmanned street view background pictures ,in and Indicates a street view image height and width;

[0100] S72: Yes Perform random horizontal cropping to make the background fill image of the unmanned street scene obtained after cropping , the length is the same as before cutting , and the aspect ratio is the same as the subject target background filter image after aspect ratio filling. ;

[0101] S73: Fill the background of the cropped street scene with the image while keeping the aspect ratio unchanged Perform proportional scaling so that the background fill image of the unmanned street scene is obtained after scaling. The subject target background filtered image after filling with aspect ratio have the same height and width;

[0102] S74: Use the zoomed background fill image of the unoccupied street scene Filter the subject target background image after aspect ratio filling Filling is performed. The specific formula used in the filling process is:

[0103]

[0104] in and , Background fill image representing the main target At coordinates The value of the pixel;

[0105] S75: Slice according to the original image The size of the main target background filling picture after the aspect ratio is filled Crop and get the original image slice x i Background filler image of the main target of the same size .

[0106] S8, extract features based on the captured pedestrian slices with the background refilled to achieve re-identification of occluded pedestrians;

[0107] In this embodiment, the TransReID pedestrian re-identification feature extraction model is used to fill the background of the subject target image. Perform feature extraction to obtain the subject target feature f i ;

[0108] S81: Obtain the pre-trained TransReID person re-identification deep learning model parameters and build the model;

[0109] S82: Use the constructed TransReID model to fill the image set of the subject target background Perform feature extraction to obtain the subject target feature set .

[0110] In another embodiment, association and querying of obscured pedestrians are implemented based on the extracted features, including:

[0111] S9, based on the subject target feature set ,Use cosine similarity to calculate the similarity of target features of different subjects, and then realize pedestrian association and query across cameras;

[0112] S91: For each subject target feature f i Normalize the channel dimension to obtain the normalized subject target feature set ;

[0113] S92: Use cosine similarity to calculate the similarity between subject target features. For each normalized subject target feature , and attach the same pedestrian id to it and the normalized subject target feature with the highest similarity, thereby realizing pedestrian association and query across cameras.

[0114] The present invention first uses an instance segmentation model to obtain each human instance segmentation mask in the pedestrian snapshot slice, calculates the subject target score of each human instance segmentation mask through the designed subject target scoring method, and selects the human instance segmentation mask with the highest subject target score as the subject target mask; then, according to the subject target mask, an unmanned street scene picture with a style similar to the background picture taken by the camera is used to filter and fill the background of the pedestrian snapshot picture to obtain a subject target background filled picture; finally, a pedestrian re-identification model is used to extract features from different subject target background filled pictures, and the feature similarity is calculated to achieve pedestrian association.

[0115] The present invention proposes an occluded pedestrian re-identification method based on subject target discrimination, which filters the interference information of the occluded pedestrian slices at the image level, effectively alleviating the interference of the background and non-target pedestrians in the image on the subject target feature learning, which is conducive to improving the association accuracy of the pedestrian re-identification model under occlusion.

[0116] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0117] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for re-identifying occluded pedestrians based on subject target discrimination, characterized in that: Obtaining pedestrian snapshot slices, performing human instance segmentation on the pedestrian snapshot slices, and obtaining a segmentation mask set; Score each mask in the set and determine the main target segmentation mask of the current pedestrian capture slice based on the mask score; Including taking the mask with the largest score as the subject of the current pedestrian capture slice; Identify obscured pedestrians based on the subject target segmentation mask, including: Obtain a street view image for target detection and obtain a set of pedestrian detection frames. Based on the pedestrian detection frame with the largest height, crop the left and right unmanned areas to obtain an unmanned street view background image with the same height as the street view image. Use the subject segmentation mask to filter the background of the pedestrian snapshot slice, and refill the background with the unmanned street scene background image; Obtain the parameters of the pre-trained TransReID pedestrian re-identification deep learning model and build a model; use the constructed TransReID model to extract features from the subject target background fill image set to obtain the subject target feature set; use cosine similarity based on the subject target feature set to calculate the similarity of different subject target features, thereby realizing pedestrian association and query across cameras.

2. The method for re-identifying occluded pedestrians based on subject target discrimination according to claim 1, characterized in that: Pedestrian snapshot slices are obtained by acquiring pedestrian snapshot images and performing human target detection on the pedestrian snapshot images.

3. The method for re-identifying occluded pedestrians based on subject target discrimination according to claim 1, characterized in that: Score each mask in the set according to the following formula: Where, represents the mask score, i represents the i-th pedestrian capture slice, j represents the j-th mask, represents the normalized distance from the jth mask to the i-th pedestrian capture slice border, Indicates the proximity of the jth mask to the center of the i-th pedestrian capture slice in the horizontal distribution; represents the pixel coverage of the j-th mask in the i-th pedestrian capture slice, λ x and λ c is the regularization coefficient.

4. The method for re-identifying occluded pedestrians based on subject target discrimination according to claim 3, characterized in that: Where w d is the weight in the d direction, d∈{up, bottom, left, right}, x i represents the i-th pedestrian capture slice, represents the jth mask in the i-th pedestrian capture slice; Where abs(·) represents the absolute value function, represents the normalized horizontal coordinate mean of the j-th mask in the i-th pedestrian capture slice; Where, Representation mask The number of pixels covered, H i and W i are the height and width of the i-th pedestrian capture slice respectively.

5. The method for re-identifying occluded pedestrians based on subject target discrimination according to claim 1, characterized in that: The steps of identifying obscured pedestrians based on the subject target segmentation mask include: Use the subject segmentation mask to filter the background of the pedestrian snapshot slice, and refill the background with the unmanned street scene background image; Feature extraction is performed based on the captured slices of pedestrians with refilled background to achieve re-identification of occluded pedestrians.

6. The method for re-identifying occluded pedestrians based on subject target discrimination according to claim 5, characterized in that: Use the subject segmentation mask to perform background filtering on the pedestrian snapshot slice as follows; Where, represents the pedestrian capture slice after background filtering, represents the subject target segmentation mask, Represents the pixel value with coordinates (h, w) on the subject target segmentation mask, x i (h, w) represents the pixel value with coordinates (h, w) in the i-th pedestrian capture slice.

7. The method for re-identifying occluded pedestrians based on subject target discrimination according to claim 6, characterized in that: When the aspect ratio of the pedestrian snapshot slice after background filtering is less than the preset threshold, the aspect ratio filling is performed simultaneously as follows: Where, represents the pedestrian capture slice filled with aspect ratio, Background filtered image representing the main target The value of the pixel at coordinate (h, w), H i Indicates the height of the pedestrian capture slice after background filtering, Indicates the height of the pedestrian capture after filling, r represents the preset aspect ratio threshold, W i Indicates the width of the i-th pedestrian capture slice.

8. The method for re-identifying occluded pedestrians based on subject target discrimination according to claim 5, characterized in that: The steps of refilling the background with the unmanned street view background image include: The unmanned street scene background image is horizontally cropped to keep the length unchanged and the aspect ratio consistent with the aspect ratio of the pedestrian snapshot slice before refilling; Scale the cropped unmanned street view background image to have the same height and width as the pedestrian snapshot slices before refilling; Refill the background with a zoomed, empty street view background image.

Images