Shielding pedestrian re-identification method based on main body target discrimination

Through the method of occluding pedestrian re-identification based on subject target discrimination, and using human instance segmentation and background filling technology, the problem of incomplete and inaccurate pedestrian feature extraction in multiple people occlusion scenarios is solved, and the accuracy and correlation accuracy of pedestrian re-identification are improved Rate.

CN119091467AActive Publication Date: 2024-12-06BEIHANG UNIV +1
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
CN202411275283.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-12
Publication Date
2024-12-06
Estimated Expiration
2044-09-12

AI Technical Summary

Technical Problem

In the complex scenario of multiple people blocking, the prior art is difficult to obtain complete and accurate pedestrian characteristics semantically, resulting in the accuracy of pedestrian re-identification being affected by noise interference.

Method used

The pedestrian re-identification method based on subject target discrimination is adopted. The segmentation mask is obtained through human instance segmentation, the subject target mask is determined by scoring, background filtering and filling are performed, interference information is reduced, and feature extraction is realized.

Benefits of technology

It effectively alleviates the interference of background and non-target pedestrians on subject target characteristics learning, and improves the accuracy and correlation accuracy of pedestrian re-identification.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119091467A_ABST
    Figure CN119091467A_ABST
Patent Text Reader

Abstract

The invention relates to a blocked pedestrian re-identification method based on main body target discrimination, and belongs to the technical field of computer vision. The method comprises the following steps: firstly, obtaining a pedestrian snapshot slice, and carrying out human body instance segmentation on the pedestrian snapshot slice to obtain a segmentation mask set; scoring each mask in the set, and determining a main target segmentation mask of the current pedestrian snapshot slice according to the score of the mask; and finally, identifying the shielded pedestrian based on the main body target segmentation mask. According to the method, the main body target can be accurately distinguished, and image-level interference information filtering is performed on the occluded pedestrian slices based on the main body target, so that the interference of the background in the picture and non-target pedestrians on the feature learning of the main body target is effectively relieved, and the association accuracy of the pedestrian re-identification model under the occlusion condition is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and more specifically to a method for re-identifying occluded pedestrians based on subject target discrimination. Background Art

[0002] Pedestrian re-identification technology aims to identify captured images of the same pedestrian from multi-camera scenes, and has important application value in fields such as security monitoring, intelligent transportation, and human-computer interaction.

[0003] In recent years, deep learning technology has made great progress in the field of computer vision. Among them, models based on convolutional neural networks (CNN) and Transformer architectures have been widely used in feature extraction of pedestrian images. These models can automatically extract useful features from images by learning a large amount of data.

[0004] However, in actual multi-camera scenarios, there are often occlusions or interfering people in the captured pedestrian images, which brings challenges to the accurate extraction of pedestrian features: first, occlusions and interfering people affect the integrity of the target pedestrian image information, which makes the target pedestrian features obtained by the feature learning model semantically incomplete; second, the image information of occlusions and interfering people brings noise interference to the feature learning process of the target pedestrian, making the extracted target pedestrian features inaccurate. In order to solve the above problems, the occluded pedestrian re-identification technology came into being.

[0005] For example, patent CN202310940241 proposed a method based on multi-feature fusion, which enables the model to learn pedestrian features that are unrelated to obstacles by enhancing the pedestrian snapshot slices in the training set. Another patent CN201810269305 uses a method that combines centralized learning with deep network learning to optimize the deep network model and reduce the interference of obstacles on pedestrian feature learning.

[0006] Although the above methods have alleviated the occlusion problem to a certain extent, they still fail to fully solve the noise interference caused by non-target people to the feature learning of the main target pedestrian in complex scenes with multiple occlusions, resulting in the model may mistakenly include the feature information of the interfering person, thereby affecting the accuracy of pedestrian re-identification.

[0007] Therefore, how to provide a method for re-identifying occluded pedestrians that can accurately extract the main target features in complex scenes with multiple occlusions is an urgent problem that technicians in this field need to solve. Summary of the invention

[0008] In view of this, in order to at least partially solve the above technical problems, the present invention provides a method for re-identifying occluded pedestrians based on subject target discrimination.

[0009] In order to achieve the above object, the present invention adopts the following technical solution:

[0010] A method for re-identifying an occluded pedestrian based on subject target discrimination, comprising:

[0011] Obtaining pedestrian snapshot slices, performing human body instance segmentation on the pedestrian snapshot slices, and obtaining a segmentation mask set;

[0012] Score each mask in the set, and determine the main target segmentation mask of the current pedestrian capture slice according to the mask score;

[0013] Recognize occluded pedestrians based on subject object segmentation mask.

[0014] Preferably, the mask with the largest score is used as the main body of the current pedestrian capture slice.

[0015] Preferably, the pedestrian snapshot slice is obtained by acquiring a pedestrian snapshot picture and performing human target detection on the pedestrian snapshot picture.

[0016] Preferably, each mask in the set is scored according to the following formula:

[0017]

[0018] In the formula, represents the mask score, i represents the i-th pedestrian capture slice, j represents the j-th mask, represents the normalized distance from the jth mask to the i-th pedestrian capture slice border, Indicates the proximity of the jth mask to the center of the i-th pedestrian snapshot slice in lateral distribution; represents the pixel coverage of the jth mask in the i-th pedestrian capture slice, and is the regularization coefficient.

[0019] Preferably,

[0020] In the formula, is the weight in the d direction, , represents the i-th pedestrian capture slice, represents the jth mask in the i-th pedestrian capture slice;

[0021]

[0022] In the formula, represents the absolute value function, represents the normalized horizontal coordinate mean of the j-th mask in the i-th pedestrian capture slice;

[0023]

[0024] In the formula, Representation mask The number of pixels covered, H i and W i are the height and width of the i-th pedestrian capture slice respectively.

[0025] Preferably, the step of identifying the obscured pedestrian based on the subject target segmentation mask comprises:

[0026] Use the subject target segmentation mask to filter the background of the pedestrian snapshot slice, and refill the background with the unmanned street view background image;

[0027] Feature extraction is performed based on the captured slices of pedestrians with background refilled to achieve re-identification of occluded pedestrians.

[0028] Preferably, the subject target segmentation mask is used to perform background filtering on the pedestrian captured slices in the following manner;

[0029]

[0030] In the formula, represents the pedestrian capture slice after background filtering, represents the subject target segmentation mask, Indicates that the coordinates on the subject target segmentation mask are The pixel value of Indicates that the coordinates of the i-th pedestrian capture slice are Pixel value.

[0031] Preferably, when the aspect ratio of the pedestrian captured slice after background filtering is less than a preset threshold, the aspect ratio filling is performed simultaneously as follows:

[0032]

[0033] In the formula, represents the pedestrian capture slice filled with aspect ratio, Background filtered image showing the main target At coordinates The value of the pixel, Indicates the height of the pedestrian capture slice after background filtering, Indicates the height of the pedestrian capture after filling, .

[0034] Preferably, the step of acquiring the unmanned street view picture includes:

[0035] Obtain street view images for target detection and obtain a set of pedestrian detection frames;

[0036] The left and right unmanned areas are cropped based on the pedestrian detection frame with the highest height to obtain an unmanned street view background image with the same height as the street view image.

[0037] Preferably, the step of refilling the background with the unmanned street view background image comprises:

[0038] The unmanned street view background image is horizontally cropped to keep the length unchanged and the aspect ratio consistent with the aspect ratio of the pedestrian snapshot slice before refilling;

[0039] The cropped unmanned street view background image is scaled proportionally to have the same height and width as the pedestrian snapshot slice before refilling;

[0040] Refill the background with a zoomed, empty street view background image.

[0041] It can be seen from the above technical solution that, compared with the prior art, the present invention discloses a method for re-identifying occluded pedestrians based on subject target discrimination. By adopting the subject target discrimination algorithm based on instance segmentation designed in this application, the instance segmentation model and the subject target scoring method can be used to effectively evaluate the score of each human body belonging to the subject target, thereby achieving accurate subject target discrimination.

[0042] Further combined with the proposed image-level background interference information filtering and filling, the interference of background and non-target pedestrians in the image on the main target feature learning can be effectively alleviated, and the noise caused by interfering people can be eliminated, so that the feature learning model can obtain more accurate main target features, thereby effectively improving the pedestrian re-identification effect in scenes with occlusions or interfering people, and improving the association accuracy of the pedestrian re-identification model under occlusion. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0044] Figure 1 A flow chart of the method for re-identifying an occluded pedestrian based on subject target discrimination provided by the present invention;

[0045] Figure 2 This is a flow chart of an example of the method for re-identifying an occluded pedestrian based on subject target discrimination provided by the present invention. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] The embodiment of the present invention discloses a method for re-identifying occluded pedestrians based on subject target discrimination, which effectively alleviates the interference problem caused by non-target people on the feature extraction of subject targets in occluded scenes without introducing additional training overhead, and is conducive to improving the recognition accuracy of pedestrian re-identification models in real complex scenes.

[0048] In the present invention, Figure 1 ,The occluded pedestrian re-identification method based on subject target discrimination includes the following steps:

[0049] Obtaining pedestrian snapshot slices, performing human body instance segmentation on the pedestrian snapshot slices, and obtaining a segmentation mask set;

[0050] Score each mask in the set, and determine the main target segmentation mask of the current pedestrian capture slice according to the mask score; preferably, the mask with the largest score is used as the main body of the current pedestrian capture slice.

[0051] Recognize occluded pedestrians based on subject object segmentation mask.

[0052] Further, in an exemplary embodiment, referring to Figure 2 ,The process of the occluded pedestrian re-identification method based on subject target discrimination is as follows:

[0053] S1, obtain pedestrian capture slices;

[0054] In this embodiment, based on the videos collected by multiple surveillance cameras, pedestrian snapshot images are obtained, and human target detection is performed on the pedestrian snapshot images to obtain pedestrian snapshot slices. In one embodiment, the steps include:

[0055] S11: Sample and collect the captured original images from the video captured by the surveillance camera to obtain a collection of the captured original images of the camera ,in The number of original images captured by the camera;

[0056] S12: Obtain the parameters of the pre-trained YOLOv8s target detection deep learning model and build the model;

[0057] S13: Use the built YOLOv8s model to capture the original image collection of the camera Perform target detection and sample P identity categories from the pedestrian snapshot slices obtained by target detection. Sample K snapshot slices x for each identity category. i , get the pedestrian capture slice collection .

[0058] S2. Performing human body instance segmentation on the captured pedestrian slices to obtain a segmentation mask set. The segmentation process includes:

[0059] S21: Get the pre-trained YOLOv8x-seg instance segmentation deep learning model parameters and build the model;

[0060] S22: Use the built YOLOv8x-seg model to capture a collection of pedestrian slices Perform human instance segmentation. For each pedestrian snapshot slice x i , get the human instance segmentation mask set , where N i Capture slices for pedestrians x i The number of pedestrians obtained by instance segmentation, H i and W i They are the pedestrian capture slices x i height and width.

[0061] In particular, if the pedestrian is captured and sliced ​​x i If no human segmentation mask is obtained during human instance segmentation, no more snapshot slice x will be taken for the pedestrian. i Perform the processing from S3 to S9, and directly .

[0062] S3, score each mask in the set;

[0063] The scoring method is:

[0064]

[0065] In the formula, represents the mask score, i represents the i-th pedestrian capture slice, j represents the j-th mask, represents the normalized distance from the jth mask to the i-th pedestrian capture slice border, Indicates the proximity of the jth mask to the center of the i-th pedestrian snapshot slice in lateral distribution; represents the pixel coverage of the jth mask in the i-th pedestrian capture slice, and is the regularization coefficient.

[0066] in,

[0067]

[0068] In the formula, is the weight in the d direction, , represents the i-th pedestrian capture slice, represents the jth mask in the i-th pedestrian capture slice;

[0069]

[0070] In the formula, represents the absolute value function, represents the normalized horizontal coordinate mean of the j-th mask in the i-th pedestrian capture slice;

[0071]

[0072] In the formula, Representation mask The number of pixels covered, H i and W i are the height and width of the i-th pedestrian capture slice respectively.

[0073] Finally, for each pedestrian snapshot slice x i A segmentation mask subject target score set can be obtained ,in .

[0074] S4. Determine the main target segmentation mask of the current pedestrian capture slice according to the score of the mask. In this embodiment, for each pedestrian capture slice x i , according to the segmentation mask subject target score , the human instance segmentation mask set corresponding to the pedestrian snapshot slice The segmentation mask corresponding to the discriminant subject target ;

[0075] As a preferred option, for each pedestrian capture slice x i The corresponding segmentation mask subject object score set , respectively select the largest Corresponding human instance segmentation mask , as the main target segmentation mask of the pedestrian capture slice , the superscript of the subject object segmentation mask The formula is defined as:

[0076]

[0077] Then, the obscured pedestrian is identified based on the subject target segmentation mask. In another embodiment, the steps include:

[0078] S5, using the subject target segmentation mask to perform background filtering on the captured slices of the pedestrian;

[0079] For each pedestrian snapshot slice x i , according to the subject target segmentation mask Slice the original snapshot x i Perform background filtering, the specific formula is:

[0080]

[0081] In the formula, represents the pedestrian capture slice after background filtering, Background filtered image showing the main target At coordinates The value of the pixel, represents the subject target segmentation mask, Indicates that the coordinates on the subject target segmentation mask are The pixel value of Indicates that the coordinates of the i-th pedestrian capture slice are Pixel value, where and .

[0082] In an optional embodiment, when the aspect ratio of the captured slice of the pedestrian after background filtering is less than the preset threshold r, the aspect ratio is filled as follows so that the half-body captured slice of the pedestrian is restored to the height of the full pedestrian. The specific formula is:

[0083]

[0084] In the formula, represents the pedestrian capture slice filled with aspect ratio, Background filtered image showing the main target At coordinates The value of the pixel, Indicates the height of the pedestrian capture slice after background filtering, Indicates the height of the pedestrian capture after filling, .

[0085] S6. Obtaining an unmanned street scene background image; the obtaining steps are:

[0086] Obtain street view images for target detection and obtain a set of pedestrian detection frames;

[0087] The left and right unmanned areas are cropped based on the pedestrian detection frame with the highest height to obtain an unmanned street view background image with the same height as the street view image.

[0088] In this embodiment, an example of obtaining an unmanned street view background image from a Cityscapes street view image dataset is used for description; the specific steps include:

[0089] S61, obtain the CityScapes street view image dataset;

[0090] S62, for each street view picture in the Cityscapes dataset Perform target detection and obtain a set of pedestrian detection frames ,in express The number of pedestrian detection boxes in and Street View Pictures The height and width of and Pedestrian detection box The height and width of

[0091] S63: In every street view picture Pedestrian detection box set Select the pedestrian detection frame with the largest height , superscript The formula is defined as:

[0092]

[0093] S64: For each street view image , according to the pedestrian detection box with the largest height The position of the left and right unmanned areas is cropped to obtain and Background image of an unmanned street scene with the same height ; Perform the above processing on all street view images to obtain a set of unmanned street view background images ,in is the number of unmanned street view background images.

[0094] S7, refilling the background with the unmanned street view background image; the filling step includes:

[0095] The unmanned street view background image is horizontally cropped to keep the length unchanged and the aspect ratio consistent with the aspect ratio of the pedestrian snapshot slice before refilling;

[0096] The cropped unmanned street view background image is scaled proportionally to have the same height and width as the pedestrian snapshot slice before refilling;

[0097] Refill the background with a zoomed, empty street view background image.

[0098] In this embodiment, for each subject target background filter image after aspect ratio filling, , using the unmanned street view background image collection Use the image in the background to fill the background, and get the main target background filling image with aspect ratio filling , where H i and W i is the length and width of the original captured image;

[0099] S71: Filter the subject background image after aspect ratio filling , randomly select a street view image from the collection of unmanned street view background images ,in and Indicates street view images The height and width of

[0100] S72: Yes Perform random horizontal cropping so that the unmanned street scene background fill image is obtained after cropping , the length is the same as before cutting. , and the aspect ratio is the same as the subject target background filter image after aspect ratio filling. ;

[0101] S73: Fill the background of the cropped street scene with the image while keeping the aspect ratio unchanged Scale proportionally so that the background fill image of the unmanned street scene obtained after scaling is The subject target background filtered image after filling with aspect ratio have the same height and width;

[0102] S74: Use the zoomed background fill image of the unmanned street scene Filter the subject background image after aspect ratio filling Filling is performed, and the specific formula used in the filling process is:

[0103]

[0104] in and , Background fill image representing the main target At coordinates The value of the pixel;

[0105] S75: Slice according to the original image The size of the main target background filling picture after the aspect ratio is filled Crop and get the original image slice x i Background fill image of the main target of the same size .

[0106] S8, feature extraction is performed based on the captured slices of pedestrians with the background refilled, so as to realize the re-identification of occluded pedestrians;

[0107] In this embodiment, the TransReID pedestrian re-identification feature extraction model is used to fill the background of the main target image. Perform feature extraction to obtain the main target feature f i ;

[0108] S81: Obtain the pre-trained TransReID pedestrian re-identification deep learning model parameters and build the model;

[0109] S82: Use the constructed TransReID model to fill the background image collection of the subject target Perform feature extraction to obtain the main target feature set .

[0110] In another embodiment, association and query of blocked pedestrians are implemented based on the extracted features, including:

[0111] S9. According to the subject target feature set ,Use cosine similarity to calculate the similarity of target features of different subjects, and then realize pedestrian association and query across cameras;

[0112] S91: For each subject target feature f i Normalize the channel dimension to obtain the normalized subject target feature set ;

[0113] S92: Use cosine similarity to calculate the similarity between subject target features. For each normalized subject target feature , and attach the same pedestrian id to it and the normalized subject target feature with the highest similarity, thereby realizing pedestrian association and query across cameras.

[0114] The present invention firstly obtains each human instance segmentation mask in the pedestrian snapshot slice by using an instance segmentation model, calculates the subject target score of each human instance segmentation mask by using a designed subject target scoring method, and selects the human instance segmentation mask with the highest subject target score as the subject target mask; then, according to the subject target mask, an unmanned street scene picture with a style similar to that of the background picture taken by the camera is used to perform background filtering and filling on the pedestrian snapshot picture to obtain a subject target background filling picture; finally, a pedestrian re-identification model is used to extract features from different subject target background filling pictures, and the feature similarity is calculated to achieve pedestrian association.

[0115] The present invention proposes an occluded pedestrian re-identification method based on subject target discrimination, which filters the interference information of the occluded pedestrian slices at the image level, effectively alleviating the interference of the background and non-target pedestrians in the picture on the subject target feature learning, which is conducive to improving the association accuracy of the pedestrian re-identification model under occlusion.

[0116] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0117] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for re-identifying occluded pedestrians based on subject target discrimination. It is characterized in that Obtaining pedestrian snapshot slices, performing human body instance segmentation on the pedestrian snapshot slices, and obtaining a segmentation mask set; Score each mask in the set, and determine the main target segmentation mask of the current pedestrian capture slice according to the mask score; Recognize occluded pedestrians based on subject object segmentation mask.

2. According to the method for re-identifying an occluded pedestrian based on subject target discrimination according to claim 1, It is characterized in that The mask with the largest score is used as the subject of the current pedestrian capture slice.

3. According to the method for re-identifying an occluded pedestrian based on subject target discrimination according to claim 1, It is characterized in that The pedestrian snapshot slices are obtained by acquiring pedestrian snapshot pictures and performing human target detection on the pedestrian snapshot pictures.

4. According to the method for re-identifying an occluded pedestrian based on subject target discrimination according to claim 1, It is characterized in that Score each mask in the set according to the following formula: ; In the formula, represents the mask score, i represents the i-th pedestrian capture slice, j represents the j-th mask, represents the normalized distance from the jth mask to the i-th pedestrian capture slice border, Indicates the proximity of the jth mask to the center of the i-th pedestrian capture slice in the horizontal distribution; represents the pixel coverage of the jth mask in the i-th pedestrian capture slice, and is the regularization coefficient.

5. According to the method for re-identifying an occluded pedestrian based on subject target discrimination according to claim 4, It is characterized in that ; In the formula, is the weight in the d direction, , represents the i-th pedestrian capture slice, represents the jth mask in the i-th pedestrian capture slice; ; In the formula, represents the absolute value function, represents the normalized horizontal coordinate mean of the j-th mask in the i-th pedestrian capture slice; ; In the formula, Representation mask The number of pixels covered, H i and W i are the height and width of the i-th pedestrian capture slice respectively.

6. According to the method for re-identifying an occluded pedestrian based on subject target discrimination according to claim 1, It is characterized in that The steps of identifying the obscured pedestrian based on the subject target segmentation mask include: Use the subject target segmentation mask to filter the background of the pedestrian snapshot slice, and refill the background with the unmanned street view background image; Feature extraction is performed based on the captured slices of pedestrians with background refilled to achieve re-identification of occluded pedestrians.

7. According to the method for re-identifying an occluded pedestrian based on subject target discrimination according to claim 6, It is characterized in that Use the subject target segmentation mask to perform background filtering on the pedestrian snapshot slice as follows; ; In the formula, represents the pedestrian capture slice after background filtering, represents the subject target segmentation mask, Indicates that the coordinates on the subject target segmentation mask are The pixel value of Indicates that the coordinates of the i-th pedestrian capture slice are Pixel value.

8. According to the method for re-identifying an occluded pedestrian based on subject target discrimination according to claim 7, It is characterized in that When the aspect ratio of the pedestrian snapshot slice after background filtering is less than the preset threshold, the aspect ratio filling is performed simultaneously as follows: ; In the formula, represents the pedestrian capture slice filled with aspect ratio, Background filtered image showing the main target At coordinates The value of the pixel, Indicates the height of the pedestrian capture slice after background filtering, Indicates the height of the pedestrian capture after filling, .

9. According to the method for re-identifying an occluded pedestrian based on subject target discrimination according to claim 6, It is characterized in that The steps for obtaining unmanned street view images include: Obtain street view images for target detection and obtain a set of pedestrian detection frames; The left and right unmanned areas are cropped based on the pedestrian detection frame with the highest height to obtain an unmanned street view background image with the same height as the street view image.

10. The method for re-identifying an occluded pedestrian based on subject target discrimination according to claim 6, It is characterized in that The steps of refilling the background with the unmanned street view background image include: The unmanned street view background image is horizontally cropped to keep the length unchanged and the aspect ratio consistent with the aspect ratio of the pedestrian snapshot slice before refilling; The cropped unmanned street view background image is scaled proportionally to have the same height and width as the pedestrian snapshot slice before refilling; Refill the background with a zoomed, empty street view background image.

Citation Information

Patent Citations

  • Image instance segmentation method and device

    CN110705558A

  • Pedestrian multi-target tracking method combined with instance segmentation

    CN116993775A

  • Pedestrian re-identification method under shielding condition

    CN117333900A

  • Electric vehicle instance retrieval method in streetscape monitoring scene and related device

    CN117746331A

  • Shielding pedestrian re-identification method based on mask self-supervision shielding pixel reconstruction

    CN117877068A