A person re-identification method based on cross-temporal data

Abstract

The present invention provides a pedestrian re-identification method based on cross-time domain data, which relates to the technical field of pedestrian identification, and includes the following steps: constructing a monitoring scene; segmenting a monitoring area and a monitoring scene; intercepting a current monitoring video frame to obtain a monitoring image; intercepting an image in the monitoring scene to obtain a background image; removing the background area in the monitoring image to obtain an image to be processed; segmenting the image to be processed to obtain an edge image and a center image; capturing a human target from the edge image to obtain a human image; giving a unique identity code to the human target; identifying the human image, marking the identification result, and associating the mark and the identity code; capturing facial features from the center image to obtain a facial image; obtaining the mark of the human target; if not obtained, giving the human target a unique identity code, and identifying the facial image to obtain and mark the identification result; if obtained, identifying the facial image according to the mark, and obtaining and updating the identification result. The pedestrian missed detection rate is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of pedestrian recognition, and in particular to a pedestrian re-recognition method based on cross-time domain data. Background Art

[0002] In the application of person re-identification technology, occlusion between pedestrians and pedestrians and between pedestrians and objects is very common, which results in that most pedestrian images used for person re-identification are incomplete and partially missing. Based on this, pedestrians are often missed in complex and dynamic monitoring environments. For example, when human models, various buildings and portraits are set up in the monitoring environment, pedestrians wearing clothes with similar colors to the background are easily misjudged as background due to mutual occlusion between pedestrians and the complexity of the background, making it difficult to distinguish individual pedestrians from the background. Pedestrian missed detection will not only directly affect the recognition accuracy of the person re-identification system for specific pedestrians, but also have a negative impact on the performance and reliability of the entire person re-identification system. Summary of the invention

[0003] The purpose of the present invention is to provide a pedestrian re-identification method based on cross-time domain data. The technical problem to be solved is how to reduce the pedestrian missed detection rate and thus improve the accuracy of pedestrian re-identification.

[0004] The present invention is achieved through the following technical solutions:

[0005] A pedestrian re-identification method based on cross-temporal data includes the following steps:

[0006] S100, obtaining background data of a monitoring area, and constructing a monitoring scene based on the background data; presetting a segmentation plane, segmenting the monitoring area by the segmentation plane, and obtaining an edge area and a center area; and mapping the segmentation plane to the monitoring scene;

[0007] Capturing the current surveillance video frame from the surveillance video to obtain a surveillance image;

[0008] Obtaining the monitoring position and monitoring angle of the monitoring image, and intercepting the image in the monitoring scene based on the monitoring position and monitoring angle to obtain a background image;

[0009] S200, using the background image, removing the background area in the monitoring image to obtain an image to be processed;

[0010] S300, mapping the segmentation surface of the monitoring scene to the image to be processed, and segmenting the image to be processed by the segmentation surface to obtain an edge image and a center image;

[0011] S400, capturing a human target from the edge image, and assigning a unique identity code to the human target; respectively selecting the captured human targets to obtain a plurality of human images;

[0012] Recognize the human body image, obtain and mark the recognition result of the human body target;

[0013] Associating the above-mentioned mark and identity code, and synchronizing the above-mentioned mark and identity code to the surveillance video;

[0014] S500, capturing facial features of a human target from the central image, and respectively selecting the captured facial features to obtain a plurality of facial images;

[0015] Acquire the mark of the above human target;

[0016] When the mark of the human target is not obtained, a unique identity code is assigned to the human target, and the facial image is recognized to obtain and mark the recognition result of the human target;

[0017] When the mark of the human target is obtained, the facial image is recognized according to the recognition result of the mark, and the recognition result of the human target is obtained and updated; the mark is synchronized to the monitoring video.

[0018] By constructing a monitoring scene and using the background image to remove the background area, pedestrian targets in the foreground can be captured more accurately, reducing the impact of background changes on detection, thereby reducing pedestrian omissions caused by background interference; the monitoring scene is divided into edge areas and central areas, and optimized according to the characteristics of different areas. The edge area contains more pedestrians entering and exiting the scene, while the central area focuses more on the facial features of pedestrians. This divide-and-conquer method ensures that both moving and stationary pedestrians can be effectively identified, improving the overall detection rate; during the recognition process, each captured human target is given a unique identity code, and the recognition results and identity codes are synchronized to the monitoring video, so that even if the pedestrian leaves the field of view for a short time and then reappears, the pedestrian's identity can be quickly matched and confirmed based on the identity code, avoiding repeated coding or misidentification, and greatly improving the accuracy of pedestrian re-identification; facial features are processed centrally in the central area, enhancing the ability to identify individual identities. Since missed detections may exist in the edge areas, detected human targets can obtain identity codes and labels in the central area, while missed human targets cannot obtain identity codes and labels in the central area. Therefore, when missed human targets are detected in the central area, they are assigned identity codes and recognition operations are performed, thereby improving the recognition accuracy while maintaining a low missed detection rate.

[0019] Furthermore, the specific steps of S200 include:

[0020] S210, performing pixel comparison between the monitoring image and the background image to determine the difference value of each pixel between the monitoring image and the background image;

[0021] S220, presetting a difference threshold, marking pixels whose difference values ​​are greater than the difference threshold as first foreground pixels, and marking pixels whose difference values ​​are less than or equal to the difference threshold as first background pixels;

[0022] S230, generating a binary first foreground mask image according to the first foreground pixel and the first background pixel;

[0023] S240 , performing a bitwise AND operation on the first foreground mask image and the monitoring image to obtain an image to be processed.

[0024] The above surveillance image is compared with the pre-built background image pixel by pixel, and the difference between the two is calculated to help distinguish static background from dynamic foreground, help locate moving objects such as pedestrians, and reduce misjudgment caused by environmental changes; set a difference threshold to determine whether a pixel belongs to the foreground or background; based on the division of foreground pixels and background pixels, a binary (black and white) foreground mask image is generated, where foreground pixels are usually represented as white (value 1) and background pixels are represented as black (value 0); the foreground mask image clearly distinguishes the foreground area that needs to be paid attention to and the background area that does not need to be considered, simplifying the complexity of subsequent processing. Use the foreground mask image as a "filter" and perform a bitwise AND operation with the original surveillance image to retain only the part considered to be the foreground, thereby obtaining the final image to be processed. This process effectively removes background interference, so that the image to be processed only contains the foreground objects of interest (such as pedestrians), greatly improving the efficiency and accuracy of subsequent feature extraction and recognition steps.

[0025] Furthermore, the specific steps of S300 include:

[0026] S310, determining a segmentation line of the background image by mapping to the segmentation plane spatial coordinates of the monitoring scene;

[0027] S320, mapping the segmentation line of the background image to the image to be processed, and segmenting the image to be processed by the segmentation line to obtain an edge image and a center image.

[0028] The above-mentioned image to be processed is segmented and different areas are processed separately. The edge image is the pedestrian entering and exiting the monitoring area, and more attention is paid to the pedestrian as a whole, while the central area focuses more on the facial features of the pedestrian. The divide-and-rule method ensures that both moving and stationary pedestrians can be effectively identified, thereby improving the overall detection rate.

[0029] Furthermore, the human body image is identified, and the identification result of the human body target is obtained and marked. The specific steps include:

[0030] S410, calling a human body matching image set, and matching the human body image with a human body matching image in the human body matching image set;

[0031] S411, if the human body image matches a human body matching image, the human body matching image is used as a recognition image; and the human body target corresponding to the human body image is marked as successfully recognized;

[0032] S412: If the human body image does not match a human body matching image, the human body target corresponding to the human body image is marked as recognition failure.

[0033] Call a pre-established human matching image set, which contains multiple human images in different postures, angles and wearing conditions, and associates the corresponding pedestrian information. Compare and match the captured human image with the human matching image, and use the known data set to identify the newly captured human image; when the human image can successfully match one or more images in the human matching image set, the matched image will be used as the recognition image, and the corresponding human target will be marked as successfully recognized. Successful recognition means that the system can confirm that the captured human image belongs to an individual already in the database; if the human image fails to find a match in the human matching image set, it will be marked as a recognition failure.

[0034] Furthermore, the specific steps of S500 include:

[0035] S510, capturing a human target from the central image, and detecting a mark of the human target;

[0036] S520: If the human target is marked as successfully recognized, output a recognition image;

[0037] S530: if the human target is marked as recognition failure, capturing facial features of the human target;

[0038] S531, if the facial features of the human target are captured, the captured facial features are framed to obtain a facial image;

[0039] Calling a facial matching image set, and matching the facial image with the facial matching images in the facial matching image set;

[0040] When the facial image matches the facial matching image, the facial matching image is used as the recognition image; the human target mark corresponding to the facial image is updated to a recognition success, and the recognition success mark is synchronized to the monitoring video;

[0041] When the facial image is not matched to a facial matching image, the similarity between the facial image and each facial matching image in the facial matching image set is determined, and the facial matching image with the greatest similarity is selected as the recognition image; the human target mark corresponding to the facial image is updated to a tentative mark, and the tentative mark is synchronized to the surveillance video;

[0042] S532: If the facial features of the human target are not captured, the next surveillance video frame is captured, the human target is extracted from the next surveillance video frame by identity coding, and the process returns to S530 to continue execution;

[0043] S540. If the mark of the human target is not detected, a unique identity code is assigned to the human target, the human target is marked as recognition failure, and S530 to S532 are executed; the recognition failure mark is synchronized to the monitoring video.

[0044] First, check whether the human target captured in the above central image already has a valid identification mark, which helps to quickly screen out known individuals, avoid repeated processing, and improve processing efficiency. If the human target has been successfully identified, the corresponding identification image is directly output to simplify the process, ensure that the confirmed identity information will not be re-evaluated, and reduce unnecessary computing resource consumption; for human targets that fail to be identified, try to capture their facial features, and introduce facial features as an auxiliary identification method, which can significantly improve the recognition success rate; select the captured facial features, generate a facial image, and call the facial matching image set for matching. If the match is successful, update the mark of the human target and synchronize it to the surveillance video; if the match is not successful, select the closest facial matching image based on similarity, and mark the human target as tentative for further verification or manual review. By using the facial matching image set, facial features are used for identification when the human body's appearance features are insufficient. At the same time, similarity matching provides a suboptimal solution in the absence of a complete match, which increases the flexibility of the system. If no valid facial features are captured, the next surveillance video frame will be captured to continue trying, ensuring that even if facial features are not visible in some frames (such as side view or back view), the system still has a chance to obtain a clear facial image in subsequent frames to complete recognition.

[0045] For newly appearing human targets for which no valid tags are detected (the human targets belong to pedestrians missed in the edge area), a unique identity code is assigned and its initial tag is set to recognition failure. The facial feature capture and matching process is then performed to ensure that every new pedestrian entering the field of view can be recorded and identified by the system, maintaining consistent tracking of all monitored objects.

[0046] Furthermore, S500 further includes the following steps:

[0047] S550, if the mark of the human target is tentative, then capture the facial features of the human target from the monitoring image according to the time sequence of the monitoring video frames; when the facial features of the human target are captured, stop capturing the facial features of the human target;

[0048] S551, frame the captured facial features to obtain a monitoring facial image; remove the background area in the monitoring facial image to obtain a provisional facial image;

[0049] S552, calling the above-mentioned face matching image set, and matching the above-mentioned tentative face image with the face matching image in the face matching image set;

[0050] S553, when the provisional facial image matches the facial matching image, the recognition image of the human target is updated to the matched facial matching image; the human target mark corresponding to the provisional facial image is updated to recognition success, and the recognition success mark is synchronized to the monitoring video;

[0051] S554: when the provisional facial image does not match the facial matching image, generating an input instruction;

[0052] S555. Collect the pedestrian information of the provisional facial image through the above input instruction, associate the provisional facial image with the pedestrian information; add the provisional facial image to the facial matching image set;

[0053] S556, using the above-mentioned provisional facial image as a recognition image; updating the human target mark corresponding to the above-mentioned provisional facial image to recognition success, and synchronizing the recognition success mark to the monitoring video.

[0054] For human targets marked as "tentative", the facial features of the human target will continue to be captured from the surveillance video frames in chronological order until the facial image is successfully captured; this ensures that the system will not give up on identifying the pedestrian due to a failed match, provides multiple recognition opportunities, and improves the success rate of the final recognition; once the facial features are captured, the system will select this area and generate a tentative facial image.

[0055] The generated provisional facial image is compared and matched with the images in the facial matching image set; the identity of the provisionally marked human target is quickly confirmed or excluded by using the existing facial matching image set; if the provisional facial image can match the facial matching image, the recognition image of the human target is updated and marked as successfully recognized, which solves the uncertainty that existed before, converts the "provisional" state into clear identity information, and enhances the reliability and accuracy of the system; when the provisional facial image fails to match any known facial matching image, the system generates an entry instruction; according to the entry instruction, the system collects the pedestrian information (such as identity information, social information, etc.) corresponding to the provisional facial image, and adds the provisional facial image to the facial matching image set, which not only expands the data volume of the facial matching image set, but also makes it easier to identify the pedestrian in the future, while maintaining the timeliness and accuracy of the data; the provisional facial image is used as the final recognition image, and its corresponding human target mark is updated to successfully recognized and synchronized to the surveillance video; even if a complete match is not found, the system can complete the recording of the new individual in this way, maintaining the consistency of pedestrian tracking.

[0056] Furthermore, the background area in the monitored facial image is removed to obtain a provisional facial image. The specific steps include:

[0057] S5511, obtaining the frame selection coordinate points of the monitoring facial image on the monitoring image, mapping the frame selection coordinate points to the background image, and frame selecting a part of the background according to the frame selection coordinate points mapped to the background image to obtain a background part image;

[0058] S5512, performing pixel comparison between the monitored facial image and the background partial image to determine the difference value of each pixel between the monitored facial image and the background partial image;

[0059] S5513, marking pixels whose difference values ​​are greater than the difference threshold as second foreground pixels, and marking pixels whose difference values ​​are less than or equal to the difference threshold as second background pixels;

[0060] S5514, generating a binary second foreground mask image according to the second foreground pixels and the second background pixels;

[0061] S5515. Perform a bitwise AND operation on the second foreground mask image and the monitored facial image to obtain a provisional facial image.

[0062] Furthermore, a two-point rectangular frame is used to select the above human targets and facial features.

[0063] The two-point rectangular frame only needs two coordinate points, which reduces the amount of calculation and improves the processing speed. Using a two-point rectangular frame to select human targets and facial features not only simplifies the image processing process, but also improves the system's operating efficiency and response speed.

[0064] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0065] By constructing a monitoring scene and using the background image to remove the background area, pedestrian targets in the foreground can be captured more accurately, reducing the impact of background changes on detection, thereby reducing pedestrian omissions caused by background interference; the monitoring scene is divided into edge areas and central areas, and optimized according to the characteristics of different areas. The edge area contains more pedestrians entering and exiting the scene, while the central area focuses more on the facial features of pedestrians. This divide-and-conquer method ensures that both moving and stationary pedestrians can be effectively identified, improving the overall detection rate; during the recognition process, each captured human target is given a unique identity code, and the recognition results and identity codes are synchronized to the monitoring video, so that even if the pedestrian leaves the field of view for a short time and then reappears, the pedestrian's identity can be quickly matched and confirmed based on the identity code, avoiding repeated coding or misidentification, and greatly improving the accuracy of pedestrian re-identification; facial features are processed centrally in the central area, enhancing the ability to identify individual identities. Since missed detections may exist in the edge areas, detected human targets can obtain identity codes and labels in the central area, while missed human targets cannot obtain identity codes and labels in the central area. Therefore, when missed human targets are detected in the central area, they are assigned identity codes and recognition operations are performed, thereby improving the recognition accuracy while maintaining a low missed detection rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and should not be regarded as limiting the scope. For those of ordinary skill in the art, other relevant drawings can be obtained based on these drawings without creative work. In the drawings:

[0067] Figure 1 As the main process diagram;

[0068] Figure 2 It is a simplified diagram after the monitoring scene is segmented by the segmentation surface;

[0069] Figure 3 The surveillance video frame is captured when the surveillance position is directly above and the surveillance angle is perpendicular to the ground;

[0070] Figure 4It is the background image when the monitoring position is directly above and the monitoring angle is perpendicular to the ground. DETAILED DESCRIPTION

[0071] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with embodiments and drawings. The exemplary embodiments of the present invention and their description are only used to explain the present invention and are not intended to limit the present invention.

[0072] First embodiment:

[0073] Combination Figures 1 to 4 , a pedestrian re-identification method based on cross-temporal data, comprising the following steps:

[0074] S100, obtaining background data of the monitoring area, and constructing a monitoring scene based on the background data; presetting a segmentation plane in the monitoring area, the segmentation line can be a logical boundary (such as a division between different areas), and segmenting the monitoring area through the segmentation plane to obtain an edge area and a central area; mapping the segmentation plane to the monitoring scene (such as Figure 2 ), Figure 2 The surface enclosed by the red line in the middle is the split surface;

[0075] Capturing the current surveillance video frame from the surveillance video to obtain a surveillance image;

[0076] Obtain the monitoring position and monitoring angle of the above monitoring image, intercept the image in the monitoring scene based on the monitoring position and monitoring angle, and obtain the background image; the background image can change according to the change of the monitoring position and monitoring angle, weaken the lighting difference between the background image and the monitoring image, thereby improving the recognition accuracy;

[0077] S200, using the background image, removing the background area in the monitoring image to obtain an image to be processed;

[0078] S300, mapping the segmentation surface of the monitoring scene to the image to be processed, and segmenting the image to be processed by the segmentation surface to obtain an edge image and a center image;

[0079] S400, capturing a human target from the edge image and assigning a unique identity code to the human target; using a two-point rectangular frame to respectively select the captured human target to obtain a plurality of human images; the two-point rectangular frame is a rectangular area determined by two diagonal points (upper left corner and lower right corner).

[0080] Recognize the human body image, obtain and mark the recognition result of the human body target;

[0081] Associating the above-mentioned mark and identity code, and synchronizing the above-mentioned mark and identity code to the surveillance video;

[0082] S500, capturing facial features of a human target from the central image, and using a two-point rectangular frame to respectively frame the captured facial features to obtain a plurality of facial images;

[0083] Acquire the mark of the above human target;

[0084] When the mark of the human target is not obtained, a unique identity code is assigned to the human target, and the facial image is recognized to obtain and mark the recognition result of the human target;

[0085] When the mark of the human target is obtained, the facial image is recognized according to the recognition result of the mark, and the recognition result of the human target is obtained and updated; the mark is synchronized to the monitoring video.

[0086] In addition to the situation where facial features of pedestrians who fail to be identified in the edge area are captured when passing through the central area, there are also situations where facial features are not captured at all; if facial features of pedestrians are not captured when passing through the central area, other features can be obtained from the surveillance image at the previous moment of the current surveillance video frame for identification, such as: hands, legs, head, torso, etc., and the corresponding matching image data set is called for matching, and the matching image with the greatest similarity is used as the recognition image, and the recognition result of the human target is updated; the above-mentioned marks are synchronized to the surveillance video.

[0087] By constructing a monitoring scene and using the background image to remove the background area, pedestrian targets in the foreground can be captured more accurately, reducing the impact of background changes on detection, thereby reducing pedestrian omissions caused by background interference; the monitoring scene is divided into edge areas and central areas, and optimized according to the characteristics of different areas. The edge area contains more pedestrians entering and exiting the scene, while the central area focuses more on the facial features of pedestrians. This divide-and-conquer method ensures that both moving and stationary pedestrians can be effectively identified, improving the overall detection rate; during the recognition process, each captured human target is given a unique identity code, and the recognition results and identity codes are synchronized to the monitoring video, so that even if the pedestrian leaves the field of view for a short time and then reappears, the pedestrian's identity can be quickly matched and confirmed based on the identity code, avoiding repeated coding or misidentification, and greatly improving the accuracy of pedestrian re-identification; facial features are processed centrally in the central area, enhancing the ability to identify individual identities. Since missed detections may exist in the edge areas, detected human targets can obtain identity codes and labels in the central area, while missed human targets cannot obtain identity codes and labels in the central area. Therefore, when missed human targets are detected in the central area, they are assigned identity codes and recognition operations are performed, thereby improving the recognition accuracy while maintaining a low missed detection rate.

[0088] In summary, the above scheme effectively solves the common missed detection problem in the pedestrian re-identification process through the fusion of multiple levels and multiple features, and the ability to dynamically adapt to environmental changes, while improving the accuracy of recognition. This method is particularly suitable for complex and ever-changing monitoring environments.

[0089] Second embodiment:

[0090] Based on the first embodiment, the specific steps of S200 include:

[0091] S210, performing pixel comparison between the monitoring image and the background image to determine the difference value of each pixel between the monitoring image and the background image;

[0092] S220, presetting a difference threshold, marking pixels whose difference values ​​are greater than the difference threshold as first foreground pixels, and marking pixels whose difference values ​​are less than or equal to the difference threshold as first background pixels;

[0093] S230, generating a binary first foreground mask image according to the first foreground pixel and the first background pixel;

[0094] S240 , performing a bitwise AND operation on the first foreground mask image and the monitoring image to obtain an image to be processed.

[0095] The above surveillance image is compared with the pre-built background image pixel by pixel, and the difference between the two is calculated to help distinguish static background from dynamic foreground, help locate moving objects such as pedestrians, and reduce misjudgment caused by environmental changes; set a difference threshold to determine whether a pixel belongs to the foreground or background; based on the division of foreground pixels and background pixels, a binary (black and white) foreground mask image is generated, where foreground pixels are usually represented as white (value 1) and background pixels are represented as black (value 0); the foreground mask image clearly distinguishes the foreground area that needs to be paid attention to and the background area that does not need to be considered, simplifying the complexity of subsequent processing. Use the foreground mask image as a "filter" and perform a bitwise AND operation with the original surveillance image to retain only the part considered to be the foreground, thereby obtaining the final image to be processed. This process effectively removes background interference, so that the image to be processed only contains the foreground objects of interest (such as pedestrians), greatly improving the efficiency and accuracy of subsequent feature extraction and recognition steps.

[0096] In summary, separating pedestrian targets in the foreground in a complex and changing environment reduces the interference of background and other irrelevant factors, which not only helps to reduce the pedestrian missed detection rate, but also provides higher quality data input for subsequent human and facial feature recognition by improving the accuracy of the preliminary processing, thereby improving the performance of the entire pedestrian re-identification.

[0097] Third embodiment:

[0098] Based on any of the above embodiments, the specific steps of S300 include:

[0099] S310, by mapping to the spatial coordinates of the segmentation plane of the monitoring scene, determine the segmentation line of the background image (such as Figure 4 ), Figure 4 The red line in the middle is the dividing line;

[0100] S320, mapping the segmentation line of the background image to the image to be processed (such as Figure 3 ), Figure 3 The red line in the middle represents the segmentation line, and the image to be processed is segmented by the segmentation line to obtain the edge image and the center image.

[0101] The above-mentioned image to be processed is segmented and different areas are processed separately. The edge image is the pedestrian entering and exiting the monitoring area, and more attention is paid to the pedestrian as a whole, while the central area focuses more on the facial features of the pedestrian. The divide-and-rule method ensures that both moving and stationary pedestrians can be effectively identified, thereby improving the overall detection rate.

[0102] Specifically, the segmentation surface in the monitoring scene is mapped to the background image in order to determine the segmentation line in the background image. The segmentation line can be a physical boundary (such as the edge of the road, the outline of the building, etc.) or a logical boundary (such as the division between different areas). The segmentation line can not only serve as a monitoring benchmark to facilitate the acquisition of the corresponding background image, so as to reduce the lighting difference between the background image and the monitoring image; it can also further help distinguish the background from the foreground, identify which parts belong to the fixed background, and which parts may be the foreground area where pedestrians or other moving objects are located, which helps to improve the accuracy and efficiency of subsequent image analysis. The determined segmentation lines are mapped to the image to be processed (i.e., the image after the background interference has been removed), and the image to be processed is segmented according to these segmentation lines to obtain edge images and center images. The edge images can be used for context understanding or anomaly detection, while the center images are mainly used for target recognition and tracking.

[0103] Fourth embodiment:

[0104] Based on any of the above embodiments, the human body image is recognized, and a recognition result of a human body target is obtained and marked. The specific steps include:

[0105] S410, calling a human body matching image set, and matching the human body image with a human body matching image in the human body matching image set; wherein the human body matching image has the background removed.

[0106] S411, if the human body image matches a human body matching image, the human body matching image is used as a recognition image; and the human body target corresponding to the human body image is marked as successfully recognized;

[0107] S412: If the human body image does not match a human body matching image, the human body target corresponding to the human body image is marked as recognition failure.

[0108] Call a pre-established human matching image set, which contains multiple human images in different postures, angles and wearing conditions, and associates the corresponding pedestrian information. Compare and match the captured human image with the human matching image, and use the known data set to identify the newly captured human image; when the human image can successfully match one or more images in the human matching image set, the matched image will be used as the recognition image, and the corresponding human target will be marked as successfully recognized. Successful recognition means that the system can confirm that the captured human image belongs to an individual already in the database; if the human image fails to find a match in the human matching image set, it will be marked as a recognition failure.

[0109] In the case of recognition failure, possible reasons include: the pedestrian appears for the first time or it is difficult to match due to lighting, angle, occlusion, etc. In this case, further measures can be taken, such as: assigning a new unique identity code to the pedestrian who appears for the first time; trying to use other features (such as facial features) for auxiliary recognition in the case of difficult matching; adding the unmatched human images to the intermediate database, and after the subsequent auxiliary recognition is successful, adding the human images in the intermediate database to the human matching image set and associating the corresponding human matching images to enhance the possibility of future recognition.

[0110] Fifth embodiment:

[0111] Based on any of the above embodiments, the specific steps of S500 include:

[0112] S510, capturing a human target from the central image, and detecting a mark of the human target;

[0113] S520: If the human target is marked as successfully recognized, output a recognition image;

[0114] S530: if the human target is marked as recognition failure, capturing facial features of the human target;

[0115] S531, if the facial features of the human target are captured, the captured facial features are framed to obtain a facial image;

[0116] The face matching image set is called to match the above face image with the face matching image in the face matching image set; wherein the face matching image has the background removed.

[0117] When the facial image matches the facial matching image, the facial matching image is used as the recognition image; the human target mark corresponding to the facial image is updated to a recognition success, and the recognition success mark is synchronized to the monitoring video;

[0118] When the facial image is not matched to a facial matching image, the similarity between the facial image and each facial matching image in the facial matching image set is determined. The similarity calculation may be performed using the following formula:

[0119] ,

[0120] in, Indicates similarity; Indicates the height of the image; Indicates the width of the image; Represents a facial image; represents a face matching image; Represents the coordinate points on the facial image The pixel value of Represents the coordinate points on the face matching image The pixel value of .

[0121] Select the facial matching image with the greatest similarity as the recognition image; update the human target mark corresponding to the facial image to a tentative mark, and synchronize the tentative mark to the surveillance video;

[0122] S532: If the facial features of the human target are not captured, the next surveillance video frame is captured, the human target is extracted from the next surveillance video frame by identity coding, and the process returns to S530 to continue execution;

[0123] S540. If the mark of the human target is not detected, a unique identity code is assigned to the human target, the human target is marked as recognition failure, and S530 to S532 are executed; the recognition failure mark is synchronized to the monitoring video.

[0124] First, check whether the human target captured in the above central image already has a valid identification mark, which helps to quickly screen out known individuals, avoid repeated processing, and improve processing efficiency. If the human target has been successfully identified, the corresponding identification image is directly output to simplify the process, ensure that the confirmed identity information will not be re-evaluated, and reduce unnecessary computing resource consumption; for human targets that fail to be identified, try to capture their facial features, and introduce facial features as an auxiliary identification method, which can significantly improve the recognition success rate; select the captured facial features, generate a facial image, and call the facial matching image set for matching. If the match is successful, update the mark of the human target and synchronize it to the surveillance video; if the match is not successful, select the closest facial matching image based on similarity, and mark the human target as tentative for further verification or manual review. By using the facial matching image set, facial features are used for identification when the human body's appearance features are insufficient. At the same time, similarity matching provides a suboptimal solution in the absence of a complete match, which increases the flexibility of the system. If no valid facial features are captured, the next surveillance video frame will be captured to continue trying, ensuring that even if facial features are not visible in some frames (such as side view or back view), the system still has a chance to obtain a clear facial image in subsequent frames to complete recognition.

[0125] For newly appearing human targets for which no valid tags are detected (the human targets belong to pedestrians missed in the edge area), a unique identity code is assigned and its initial tag is set to recognition failure. The facial feature capture and matching process is then performed to ensure that every new pedestrian entering the field of view can be recorded and identified by the system, maintaining consistent tracking of all monitored objects.

[0126] In a specific embodiment, S500 further includes the following steps:

[0127] S550, if the mark of the human target is tentative, then capture the facial features of the human target from the monitoring image according to the time sequence of the monitoring video frames; when the facial features of the human target are captured, stop capturing the facial features of the human target;

[0128] S551, frame the captured facial features to obtain a monitoring facial image; remove the background area in the monitoring facial image to obtain a provisional facial image;

[0129] S552, calling the above-mentioned face matching image set, and matching the above-mentioned tentative face image with the face matching image in the face matching image set;

[0130] S553, when the provisional facial image matches the facial matching image, the recognition image of the human target is updated to the matched facial matching image; the human target mark corresponding to the provisional facial image is updated to recognition success, and the recognition success mark is synchronized to the monitoring video;

[0131] S554: when the provisional facial image does not match the facial matching image, generating an input instruction;

[0132] S555. Collect the pedestrian information of the provisional facial image through the above input instruction, associate the provisional facial image with the pedestrian information; add the provisional facial image to the facial matching image set;

[0133] S556, using the above-mentioned provisional facial image as a recognition image; updating the human target mark corresponding to the above-mentioned provisional facial image to recognition success, and synchronizing the recognition success mark to the monitoring video.

[0134] For human targets marked as "tentative", a subsequent verification and update mechanism is provided in the event that the identity of the pedestrian cannot be fully confirmed at the first attempt. For human targets marked as "tentative", the facial features of the human target are continuously captured from the surveillance video frames in chronological order until the facial image is successfully captured; this ensures that the system will not give up on identifying the pedestrian due to a failed match, provides multiple recognition opportunities, and improves the success rate of the final recognition; once the facial features are captured, the system will select this area to obtain the surveillance facial image, and obtain the tentative facial image after removing the background area, reducing background interference and improving the accuracy of subsequent matching.

[0135] The above-mentioned provisional facial image is compared and matched with the images in the facial matching image set; the identity of the tentatively marked human target is quickly confirmed or excluded by using the existing facial matching image set; if the provisional facial image can match the facial matching image, the recognition image of the human target is updated and marked as successfully recognized, which solves the uncertainty that existed before, converts the "tentative" state into clear identity information, and enhances the reliability and accuracy of the system; when the provisional facial image fails to match any known facial matching image, the system generates an entry instruction (the entry instruction solves the non-matching situation and adds the new pedestrian information to the database); according to the entry instruction, the system collects the pedestrian information (such as identity information, social information, etc.) corresponding to the provisional facial image, and adds the provisional facial image to the facial matching image set, which not only expands the data volume of the facial matching image set, but also makes it easier to recognize the pedestrian in the future, while maintaining the timeliness and accuracy of the data; the provisional facial image is used as the final recognition image, and its corresponding human target mark is updated to be successfully recognized and synchronized to the surveillance video; even if a complete match is not found, the system can complete the recording of the new individual in this way, maintaining the consistency of pedestrian tracking.

[0136] In a specific embodiment, the background area in the above-mentioned monitored facial image is removed to obtain a provisional facial image, and the specific steps include:

[0137] S5511, obtaining the frame selection coordinate points of the monitoring facial image on the monitoring image, mapping the frame selection coordinate points to the background image, frame-selecting the background part according to the frame selection coordinate points mapped to the background image, and obtaining the background part image; ensuring that the background part image and the monitoring facial image are of the same size;

[0138] S5512, performing pixel comparison between the monitored facial image and the background partial image to determine the difference value of each pixel between the monitored facial image and the background partial image;

[0139] S5513, marking pixels whose difference values ​​are greater than the difference threshold as second foreground pixels, and marking pixels whose difference values ​​are less than or equal to the difference threshold as second background pixels;

[0140] S5514, generating a binary second foreground mask image according to the second foreground pixels and the second background pixels;

[0141] S5515. Perform a bitwise AND operation on the second foreground mask image and the monitored facial image to obtain a provisional facial image.

[0142] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A pedestrian re-identification method based on cross-temporal data, characterized in that: The following steps are involved: S100, obtaining background data of a monitoring area, and constructing a monitoring scene based on the background data; A pre-set segmentation plane is used to segment the monitoring area to obtain an edge area and a center area; Mapping the segmentation surface to a monitoring scene; Capturing the current surveillance video frame from the surveillance video to obtain a surveillance image; Acquire the monitoring position and monitoring angle of the monitoring image, and intercept the image in the monitoring scene based on the monitoring position and monitoring angle to obtain a background image; S200, removing the background area in the monitoring image through the background image to obtain an image to be processed; S300, mapping the segmentation surface of the monitoring scene to the image to be processed, and segmenting the image to be processed by the segmentation surface to obtain an edge image and a center image; S400, capturing a human target from the edge image, and assigning a unique identity code to the human target; Select the captured human targets respectively to obtain a number of human body images; Recognize the human body image, obtain and mark the recognition result of the human body target; Associating the mark and the identity code, and synchronizing the mark and the identity code to the surveillance video; S500, capturing facial features of a human target from the central image, and respectively selecting the captured facial features to obtain a plurality of facial images; Acquire a mark of the human target; When the mark of the human target is not obtained, a unique identity code is assigned to the human target, and the facial image is recognized to obtain and mark the recognition result of the human target; When the mark of the human target is obtained, the facial image is recognized according to the recognition result of the mark, and the recognition result of the human target is obtained and updated; The marker is synchronized to the surveillance video.

2. The pedestrian re-identification method based on cross-temporal data according to claim 1, characterized in that: The specific steps of S200 include: S210, performing pixel comparison between the monitoring image and the background image to determine the difference value of each pixel between the monitoring image and the background image; S220, presetting a difference threshold, marking pixels whose difference values ​​are greater than the difference threshold as first foreground pixels, and marking pixels whose difference values ​​are less than or equal to the difference threshold as first background pixels; S230, generating a binary first foreground mask image according to the first foreground pixels and the first background pixels; S240: Perform a bitwise AND operation on the first foreground mask image and the monitoring image to obtain an image to be processed.

3. The pedestrian re-identification method based on cross-temporal data according to claim 1, characterized in that: The specific steps of S300 include: S310, determining a segmentation line of the background image by mapping the segmentation plane spatial coordinates to the monitoring scene; S320: Map the segmentation line of the background image to the image to be processed, and segment the image to be processed by the segmentation line to obtain an edge image and a center image.

4. The pedestrian re-identification method based on cross-temporal data according to claim 1, characterized in that: Recognize the human body image, obtain and mark the recognition result of the human body target, and the specific steps include: S410, calling a human body matching image set, and matching the human body image with a human body matching image in the human body matching image set; S411, if the human body image matches a human body matching image, use the human body matching image as a recognition image; and mark the human body target corresponding to the human body image as successfully recognized; S412: If the human body image does not match a human body matching image, the human body target corresponding to the human body image is marked as recognition failure.

5. The pedestrian re-identification method based on cross-temporal data according to claim 1, characterized in that: The specific steps of S500 include: S510, capturing a human target from the central image, and detecting a mark of the human target; S520: If the human target is marked as successfully recognized, output a recognition image; S530: If the human target is marked as recognition failure, capturing facial features of the human target; S531, if the facial features of the human target are captured, the captured facial features are framed to obtain a facial image; Calling a facial matching image set, and matching the facial image with a facial matching image in the facial matching image set; When the facial image matches the facial matching image, the facial matching image is used as the recognition image; the human target mark corresponding to the facial image is updated to a recognition success, and the recognition success mark is synchronized to the monitoring video; When the facial image does not match a facial matching image, determine the similarity between the facial image and each facial matching image in the facial matching image set, and select the facial matching image with the greatest similarity as the recognition image; update the human target mark corresponding to the facial image to provisional, and synchronize the provisional mark to the monitoring video; S532: If the facial features of the human target are not captured, the next surveillance video frame is intercepted, the human target is extracted from the next surveillance video frame by identity coding, and the process returns to S530 to continue execution; S540. If the mark of the human target is not detected, a unique identity code is assigned to the human target, the human target is marked as recognition failure, and S530 to S532 are executed; the recognition failure mark is synchronized to the monitoring video.

6. The pedestrian re-identification method based on cross-temporal data according to claim 5, characterized in that: S500 also includes the following steps: S550, if the mark of the human target is tentative, then capture the facial features of the human target from the monitoring image according to the time sequence of the monitoring video frames; when the facial features of the human target are captured, stop capturing the facial features of the human target; S551, selecting the captured facial features to obtain a monitoring facial image; removing the background area in the monitoring facial image to obtain a provisional facial image; S552, calling the face matching image set, and matching the tentative face image with the face matching image in the face matching image set; S553, when the provisional facial image matches the facial matching image, the recognition image of the human target is updated to the matched facial matching image; the human target mark corresponding to the provisional facial image is updated to recognition success, and the recognition success mark is synchronized to the monitoring video; S554, when the provisional facial image does not match the facial matching image, generating an input instruction; S555, collecting pedestrian information of the provisional facial image through the input instruction, associating the provisional facial image with the pedestrian information; adding the provisional facial image to the facial matching image set; S556, using the provisional facial image as a recognition image; updating the human target mark corresponding to the provisional facial image to a recognition success, and synchronizing the recognition success mark to the monitoring video.

7. The pedestrian re-identification method based on cross-temporal data according to claim 6, characterized in that: The background area in the monitored facial image is removed to obtain a provisional facial image, and the specific steps include: S5511, obtaining frame selection coordinate points of the monitoring facial image on the monitoring image, mapping the frame selection coordinate points to the background image, frame selecting a part of the background according to the frame selection coordinate points mapped to the background image, and obtaining a background part image; S5512, performing pixel comparison between the monitored facial image and the background partial image to determine the difference value of each pixel between the monitored facial image and the background partial image; S5513, marking pixels whose difference values ​​are greater than the difference threshold as second foreground pixels, and marking pixels whose difference values ​​are less than or equal to the difference threshold as second background pixels; S5514, generating a binary second foreground mask image according to the second foreground pixels and the second background pixels; S5515. Perform a bitwise AND operation on the second foreground mask image and the monitored facial image to obtain a provisional facial image.

8. The pedestrian re-identification method based on cross-temporal data according to claim 7, characterized in that: A two-point rectangular frame is used to select the human target and facial features.

Images