Human body photo dynamic warehousing method, system and computer readable medium for pedestrian re-identification

Abstract

The application provides a human body photo dynamic storage method and system for pedestrian re-identification, which comprises: when a certain pedestrian S enters the collection area of a first camera P1 deployed in a monitoring scene, pedestrian identification and human body ID tracking are performed, and human body photos are continuously captured to form a human body photo sequence of the pedestrian S in the collection area of the first camera P1; the human body photo sequence and the human body coordinate values corresponding to the photos are summarized and uploaded; the server platform determines the moving track of the pedestrian S according to the human body coordinate values corresponding to the human body photos, calculates the human body moving speed and direction, and obtains the nearest two camera points and the nearest expected arrival time point of the pedestrian S according to the nearest two camera points; the human body number in the human body database is updated or the human body database is maintained unchanged by combining the comparison of the characteristic values and whether the time error of the human body passing through the adjacent two cameras is within the allowed range, so as to reduce the size of the human body dynamic database and improve the efficiency and accuracy of cross-camera identification.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of cross-border identification, in particular to a human photo dynamic storage method for pedestrian re-identification (Reid). BACKGROUND

[0002] Cross-camera identification, also known as shape cross-camera identification technology, refers to deploying multiple cameras in a certain area, capturing the human body graph passing through the camera and storing it into the human body library, and completing the data update of the human body dynamic library. When used, based on the Reid technology, the human photo data meeting the matching degree is retrieved from the human body dynamic library through the uploaded single human photo and the specified matching degree, realizing the function of finding the target person through a photo. The essence is to use computer vision technology to search whether a specific person exists in an image or video sequence, which can be used in the fields of intelligent video monitoring and intelligent security. For example, in the public security scene, the corresponding person and the action track of the suspect in the monitoring video are quickly found through the human photo of the suspect, the identity of the suspect is identified, and the case is facilitated.

[0003] The traditional Reid identification method mainly matches the human body through the characteristic value attribute of the human body. Due to the different heights and angles of the camera installation, the same person may be identified as different personnel for storage, resulting in an abnormally large human body base library, thereby affecting the identification accuracy and making it difficult to quickly and accurately find the target object. SUMMARY

[0004] The present application aims to provide a human photo dynamic storage method for pedestrian re-identification, which excludes invalid human photos, reduces the size of the human body dynamic library, and improves the efficiency and accuracy of cross-camera identification.

[0005] According to the first aspect of the object of the present application, a human photo dynamic storage method for pedestrian re-identification is provided, characterized in that it comprises:

[0006] When a certain pedestrian S enters the collection area of the first camera P1 deployed in the monitoring scene, the first camera P1 performs pedestrian identification and tracks the human body ID based on the collected video stream, and continuously captures human photos according to a preset period until the pedestrian S disappears in the collection area of the first camera P1, thereby forming a human photo sequence of the pedestrian S in the collection area of the first camera P1, defined as [p1, p2, p3,..., pn], where pn represents the nth captured human photo;

[0007] The first camera P1 uploads the human photo sequence and the human body coordinate value corresponding to each human photo to the server platform;

[0008] The server platform determines the moving track of the pedestrian S according to the human body coordinate value corresponding to each human body photo;

[0009] The human body moving speed and direction are calculated based on the moving track;

[0010] The nearest two camera points and the expected arrival time points of the pedestrian S to the nearest two camera points are obtained based on the human body moving speed and direction of the pedestrian S.

[0011] The server platform extracts the human body feature value data from the human body photo sequence, and compares and judges the feature value in the human body base library, combines the matching degree and whether the time error of the human body passing through the adjacent two cameras is within the allowed range, and updates the number of human bodies in the human body base library or maintains the human body base library unchanged.

[0012] According to the second aspect of the object of the application, a human photo dynamic warehousing system for pedestrian re-identification is provided, comprising:

[0013] One or more processors;

[0014] A memory stores instructions that can be operated, and the instructions make the one or more processors execute operations when executed by the one or more processors, and the operations include the flow of the human photo dynamic warehousing method for pedestrian re-identification as described above.

[0015] According to the third aspect of the object of the application, a computer readable medium storing software is provided, and the software includes instructions that can be executed by one or more computers, and the instructions make the one or more computers execute operations when executed, and the operations include the flow of the human photo dynamic warehousing method for pedestrian re-identification as described above.

[0016] The human photo dynamic warehousing method for pedestrian re-identification provided by the application combines the advantages of Reid identification technology, and on this basis, the position of the human body, the moving direction of the human body, and the expected time to reach the target node are comprehensively analyzed and judged, the human body library is dynamically updated, invalid human photos are excluded, the size of the base library is greatly reduced, and the accuracy of cross-camera identification is improved.

[0017] It should be understood that all combinations of the aforementioned concepts and additional concepts described in greater detail below can be seen as part of the subject matter of the present disclosure as long as such concepts are not mutually inconsistent. In addition, all combinations of the claimed subject matter are considered as part of the subject matter of the present disclosure.

[0018] The foregoing and other aspects, embodiments and features of the present teachings can be better understood from the following detailed description taken in conjunction with the drawings. Other aspects, embodiments and features of the present teachings will be apparent from the detailed description and the drawings. BRIEF DESCRIPTION OF DRAWINGS

[0019] The accompanying drawings are not intended to be drawn to scale. In the drawings, each identical, or nearly identical, component that is illustrated in various figures is represented with a like numeral. For purposes of clarity, not every component is called out in every drawing. Embodiments of various aspects of the present teachings will now be described, by way of example only, with reference to the drawings in which:

[0020] Figure 1 is a flowchart of a human photo dynamic entry method for pedestrian re-identification according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0021] In order to more clearly understand the technical content of the present disclosure, specific embodiments are shown and described below with reference to the accompanying drawings.

[0022] Aspects of the present teachings are described in the disclosure by reference to the drawings, in which a number of illustrative embodiments are shown. The embodiments of the disclosure need not necessarily include all aspects of the present teachings. It should be understood that various concepts and embodiments introduced above and those described below can be implemented in any of numerous ways, as the disclosed concepts and embodiments are not limited to any particular manner of implementation. Additionally, some aspects of the present teachings can be utilized independently, or in any suitable combination with other aspects of the present teachings.

[0023] In conjunction with Figure 1 The flow of the human photo dynamic entry method for pedestrian re-identification according to the exemplary embodiment shown includes the following processes:

[0024] When a certain pedestrian S enters the collection area of the first camera P1 deployed in the monitoring scene, the first camera P1 performs pedestrian recognition and tracks the human ID based on the collected video stream, and continuously captures human photos according to a preset period until the pedestrian S disappears in the collection area of the first camera P1, thereby forming a human photo sequence of the pedestrian S in the collection area of the first camera P1, defined as [p1, p2, p3,..., pn], where pn represents the nth captured human photo;

[0025] The first camera P1 uploads the human photo sequence and the human coordinate value corresponding to each human photo to the server platform;

[0026] The server platform determines the movement trajectory of pedestrian S based on the human coordinates corresponding to each human photograph;

[0027] Calculate human movement speed and direction based on movement trajectory;

[0028] Based on the human movement speed and direction of pedestrian S, obtain the two nearest camera locations and the estimated arrival time of pedestrian S to the two nearest camera locations;

[0029] The server platform extracts human feature data from the sequence of human photos and compares it with the feature values ​​in the human database. Based on the matching degree and whether the time error of the human body passing through two adjacent cameras is within the allowable range, the platform updates the number of human bodies in the human database or keeps the human database unchanged.

[0030] Specifically, when continuously capturing human body photos, a human body photo is captured every predetermined n frames, where n is greater than or equal to 2. In an embodiment of the present invention, it is configured to capture a human body photo every 3 frames.

[0031] In a preferred embodiment, the server platform extracts human feature data by taking the highest quality human image from the human image sequence as a benchmark.

[0032] Among them, the quality assessment of human body photos can be achieved based on existing commercial human body photo quality assessment methods, such as human body quality assessment based on key features such as human body-photo ratio, head and shoulder ratio, left and right symmetry, blur, and brightness.

[0033] As an optional embodiment, the acquisition of the human body coordinate values ​​corresponding to each human body photograph includes:

[0034] For the i-th human body photo in the human body photo sequence, i = 1, 2, 3, ..., n, a bounding box is marked based on the pedestrian detection model;

[0035] Obtain the coordinates of the center point of the rectangle in the i-th human body photo, and use them as the human body coordinates of the i-th human body photo.

[0036] The pedestrian detection model uses a pedestrian target detection model based on YOLOv3.

[0037] As an optional embodiment, the aforementioned server platform determines the movement trajectory of pedestrian S based on the human coordinate values ​​corresponding to each human photograph, including:

[0038] For any two adjacent human images, the movement trajectory is determined by connecting their human coordinates, and each movement trajectory is connected end to end in chronological order to obtain the movement trajectory of pedestrian S within the acquisition area of ​​the first camera P1.

[0039] As an optional embodiment, the aforementioned calculating the human body moving speed and direction based on the moving trajectory comprises:

[0040] For each moving trajectory, taking the XY coordinate system of the photo as the reference, the distance d between the ith human body photo and the i+1th human body photo is calculated as:

[0041]

[0042] wherein, (x i ,y i ) represents the human body coordinate value of the ith human body photo, (x i+1 ,y i+1 ) represents the human body coordinate value of the i+1th human body photo;

[0043] A right triangle with the diagonal of the point (x i ,y i ) and the point (x i+1 ,y i+1 ) is constructed, and the internal angle is calculated to determine the moving direction of the pedestrian in the photo XY coordinate system.

[0044] As an optional embodiment, the aforementioned obtaining the nearest two camera positions and the estimated arrival time points of the pedestrian S to the nearest two camera positions based on the human body moving speed and direction of the pedestrian S comprises:

[0045] The position coordinates of the multiple cameras deployed in the monitoring scene are obtained, and the distance of the line connecting any two cameras is calculated;

[0046] According to the last moving trajectory in the moving trajectory of the pedestrian S, it is determined which line is first contacted, so as to determine the two cameras corresponding to the line;

[0047] According to the human body coordinate value of the last human body photo in the human body photo sequence of the pedestrian S and the midpoint of the first contacted line, the distance of the pedestrian S from the first camera P1 to the collection area of the nearest two cameras is determined, and the estimated arrival time point is estimated based on the human body moving speed of the pedestrian S.

[0048] As an optional embodiment, the aforementioned updating the number of human bodies in the human body database or maintaining the human body database unchanged in combination with the matching degree and whether the time error of the human body passing through the adjacent two cameras is within the allowable range comprises:

[0049] The extracted human body feature value data of the pedestrian S1 is compared with the feature values in the human body database, and the matching degree threshold is determined:

[0050] When the matching degree of the feature value of the person in the human body library exceeds the first threshold f1, it is considered to be the same person, the appearance frequency of the person is incremented by 1, and the number of the human body library remains unchanged.

[0051] When the matching degree is between the second threshold f2 and the first threshold f1, it is determined whether the predicted arrival time point is within the time error allowable range. If it is within the time error allowable range, it is considered to be the same person, the appearance frequency of the person is incremented by 1, and the number of the human body library remains unchanged. Otherwise, it is considered to be a different person, the number of the human body library is incremented by 1, and the pedestrian S is added to the human body library. The second threshold f2 is less than the first threshold f1.

[0052] When the matching degree is less than the second threshold f2, it is considered to be a different person, the number of the human body library is incremented by 1, and the pedestrian S is added to the human body library.

[0053] For example, the first threshold f1 is configured to be 85%, and the second threshold f2 is configured to be 70%. Then:

[0054] When the matching degree of the feature value of the person in the human body library exceeds 85%, it is considered to be the same person, the appearance frequency of the person is incremented by 1, and the number of the human body library remains unchanged.

[0055] When the matching degree is between 70% and 85%, it is determined whether the predicted arrival time point is within the time error allowable range. If it is within the time error allowable range, it is considered to be the same person, the appearance frequency of the person is incremented by 1, and the number of the human body library remains unchanged. Otherwise, it is considered to be a different person, the number of the human body library is incremented by 1, and the pedestrian S is added to the human body library.

[0056] When the matching degree is less than 70%, it is considered to be a different person, the number of the human body library is incremented by 1, and the pedestrian S is added to the human body library.

[0057] It should be understood that the extraction of the feature value of the human body and the processing of the comparison and identification in the foregoing embodiments of the present application can be realized based on the existing Reid algorithm, for example, a Reid identification model based on a deep learning network, in particular, a Reid identification model based on a CNN.

[0058] According to an embodiment of the present application, a human body dynamic warehousing system for pedestrian re-identification is also provided, which comprises one or more processors and a memory storing executable instructions. When the instructions are executed by the one or more processors, the one or more processors perform operations, and the operations comprise the flow of the human body dynamic warehousing method for pedestrian re-identification in the foregoing embodiments.

[0059] According to an embodiment of the present application, a computer readable medium storing software is also provided, the software comprising instructions executable by one or more computers, the instructions, when executed, causing the one or more computers to perform operations comprising the flow of the human body photo dynamic storage method for pedestrian re-identification of the aforementioned embodiment.

[0060] The present application has been disclosed with reference to the preferred embodiments thereof. It is understood, however, that variations and modifications can be effected within the spirit and scope of the application to the full extent of the prior art concept; therefore, other variations being intended to be within the scope of the present application.

Claims

1. A human body photo dynamic warehousing method for pedestrian re-identification, characterized in that, The application relates to a method for tracking a pedestrian in a monitoring scene. When a pedestrian S enters the collection area of a first camera P1 arranged in a monitoring scene, the first camera P1 performs pedestrian recognition and tracks the human body ID based on a collected video stream, continuously captures human body photos according to a preset period, and forms a human body photo sequence of the pedestrian S in the collection area of the first camera P1, defined as [p1, p2, p3,..., pn], wherein pn represents the nth captured human body photo. The first camera P1 uploads the human body photo sequence and the human body coordinate value corresponding to each human body photo to a server platform. The server platform determines the moving track of the pedestrian S according to the human body coordinate value corresponding to each human body photo. The moving speed and direction of the human body are calculated based on the moving track. The nearest two camera points and the estimated arrival time points of the pedestrian S to the nearest two camera points are obtained based on the moving speed and direction of the human body. The server platform extracts the human body feature value data from the human body photo sequence, compares the human body feature value data with the feature values in a human body database, judges the matching degree, and determines whether the time error of the human body passing through adjacent two cameras is within an allowable range, so as to update the number of human bodies in the human body database or maintain the human body database unchanged.

2. The human photo dynamic warehousing method for pedestrian re-identification according to claim 1, characterized in that, The human body coordinate value corresponding to each human body photo is obtained by: For the first i human photo sequence, i=1,2,3,...,n , a rectangular frame is labeled based on a pedestrian detection model; The center point of the rectangular frame is acquired in the first i The coordinates of the person in the photo are acquired as the first i The body coordinate values of the person in the photo are acquired.

3. The human photo dynamic warehousing method for pedestrian re-identification according to claim 1, characterized in that, The server platform determines the moving track of the pedestrian S according to the human body coordinate value corresponding to each human body photo, which includes: For any two adjacent human body photos in the sequence, the moving track of each segment is determined according to the line connecting the human body coordinate values, and each segment of the moving track is sequentially connected in time sequence, so as to obtain the moving track of the pedestrian S in the collection area of the first camera P1. 4.The human body photo dynamic warehousing method for pedestrian re-identification according to claim 1, wherein, The moving speed and direction of the human body are calculated based on the moving track, which includes: For each segment of the moving track, the distance between the first photo and the second photo is calculated as follows: i The distance between the first photo and the second photo is calculated as follows: i+1 The distance between the first photo and the second photo is calculated as follows: d The distance between the first photo and the second photo is calculated as follows: ； wherein, x i , y i denotes i the body coordinate values of a person in a photograph, wherein, x i+1 , y i+1 denotes i+1 the body coordinate values of a person in a photograph. Construct a right triangle with point ( x i , y i )- point ( x i+1 , y i+1 ) as the hypotenuse, and calculate the internal angle to determine the direction of movement of the pedestrian in the photo XY coordinate system.

5. The human photo dynamic warehousing method for pedestrian re-identification according to claim 1, characterized in that, The nearest two camera points and the estimated arrival time points of the pedestrian S to the nearest two camera points are obtained based on the moving speed and direction of the human body, which includes: The position coordinates of the multiple cameras arranged in the monitoring scene are obtained, any two cameras are connected, and the line distance between any two cameras is calculated. According to the last segment of the moving track of the pedestrian S, it is judged which line is first contacted, so as to determine the two cameras corresponding to the line. The distance of the pedestrian S from the first camera P1 to the collection area of the nearest two cameras is determined according to the midpoint of the first contacted line and the human body coordinate value of the last human body photo in the human body photo sequence of the pedestrian S, and the estimated arrival time point is estimated based on the moving speed of the human body.

6. The human photo dynamic warehousing method for pedestrian re-identification according to claim 1, characterized in that, The matching degree and whether the time error of the human body passing through adjacent two cameras is within an allowable range are combined to update the number of human bodies in the human body database or maintain the human body database unchanged, which includes: The extracted human body feature value data of the pedestrian S1 is compared with the feature values in the human body database, and the matching degree threshold is judged. When the matching degree of the feature value of a certain person in the human body library exceeds the first threshold f1, it is considered to be the same person, the human body appearance frequency +1, and the human body library quantity remains unchanged; When the matching degree is between the second threshold f2 and the first threshold f1, it is determined whether the predicted arrival time point is within the time error allowable range. If it is within the time error allowable range, it is considered to be the same person, the human body appearance frequency +1, and the human body library quantity remains unchanged. Otherwise, it is considered to be a different human body, the human body library quantity +1, and the pedestrian S is updated and added to the human body library. The second threshold f2 is less than the first threshold f1. When the matching degree is lower than the second threshold f2, it is considered to be a different human body, the human body library quantity +1, and the pedestrian S is updated and added to the human body library.

7. The human photo dynamic warehousing method for pedestrian re-identification according to claim 1, characterized in that, The server platform takes the human body photo with the highest quality as the reference according to the quality of the human body photos in the human body photo sequence, and extracts the human body feature value data. 8.The human body photo dynamic warehousing method for pedestrian re-identification according to claim 1, characterized in that, When continuously capturing human body photos, a human body photo is captured every n frames, and n is greater than or equal to 2.

9. A human photo dynamic warehousing system for pedestrian re-identification, characterized in that, It comprises: One or more processors; A memory storing instructions operable to cause the one or more processors to perform operations when executed by the one or more processors, the operations comprising the flow of the human body photo dynamic library method for pedestrian re-identification according to any one of claims 1-8.

10. A computer readable medium storing software, characterized in that, The software comprises instructions executable by one or more computers, the instructions when executed cause the one or more computers to perform operations, the operations comprising the flow of the human body photo dynamic library method for pedestrian re-identification according to any one of claims 1-8.

Images