Pedestrian recognition method and device

Abstract

The present invention provides a method and device for pedestrian recognition. The method comprises: after determining that the tracking identifier trackID of a target person is lost during the tracking process of a frame image in a target video, obtaining a predicted trajectory of the target person based on a target candidate sequence and a historical position sequence; and determining whether the target person is occluded or has left a boundary; performing different processing on the target candidate sequence based on different states of the target person; and identifying the target person using a REID algorithm and face recognition based on the target candidate sequence during the period when the trackID is lost. The present invention processes the target candidate sequence by predicting the pedestrian's motion trajectory during the period when the pedestrian tracking identifier trackID is lost in the video frame image, thereby improving the stability and continuity of target tracking in the occluded state. In the state when the pedestrian has left a boundary, video splicing technology is used to verify the correctness of the pedestrian's motion trajectory, thereby improving the accuracy of pedestrian recognition.

Description

Technical Field

[0001] The present invention relates to the field of Internet technology application technology, and in particular to a method and device for pedestrian recognition. Background Art

[0002] With the development of modern cities, surveillance camera coverage has increased year by year. Image and video applications have become increasingly prominent in people's daily lives, greatly facilitating the search for missing persons and the tracking of suspicious individuals. However, relying on manual labor to locate a target person from surveillance video is a huge workload and prone to missed detections.

[0003] Currently, facial recognition is the primary method used in surveillance and security to locate and track people. However, surveillance video images are limited by camera resolution and shooting angles, making it difficult to obtain high-quality facial images. Person re-identification (REID) technology has become increasingly relevant to our daily lives in recent years. Person re-identification (REID) involves retrieving images of the same person across multiple cameras, given an image of a person captured by a specific camera. In practical applications, person re-identification and facial recognition often serve as alternatives. Facial recognition relies on capturing facial features for identification and matching. In real-world scenarios, capturing high-quality facial images is often difficult, resulting in uncertainties in the success rate of finding people. Furthermore, person re-identification can only perform simple motion trajectory mapping and cannot link the motion trajectories of pedestrians captured by each surveillance camera to analyze the target's potential intentions. Summary of the Invention

[0004] In view of the problems existing in the prior art, the present invention provides a method and device for pedestrian recognition.

[0005] In a first aspect, the present invention provides a method for pedestrian recognition, comprising:

[0006] After determining that the tracking ID trackID of the target person is lost during the tracking process of the frame image in the target video, the predicted trajectory of the target person is obtained based on the target candidate sequence and the historical position sequence;

[0007] Based on the predicted trajectory of the target person, determining whether the target person is blocked or leaves the boundary;

[0008] If the target person is blocked, the target candidate sequence is corrected and supplemented based on the predicted trajectory of the target person;

[0009] If the target person leaves the boundary, the videos with the same spatial relationship as the predicted start time are spliced ​​together. Based on the spliced ​​video frame images, the tracking identifier trackID of the pedestrian is obtained to generate a target candidate sequence;

[0010] Based on the target candidate sequence during the trackID loss period, the REID algorithm and face recognition are used to identify the target person;

[0011] The target candidate sequence is obtained by obtaining the tracking identifier trackID of the pedestrian based on the frame image in the target video, including the trackID and bounding box of each pedestrian;

[0012] The historical position sequence is obtained after the target person is determined based on the position information of the target person in a preset number of consecutive frame images in the target video.

[0013] In one embodiment, the method for obtaining a pedestrian's tracking identifier trackID includes:

[0014] Perform pedestrian detection on the frame images in the target video to determine the trackID and bounding box of each pedestrian;

[0015] The pedestrians are tracked, similarities between the pedestrians in frame images at adjacent times are determined, and the pedestrians that meet the similarity threshold are associated and assigned the same trackID.

[0016] In one embodiment, obtaining a predicted trajectory of a target person based on the target candidate sequence and the historical position sequence includes:

[0017] The starNet algorithm based on global information interaction combines the target candidate sequence and the historical position sequence to predict the trajectory of the target person;

[0018] The trajectory prediction of the target person includes location information.

[0019] In one embodiment, if the target person is occluded, before correcting and supplementing the target candidate sequence based on the predicted trajectory of the target person, the method further includes:

[0020] Determine an occlusion state based on the position information of the predicted trajectory of the target person and the pedestrian detection information of the current frame image, wherein the occlusion state includes a severe occlusion state and a mild occlusion state;

[0021] If the occlusion state is serious, the bounding box of the target person at the current moment is predicted based on the position information of the target person at the current moment in the predicted trajectory and the bounding box of the target person at the previous moment;

[0022] If the occlusion state is not serious, the bounding box of the target person at the current moment is predicted based on the position information of the target person at the current moment in the predicted trajectory and the bounding boxes of the target person at the current moment and the previous moment in the actual frame image.

[0023] In one embodiment, if the occlusion state is serious, predicting the bounding box of the target person at the current moment based on the position information of the target person at the current moment in the predicted trajectory and the bounding box of the target person at the previous moment includes:

[0024] Determine a first width and a first height based on a bounding box of the target person at a previous moment;

[0025] Predicting a bounding box of the target person at the current moment based on the position information of the target person at the current moment in the predicted trajectory, the first width, and the first height;

[0026] If the occlusion state is not serious, based on the position information of the target person at the current moment in the predicted trajectory and the bounding boxes of the target person at the current moment and the previous moment in the actual frame image, predicting the bounding box of the target person at the current moment includes:

[0027] Determining a first width, a first height, and a first aspect ratio based on a bounding box of the target person at a previous moment;

[0028] Determining a second width and a second height, as well as a second aspect ratio, based on the bounding box of the target person at the current moment;

[0029] Based on the first aspect ratio and the second aspect ratio, and the position information of the target person at the current moment in the predicted trajectory, a bounding box of the target person at the current moment is predicted.

[0030] In one embodiment, if the target person is occluded, the target candidate sequence is corrected and supplemented based on the predicted trajectory of the target person, including:

[0031] Based on the position information of the predicted trajectory of the target person and the position information of pedestrian detection in the actual frame image, the target candidate sequence is corrected;

[0032] The REID algorithm and historical position sequence are used to verify the corrected target candidate sequence;

[0033] Based on the verified target candidate sequence, determine the correct predicted trajectory of the target person;

[0034] The target candidate sequence is supplemented based on the correct predicted trajectory of the target person and the predicted bounding box of the target person.

[0035] In one embodiment, the target candidate sequence during the trackID loss period is used to identify the target person using the REID algorithm and face recognition, including:

[0036] Obtain a target query graph, determine a target person in the target candidate sequence based on the REID algorithm, face recognition, and similarity ranking, and determine a historical location sequence of the target person;

[0037] Among them, the similarity between the target query graph and the determined target person meets a preset threshold and ranks highest.

[0038] In a second aspect, the present invention further provides a pedestrian recognition device, comprising a memory, a transceiver, and a processor;

[0039] A memory for storing a computer program; a transceiver for transmitting and receiving data under the control of the processor; and a processor for executing the computer program in the memory and implementing the following steps:

[0040] After determining that the tracking ID trackID of the target person is lost during the tracking process of the frame image in the target video, the predicted trajectory of the target person is obtained based on the target candidate sequence and the historical position sequence;

[0041] Based on the predicted trajectory of the target person, determining whether the target person is blocked or leaves the boundary;

[0042] If the target person is blocked, the target candidate sequence is corrected and supplemented based on the predicted trajectory of the target person;

[0043] If the target person leaves the boundary, the videos with the same spatial relationship as the predicted start time are spliced ​​together. Based on the spliced ​​video frame images, the tracking identifier trackID of the pedestrian is obtained to generate a target candidate sequence;

[0044] Based on the target candidate sequence during the trackID loss period, the REID algorithm and face recognition are used to identify the target person;

[0045] The target candidate sequence is obtained by obtaining the tracking identifier trackID of the pedestrian based on the frame image in the target video, including the trackID and bounding box of each pedestrian;

[0046] The historical position sequence is obtained after the target person is determined based on the position information of the target person in a preset number of consecutive frame images in the target video.

[0047] In a third aspect, the present invention further provides a pedestrian recognition device, the device comprising:

[0048] The trajectory prediction module is used to obtain the predicted trajectory of the target person based on the target candidate sequence and the historical position sequence after determining that the tracking identifier trackID of the target person is lost during the tracking process of the frame image in the target video;

[0049] A status confirmation module is used to determine whether the target person is blocked or leaves the boundary based on the predicted trajectory of the target person;

[0050] A correction and supplement module is used to correct and supplement the target candidate sequence based on the predicted trajectory of the target person if the target person is occluded;

[0051] A generation module is used to stitch together the videos with the same spatial relationship as the prediction start time if the target person leaves the boundary, obtain the tracking identifier trackID of the pedestrian based on the stitched video frame images, and generate a target candidate sequence;

[0052] The recognition module is used to identify the target person based on the target candidate sequence during the trackID loss period, using the REID algorithm and face recognition;

[0053] The target candidate sequence is obtained by obtaining the tracking identifier trackID of the pedestrian based on the frame image in the target video, including the trackID and bounding box of each pedestrian;

[0054] The historical position sequence is obtained after the target person is determined based on the position information of the target person in a preset number of consecutive frame images in the target video.

[0055] In a fourth aspect, the present invention further provides a processor-readable storage medium, wherein the processor-readable storage medium stores a computer program, and the computer program is used to enable the processor to execute the steps of the pedestrian recognition method described in the first aspect above.

[0056] The pedestrian recognition method and device provided by the present invention predict the pedestrian's motion trajectory after the pedestrian tracking identifier trackID in the video frame image is lost, realizes the processing of the target candidate sequence during the period of trackID loss, improves the stability and continuity of target tracking in the occlusion state, and adopts video splicing technology to verify the correctness of the pedestrian's motion trajectory when leaving the boundary state, thereby improving the accuracy of pedestrian recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0058] Figure 1 1 is a flow chart of a pedestrian recognition method provided by the present invention;

[0059] Figure 2 This is a network structure diagram for implementing REID using deep network technology;

[0060] Figure 3This is the structure diagram of the starNet network;

[0061] Figure 4 This is an overall flow chart of the pedestrian recognition method provided by the present invention;

[0062] Figure 5 It is a structural diagram of the pedestrian recognition device provided by the present invention;

[0063] Figure 6 It is a structural schematic diagram of the pedestrian recognition device provided by the present invention. DETAILED DESCRIPTION

[0064] In this disclosure, the term "and / or" describes the relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0065] In the present invention, the term "plurality" refers to two or more than two, and other quantifiers are similar to it.

[0066] The following will clearly and completely describe the technical solutions of the present invention in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention and not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0067] Figure 1 FIG. 1 is a flow chart of the pedestrian recognition method provided by the present invention. Figure 1 As shown, the method includes the following steps:

[0068] Step 101: After determining that the tracking identifier trackID of the target person is lost during the tracking process of the frame image in the target video, a predicted trajectory of the target person is obtained based on the target candidate sequence and the historical position sequence;

[0069] The target candidate sequence is obtained by obtaining the tracking identifier trackID of the pedestrian based on the frame image in the target video, including the trackID and bounding box of each pedestrian;

[0070] The historical position sequence is obtained after the target person is determined based on the position information of the target person in a preset number of consecutive frame images in the target video.

[0071] Specifically, the widespread use of surveillance cameras has made pedestrian identification and person retrieval increasingly easier. Captured facial images can more accurately identify pedestrians. However, in real-world scenarios, it's often difficult to capture high-quality facial images, resulting in a low success rate for person retrieval. Alternatively, occlusion may cause a person to be outside a camera's monitoring range, making it impossible to locate the person.

[0072] Therefore, the present invention obtains the target video, detects each frame image of the target video, obtains the bounding box of each pedestrian in each frame image, and assigns a trackID to each pedestrian to identify each pedestrian in the frame image of the target video. The range of the bounding box of each pedestrian must completely cover the human body features of the pedestrian object. There can be multiple representations, as long as the determined bounding box is unique. For example, the bounding box can be a quadrilateral, a circle, or other irregular shapes. Taking a quadrilateral as an example, the coordinate position of each vertex (x1, y1, x2, y2, x3, y3, x4, y4), or the coordinate position of the quadrilateral opposite the vertex (x5, y5, x6, y6), or the coordinates of the center point of the four deformations and the width and height of the quadrilateral, etc. In the present invention, the coordinate position of the quadrilateral opposite the vertex is mainly used to represent the bounding box of a pedestrian. The target candidate sequence is composed of the trackID of each pedestrian and the corresponding bounding box.

[0073] The target person is tentatively identified as the pedestrian with the highest object similarity. After the target person is identified, the target person's location information in a preset number of consecutive frames is saved to form a historical position sequence. The target person's location information here can be the coordinate position of the target person's center or the target person's bounding box information. Both have equivalent functions and can identify the target person's position in the corresponding video frame image. The preset number can be selected based on actual circumstances and is not limited here.

[0074] If a pedestrian's trackID is missing in a target video frame, the target's trajectory needs to be predicted based on the target candidate sequence and historical position sequence. The predicted trajectory includes all the target's position information during the period when the trackID was lost. This position information primarily consists of the target's coordinates in the video frame.

[0075] Step 102: Based on the predicted trajectory of the target person, determine whether the target person is blocked or leaves the boundary;

[0076] Specifically, the target person's predicted trajectory is used to determine their position coordinates within the video frame. If the position coordinates exceed the boundaries of the video frame, the target person is determined to have left the boundary; if the position information does not exceed the boundaries of the video frame, the target person is determined to be obscured.

[0077] Step 103: If the target person is blocked, the target candidate sequence is corrected and supplemented based on the predicted trajectory of the target person;

[0078] Specifically, if it is determined that the target person is occluded, the position coordinates in the predicted trajectory of the target person are compared with the motion trajectories of all pedestrians obtained by actual pedestrian detection and tracking to achieve correction and supplement of the target candidate sequence.

[0079] Step 104: If the target person leaves the boundary, the videos with the same predicted start time and adjacent spatial relationships are spliced ​​together, and based on the spliced ​​video frame images, the tracking identifier trackID of the pedestrian is obtained to generate a target candidate sequence;

[0080] Specifically, if the target person is determined to have left the boundary, meaning their current location is outside the range of the video frame, the video file is spliced ​​with the video from the same geographically adjacent cameras at the same time as the prediction start time. This is done using the Parallax-Robust Surveillance Video Stitching algorithm, which detects whether a moving object crosses the seam and updates the camera pixel mapping matrix if so, improving the stitching quality.

[0081] Step 105: Based on the target candidate sequence during the trackID loss period, the target person is identified using the REID algorithm and face recognition;

[0082] Specifically, the present invention uses deep network technology to implement REID, and the network structure is as follows Figure 2 .like Figure 2 The resnet50 (residual 50) shown is the basic network. It obtains high-resolution feature images through upsampling and then obtains fused features through convolution operations. A BatchNorm (batch normalization) layer is added before the fully connected layer to increase the convergence ability of the training stage.

[0083] The use of REID technology as the main method and face recognition as the auxiliary method can reduce the impact of face capture quality on target determination, while making full use of time correlation and space correlation to identify the target person.

[0084] The pedestrian recognition method and device provided by the present invention predict the pedestrian's motion trajectory after the pedestrian tracking identifier trackID in the video frame image is lost, realizes the processing of the target candidate sequence during the period of trackID loss, improves the stability and continuity of target tracking in the occlusion state, and adopts video splicing technology to verify the correctness of the pedestrian's motion trajectory when leaving the boundary state, thereby improving the accuracy of pedestrian recognition.

[0085] Optionally, the method for obtaining a pedestrian's tracking identifier trackID includes:

[0086] Perform pedestrian detection on the frame images in the target video to determine the trackID and bounding box of each pedestrian;

[0087] The pedestrians are tracked, similarities between the pedestrians in frame images at adjacent times are determined, and the pedestrians that meet the similarity threshold are associated and assigned the same trackID.

[0088] Specifically, before performing target person retrieval, pedestrian detection and tracking are performed on all persons in the video frame image to form a target candidate sequence.

[0089] 1) The pedestrian detection step is implemented using a deep convolutional network to detect the presence of pedestrians in the frame image in real time and accurately locate them, that is, to determine the bounding box of each pedestrian and assign a trackID to each new pedestrian;

[0090] The general workflow of a tracking algorithm is as follows:

[0091] 1.1 Given the original frame of the video;

[0092] 1.2 Run the object detector to obtain the bounding boxes of all pedestrians in the video frame image;

[0093] 1.3 For each detected pedestrian, different features are calculated, usually visual and motion features;

[0094] 1.4 Use the similarity calculation formula to determine the probability that two pedestrians belong to the same target;

[0095] 1.5 If the similarity between two pedestrians is greater than the preset threshold, the two pedestrians are associated and the same trackID is assigned to them.

[0096] The pedestrian recognition method and device provided by the present invention predict the pedestrian's motion trajectory after the pedestrian tracking identifier trackID in the video frame image is lost, realizes the processing of the target candidate sequence during the period of trackID loss, improves the stability and continuity of target tracking in the occlusion state, and adopts video splicing technology to verify the correctness of the pedestrian's motion trajectory when leaving the boundary state, thereby improving the accuracy of pedestrian recognition.

[0097] Optionally, obtaining a predicted trajectory of a target person based on the target candidate sequence and the historical position sequence includes:

[0098] The starNet algorithm based on global information interaction combines the target candidate sequence and the historical position sequence to predict the trajectory of the target person;

[0099] The trajectory prediction of the target person includes location information.

[0100] Specifically, in the process of tracking pedestrians, if no associated object is found in N consecutive frame images, it is preliminarily judged that the trackID of the pedestrian or target person is lost. The N value here can be set according to the actual scene and is not limited here. When the trackID is lost, in order to determine whether it is occluded or leaving the boundary, the motion trajectory of the target person needs to be predicted. If the prediction is simply based on the motion information of a single person, the final prediction information will have a large error. Therefore, the present invention uses the starNet algorithm based on global information interaction to predict the human motion trajectory. The algorithm can form a static "map" through the positions of all obstacles at each moment. As time changes, these static maps become a dynamic map with time series information. This dynamic map records the obstacle motion information in each area, where the motion information is obtained by the influence of all obstacles together, rather than formed by individual interactions between two obstacles. By using the "shared global interactive map + individual query" method, it is possible to calculate global interactions and compress computing overhead. The starNet network structure is as follows Figure 3 As shown in Figure 1, the starNet network is divided into the trajectory prediction network Host NetWork and the global time series interactive computing network Hub NetWork.

[0101] The pedestrian recognition method and device provided by this invention predicts the pedestrian's motion trajectory after the pedestrian tracking identifier (trackID) in a video frame is lost. This trajectory prediction can be used to determine the pedestrian's position within a certain period of time, allowing for determination of whether the target person is obscured or has left the boundary. Furthermore, cross-validation is performed using the predicted trajectory, tracking results, and re-identification results, allowing the algorithm to be implemented in stages, reducing computing resource consumption and improving algorithm accuracy.

[0102] Optionally, if the target person is blocked, before correcting and supplementing the target candidate sequence based on the predicted trajectory of the target person, the method further includes:

[0103] Determine an occlusion state based on the position information of the predicted trajectory of the target person and the pedestrian detection information of the current frame image, wherein the occlusion state includes a severe occlusion state and a mild occlusion state;

[0104] If the occlusion state is serious, the bounding box of the target person at the current moment is predicted based on the position information of the target person at the current moment in the predicted trajectory and the bounding box of the target person at the previous moment;

[0105] If the occlusion state is not serious, the bounding box of the target person at the current moment is predicted based on the position information of the target person at the current moment in the predicted trajectory and the bounding boxes of the target person at the current moment and the previous moment in the actual frame image.

[0106] Specifically, the system determines the position coordinates of all pedestrians based on the bounding box of pedestrian detection in the current frame image. Each pedestrian's position coordinates are then compared with the coordinates of the target person's predicted track. If the distance between any two positions is less than a set threshold, the missing trackID is considered to be present in the current frame, indicating that the target person's occlusion is not severe. If the distance between both positions is greater than the set threshold, the target person's occlusion is considered severe.

[0107] In the case of severe occlusion, such as in the pedestrian detection stage, the target person is completely blocked by a car or other object and its corresponding bounding box cannot be obtained. In this case, the bounding box of the target person at the current moment is predicted based on the position information of the target person at the current moment in the predicted trajectory and the bounding box of the target person in the video frame image at the previous moment.

[0108] When the occlusion state is not serious, for example, in the pedestrian detection stage, some human features of the target person are blocked, and the corresponding bounding box can be obtained, but the human features of the target person in the bounding box are incomplete. Based on the position information of the target person at the current moment in the predicted trajectory and the bounding boxes of the target person at the current moment and the previous moment in the actual frame image, the bounding box of the target person at the current moment is predicted.

[0109] The pedestrian recognition method and device provided by the present invention predict the pedestrian's motion trajectory after the pedestrian tracking identifier trackID in the video frame image is lost, and determine the occlusion status based on the position information of the target person's predicted trajectory and the pedestrian detection information of the current frame image. According to different occlusion states, the target person's bounding box is predicted in different ways, providing a prerequisite for the next step of processing the target candidate sequence.

[0110] Optionally, if the occlusion state is serious, predicting the bounding box of the target person at the current moment based on the position information of the target person at the current moment in the predicted trajectory and the bounding box of the target person at the previous moment includes:

[0111] Determine a first width and a first height based on a bounding box of the target person at a previous moment;

[0112] Based on the position information of the target person at the current moment in the predicted trajectory, the first width and the first height, a bounding box of the target person at the current moment is predicted.

[0113] Optionally, if the occlusion state is not serious, based on the position information of the target person at the current moment in the predicted trajectory and the bounding boxes of the target person at the current moment and the previous moment in the actual frame image, predicting the bounding box of the target person at the current moment includes:

[0114] Determining a first width, a first height, and a first aspect ratio based on a bounding box of the target person at a previous moment;

[0115] Determining a second width and a second height, as well as a second aspect ratio, based on the bounding box of the target person at the current moment;

[0116] Based on the first aspect ratio and the second aspect ratio, and the position information of the target person at the current moment in the predicted trajectory, a bounding box of the target person at the current moment is predicted.

[0117] Specifically, the position information obtained from the predicted trajectory of the target person is compared with the position information of the detection result in the current frame, and the distance between the position in the predicted trajectory and the position in the detection result is calculated. If the distance is less than the set threshold, it is determined that the position where the trackID appears in the current frame is lost, which means that the occlusion is not serious. Otherwise, it is a serious occlusion.

[0118] If the target person's occlusion status is severely determined through trajectory prediction, the bounding box of the target person will be calculated using the aspect ratio of the bounding box of the previous frame video and the predicted target person's position coordinates (see Formula 2 for details). For pedestrians whose occlusion status is not severely determined (the pedestrian detection algorithm can detect the bounding box covering part of the pedestrian's body), the bounding box of the target person will be calculated using the aspect ratio of the bounding box detected in the image frame of the previous moment in the video, the width or height of the bounding box of the pedestrian detected at the current moment, and the predicted center coordinates of the target person (see Formula 4 for details).

[0119] Assuming that the pedestrian detection algorithm obtains the bounding box of the target person at the previous moment as box1 = (x1, y1, x2, y2), then convert it into a form to obtain box2 = (x, y, w, h), where w is the first width and h is the first height. The corresponding relationship is shown in Formula 1:

[0120]

[0121] Assuming that the center coordinates of the predicted target person's position are (xc, yc), the bounding box box3 = (x3, y3, x4, y4) of the severely occluded predicted target person is calculated as follows:

[0122]

[0123] Assuming that the occlusion is not serious, the bounding box of the pedestrian detection of the target person at the current moment is box = (x5, y5, x6, y6). Convert it to a form to get box0 = (x*, y*, w*, h*), where w* is the second width and h* is the second height. The corresponding relationship is as follows:

[0124]

[0125] The calculation formula for predicting the target person's bounding box box4 = (x7, y7, x8, y8) is as follows:

[0126]

[0127] in, is the first aspect ratio, The second aspect ratio.

[0128] The pedestrian recognition method and device provided by the present invention predict the pedestrian's motion trajectory after the pedestrian tracking identifier trackID in the video frame image is lost, realizes the processing of the target candidate sequence during the period of trackID loss, improves the stability and continuity of target tracking in the occlusion state, and adopts video splicing technology to verify the correctness of the pedestrian's motion trajectory when leaving the boundary state, thereby improving the accuracy of pedestrian recognition.

[0129] Optionally, if the target person is blocked, the target candidate sequence is corrected and supplemented based on the predicted trajectory of the target person, including:

[0130] Based on the position information of the predicted trajectory of the target person and the position information of pedestrian detection in the actual frame image, the target candidate sequence is corrected;

[0131] The REID algorithm and historical position sequence are used to verify the corrected target candidate sequence;

[0132] Based on the verified target candidate sequence, determine the correct predicted trajectory of the target person;

[0133] The target candidate sequence is supplemented based on the correct predicted trajectory of the target person and the predicted bounding box of the target person.

[0134] Specifically, to avoid target confusion caused by pedestrian occlusion during target tracking, a tracking, trajectory prediction, and cross-validation approach for human weight recognition are employed. Throughout system operation, target tracking is performed continuously. For example, the trajectory prediction algorithm predicts the pedestrian's trajectory for the next 3 seconds of video using 5 seconds of historical video frames, or for the next 10 seconds of video using 10 seconds of historical video frames. The predicted trajectory of the target person is compared with the trajectory obtained by tracking the pedestrian to correct the target candidate sequence. If a trackID is lost, the pedestrian's position (i.e., the center coordinates of each bounding box) is determined based on the actual pedestrian detection and tracking, along with the trackID. The coordinates of the target person's predicted trajectory are compared with the center coordinates of the actual detected pedestrian. If the distance between the two positions is less than a preset threshold, and the distances between the positions of a preset number of target persons and the same pedestrian are all less than the preset threshold, the predicted trajectory of the target person is determined to have the same trajectory trend as the corresponding pedestrian, and the pedestrian is identified as the target person. Simultaneously, the target candidate sequence, including the target person, is corrected using the target person's predicted trajectory.

[0135] Based on the corrected target candidate sequence, the REID algorithm is used to identify the target candidate sequence and the people in the target query graph. If a pedestrian in the target candidate sequence and the people in the target query graph do not match, that is, the similarity between the pedestrian in the target candidate sequence and the people in the target query graph does not meet the preset threshold, and the people in the target query graph cannot be identified, the corresponding position of the target person is found from the historical position sequence. The bounding box of the tracking target is determined through trajectory prediction. The bounding box of the people in the target query graph is combined with the bounding box of the people in the target query graph. The IOU (intersection over union) of the two bounding boxes is compared. This is to calculate the ratio of the intersection and union of the bounding boxes of the tracking target and the people in the target query graph. The person with the larger IOU is selected as the correct target person. This completes the verification of the corrected target candidate sequence.

[0136] After confirming that the corrected target candidate sequence is correct, the starNet algorithm is used to predict the human motion trajectory based on the target candidate sequence and the historical position sequence to determine the predicted trajectory of the target person;

[0137] Based on the predicted trajectory's location information and the bounding box of the target person, the candidate sequence for the target person is supplemented. This cross-validation approach allows the algorithm to be deployed in stages, reducing computing resource consumption while minimizing errors caused by false positives, thereby reducing the impact of scene complexity on algorithm accuracy.

[0138] The pedestrian recognition method and device provided by the present invention predict the pedestrian's motion trajectory after the pedestrian tracking identifier trackID in the video frame image is lost, realizes the processing of the target candidate sequence during the period of trackID loss, improves the stability and continuity of target tracking in the occlusion state, and adopts video splicing technology to verify the correctness of the pedestrian's motion trajectory when leaving the boundary state, thereby improving the accuracy of pedestrian recognition.

[0139] Optionally, the target candidate sequence during the trackID loss period is identified by using a REID algorithm and face recognition to identify the target person, including:

[0140] Obtain a target query graph, determine a target person in the target candidate sequence based on the REID algorithm, face recognition, and similarity ranking, and determine a historical location sequence of the target person;

[0141] Among them, the similarity between the target query graph and the determined target person meets a preset threshold and ranks highest.

[0142] Specifically, a target query graph is obtained, which includes the human body and facial feature information of the human object to be found. Based on the target candidate sequence, the REID algorithm is used for human recognition. By comparing the features of the human objects in the target query graph with the features of the pedestrians in the target candidate sequence, the similarity ranking of the pedestrians in the target candidate sequence is obtained, and it is determined whether the similarity meets the preset threshold. If it does, the pedestrian with the highest ranking is temporarily used as the target person; if it does not, it is determined that there is no pedestrian in the target candidate sequence that matches the human object in the target query graph. In subsequent video frames, if a face is captured, a face comparison is performed. After the top five pedestrians in the target candidate sequence are compared with the human objects in the target query graph, the target person is finally determined. After the target person is determined, the coordinate position of the target person in several consecutive frames is saved as a historical position sequence.

[0143] The pedestrian recognition method and device provided by the present invention predict the pedestrian's motion trajectory after the pedestrian tracking identifier trackID in the video frame image is lost, realizes the processing of the target candidate sequence during the period of trackID loss, improves the stability and continuity of target tracking in the occlusion state, and adopts video splicing technology to verify the correctness of the pedestrian's motion trajectory when leaving the boundary state, thereby improving the accuracy of pedestrian recognition.

[0144] The overall process of the present invention is described below. Figure 4 This is the overall flow chart of the pedestrian recognition method provided by the present invention. Figure 4 As shown:

[0145] Step 401: Pedestrian detection and tracking. During the pedestrian detection phase, all pedestrians are detected using frame images in the video. Each new pedestrian is assigned a trackID, and a bounding box is defined for each pedestrian based on its body area. During the pedestrian tracking phase, the probability of two objects belonging to the same target is determined by comparing the similarity of objects within different bounding boxes. If the similarity is greater than a preset threshold, the two objects are associated and assigned the same trackID. A candidate target sequence is also generated.

[0146] Step 402: Determine the target pedestrian. Using the REID algorithm, similarity is calculated between the target query image and the pedestrians in the target candidate sequence. The highest-ranked pedestrian is tentatively designated as the target person, i.e., the person object in the target query image. In subsequent video frames, if a face is captured, the top five pedestrians with the highest similarity are compared with the facial image for face recognition, ultimately determining the target person. After the target person is determined, the coordinates of the target person in a preset number of consecutive frames are saved as a historical position sequence.

[0147] Step 403: Preliminary determination of occlusion and crossing of boundaries: If a trackID has no related object in N consecutive frames during the tracking process, the trackID of the pedestrian or target person is determined to be lost, where N can be set according to the actual situation.

[0148] Step 404: Pedestrian trajectory prediction is performed based on global information interaction. A preliminary determination is made that the target person's trackID is lost. Based on the target candidate sequence obtained in step 401 and the historical position sequence obtained in step 402, the predicted trajectory of the target person is determined using the starNet algorithm, including the target person's position information during the period when the trackID was lost.

[0149] Step 405: Predict the human frame for the occluded target person. Based on the predicted trajectory of the target person, determine whether the target person is occluded or leaving the boundary. If the target person is occluded, further determine the occlusion status, whether the occlusion is severe or not.

[0150] In the case of severe occlusion, the bounding box of the target person at the current moment is predicted based on the bounding box of the target person at the previous moment and the predicted center coordinates.

[0151] When the occlusion is not serious, the bounding box of the target person at the current moment is predicted based on the bounding box of the target person at the previous moment, the bounding box at the current moment, and the predicted center coordinates.

[0152] Step 406: Tracking, Prediction, and Re-ID Cross-Validation. Based on the position of the target person in the predicted trajectory and the position in the actual tracked pedestrian trajectory, the target candidate sequence is corrected. Based on the corrected target candidate sequence, the REID algorithm is used to re-identify the object in the target query graph. If no matching object is found, the location information corresponding to the target person is found in the historical location sequence. The corresponding bounding box is obtained through trajectory prediction. The ratio of the intersection and union of the bounding box of the target person in the predicted trajectory and the bounding box of the actual tracked pedestrian is calculated. The person with the larger ratio is selected as the target person.

[0153] Step 407: Perform video stitching for boundary crossing. Based on the target person's predicted trajectory, if the target person's position is determined to have exceeded the video range, then the target person has left the boundary, i.e., crossed the boundary. Using the Parallax-RobustSurveillance Video Stitching algorithm, video files from spatially adjacent cameras with the same predicted start time are stitched together with the current video. Based on this stitched video, repeat steps 401-402 to re-identify the objects in the target query graph.

[0154] Figure 5 FIG. 1 is a schematic diagram of the structure of the pedestrian recognition device provided by the present invention. Figure 5 As shown, the audio jamming assessment device includes a memory 520, a transceiver 510 and a processor 500; wherein, the processor 500 and the memory 520 can also be arranged physically separately.

[0155] The memory 520 is used to store computer programs; the transceiver 510 is used to send and receive data under the control of the processor 500.

[0156] Specifically, the transceiver 510 is configured to receive and send data under the control of the processor 500 .

[0157] Among them, Figure 5 In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically linking together various circuits of one or more processors represented by processor 500 and memory represented by memory 520. The bus architecture may also link together various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are all well known in the art and therefore will not be further described in the present invention. The bus interface provides an interface. The transceiver 510 may be a plurality of components, namely, a transmitter and a receiver, providing a unit for communicating with various other devices over a transmission medium, such as a wireless channel, a wired channel, an optical cable, or the like.

[0158] The processor 500 is responsible for managing the bus architecture and general processing, and the memory 520 can store data used by the processor 500 when performing operations.

[0159] The processor 500 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or a complex programmable logic device (CPLD), and the processor may also adopt a multi-core architecture.

[0160] The processor 500 calls the computer program stored in the memory 520 to execute any of the methods provided by the present invention according to the obtained executable instructions, for example:

[0161] After determining that the tracking ID trackID of the target person is lost during the tracking process of the frame image in the target video, the predicted trajectory of the target person is obtained based on the target candidate sequence and the historical position sequence;

[0162] Based on the predicted trajectory of the target person, determining whether the target person is blocked or leaves the boundary;

[0163] If the target person is blocked, the target candidate sequence is corrected and supplemented based on the predicted trajectory of the target person;

[0164] If the target person leaves the boundary, the videos with the same spatial relationship as the predicted start time are spliced ​​together. Based on the spliced ​​video frame images, the tracking identifier trackID of the pedestrian is obtained to generate a target candidate sequence;

[0165] Based on the target candidate sequence during the trackID loss period, the target person is identified using the REID algorithm and face recognition;

[0166] The target candidate sequence is obtained by obtaining the tracking identifier trackID of the pedestrian based on the frame image in the target video, including the trackID and bounding box of each pedestrian;

[0167] The historical position sequence is obtained after the target person is determined based on the position information of the target person in a preset number of consecutive frame images in the target video.

[0168] It should be noted here that the above-mentioned pedestrian recognition device provided by the present invention can implement all the method steps implemented in the above-mentioned pedestrian recognition method embodiment, and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as the method embodiment will not be described in detail here.

[0169] Figure 6 Schematic diagram of the structure of the pedestrian recognition device provided by the present invention. Figure 6 As shown, the device includes:

[0170] The trajectory prediction module 601 is used to obtain the predicted trajectory of the target person based on the target candidate sequence and the historical position sequence after determining that the tracking identifier trackID of the target person is lost during the tracking process of the frame image in the target video;

[0171] A status confirmation module 602 is configured to determine whether the target person is blocked or has left the boundary based on the predicted trajectory of the target person;

[0172] The correction and supplement module 603 is used to correct and supplement the target candidate sequence based on the predicted trajectory of the target person if the target person is blocked;

[0173] The generation module 604 is configured to, if the target person leaves the boundary, stitch the videos that are spatially adjacent to the prediction start time, obtain the tracking identifier trackID of the pedestrian based on the stitched video frame images, and generate a target candidate sequence;

[0174] Identification module 605, for identifying the target person based on the target candidate sequence during the trackID loss period using REID algorithm and face recognition;

[0175] The target candidate sequence is obtained by obtaining the tracking identifier trackID of the pedestrian based on the frame image in the target video, including the trackID and bounding box of each pedestrian;

[0176] The historical position sequence is obtained after the target person is determined based on the position information of the target person in a preset number of consecutive frame images in the target video.

[0177] The generation module 604 is further configured to perform pedestrian detection on the frame images in the target video and determine the trackID and bounding box of each pedestrian;

[0178] The pedestrians are tracked, similarities between the pedestrians in frame images at adjacent times are determined, and the pedestrians that meet the similarity threshold are associated and assigned the same trackID.

[0179] The trajectory prediction module 601 is further configured to predict the trajectory of the target person by combining the target candidate sequence and the historical position sequence based on the starNet algorithm of global information interaction;

[0180] The trajectory prediction of the target person includes location information.

[0181] The trajectory prediction module 601 is further configured to determine an occlusion state based on the position information of the predicted trajectory of the target person and the pedestrian detection information of the current frame image, wherein the occlusion state includes a severe occlusion state and a mild occlusion state;

[0182] If the occlusion state is serious, the bounding box of the target person at the current moment is predicted based on the position information of the target person at the current moment in the predicted trajectory and the bounding box of the target person at the previous moment;

[0183] If the occlusion state is not serious, the bounding box of the target person at the current moment is predicted based on the position information of the target person at the current moment in the predicted trajectory and the bounding boxes of the target person at the current moment and the previous moment in the actual frame image.

[0184] The correction and supplement module 603 is further configured to predict the bounding box of the target person at the current moment based on the position information of the target person at the current moment in the predicted trajectory and the bounding box of the target person at the previous moment if the occlusion state is serious, including:

[0185] Determine a first width and a first height based on a bounding box of the target person at a previous moment;

[0186] Based on the position information of the target person at the current moment in the predicted trajectory, the first width and the first height, a bounding box of the target person at the current moment is predicted.

[0187] The correction and supplement module 603 is further configured to predict the target person's bounding box at the current moment based on the position information of the target person at the current moment in the predicted trajectory and the target person's bounding boxes at the current moment and the previous moment in the actual frame image if the occlusion state is not serious, including:

[0188] Determining a first width, a first height, and a first aspect ratio based on a bounding box of the target person at a previous moment;

[0189] Determining a second width and a second height, as well as a second aspect ratio, based on the bounding box of the target person at the current moment;

[0190] Based on the first aspect ratio and the second aspect ratio, and the position information of the target person at the current moment in the predicted trajectory, a bounding box of the target person at the current moment is predicted.

[0191] The correction and supplement module 603 is further used to correct and supplement the target candidate sequence based on the predicted trajectory of the target person if the target person is blocked, including:

[0192] Based on the position information of the predicted trajectory of the target person and the position information of pedestrian detection in the actual frame image, the target candidate sequence is corrected;

[0193] The REID algorithm and historical position sequence are used to verify the corrected target candidate sequence;

[0194] Based on the verified target candidate sequence, determine the correct predicted trajectory of the target person;

[0195] The target candidate sequence is supplemented based on the correct predicted trajectory of the target person and the predicted bounding box of the target person.

[0196] The identification module 605 is also used to identify the target person based on the target candidate sequence during the trackID loss period using the REID algorithm and face recognition, including:

[0197] Obtain a target query graph, determine a target person in the target candidate sequence based on the REID algorithm, face recognition, and similarity ranking, and determine a historical location sequence of the target person;

[0198] Among them, the similarity between the target query graph and the determined target person meets a preset threshold and ranks highest.

[0199] It should be noted that the division of units in the present invention is illustrative and represents only a logical functional division. In actual implementation, other divisions may be employed. Furthermore, the functional units in various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically as a separate unit, or two or more units may be integrated into a single unit. These integrated units may be implemented in either hardware or software functional units.

[0200] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a processor-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0201] It should be noted here that the above-mentioned device provided by the present invention can implement all the method steps implemented in the above-mentioned method embodiment and can achieve the same technical effects. The parts and beneficial effects that are the same as those in the method embodiment will not be described in detail here.

[0202] In another aspect, the present invention further provides a computer program product, comprising a computer program stored on a non-transitory computer-readable storage medium, wherein the computer program comprises program instructions. When the program instructions are executed by a computer, the computer can perform the steps of the pedestrian recognition method provided by the above methods, for example, including:

[0203] After determining that the tracking ID trackID of the target person is lost during the tracking process of the frame image in the target video, the predicted trajectory of the target person is obtained based on the target candidate sequence and the historical position sequence;

[0204] Based on the predicted trajectory of the target person, determining whether the target person is blocked or leaves the boundary;

[0205] If the target person is blocked, the target candidate sequence is corrected and supplemented based on the predicted trajectory of the target person;

[0206] If the target person leaves the boundary, the videos with the same spatial relationship as the predicted start time are spliced ​​together. Based on the spliced ​​video frame images, the tracking identifier trackID of the pedestrian is obtained to generate a target candidate sequence;

[0207] Based on the target candidate sequence during the trackID loss period, the REID algorithm and face recognition are used to identify the target person;

[0208] The target candidate sequence is obtained by detecting and tracking pedestrians in the frame images of the target video, including the trackID and bounding box of each pedestrian;

[0209] The historical position sequence is obtained after the target person is determined based on the position information of the target person in a preset number of consecutive frame images in the target video.

[0210] On the other hand, the present invention further provides a processor-readable storage medium storing a computer program, wherein the computer program is configured to cause the processor to execute the pedestrian recognition method provided in each of the above embodiments, for example, including:

[0211] After determining that the tracking ID trackID of the target person is lost during the tracking process of the frame image in the target video, the predicted trajectory of the target person is obtained based on the target candidate sequence and the historical position sequence;

[0212] Based on the predicted trajectory of the target person, determining whether the target person is blocked or leaves the boundary;

[0213] If the target person is blocked, the target candidate sequence is corrected and supplemented based on the predicted trajectory of the target person;

[0214] If the target person leaves the boundary, the videos with the same spatial relationship as the predicted start time are spliced ​​together. Based on the spliced ​​video frame images, the tracking identifier trackID of the pedestrian is obtained to generate a target candidate sequence;

[0215] Based on the target candidate sequence during the trackID loss period, the REID algorithm and face recognition are used to identify the target person;

[0216] The target candidate sequence is obtained by detecting and tracking pedestrians in the frame images of the target video, including the trackID and bounding box of each pedestrian;

[0217] The historical position sequence is obtained after the target person is determined based on the position information of the target person in a preset number of consecutive frame images in the target video.

[0218] The processor-readable storage medium can be any available medium or data storage device that can be accessed by the processor, including but not limited to magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO)), optical storage (such as CDs, DVDs, BDs, HVDs, etc.), and semiconductor storage (such as ROMs, EPROMs, EEPROMs, non-volatile memories (NANDFLASH), solid-state drives (SSDs)), etc.

[0219] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage and optical storage, etc.) containing computer-usable program code.

[0220] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer-executable instructions. These computer-executable instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0221] These processor-executable instructions may also be stored in a processor-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the processor-readable memory produce an article of manufacture comprising an instruction device that implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0222] These processor-executable instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are performed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0223] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A method for pedestrian recognition, characterized in that: include: After determining that the tracking ID of the target person is lost during the tracking process of the frame image in the target video, the starNet algorithm based on global information interaction combines the target candidate sequence and the historical position sequence to predict the trajectory of the target person; wherein the trajectory prediction of the target person includes position information; the starNet algorithm is composed of a global temporal interaction computing network and a trajectory prediction network; Based on the predicted trajectory of the target person, determining whether the target person is blocked or leaves the boundary; If the target person is blocked, the target candidate sequence is corrected and supplemented based on the predicted trajectory of the target person; If the target person leaves the boundary, the videos with the same spatial relationship as the predicted start time are spliced ​​together. Based on the spliced ​​video frame images, the tracking identifier trackID of the pedestrian is obtained to generate a target candidate sequence; Based on the target candidate sequence during the trackID loss period, the target person is identified using the REID algorithm and face recognition; The target candidate sequence is obtained by obtaining the tracking identifier trackID of the pedestrian based on the frame image in the target video, including the trackID and bounding box of each pedestrian; The historical position sequence is obtained after the target person is determined based on the position information of the target person in a preset number of consecutive frame images in the target video.

2. The method for pedestrian recognition according to claim 1, characterized in that: The method for obtaining the pedestrian's tracking identifier trackID includes: Perform pedestrian detection on the frame images in the target video to determine the trackID and bounding box of each pedestrian; The pedestrians are tracked, similarities between the pedestrians in frame images at adjacent times are determined, and the pedestrians that meet the similarity threshold are associated and assigned the same trackID.

3. The method for pedestrian recognition according to claim 1, characterized in that: If the target person is blocked, before correcting and supplementing the target candidate sequence based on the predicted trajectory of the target person, the method further includes: Determine an occlusion state based on the position information of the predicted trajectory of the target person and the pedestrian detection information of the current frame image, wherein the occlusion state includes a severe occlusion state and a mild occlusion state; If the occlusion state is serious, the bounding box of the target person at the current moment is predicted based on the position information of the target person at the current moment in the predicted trajectory and the bounding box of the target person at the previous moment; If the occlusion state is not serious, the bounding box of the target person at the current moment is predicted based on the position information of the target person at the current moment in the predicted trajectory and the bounding boxes of the target person at the current moment and the previous moment in the actual frame image.

4. The method for pedestrian recognition according to claim 3, characterized in that: If the occlusion state is serious, predicting the bounding box of the target person at the current moment based on the position information of the target person at the current moment in the predicted trajectory and the bounding box of the target person at the previous moment includes: Determine a first width and a first height based on a bounding box of the target person at a previous moment; Predicting a bounding box of the target person at the current moment based on the position information of the target person at the current moment in the predicted trajectory, the first width, and the first height; If the occlusion state is not serious, based on the position information of the target person at the current moment in the predicted trajectory and the bounding boxes of the target person at the current moment and the previous moment in the actual frame image, predicting the bounding box of the target person at the current moment includes: Determining a first width, a first height, and a first aspect ratio based on a bounding box of the target person at a previous moment; Determining a second width and a second height, as well as a second aspect ratio, based on the bounding box of the target person at the current moment; Based on the first aspect ratio and the second aspect ratio, and the position information of the target person at the current moment in the predicted trajectory, a bounding box of the target person at the current moment is predicted.

5. The method for pedestrian recognition according to claim 3, characterized in that: If the target person is blocked, the target candidate sequence is corrected and supplemented based on the predicted trajectory of the target person, including: Based on the position information of the predicted trajectory of the target person and the position information of pedestrian detection in the actual frame image, the target candidate sequence is corrected; The REID algorithm and historical position sequence are used to verify the corrected target candidate sequence; Based on the verified target candidate sequence, determine the correct predicted trajectory of the target person; The target candidate sequence is supplemented based on the correct predicted trajectory of the target person and the predicted bounding box of the target person.

6. The method for pedestrian recognition according to claim 1, characterized in that: The target candidate sequence based on the trackID loss period uses the REID algorithm and face recognition to identify the target person, including: Obtain a target query graph, determine a target person in the target candidate sequence based on the REID algorithm, face recognition, and similarity ranking, and determine a historical location sequence of the target person; Among them, the similarity between the target query graph and the determined target person meets a preset threshold and ranks highest.

7. An electronic device for pedestrian recognition, comprising a memory, a transceiver, and a processor; A memory for storing a computer program; a transceiver for transmitting and receiving data under the control of the processor; and a processor for executing the computer program in the memory and implementing the following steps: After determining that the tracking ID trackID of the target person is lost during the tracking process of the frame image in the target video, the starNet algorithm based on global information interaction is combined with the target candidate sequence and the historical position sequence to predict the trajectory of the target person; wherein, The trajectory prediction of the target person includes location information; the starNet algorithm is composed of a global temporal interactive computing network and a trajectory prediction network; Based on the predicted trajectory of the target person, determining whether the target person is blocked or leaves the boundary; If the target person is blocked, the target candidate sequence is corrected and supplemented based on the predicted trajectory of the target person; If the target person leaves the boundary, the videos with the same spatial relationship as the predicted start time are spliced ​​together. Based on the spliced ​​video frame images, the tracking identifier trackID of the pedestrian is obtained to generate a target candidate sequence; Based on the target candidate sequence during the trackID loss period, the target person is identified using the REID algorithm and face recognition; The target candidate sequence is obtained by obtaining the tracking identifier trackID of the pedestrian based on the frame image in the target video, including the trackID and bounding box of each pedestrian; The historical position sequence is obtained after the target person is determined based on the position information of the target person in a preset number of consecutive frame images in the target video.

8. A pedestrian recognition device, characterized in that: The device comprises: A trajectory prediction module is configured to predict the trajectory of a target person based on the starNet algorithm using global information interaction, after determining that the tracking identifier trackID of the target person is lost during the tracking process of the frame image in the target video, by combining the target candidate sequence and the historical position sequence; wherein the trajectory prediction of the target person includes position information; the starNet algorithm is composed of a global temporal interaction computing network and a trajectory prediction network; A status confirmation module is used to determine whether the target person is blocked or leaves the boundary based on the predicted trajectory of the target person; A correction and supplement module is used to correct and supplement the target candidate sequence based on the predicted trajectory of the target person if the target person is occluded; A generation module is used to stitch together the videos with the same spatial relationship as the prediction start time if the target person leaves the boundary, obtain the tracking identifier trackID of the pedestrian based on the stitched video frame images, and generate a target candidate sequence; An identification module is used to identify the target person based on the target candidate sequence during the trackID loss period using the REID algorithm and face recognition; The target candidate sequence is obtained by obtaining the tracking identifier trackID of the pedestrian based on the frame image in the target video, including the trackID and bounding box of each pedestrian; The historical position sequence is obtained after the target person is determined based on the position information of the target person in a preset number of consecutive frame images in the target video.

9. A processor-readable storage medium, characterized in that: The processor-readable storage medium stores a computer program, and the computer program is used to enable the processor to execute the pedestrian recognition method according to any one of claims 1 to 6.

Images