A method, system and storage medium for identifying obstacles on a moving walkway

Abstract

The present invention relates to the field of obstacle recognition, and in particular to a method, system and storage medium for obstacle recognition of an automatic sidewalk, which recognize a target object in a current video frame, obtain a predicted trajectory in the current video frame, calculate the previous IOU score of an unconfirmed trajectory and a target object, calculate the appearance feature change of the unconfirmed trajectory, and divide the unconfirmed trajectory into two subsets based on the IOU score and the appearance change; cascade matching is performed on the target object, the confirmed trajectory and the first subset to obtain a matching result of the trajectory and the target object, and if the unconfirmed trajectory and the object match, the unconfirmed trajectory is converted into a confirmed trajectory; IOU matching is performed on the second subset and the unmatched trajectory and the unmatched target object in the cascade matching to complete target tracking of people and objects in the automatic sidewalk; the relative distance and movement traces of people and objects in the video are calculated, and obstacles of the automatic sidewalk are recognized based on the relative distance and the movement traces.

Description

Technical Field

[0001] The present invention relates to the field of obstacle recognition, and in particular to a method, system and storage medium for identifying obstacles on a moving walkway. Background Art

[0002] In large areas such as airports, train stations, and tourist attractions, walking will not only consume a lot of physical energy, but also bring a bad experience to customers. Moving walkways are similar to escalators, but they are generally horizontal or have a slight slope. Moving walkways can not only speed up people's movement in large spaces, but also reduce walking time in places such as airports, train stations, and shopping malls where long distances need to be walked, allowing passengers, shoppers, etc. to reach their destinations faster. Moreover, moving walkways provide greater convenience for people with limited mobility, especially the elderly, pregnant women, and the physically disabled, who can more easily pass through large places.

[0003] Compared with elevators and escalators, moving walkways have longer distances, more people riding at the same time, and higher safety requirements. If there is luggage, debris or other objects on the sidewalk that are not removed in time, it may cause equipment failure, downtime or even accidents. Especially in places such as airports and stations, when obstacles are detected, the system can automatically slow down, stop or issue a warning to prevent further damage to the equipment, thereby reducing maintenance and downtime. It is very necessary. However, there are many types of obstacles that may appear on moving walkways, including small items such as luggage, beverage cups, etc., and there may also be personal belongings left behind by pedestrians. Obstacles vary in size, shape and material, which makes accurate identification difficult. Summary of the invention

[0004] In order to solve the above problems, the present invention provides a method for identifying obstacles on a moving walkway, the method comprising the following steps:

[0005] Collect video of the moving walkway in real time, identify the target object in the current video frame, obtain the predicted trajectory in the current video frame, calculate the previous IOU score of the unconfirmed trajectory and the target object, and calculate the change of the appearance features of the unconfirmed trajectory, and divide the unconfirmed trajectory into two subsets based on the IOU score and the appearance change;

[0006] The target object, the confirmed trajectory, and the first subset are cascade matched to obtain the trajectory and target object matching result. If the unconfirmed trajectory and the object match, the unconfirmed trajectory is converted into a confirmed trajectory. The second subset and the unmatched trajectory in the cascade matching are matched with the unmatched target object to complete the target tracking of people and objects in the moving sidewalk.

[0007] The relative distance and movement traces of people and objects in the video are calculated, and obstacles on the moving walkway are identified based on the relative distance and movement traces.

[0008] Preferably, the calculation of the change of the appearance features of the unconfirmed trajectory is specifically as follows:

[0009] All feature vectors extracted by using the ReID network in the unconfirmed trajectory are obtained, the average value of the Euclidean distances of all feature vectors is calculated, and the average value is used as the appearance feature change.

[0010] Preferably, the non-confirmed state trajectories are divided into two subsets based on the IOU score and the appearance change, specifically:

[0011] Obtain the video frame closest to the current video frame after cascade matching, the average IOU of the confirmed track and the target object that are successfully cascade matched, and obtain the appearance feature vector of the confirmed track that is successfully cascade matched, calculate the Euclidean distance between the two appearance feature vectors closest to the current video frame among all the appearance feature vectors of the confirmed track that are successfully cascade matched, and then calculate the average Euclidean distance of all the confirmed tracks that are successfully cascade matched;

[0012] If the IOU score is greater than the weighted IOU average and the appearance feature change is less than the weighted Euclidean distance average, the unconfirmed trajectory is added to the first subset, otherwise it is added to the second subset.

[0013] Preferably, the method of identifying obstacles on the moving walkway based on relative distance and movement traces is specifically:

[0014] Establish the relationship between people and objects based on their relative distance and movement traces, and mark objects that are not associated with people as suspected obstacles;

[0015] The obstacle is determined based on the movement traces of the suspected obstacle and the changes in the relationship between the suspected obstacle and the person.

[0016] Preferably, the obstacle is determined based on the movement trace of the suspected obstacle and the change of the association between the suspected obstacle and the person, specifically:

[0017] If the trajectory of the suspected obstacle is a confirmed trajectory and the duration of the confirmed trajectory is greater than the threshold, and it has not established an association relationship with a person or has established an association relationship with multiple people, the suspected obstacle is determined to be an obstacle.

[0018] Preferably, the establishing of the association relationship between the person and the object based on the relative distance and movement traces between the person and the object is specifically as follows:

[0019] For people and objects whose relative distance is less than the distance threshold, the similarity of the trajectories is further calculated;

[0020] The relevance score of the person and the object is calculated according to the relative distance and the similarity of the trajectory, and the person and the object with the largest relevance score are associated.

[0021] Preferably, the similarity of the calculated trajectories is specifically:

[0022] The target frame center points of the same person and the same object in different video frames are obtained, and the trajectory changes are calculated according to the coordinates of the target frame center points of the person and the object in different video frames. The similarity of the trajectories is calculated based on the trajectory changes.

[0023] Preferably, the calculation of the correlation score between the person and the object based on the relative distance and the similarity of the trajectory is specifically as follows:

[0024] Based on the formula The correlation score is calculated, where , Respectively represent the weights of relative distance and trajectory similarity, represents the relative distance, Indicates the similarity of the trajectories.

[0025] In another aspect, the present invention provides a moving walkway obstacle recognition system, the system comprising the following modules:

[0026] The trajectory division module is used to collect the video of the moving walkway in real time, identify the target object in the current video frame, obtain the predicted trajectory in the current video frame, calculate the previous IOU score of the unconfirmed trajectory and the target object, and calculate the change of the appearance features of the unconfirmed trajectory, and divide the unconfirmed trajectory into two subsets based on the IOU score and the appearance change;

[0027] The target tracking module is used to perform cascade matching on the target object, the confirmed trajectory and the first subset to obtain a matching result of the trajectory and the target object. If the unconfirmed trajectory and the object match, the unconfirmed trajectory is converted into a confirmed trajectory; the second subset and the unmatched trajectory in the cascade matching are matched with the unmatched target object to complete the target tracking of people and objects in the moving walkway;

[0028] The obstacle recognition module is used to calculate the relative distance and movement traces of people and objects in the video, and identify obstacles on the moving walkway based on the relative distance and movement traces.

[0029] Preferably, the calculation of the change of the appearance features of the unconfirmed trajectory is specifically as follows:

[0030] All feature vectors extracted by using the ReID network in the unconfirmed trajectory are obtained, the average value of the Euclidean distances of all feature vectors is calculated, and the average value is used as the appearance feature change.

[0031] Preferably, the non-confirmed state trajectories are divided into two subsets based on the IOU score and the appearance change, specifically:

[0032] Obtain the video frame closest to the current video frame after cascade matching, the average IOU of the confirmed track and the target object that are successfully cascade matched, and obtain the appearance feature vector of the confirmed track that is successfully cascade matched, calculate the Euclidean distance between the two appearance feature vectors closest to the current video frame among all the appearance feature vectors of the confirmed track that are successfully cascade matched, and then calculate the average Euclidean distance of all the confirmed tracks that are successfully cascade matched;

[0033] If the IOU score is greater than the weighted IOU average and the appearance feature change is less than the weighted Euclidean distance average, the unconfirmed trajectory is added to the first subset, otherwise it is added to the second subset.

[0034] Preferably, the method of identifying obstacles on the moving walkway based on relative distance and movement traces is specifically:

[0035] Establish the relationship between people and objects based on their relative distance and movement traces, and mark objects that are not associated with people as suspected obstacles;

[0036] The obstacle is determined based on the movement traces of the suspected obstacle and the changes in the relationship between the suspected obstacle and the person.

[0037] Preferably, the obstacle is determined based on the movement trace of the suspected obstacle and the change of the association between the suspected obstacle and the person, specifically:

[0038] If the trajectory of the suspected obstacle is a confirmed trajectory and the duration of the confirmed trajectory is greater than the threshold, and it has not established an association relationship with a person or has established an association relationship with multiple people, the suspected obstacle is determined to be an obstacle.

[0039] Preferably, the establishing of the association relationship between the person and the object based on the relative distance and movement traces between the person and the object is specifically as follows:

[0040] For people and objects whose relative distance is less than the distance threshold, the similarity of the trajectories is further calculated;

[0041] The relevance score of the person and the object is calculated according to the relative distance and the similarity of the trajectory, and the person and the object with the largest relevance score are associated.

[0042] Preferably, the similarity of the calculated trajectories is specifically:

[0043] The target frame center points of the same person and the same object in different video frames are obtained, and the trajectory changes are calculated according to the coordinates of the target frame center points of the person and the object in different video frames. The similarity of the trajectories is calculated based on the trajectory changes.

[0044] Preferably, the calculation of the correlation score between the person and the object based on the relative distance and the similarity of the trajectory is specifically as follows:

[0045] Based on the formula The correlation score is calculated, where , Respectively represent the weights of relative distance and trajectory similarity, represents the relative distance, represents the trajectory similarity.

[0046] In addition, the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method described above is implemented.

[0047] In view of the problem of complex personnel and many objects in obstacle identification on moving walkways, the present invention includes relatively stable non-confirmed state trajectories into the matching process by screening the IOU score and appearance feature changes, which can improve the tracking continuity and accuracy of the system in complex scenes, reduce the risk of target loss or drift, and thus facilitate subsequent obstacle identification to provide more accurate results. In addition, the present invention also uses the relative distance and movement traces of people and objects in the video to identify obstacles on moving walkways, reducing the impact of passengers' carrying items on obstacle identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 This is a flow chart of Embodiment 1;

[0049] Figure 2 Schematic diagram of the target object and trajectory matching process;

[0050] Figure 3 This is a structural diagram of the second embodiment. DETAILED DESCRIPTION

[0051] In this article, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0052] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] In a first embodiment of the present invention, Figure 1 As shown, the present invention provides a method for identifying obstacles on a moving walkway, the method comprising the following steps:

[0054] S1, real-time video of the moving walkway is collected, the target object in the current video frame is identified, and the predicted trajectory in the current video frame is obtained, the IOU score of the previous unconfirmed trajectory and the target object is calculated, and the appearance feature change of the unconfirmed trajectory is calculated, and the unconfirmed trajectory is divided into two subsets based on the IOU score and the appearance change;

[0055] When DeepSort is used to track the moving walkway, the Mahalanobis distance and the apparent feature cosine distance are used in the cascade matching, which can more accurately match the track (Tracks) and the target object (Detections). The unconfirmed track will not participate in the cascade matching, but only use IOU matching. It will be converted to the confirmed state only after the unconfirmed track is successfully matched for several times. However, the number of successful matches is set too small. Due to premature confirmation, the situation of mismatch may increase, and too late confirmation may increase the possibility of target loss. The present invention sets a larger number of successful matches, and then classifies the unconfirmed tracks, and adds the unconfirmed tracks with a high IOU score with the target object and little change in appearance features to the cascade matching. The high IOU score indicates that the spatial position of the track is highly consistent with the target, and the small change in appearance features means that the appearance features of the target are stable. The combination of these two can ensure that the selected track has a high degree of credibility, filter out unstable or noisy unconfirmed tracks, and reduce the risk of mismatch.

[0056] The newly generated trajectory generates a new trajectory after passing through the Kalman filter. The new trajectory is in an unconfirmed state and will directly perform IOU matching. After matching, there is an IOU score. The higher the score, the more accurate the Kalman filter prediction is and the more stable the trajectory is. The appearance feature change is calculated through the appearance feature vector in the trajectory. In one embodiment, all feature vectors extracted by the ReID network in the unconfirmed trajectory are obtained, and the average value of the Euclidean distance of all feature vectors is calculated, and the average value is used as the appearance feature change. If there are two feature vectors extracted by the ReID network for the trajectory, the Euclidean distance of the two feature vectors is calculated. In another embodiment, the average value of the cosine distance of the feature vectors is calculated, and the average value is used as the appearance feature change. The above-mentioned appearance feature change is the similarity of the appearance descriptors of the trajectory, rather than the similarity of the appearance features of the trajectory and the target object.

[0057] In one embodiment, the unconfirmed state trajectories are divided into two subsets based on the IOU score and the appearance change, specifically:

[0058] Obtain the video frame closest to the current video frame after cascade matching, the average IOU of the confirmed track and the target object that are successfully cascade matched, and obtain the appearance feature vector of the confirmed track that is successfully cascade matched, calculate the Euclidean distance between the two appearance feature vectors closest to the current video frame among all the appearance feature vectors of the confirmed track that are successfully cascade matched, and then calculate the average Euclidean distance of all the confirmed tracks that are successfully cascade matched;

[0059] Cascade matching and IOU matching are performed on each frame. For the frame closest to the current video frame, the average IOU of the successfully matched confirmed trajectories and target objects of the frame closest to the current one after cascade matching is calculated. If there are 10 pairs of successfully matched confirmed trajectories and target objects in the frame, the average IOU of these 10 pairs of successfully matched confirmed trajectories and target objects is calculated.

[0060] Similarly, for each confirmed state trajectory with successful cascade matching, obtain the average Euclidean distance of the appearance feature vector. If a confirmed state trajectory with successful cascade matching has 8 appearance feature vectors, and these 8 appearance feature vectors come from different video frames, find the two appearance feature vectors closest to the current video frame, calculate the Euclidean distance of the appearance feature vector, and then calculate the average Euclidean distance of all confirmed state trajectories with successful cascade matching after the cascade matching with the frame closest to the current frame.

[0061] If the IOU score is greater than the weighted IOU average and the appearance feature change is less than the weighted Euclidean distance average, the unconfirmed trajectory is added to the first subset, otherwise it is added to the second subset.

[0062] For non-confirmed tracks with large IOU and little appearance change, they are added to the first subset, otherwise they are added to the second subset.

[0063] S2, cascade matching is performed on the target object, the confirmed trajectory, and the first subset to obtain a matching result of the trajectory and the target object. If the unconfirmed trajectory matches the object, the unconfirmed trajectory is converted into a confirmed trajectory; the second subset and the unmatched trajectory in the cascade matching are matched with the unmatched target object by IOU matching to complete the target tracking of people and objects in the moving walkway;

[0064] DeepSORT multi-target tracking includes two matching processes, one is cascade matching and the other is IOU matching. The present invention uses the trajectories in the first subset and the confirmed trajectories as cascade matching trajectories, and uses the trajectories that failed the cascade matching and the trajectories in the second subset as IOU matching trajectories. Figure 2 As shown in the figure, by adding the stable non-confirmation state to the cascade matching in advance, the matching accuracy is improved. When the matching is completed, the people and objects in the moving sidewalk will be marked on the video and have corresponding numbers. The target tracking of people and objects in the moving sidewalk is realized, and the target tracking includes the tracking of moving people and objects, as well as the tracking of non-moving people and objects.

[0065] S3 calculates the relative distance and movement traces of people and objects in the video, and identifies obstacles on the moving walkway based on the relative distance and movement traces.

[0066] In a moving walkway, people (passengers) and objects move with the movement of the moving walkway, wherein objects may not move, for example, obstacles at the entrance and exit of the moving walkway. If the objects are objects carried by people, the objects are close to the people and will move with the movement of the people. If the obstacles are on the moving walkway, the distance between the objects and the people is far, and the similarity of the movement traces and the people is low. First, the association relationship between people and objects is established. For people and objects whose relative distance is less than the distance threshold, the similarity of the trajectories is further calculated; the association score of people and objects is calculated according to the relative distance and the similarity of the trajectories, and the people and objects with the largest association score are associated. If the relative distance between people and objects is less than the distance threshold, it means that the distance between people and objects is close. The distance threshold is determined according to the height of the target frame of the person, for example, the distance threshold is half the height of the target frame of the person. The similarity of the trajectories of people and objects is further calculated. The similarity of the trajectories is calculated by using the center point of the target frame of the same person and the center point of the target frame of the same object in different video frames. The trajectory change is calculated according to the coordinates of the center point of the target frame of people and objects in different video frames, and the similarity of the trajectory is calculated based on the trajectory change. For example, in a certain video frame, the distance between a person and an object is less than a distance threshold, and then the similarity between the person's motion trajectory and the object's motion trajectory is calculated, and a correlation score is calculated based on the relative distance and the similarity of the motion trajectory, and the object and the person with the largest correlation score are selected for association. In a preferred embodiment, the correlation score between the person and the object is calculated based on the relative distance and the similarity of the trajectory, specifically: based on the formula The correlation score is calculated, where , Respectively represent the weights of relative distance and trajectory similarity, represents the relative distance, represents the trajectory similarity.

[0067] If an object is far away from a person or has a low association score, the object is identified as a suspected obstacle. If the trajectory of the suspected obstacle is a confirmed trajectory and the duration of the confirmed trajectory is greater than the threshold, and it has not established an association relationship with a person or has established an association relationship with multiple people, the suspected obstacle is determined to be an obstacle.

[0068] In another embodiment, the calculating the relative distance and movement traces of people and objects in the video, and identifying obstacles of the moving walkway based on the relative distance and movement traces, is specifically:

[0069] Calculate and obtain objects whose relative distance between people and objects in each frame is greater than the distance threshold and put them into the suspected obstacle set, and put objects whose movement trace similarity between people and objects is less than the threshold into the suspected obstacle set, count the frequency of the same object appearing in the suspected obstacle set over a period of time, and determine the obstacles of the moving walkway based on the frequency. Among them, the suspected obstacle set is a set in the form of a list (List), which allows repeated elements, and after a period of time, the suspected obstacle set is cleared and re-counted.

[0070] In a second embodiment of the present invention, Figure 3 As shown, the present invention provides a moving walkway obstacle recognition system, the system comprising the following modules:

[0071] The trajectory division module is used to collect the video of the moving walkway in real time, identify the target object in the current video frame, obtain the predicted trajectory in the current video frame, calculate the previous IOU score of the unconfirmed trajectory and the target object, and calculate the change of the appearance features of the unconfirmed trajectory, and divide the unconfirmed trajectory into two subsets based on the IOU score and the appearance change;

[0072] The target tracking module is used to perform cascade matching on the target object, the confirmed trajectory and the first subset to obtain a matching result of the trajectory and the target object. If the unconfirmed trajectory and the object match, the unconfirmed trajectory is converted into a confirmed trajectory; the second subset and the unmatched trajectory in the cascade matching are matched with the unmatched target object to complete the target tracking of people and objects in the moving walkway;

[0073] The obstacle recognition module is used to calculate the relative distance and movement traces of people and objects in the video, and identify obstacles on the moving walkway based on the relative distance and movement traces.

[0074] Preferably, the calculation of the change of the appearance features of the unconfirmed trajectory is specifically as follows:

[0075] All feature vectors extracted by using the ReID network in the unconfirmed trajectory are obtained, the average value of the Euclidean distances of all feature vectors is calculated, and the average value is used as the appearance feature change.

[0076] Preferably, the non-confirmed state trajectories are divided into two subsets based on the IOU score and the appearance change, specifically:

[0077] Obtain the video frame closest to the current video frame after cascade matching, the average IOU of the confirmed track and the target object that are successfully cascade matched, and obtain the appearance feature vector of the confirmed track that is successfully cascade matched, calculate the Euclidean distance between the two appearance feature vectors closest to the current video frame among all the appearance feature vectors of the confirmed track that are successfully cascade matched, and then calculate the average Euclidean distance of all the confirmed tracks that are successfully cascade matched;

[0078] If the IOU score is greater than the weighted IOU average and the appearance feature change is less than the weighted Euclidean distance average, the unconfirmed trajectory is added to the first subset, otherwise it is added to the second subset.

[0079] Preferably, the method of identifying obstacles on the moving walkway based on relative distance and movement traces is specifically:

[0080] Establish the relationship between people and objects based on their relative distance and movement traces, and mark objects that are not associated with people as suspected obstacles;

[0081] The obstacle is determined based on the movement traces of the suspected obstacle and the changes in the relationship between the suspected obstacle and the person.

[0082] Preferably, the obstacle is determined based on the movement trace of the suspected obstacle and the change of the association between the suspected obstacle and the person, specifically:

[0083] If the trajectory of the suspected obstacle is a confirmed trajectory and the duration of the confirmed trajectory is greater than the threshold, and it has not established an association relationship with a person or has established an association relationship with multiple people, the suspected obstacle is determined to be an obstacle.

[0084] Preferably, the establishing of the association relationship between the person and the object based on the relative distance and movement traces between the person and the object is specifically as follows:

[0085] For people and objects whose relative distance is less than the distance threshold, the similarity of the trajectories is further calculated;

[0086] The relevance score of the person and the object is calculated according to the relative distance and the similarity of the trajectory, and the person and the object with the largest relevance score are associated.

[0087] Preferably, the similarity of the calculated trajectories is specifically:

[0088] The target frame center points of the same person and the same object in different video frames are obtained, and the trajectory changes are calculated according to the coordinates of the target frame center points of the person and the object in different video frames. The similarity of the trajectories is calculated based on the trajectory changes.

[0089] Preferably, the calculation of the correlation score between a person and an object according to the relative distance and the trajectory similarity is specifically as follows:

[0090] Based on the formula The correlation score is calculated, where , Respectively represent the weights of relative distance and trajectory similarity, represents the relative distance, represents the trajectory similarity.

[0091] In a third embodiment of the present invention, the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method described in the first embodiment is implemented.

[0092] In a fourth embodiment of the present invention, the present invention further provides a computer device, which includes at least a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the method described in the first embodiment is implemented.

[0093] Through the description of the above implementation methods, technicians in this field can clearly understand that each implementation method can be implemented by adding a necessary general hardware platform, and of course can also be implemented by combining hardware and software. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a computer product, and the present invention can be in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0094] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it, and other embodiments may also be used. Although the present invention has been described in detail with reference to the aforementioned embodiments, ordinary technicians in this field should understand that they can still modify the technical solutions recorded in the aforementioned embodiments, or replace some of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for identifying obstacles on a moving walkway, characterized in that: The method comprises the following steps: Collect video of the moving walkway in real time, identify the target object in the current video frame, obtain the predicted trajectory in the current video frame, calculate the previous IOU score of the unconfirmed trajectory and the target object, and calculate the change of the appearance features of the unconfirmed trajectory, and divide the unconfirmed trajectory into two subsets based on the IOU score and the appearance change; The target object, the confirmed trajectory, and the first subset are cascade matched to obtain the trajectory and target object matching result. If the unconfirmed trajectory and the object match, the unconfirmed trajectory is converted into a confirmed trajectory. The second subset and the unmatched trajectory in the cascade matching are matched with the unmatched target object to complete the target tracking of people and objects in the moving sidewalk. Calculate the relative distance and movement traces of people and objects in the video, and identify obstacles on the moving walkway based on the relative distance and movement traces; The non-confirmed state trajectories are divided into two subsets based on the IOU score and the appearance change, specifically: Obtain the video frame closest to the current video frame after cascade matching, the average IOU of the confirmed track and the target object that are successfully cascade matched, and obtain the appearance feature vector of the confirmed track that is successfully cascade matched, calculate the Euclidean distance between the two appearance feature vectors closest to the current video frame among all the appearance feature vectors of the confirmed track that are successfully cascade matched, and then calculate the average Euclidean distance of all the confirmed tracks that are successfully cascade matched; If the IOU score is greater than the weighted IOU average and the appearance feature change is less than the weighted Euclidean distance average, the unconfirmed trajectory is added to the first subset, otherwise it is added to the second subset.

2. The method according to claim 1, characterized in that The calculation of the change in appearance features of the unconfirmed trajectory is specifically as follows: All feature vectors extracted by using the ReID network in the unconfirmed trajectory are obtained, the average value of the Euclidean distances of all feature vectors is calculated, and the average value is used as the appearance feature change.

3. The method according to claim 1, characterized in that The obstacle identification of the moving walkway based on relative distance and movement traces is specifically as follows: Establish the relationship between people and objects based on their relative distance and movement traces, and mark objects that are not associated with people as suspected obstacles; The obstacle is determined based on the movement traces of the suspected obstacle and the changes in the relationship between the suspected obstacle and the person.

4. The method according to claim 3, characterized in that The obstacle is determined based on the movement trace of the suspected obstacle and the change of the association between the suspected obstacle and the person, specifically: If the trajectory of the suspected obstacle is a confirmed trajectory and the duration of the confirmed trajectory is greater than the threshold, and it has not established an association relationship with a person or has established an association relationship with multiple people, the suspected obstacle is determined to be an obstacle.

5. The method according to claim 3, characterized in that The establishment of the association relationship between people and objects based on the relative distance and movement traces of people and objects is specifically as follows: For people and objects whose relative distance is less than the distance threshold, the similarity of the trajectories is further calculated; The relevance score of the person and the object is calculated according to the relative distance and the similarity of the trajectory, and the person and the object with the largest relevance score are associated.

6. The method according to claim 5, characterized in that The similarity of the calculation trajectory is specifically: The target frame center points of the same person and the same object in different video frames are obtained, and the trajectory changes are calculated according to the coordinates of the target frame center points of the person and the object in different video frames. The similarity of the trajectories is calculated based on the trajectory changes.

7. The method according to claim 5, characterized in that The calculation of the correlation score between the person and the object according to the relative distance and the similarity of the trajectory is specifically as follows: Based on the formula The correlation score is calculated, where , They represent the weights of relative distance and trajectory similarity respectively, d represents the relative distance, Indicates the similarity of the trajectories.

8. A moving walkway obstacle recognition system, characterized in that: The system includes the following modules: The trajectory division module is used to collect the video of the moving walkway in real time, identify the target object in the current video frame, obtain the predicted trajectory in the current video frame, calculate the previous IOU score of the unconfirmed trajectory and the target object, and calculate the change of the appearance features of the unconfirmed trajectory, and divide the unconfirmed trajectory into two subsets based on the IOU score and the appearance change; The target tracking module is used to perform cascade matching on the target object, the confirmed trajectory and the first subset to obtain a matching result of the trajectory and the target object. If the unconfirmed trajectory and the object match, the unconfirmed trajectory is converted into a confirmed trajectory; the second subset and the unmatched trajectory in the cascade matching are matched with the unmatched target object to complete the target tracking of people and objects in the moving walkway; The obstacle recognition module is used to calculate the relative distance and movement traces of people and objects in the video, and recognize obstacles on the moving walkway based on the relative distance and movement traces; The non-confirmed state trajectories are divided into two subsets based on the IOU score and the appearance change, specifically: Obtain the video frame closest to the current video frame after cascade matching, the average IOU of the confirmed track and the target object that are successfully cascade matched, and obtain the appearance feature vector of the confirmed track that is successfully cascade matched, calculate the Euclidean distance between the two appearance feature vectors closest to the current video frame among all the appearance feature vectors of the confirmed track that are successfully cascade matched, and then calculate the average Euclidean distance of all the confirmed tracks that are successfully cascade matched; If the IOU score is greater than the weighted IOU average and the appearance feature change is less than the weighted Euclidean distance average, the unconfirmed trajectory is added to the first subset, otherwise it is added to the second subset.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the computer program implements the method according to any one of claims 1 to 7.

Images