Method and system for early warning monitoring of pedestrian flow on moving sidewalk

Abstract

The present invention relates to the field of monitoring, and in particular to a method and system for monitoring pedestrian flow early warning on a moving walkway. A heat map, a center point offset and an apparent feature are obtained based on the video frame features of the moving walkway, and the updated apparent feature of the center point is obtained according to the apparent feature of the center point and the apparent feature after the center point is offset; a trajectory whose number of successful matches with the center point is less than a preset value is obtained, and the trajectory is matched with all center points in the current video frame by using the apparent feature, and if the match is successful and the trajectory is in an unconfirmed state, the trajectory is converted into a confirmed state; for the center point and trajectory that are not successfully matched, the center point is matched with a predicted trajectory based on the position and the apparent feature of the center point; whether to convert the unconfirmed state trajectory into a confirmed state trajectory is determined based on the number of successful matches of the unconfirmed state trajectory and the matching process; crowd flow statistical information is obtained according to the matching results of the center point and the trajectory, and a warning prompt is issued when the crowd flow reaches a warning value.

Description

Technical Field

[0001] The present invention relates to the field of monitoring, and in particular to a method and system for early warning monitoring of pedestrian flow on a moving pedestrian walkway. Background Art

[0002] With the acceleration of urbanization and the increase in the frequency of use of public transportation hubs, the high density of people has become more and more obvious. The emergence of moving walkways has greatly improved people's travel efficiency. Moving walkways, also known as moving walkways or moving walkways, are flat moving devices that are usually found in crowded places such as airports, subway stations, exhibition halls, and shopping malls to quickly transport people. Its function is similar to that of elevators and escalators, but the difference is that it is mainly used for horizontal transportation. Moreover, in some long-distance or special environments, moving walkways can also reduce people's physical exertion and improve the convenience of travel. Moving walkways are not only an important part of modern urban infrastructure construction, but also one of the core facilities for efficient crowd management.

[0003] Traditional methods of monitoring moving walkways often rely on manual management and simple sensor equipment, which makes it difficult to timely and accurately judge and warn of crowd conditions, especially during peak hours, when congestion, trampling and other safety hazards are prone to occur. Crowd flow monitoring methods based on target detection models, especially target detection algorithms in deep learning, can monitor and analyze the density and direction of crowds on moving walkways in real time, thereby realizing intelligent early warning and management. With the help of visual devices such as cameras, the system can automatically identify and track people, evaluate crowd conditions, and issue early warnings when congestion or overload may occur. However, existing crowd flow monitoring methods have low recognition rates in crowded places and are easily affected by occlusion. Summary of the invention

[0004] In order to accurately identify the flow of people on a moving walkway, the present invention provides a method for early warning and monitoring the flow of people on a moving walkway, the method comprising the following steps:

[0005] Obtain the captured moving walkway video, pass the video frame through the backbone network to obtain the video frame features, and obtain the heat map, center point offset and appearance features based on the video frame features. Determine the center point from the heat map, and obtain the updated appearance features of the center point based on the appearance features of the center point and the appearance features after the center point offset.

[0006] Acquire a trajectory whose number of successful matches with the center point is less than a preset value, and use the appearance feature to match the trajectory with all the center points in the current video frame. If the match is successful and the trajectory is in an unconfirmed state, convert the trajectory to a confirmed state;

[0007] For the center point and the trajectory that are not successfully matched, the center point and the predicted trajectory are matched based on the position and appearance features of the center point; if the center point is not successfully matched with the trajectory, a new unconfirmed trajectory is obtained according to the center point; and whether to convert the unconfirmed trajectory into a confirmed trajectory is determined based on the number of successful matches of the unconfirmed trajectory and the matching process;

[0008] The crowd flow statistics information is obtained based on the matching results of the center point and the trajectory, and an early warning is issued when the crowd flow reaches the warning value.

[0009] Preferably, the updated apparent features of the center point are obtained according to the apparent features of the center point and the apparent features of the center point after being offset, specifically:

[0010] Obtaining the apparent features of the center point, and obtaining the pixel point closest to the center point after the offset in the heat map and not the center point, and obtaining the apparent features of the nearest pixel point;

[0011] The apparent features of the center point and the apparent features of the nearest pixel point are fused as the updated apparent features of the center point.

[0012] Preferably, the matching of the trajectory with all center points in the current video frame is specifically as follows:

[0013] Calculate the similarity between the apparent features in the trajectory and the apparent features of each center point in the current video frame;

[0014] If the similarity is greater than the threshold and the distance between the last position in the trajectory and the center point meets the preset conditions, the trajectory and the center point are matched.

[0015] Preferably, the center point and the predicted trajectory are matched based on the position and appearance features of the center point, specifically:

[0016] The confirmed state trajectory and the center point are matched by cascade matching;

[0017] Get the unconfirmed trajectories and the confirmed trajectories that have not been matched successfully and put them into the list of trajectories matched by IOU, and put the unmatched center points into the list of center points matched by IOU;

[0018] The center point in the center point list and the trajectory in the trajectory list are matched using IOU matching.

[0019] Preferably, the determining whether to convert the unconfirmed track into the confirmed track based on the number of successful matches of the unconfirmed track and the matching process is specifically:

[0020] For each unconfirmed trajectory, accumulate the value corresponding to the unconfirmed trajectory in the IOU cost matrix each time the match is successful;

[0021] When the accumulated value or the number of successful matches meets the conditions, the unconfirmed state track is converted to the confirmed state.

[0022] On the other hand, the present invention also provides a pedestrian flow early warning monitoring system for a moving walkway, the system comprising the following modules:

[0023] The feature extraction module is used to obtain the captured moving walkway video, pass the video frame through the backbone network to obtain the video frame features, and obtain the heat map, center point offset and appearance features based on the video frame features, determine the center point from the heat map, and obtain the updated appearance features of the center point based on the appearance features of the center point and the appearance features after the center point offset;

[0024] An appearance matching module is used to obtain a trajectory whose number of successful matches with the center point is less than a preset value, and use the appearance features to match the trajectory with all the center points in the current video frame. If the match is successful and the trajectory is in an unconfirmed state, the trajectory is converted to a confirmed state;

[0025] The trajectory matching and updating module is used to match the center point with the predicted trajectory based on the position and appearance characteristics of the center point for the center point and trajectory that have not been successfully matched; if the center point is not successfully matched with the trajectory, a new unconfirmed trajectory is obtained according to the center point; and whether to convert the unconfirmed trajectory into a confirmed trajectory based on the number of successful matches of the unconfirmed trajectory and the matching process;

[0026] The early warning module is used to obtain the statistical information of the flow of people according to the matching results of the center point and the trajectory, and to issue an early warning when the flow of people reaches the warning value.

[0027] Preferably, the updated apparent features of the center point are obtained according to the apparent features of the center point and the apparent features of the center point after being offset, specifically:

[0028] Obtaining the apparent features of the center point, and obtaining the pixel point closest to the center point after the offset in the heat map and not the center point, and obtaining the apparent features of the nearest pixel point;

[0029] The apparent features of the center point and the apparent features of the nearest pixel point are fused as the updated apparent features of the center point.

[0030] Preferably, the matching of the trajectory with all center points in the current video frame is specifically as follows:

[0031] Calculate the similarity between the apparent features in the trajectory and the apparent features of each center point in the current video frame;

[0032] If the similarity is greater than the threshold and the distance between the last position in the trajectory and the center point meets the preset conditions, the trajectory and the center point are matched.

[0033] Preferably, the center point and the predicted trajectory are matched based on the position and appearance features of the center point, specifically:

[0034] The confirmed state trajectory and the center point are matched by cascade matching;

[0035] Get the unconfirmed trajectories and the confirmed trajectories that have not been matched successfully and put them into the list of trajectories matched by IOU, and put the unmatched center points into the list of center points matched by IOU;

[0036] The center point in the center point list and the trajectory in the trajectory list are matched using IOU matching.

[0037] Preferably, the determining whether to convert the unconfirmed track into the confirmed track based on the number of successful matches of the unconfirmed track and the matching process is specifically:

[0038] For each unconfirmed trajectory, accumulate the value corresponding to the unconfirmed trajectory in the IOU cost matrix each time the match is successful;

[0039] When the accumulated value or the number of successful matches meets the conditions, the unconfirmed state track is converted to the confirmed state.

[0040] In addition, the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method described above is implemented.

[0041] The present invention improves the appearance features corresponding to the center point of the heat map by shifting the center point. Specifically, the updated appearance features of the center point are obtained based on the apparent features of the center point and the apparent features after the center point is shifted. Since the center point is at a more accurate position after the center point is shifted, the appearance features of the center point obtained in the above manner can more accurately represent the features of the center point, thereby improving the accuracy of subsequent center point and trajectory matching. In addition, the present invention only uses appearance features for matching trajectories with a small number of Kalman filter predictions, thereby reducing the problem of object matching caused by inaccurate predictions when the Kalman filter is just starting to predict. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 This is a flow chart of Embodiment 1;

[0043] Figure 2 is the relationship between the center point and the offset in the heat map;

[0044] Figure 3 Schematic diagram of the trajectory and center point matching process;

[0045] Figure 4 This is a structural diagram of the second embodiment. DETAILED DESCRIPTION

[0046] In this article, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0047] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] In the first embodiment of the present invention, the present invention provides a method for monitoring pedestrian flow on a moving pedestrian path. Figure 1 As shown, the method comprises the following steps:

[0049] S1, obtain the captured video of the moving walkway, pass the video frame through the backbone network to obtain the video frame features, and obtain the heat map, center point offset and appearance features based on the video frame features, determine the center point from the heat map, and obtain the updated appearance features of the center point based on the appearance features of the center point and the appearance features after the center point offset;

[0050] The video of the moving sidewalk shot has multiple frames, and each video frame will be input into the backbone network, which includes but is not limited to ResNet, DLA-34, etc. The backbone network will output the features of the video frame. Since the video of the moving sidewalk shot may include other parts, in an optional embodiment, the moving sidewalk is first extracted for each video frame, and the image of the moving sidewalk part is used as the input of the backbone network. After obtaining the video frame features, a heatmap, a center offset, a detection box size, and an apparent feature (Re-ID feature) are obtained through different heads (such as a regression head). Among them, each pixel in the heatmap represents the possibility of the existence of a person, and the center point can be obtained through the heatmap, and the center point indicates that there is a pedestrian here; since the size of the heatmap is smaller than the size of the video frame input to the backbone network, a point in the heatmap is actually an area in the video frame, and the position of the pedestrian represented by the center point is not necessarily accurate. The center point offset is used to correct the position of the center point obtained by the heatmap. For example, if a center point is (4, 10) and the center point offset is (0.1, -0.3), the corrected center point is (4.1, 9.7). The appearance feature represents the characteristics of the target, that is, the pedestrian. The appearance feature can be used to determine whether the pedestrians in different frames are the same pedestrian. Each target (pedestrian) has a detected center point in the video frame. The appearance feature corresponding to the center point is the appearance information of the target, which represents the visual attributes of the target, such as color, shape, texture, etc.

[0051] Since the center point is not an accurate position, simply using the appearance features of the center point is prone to deviation. For example, if the center point is (4, 10), its corresponding appearance features are different from the appearance features of the corrected position (4.1, 9.7), which will cause errors in the subsequent judgment of whether they are the same pedestrian. In one embodiment, the updated appearance features of the center point are obtained based on the appearance features of the center point and the appearance features after the center point is offset, specifically:

[0052] Obtaining the apparent features of the center point, and obtaining the pixel point closest to the center point after the offset in the heat map and not the center point, and obtaining the apparent features of the nearest pixel point;

[0053] For each center point, there is a center point offset. The center point is a specific pixel in the heat map. Assuming the heat map is 128×128, the position of a center point in the heat map is (4, 10), and the position of the center point after offset is (4.1, 9.7). Figure 2The relationship between the center point and the offset in the heat map is shown. The pixels closest to the offset position of the center point are (4, 10) and (4, 9). Since (4, 10) is the center point, the pixel closest to the offset position of the center point and not the center point is (4, 9).

[0054] The apparent features of the center point and the apparent features of the nearest pixel are fused as the updated apparent features of the center point. The apparent features of the center point and the apparent features of the nearest pixel are fused, that is, the apparent features corresponding to the positions (4, 10) and (4, 9) in the heat map are fused. The fusion method includes but is not limited to splicing, fusion through a fully connected layer, etc. Unless otherwise specified, the apparent features of the center point described below are all updated apparent features. The apparent features are also stored in the trajectory. The apparent features in the trajectory are the updated apparent features of the center point after the trajectory and the center point are successfully matched.

[0055] S2, obtaining a trajectory whose number of successful matches with the center point is less than a preset value, and matching the trajectory with all the center points in the current video frame using the appearance feature, and if the match is successful and the trajectory is in an unconfirmed state, converting the trajectory to a confirmed state;

[0056] The status of the trajectory is divided into a confirmed state and an unconfirmed state. If an unconfirmed trajectory is not successfully matched with the center point, the trajectory will be deleted. If the match is successful, the position of the trajectory in the next frame will be obtained by using methods such as Kalman filtering. Prediction methods such as Kalman filtering will be continuously corrected with the number of successful matches. That is, the more successful matches there are, the higher the accuracy of the Kalman filter prediction. However, when the number of successful matches is small, such as the first prediction of a trajectory, the prediction accuracy of the Kalman filter is low. In order to prevent unsuccessful matching due to errors in the Kalman filter, the present invention first uses apparent features to match trajectories that have successfully matched the center point less than a preset value. This process will match the center point and trajectory with a large similarity in apparent features. In one embodiment, the trajectory is matched with all the center points in the current video frame, specifically:

[0057] Calculate the similarity between the apparent features in the trajectory and the apparent features of each center point in the current video frame;

[0058] If the similarity is greater than the threshold and the distance between the last position in the trajectory and the center point meets the preset conditions, the trajectory and the center point are matched.

[0059] After obtaining all the center points and the apparent features of the center points in the current video frame, all existing trajectories and the apparent features of the trajectories are obtained. If the apparent features of a center point and the apparent features of a trajectory with a number of successful matches less than a preset value are greater than a threshold, and the distance between the two meets the preset conditions, the trajectory and the center point are matched. Since the matching is based only on the apparent features, the errors in the Kalman filter in the previous predictions of the trajectory are avoided. This process requires a large threshold value. In one embodiment, the threshold value is greater than 0.9. Among them, the previous position in the trajectory is the actual position of the target corresponding to the center point in the previous frame, that is, the actual position of the pedestrian. In one embodiment, the actual position of the pedestrian is the position after the center point plus the offset, and is not the position predicted by the Kalman filter. Among them, the preset condition is the distance mapped to the heat map from the maximum moving distance of the pedestrian in the interval between two video frames. In a specific embodiment, the maximum moving distance of the pedestrian is the moving distance at the fastest moving speed of the pedestrian within the interval between two video frames plus the moving distance at the speed of the moving sidewalk. For example, the maximum moving speed of the pedestrian is 12m / s, the maximum moving speed of the moving sidewalk is 2m / s, and the interval between two video frames is 30ms, then the maximum moving distance of the pedestrian is 0.42m.

[0060] S3, for the center point and the trajectory that have not been successfully matched, match the center point with the predicted trajectory based on the position and appearance features of the center point; if the center point does not successfully match the trajectory, obtain a new unconfirmed trajectory based on the center point; determine whether to convert the unconfirmed trajectory into a confirmed trajectory based on the number of successful matches of the unconfirmed trajectory and the matching process;

[0061] For center points and trajectories that are not successfully matched, further matching is performed based on the trajectory and apparent features of the center point. Specifically, the confirmed trajectory and the center point are matched by cascade matching. The non-confirmed trajectory and the confirmed trajectory that are not successfully matched are obtained and put into the IOU matched trajectory list, and the unmatched center points are put into the IOU matched center point list. The center points in the center point list are matched with the trajectories in the trajectory list by IOU matching.

[0062] First, the confirmed trajectory and the center point are matched by cascade matching. After cascade matching, there may be two trajectories that are not matched to the center point, one is the confirmed trajectory and the other is the unconfirmed trajectory. There may also be center points that are not successfully matched. At this time, for these remaining trajectories and center points, the IOU matching method is used for further matching.

[0063] After IOU matching, the unconfirmed state that meets the conditions may be converted to the confirmed state. Figure 3In one embodiment, the determination of whether to convert a non-confirmed state track into a confirmed state track based on the number of successful matches of the non-confirmed state track and the matching process is as follows: if a non-confirmed state track is successfully matched more than three times in a row, the non-confirmed state track is converted into a confirmed state track.

[0064] In another embodiment, the determining whether to convert the unconfirmed track into the confirmed track based on the number of successful matches of the unconfirmed track and the matching process is specifically:

[0065] For each unconfirmed trajectory, accumulate the value corresponding to the unconfirmed trajectory in the IOU cost matrix each time the match is successful;

[0066] When the accumulated value or the number of successful matches meets the conditions, the unconfirmed state track is converted to the confirmed state.

[0067] If a non-confirmed trajectory is successfully matched to a center point, the value of the trajectory in the IOU cost matrix is ​​calculated and accumulated when the IOU match is successful; when the accumulated value or the number of successful matches meets the conditions, the non-confirmed state is converted to the confirmed state. For example, if the column addition value is greater than 1.8, or the number of consecutive successful matches is greater than 3, the non-confirmed state is converted to the confirmed state.

[0068] If a center point does not match any trajectory after the above steps, a trajectory with a non-confirmed state is initialized based on this center point; if a trajectory does not match any center point, if the trajectory is in a non-confirmed state, the trajectory will be deleted; if the trajectory is in a confirmed state, if the number of consecutive unmatched successful matches exceeds the maximum value, the trajectory will be deleted directly.

[0069] S4, obtains crowd flow statistics information based on the matching results of the center point and the trajectory, and issues a warning prompt when the crowd flow reaches a warning value.

[0070] After the center point and the trajectory are matched, the detection frame can be obtained according to the detection frame size corresponding to the center point, the offset of the center point, and the position of the center point. The detection frame is displayed on the video and the crowd flow information is counted. If the crowd flow reaches the warning value, a prompt will be given on the monitoring screen or through a buzzer.

[0071] In a second embodiment of the present invention, the present invention also provides a pedestrian flow early warning monitoring system for a moving pedestrian walkway, such as Figure 4 As shown, the system includes the following modules:

[0072] The feature extraction module is used to obtain the captured moving walkway video, pass the video frame through the backbone network to obtain the video frame features, and obtain the heat map, center point offset and appearance features based on the video frame features, determine the center point from the heat map, and obtain the updated appearance features of the center point based on the appearance features of the center point and the appearance features after the center point offset;

[0073] An appearance matching module is used to obtain a trajectory whose number of successful matches with the center point is less than a preset value, and use the appearance features to match the trajectory with all the center points in the current video frame. If the match is successful and the trajectory is in an unconfirmed state, the trajectory is converted to a confirmed state;

[0074] The trajectory matching and updating module is used to match the center point with the predicted trajectory based on the position and appearance characteristics of the center point for the center point and trajectory that have not been successfully matched; if the center point is not successfully matched with the trajectory, a new unconfirmed trajectory is obtained according to the center point; and whether to convert the unconfirmed trajectory into a confirmed trajectory based on the number of successful matches of the unconfirmed trajectory and the matching process;

[0075] The early warning module is used to obtain the statistical information of the flow of people according to the matching results of the center point and the trajectory, and to issue an early warning when the flow of people reaches the warning value.

[0076] Preferably, the updated apparent features of the center point are obtained according to the apparent features of the center point and the apparent features of the center point after being offset, specifically:

[0077] Obtaining the apparent features of the center point, and obtaining the pixel point closest to the center point after the offset in the heat map and not the center point, and obtaining the apparent features of the nearest pixel point;

[0078] The apparent features of the center point and the apparent features of the nearest pixel point are fused as the updated apparent features of the center point.

[0079] Preferably, the matching of the trajectory with all center points in the current video frame is specifically as follows:

[0080] Calculate the similarity between the apparent features in the trajectory and the apparent features of each center point in the current video frame;

[0081] If the similarity is greater than the threshold and the distance between the last position in the trajectory and the center point meets the preset conditions, the trajectory and the center point are matched.

[0082] Preferably, the center point and the predicted trajectory are matched based on the position and appearance features of the center point, specifically:

[0083] The confirmed state trajectory and the center point are matched by cascade matching;

[0084] Get the unconfirmed trajectories and the confirmed trajectories that have not been matched successfully and put them into the list of trajectories matched by IOU, and put the unmatched center points into the list of center points matched by IOU;

[0085] The center point in the center point list and the trajectory in the trajectory list are matched using IOU matching.

[0086] Preferably, the determining whether to convert the unconfirmed track into the confirmed track based on the number of successful matches of the unconfirmed track and the matching process is specifically:

[0087] For each unconfirmed trajectory, accumulate the value corresponding to the unconfirmed trajectory in the IOU cost matrix each time the match is successful;

[0088] When the accumulated value or the number of successful matches meets the conditions, the unconfirmed state track is converted to the confirmed state.

[0089] In a third embodiment of the present invention, the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method described in the first embodiment is implemented.

[0090] In a fourth embodiment of the present invention, the present invention further provides a computer device, which includes at least a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the method described in the first embodiment is implemented.

[0091] Through the description of the above implementation methods, technicians in this field can clearly understand that each implementation method can be implemented by adding a necessary general hardware platform, and of course can also be implemented by combining hardware and software. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a computer product, and the present invention can be in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0092] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it, and other embodiments may also be used. Although the present invention has been described in detail with reference to the aforementioned embodiments, ordinary technicians in this field should understand that they can still modify the technical solutions recorded in the aforementioned embodiments, or replace some of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for early warning and monitoring of pedestrian flow on a moving sidewalk, characterized in that: The method comprises the following steps: Obtain the captured moving walkway video, pass the video frame through the backbone network to obtain the video frame features, and obtain the heat map, center point offset and appearance features based on the video frame features. Determine the center point from the heat map, and obtain the updated appearance features of the center point based on the appearance features of the center point and the appearance features after the center point offset. Acquire a trajectory whose number of successful matches with the center point is less than a preset value, and use the appearance feature to match the trajectory with all the center points in the current video frame. If the match is successful and the trajectory is in an unconfirmed state, convert the trajectory to a confirmed state; For the center point and the trajectory that are not successfully matched, the center point and the predicted trajectory are matched based on the position and appearance features of the center point; if the center point is not successfully matched with the trajectory, a new unconfirmed trajectory is obtained according to the center point; and whether to convert the unconfirmed trajectory into a confirmed trajectory is determined based on the number of successful matches of the unconfirmed trajectory and the matching process; The crowd flow statistics information is obtained based on the matching results of the center point and the trajectory, and an early warning is issued when the crowd flow reaches the warning value.

2. The method according to claim 1, characterized in that The updated apparent features of the center point are obtained according to the apparent features of the center point and the apparent features of the center point after the center point is shifted, specifically: Obtaining the apparent features of the center point, and obtaining the pixel point closest to the center point after the offset in the heat map and not the center point, and obtaining the apparent features of the nearest pixel point; The apparent features of the center point and the apparent features of the nearest pixel point are fused as the updated apparent features of the center point.

3. The method according to claim 1, characterized in that The track is matched with all the center points in the current video frame, specifically: Calculate the similarity between the apparent features in the trajectory and the apparent features of each center point in the current video frame; If the similarity is greater than the threshold and the distance between the last position in the trajectory and the center point meets the preset conditions, the trajectory and the center point are matched.

4. The method according to claim 1, characterized in that The center point and the predicted trajectory are matched based on the position and appearance features of the center point, specifically: The confirmed state trajectory and the center point are matched by cascade matching; Get the unconfirmed trajectories and the confirmed trajectories that have not been matched successfully and put them into the list of trajectories matched by IOU, and put the unmatched center points into the list of center points matched by IOU; The center point in the center point list and the trajectory in the trajectory list are matched using IOU matching.

5. The method according to claim 1, characterized in that The determining whether to convert the unconfirmed track into the confirmed track based on the number of successful matches of the unconfirmed track and the matching process is specifically: For each unconfirmed trajectory, accumulate the value corresponding to the unconfirmed trajectory in the IOU cost matrix each time the match is successful; When the accumulated value or the number of successful matches meets the conditions, the unconfirmed state track is converted to the confirmed state.

6. A pedestrian flow early warning monitoring system for moving walkways, characterized in that: The system includes the following modules: The feature extraction module is used to obtain the captured moving walkway video, pass the video frame through the backbone network to obtain the video frame features, and obtain the heat map, center point offset and appearance features based on the video frame features, determine the center point from the heat map, and obtain the updated appearance features of the center point based on the appearance features of the center point and the appearance features after the center point offset; An appearance matching module is used to obtain a trajectory whose number of successful matches with the center point is less than a preset value, and use the appearance features to match the trajectory with all the center points in the current video frame. If the match is successful and the trajectory is in an unconfirmed state, the trajectory is converted to a confirmed state; The trajectory matching and updating module is used to match the center point with the predicted trajectory based on the position and appearance characteristics of the center point for the center point and trajectory that have not been successfully matched; if the center point is not successfully matched with the trajectory, a new unconfirmed trajectory is obtained according to the center point; and whether to convert the unconfirmed trajectory into a confirmed trajectory based on the number of successful matches of the unconfirmed trajectory and the matching process; The early warning module is used to obtain the statistical information of the flow of people according to the matching results of the center point and the trajectory, and to issue an early warning when the flow of people reaches the warning value.

7. The system according to claim 6, characterized in that The updated apparent features of the center point are obtained according to the apparent features of the center point and the apparent features of the center point after the center point is shifted, specifically: Obtaining the apparent features of the center point, and obtaining the pixel point closest to the center point after the offset in the heat map and not the center point, and obtaining the apparent features of the nearest pixel point; The apparent features of the center point and the apparent features of the nearest pixel point are fused as the updated apparent features of the center point.

8. The system according to claim 6, characterized in that The track is matched with all the center points in the current video frame, specifically: Calculate the similarity between the apparent features in the trajectory and the apparent features of each center point in the current video frame; If the similarity is greater than the threshold and the distance between the last position in the trajectory and the center point meets the preset conditions, the trajectory and the center point are matched.

9. The system according to claim 6, characterized in that The determining whether to convert the unconfirmed track into the confirmed track based on the number of successful matches of the unconfirmed track and the matching process is specifically: For each unconfirmed trajectory, accumulate the value corresponding to the unconfirmed trajectory in the IOU cost matrix each time the match is successful; When the accumulated value or the number of successful matches meets the conditions, the unconfirmed state track is converted to the confirmed state.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the computer program implements the method according to any one of claims 1 to 5.

Images