A lightweight personnel crossing line detection method and device suitable for edge computing power limited devices

Abstract

The present application relates to the technical field of computer vision and edge computing. A lightweight personnel crossing line detection method and device suitable for edge computing devices with limited computing power are disclosed. The method aims to solve the problems of delay and concurrency bottleneck caused by the dependence on cloud transmission or edge overload model in the prior art, and the high false alarm rate caused by the traditional single-frame geometric determination being easily disturbed by target occlusion and posture change, which makes it difficult to achieve stable detection on the edge. The method comprises: collecting and decoding video streams to obtain continuous single-frame images; inputting a lightweight target detection model to output personnel bounding boxes; performing multi-target tracking based on the bounding box set and assigning a unique identity; calculating the crossing judgment point coordinates based on the top-left corner coordinates, width and height of the bounding box and a preset coefficient; maintaining the cross-frame historical trajectory, and determining the crossing by judging the vector cross product sign of the adjacent frame judgment points relative to the end points of the preset detection line and the line segment intersection condition; triggering an alarm and recording the crossing event, and starting a repeated alarm suppression period for the crossing identifier.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and edge computing technology, and more specifically, to a lightweight method and apparatus for detecting people crossing the line that is adapted to devices with limited edge computing power. Background Technology

[0002] With the rapid development of computer vision and intelligent security technologies, personnel crossing detection (or boundary detection) has become a core basic function in scenarios such as video surveillance, park security, traffic intersections, and industrial safety production. Currently, the technical approaches to achieving personnel crossing detection are mainly divided into three categories: centralized cloud processing, edge model deployment, and traditional computer vision algorithms. However, existing technologies all have varying degrees of technical defects and bottlenecks in practical applications.

[0003] The first category is cloud-based centralized personnel crossing detection solutions (such as the solution disclosed in patent document CN112927306A). This solution, as the mainstream implementation method for traditional security video analysis, mainly relies on front-end acquisition devices to upload the entire video stream to cloud computing nodes via public or private networks. Large-scale deep learning object detection models (such as Faster R-CNN, YOLOv5x, etc.) deployed in the cloud then complete personnel target identification and crossing behavior determination. However, this solution suffers from high end-to-end latency and weak data security. Specifically, the uplink transmission of the video stream is highly dependent on the network link; even in stable scenarios, the end-to-end detection latency is usually higher than 200ms. When network fluctuations or insufficient bandwidth occur, detection frame loss and inference interruptions are highly likely. Furthermore, the transmission of raw video data across devices and networks poses a high risk of data leakage.

[0004] The second category is the direct deployment solution of heavy-duty models at the edge (such as edge deployment projects based on open-source models like YOLOv5-Lite). This solution deploys a general object detection model directly on edge IPCs (network cameras), edge gateways, and other end-side devices after simple quantization and compression, performing video inference and line-crossing detection locally. However, this solution suffers from technical bottlenecks such as poor computing power adaptability and insufficient concurrency. General object detection models typically have a large number of parameters and high floating-point computation. On edge devices with limited computing power (e.g., ≤1 TOPS) due to ARM architecture or NPU limitations, the inference frame rate of a single 1080P video stream is usually less than 10fps, and memory usage easily exceeds the device's available threshold, making it impossible to support concurrent detection of multiple video streams and failing to meet the frame rate requirements of real-time monitoring.

[0005] The third category is traditional computer vision-based line-crossing detection schemes. This approach is commonly found in early low-computing-power embedded surveillance equipment. It primarily relies on manually designed low-level image features, such as background subtraction, inter-frame subtraction, and optical flow, to extract moving targets, and then combines these with preset warning line coordinates to determine line-crossing behavior. However, this scheme suffers from poor detection robustness and insufficient generalization ability. Due to its over-reliance on manually designed low-level pixel features, it is poorly adaptable to complex scenarios such as sudden changes in lighting, partial target occlusion, changes in personnel posture, and dynamic background interference (e.g., swaying trees, moving light and shadow). This results in false positive and false negative rates typically exceeding 15%, making it unsuitable for complex outdoor or semi-outdoor surveillance scenarios.

[0006] Furthermore, in the specific implementation of edge-end boundary crossing detection, some existing edge detection schemes (such as the patent document with publication number CN111738203B) still have obvious architectural and algorithmic limitations. At the hardware architecture level, existing schemes mostly rely on general-purpose CPUs for processing, lacking dedicated VPU (video processing unit) hardware encoding and decoding acceleration and zero-copy data transmission mechanisms. Not only is the CPU computing power insufficient to support multi-channel concurrency, but multiple memory copies also lead to high end-to-end processing latency. At the algorithm logic level, existing schemes mostly use single-frame image judgment logic, lacking multi-frame trajectory verification mechanisms and multi-channel concurrency optimization. When the camera shifts slightly or the target detection box shakes due to changes in pedestrian posture, the single-frame judgment logic is prone to generating a large number of false alarms. Moreover, in densely populated and frequently obstructed scenes, due to the lack of cross-frame target identity maintenance capabilities, target identity is easily lost, leading to missed boundary crossings.

[0007] In summary, overcoming the high latency and security risks of cloud solutions, the high computational consumption of heavy-load models, and the low robustness of traditional vision solutions under the hardware conditions of limited edge computing power, and solving the problems of high false alarms and occlusion missed detection caused by the lack of hardware-level zero-copy acceleration and single-frame judgment in existing edge solutions, and achieving real-time personnel crossing detection with low latency, high concurrency, and high robustness are technical challenges that urgently need to be solved by those skilled in the art. Summary of the Invention

[0008] The purpose of this invention is to solve the technical problems of high latency and computing power concurrency bottleneck caused by the reliance on cloud transmission or edge heavy-load models in existing line crossing detection technologies, as well as the high false alarm and false alarm rates caused by the sensitivity of traditional single-frame geometric determination to target occlusion and attitude changes, making it difficult to achieve highly robust real-time line crossing detection on edge-constrained devices.

[0009] To achieve the above objectives, the present invention employs the following technical means:

[0010] This invention provides a real-time person crossing-the-line detection method based on a lightweight edge model, comprising the following steps:

[0011] Step S1: Video stream acquisition: The scene video stream is acquired through the camera connected to the edge computing device, and the scene video stream is decoded by the hardware decoding unit of the edge computing device to obtain continuous single-frame digital images;

[0012] Step S2: Pedestrian detection: Input the single-frame digital image of each frame into the lightweight target detection model for forward inference, and output the bounding box coordinates and confidence scores of all target persons in the current frame;

[0013] Step S3: Multi-target tracking: Based on the bounding box set of the current frame output in step S2, data association is performed with the tracked target personnel in the previous frame. A unique identity ID is assigned and maintained for each target personnel, and a sequence of target personnel bounding boxes with the unique identity ID is output. Through multi-target tracking, pedestrians are stably associated across frames. Even after brief occlusion or detection interruption, the identity continuity of the same pedestrian can still be maintained, avoiding missed detection of crossing the line due to missed detection in a single frame.

[0014] Step S4: Calculation of the crossing judgment point: For each target personnel bounding box with the unique identification ID output in step S3, calculate the coordinates of the crossing judgment point P of the target personnel bounding box according to the following formula. coordinate:

[0015]

[0016] in, and The coordinates of the top left corner of the bounding box of the target person are given, width and height are the width and height of the bounding box of the target person, and K is a preset height ratio empirical coefficient.

[0017] Step S5: Crossing the Line Determination: Using the target personnel bounding box sequence obtained in Step S3, maintain the cross-frame historical trajectory. Based on the geometric relationship of the crossing judgment points of target personnel with the same unique identification ID in adjacent frames, cross-line determination is performed, specifically including:

[0018] Calculate the crossing point between adjacent frames and Relative to the endpoints of the preset detection line and endpoints The position symbol is used to determine the crossing point through the cross product of vectors cross1 and cross2. and Whether they are distributed on both sides of the preset detection line;

[0019] When the cross products of the vectors cross1 and cross2 have opposite signs, further determination of the line segment is needed. Whether the line segment AB actually intersects with the target person, if the conditions of being distributed on both sides of the preset detection line and the line segment intersection condition are met, then the vertical distance from the crossing judgment point of the current frame to the preset detection line is further calculated. Only when the vertical distance is greater than 0.2 times the width of the target person's bounding box in the current frame is the target person finally determined to have crossed the line.

[0020] Step S6: Alarm and Result Output: When the boundary crossing event determined in step S5 is established, an alarm is triggered and the boundary crossing event is recorded. At the same time, a suppression period is started for the unique identification ID of the target personnel who caused the boundary crossing event. During the suppression period, the same unique identification ID will not be repeatedly triggered with an alarm.

[0021] In the above scheme, step S1 includes:

[0022] Step S1.1: Continuously collect scene video streams using surveillance cameras;

[0023] Step S1.2: The video stream is decoded using the hardware video processing unit (VPU) built into the edge computing device to obtain consecutive single-frame digital images one by one. The image format is RGB or YUV, and the output is image data in hardware buffer format.

[0024] Step S1.3: The image data in the hardware buffer format is directly transmitted to the input memory of the neural network processor (NPU) using a zero-copy method to obtain the single-frame digital image for inference, thereby eliminating the latency and bandwidth occupation caused by CPU memory copying.

[0025] In the above scheme, step S2 specifically includes the following sub-steps:

[0026] Step S2.1: Input each frame of the single-frame digital image into the pre-trained and deployed target detection model for inference. The target detection model is a lightweight YOLO target detection network. The inference process is accelerated by INT8 integer quantization or FP16 half-precision execution in the graphics processing unit (GPU) or neural network processing unit (NPU) of the edge computing device to reduce the computing power consumption of a single inference and meet the real-time requirements in the edge computing power-constrained environment.

[0027] Step S2.2: Output the bounding box parameters and confidence score for each detected target person in the current frame using the target detection model. The bounding box parameters are represented as follows: or ;in, This represents the x-coordinate of the left edge of the bounding box. The ordinate represents the top edge of the bounding box. Indicates the width of the bounding box. Indicates the height of the bounding box. This represents the x-coordinate of the right edge of the bounding box. Represents the ordinate of the bottom edge of the bounding box;

[0028] Step S2.3: Filter the confidence scores according to the preset confidence threshold, and retain only the bounding box parameters of the targets whose confidence scores are higher than the preset confidence threshold and whose category is the target personnel, so as to obtain the bounding box coordinates and confidence scores of all target personnel in the current frame.

[0029] In the above scheme, step S3 includes:

[0030] Step S3.1: For each tracked target person, use a Kalman filter to predict its bounding box position in the current frame, obtaining the predicted bounding box. The state vector of the Kalman filter is represented as:

[0031]

[0032] in, Indicates the x-coordinate of the bounding box center. Indicates the y-coordinate of the bounding box center. Indicates the aspect ratio of the bounding box. Indicates the height of the bounding box. Indicates the rate of change of the central x-coordinate. Indicates the rate of change of the central ordinate. Indicates the rate of change of aspect ratio. Indicates the rate of change of altitude;

[0033] Step S3.2: Calculate the similarity between the bounding box of the current frame output in step S2 and the predicted bounding box. The similarity calculation includes motion matching degree based on Mahalanobis distance and appearance matching degree based on cosine distance calculated from depth appearance features.

[0034] Step S3.3: Perform optimal matching of the bipartite graph using the Hungarian algorithm to associate the bounding box of the current frame with the tracked target personnel;

[0035] Step S3.4: Initialize new tracking trajectories for unmatched bounding boxes and assign new unique identifiers (IDs). Delete tracked target personnel that have not been matched for a long time and output a sequence of target personnel bounding boxes with the unique identifiers (IDs).

[0036] Step S3.5: When the target person is occluded in the current frame and no matching bounding box is obtained, the bounding box position predicted by the Kalman filter is used to maintain the tracking trajectory of the target person, and the unique identity ID is retained to ensure the continuity of identity across frames and overcome tracking loss and missed detection of crossing the line caused by occlusion.

[0037] In the above scheme, step S4: calculating the line crossing judgment point, specifically includes the following sub-steps:

[0038] Step S4.1: For each target personnel bounding box with the unique identification ID output in step S3, obtain the coordinates of the upper left corner, as well as the width and height of the target personnel bounding box;

[0039] Step S4.2: Calculate the coordinates of the boundary judgment point P of the target person's bounding box according to the following formula. :

[0040]

[0041]

[0042] Wherein, 0.7 is a height ratio coefficient, which is used to make the crossing judgment point P close to the contact position between the target person's leg and the ground, so as to eliminate the deformation of the bounding box and the jump of the center point caused by the swaying of the target person's torso or the swinging of the arm, and overcome the false crossing caused by posture changes.

[0043] The crossing point P is considered as an approximate projection point of the target person's standing position on the image plane.

[0044] Instead of using the center point of the bounding box, a point 0.7 times the width and height of the pedestrian bounding box is chosen as the crossing detection point. This point is closer to the contact point between the pedestrian's legs and the ground, making it less sensitive to posture changes such as torso swaying and arm swinging. The positional jitter is much smaller than that of the bounding box centroid, thus significantly reducing false alarms caused by bounding box deformation. The combination of these two methods results in crossing detection that offers both high recall in occluded scenes and high accuracy for complex postures, improving the overall practical performance of the system.

[0045] In the above scheme, step S5: line crossing determination, specifically includes the following sub-steps:

[0046] Step S5.1: Maintain a fixed-length historical queue of crossing judgment points in memory for each target person with the unique identification ID. Whenever a new crossing judgment point is calculated, add it to the corresponding historical queue of crossing judgment points.

[0047] If the target person in the current frame does not have a bounding box, then according to the bounding box position predicted by the Kalman filter in step S3, the crossing judgment point is recalculated and supplemented according to the formula in step S4, or the position of the previous frame is maintained to ensure the continuity of the trajectory.

[0048] Step S5.2: Pre-define at least one preset detection line in the image coordinate system. Each preset detection line is defined by two endpoints A and B and has a direction attribute.

[0049] Step S5.3: For each tracked target person with the unique identification ID, read the line-crossing judgment point of the current frame. The point of crossing the line in the previous frame Calculate the crossing point and The following cross product formula is used to determine the two sides relative to the position sign of the preset detection line:

[0050]

[0051]

[0052] in, and These are the points for determining the crossing line. and The cross product value of the vector relative to the preset detection line, and Let x and y be the x and y coordinates of endpoint A, respectively. and Let x and y be the x and y coordinates of endpoint B, respectively. and These are the points for determining the crossing line. x and y coordinates and These are the points for determining the crossing line. The x and y coordinates; if and A different sign indicates the point where the line was crossed. and Distributed on both sides of the preset detection line;

[0053] Step S5.4: When cross1 and cross2 have opposite signs, further determine the line segment through rapid repulsion test and straddle test. The system checks whether the line segment AB actually intersects with the target person to rule out false alarms where the crossing point is located on the extension of the preset detection line. If both the conditions of being distributed on both sides of the preset detection line and the line segment intersection condition are met, then a crossing anti-shake judgment is performed: the projection point of the crossing point Pt in the current frame onto the straight line determined by endpoints A and B is calculated, and the distance between the crossing point Pt and the projection point is calculated as the crossing vertical distance. When the crossing vertical distance is greater than 0.2 times the width of the target person's bounding box in the current frame, the anti-shake distance requirement is met, and only then is it finally determined that the target person has crossed the line. The accuracy and robustness of the crossing judgment are improved by using dual or multiple verification logic, including the judgment on both sides, the intersection judgment, and the anti-shake distance verification with adaptive size.

[0054] In the above scheme, step S6: alarm and result output, specifically includes the following sub-steps:

[0055] Step S6.1: Based on the sign change direction of the cross product cross1 to cross2 of the vectors, and combined with the line segment direction attribute of the preset detection line, determine the crossing direction of the target person; wherein, the line segment direction attribute is the attribute determined by the vector direction from endpoint A to endpoint B;

[0056] Step S6.2: When the line crossing event is established, record the line crossing event and trigger an alarm;

[0057] If a target person with the same unique identification ID crosses multiple preset detection lines in the current frame, the crossing events will be merged into a single event and reported.

[0058] Step S6.3: Initiate a preset duration suppression period for the unique identification ID of the target person who has experienced the crossing event. During the suppression period, do not trigger alarms repeatedly for the same unique identification ID, so as to block multiple false alarms caused by single-frame image jitter or light and shadow interference.

[0059] In the above scheme, the method is applied to an edge computing device containing multiple system-on-a-chip (SoC). The method further includes a multi-channel video concurrent processing step: dynamically binding and allocating multiple scene video streams to different independent SoC computing cores in the edge computing device; each independent SoC computing core executes the pipeline processing of steps S1 to S6 in parallel, so that the decoding, inference, and tracking computing power of each video stream does not compete with each other, realizing multi-channel high frame rate real-time concurrent detection in edge computing power-constrained environments.

[0060] This invention also provides a real-time personnel crossing detection module based on a lightweight edge model, comprising:

[0061] Video stream acquisition module: Acquires scene video streams through cameras connected to edge computing devices, decodes the scene video streams, and obtains continuous single-frame digital images;

[0062] Pedestrian detection module: Input the single-frame digital image of each frame into the lightweight target detection model for forward inference, and output the bounding box coordinates and confidence scores of all target persons in the current frame;

[0063] Multi-target tracking module: Based on the set of bounding boxes of the current frame output by the pedestrian detection module, it associates data with the target personnel already tracked in the previous frame, assigns and maintains a unique identity ID for each target personnel, and outputs a sequence of target personnel bounding boxes with the unique identity ID;

[0064] The line-crossing judgment point calculation module calculates the coordinates of the line-crossing judgment point P of each target person bounding box output by the multi-target tracking module, using the following formula: coordinate:

[0065]

[0066] in, and The coordinates of the top left corner of the bounding box of the target person are given, width and height are the width and height of the bounding box of the target person, and K is a preset height ratio empirical coefficient.

[0067] The line-crossing determination module: Maintains cross-frame historical trajectories using the target personnel bounding box sequence obtained from the multi-target tracking module. Line-crossing determination is performed based on the geometric relationship of line-crossing judgment points of target personnel with the same unique identification ID in adjacent frames. Specifically, this includes:

[0068] Calculate the crossing point between adjacent frames and Relative to the endpoints of the preset detection line and endpoints The position symbol is used to determine the crossing point through the cross product of vectors cross1 and cross2. and Whether they are distributed on both sides of the preset detection line;

[0069] When the cross products of the vectors cross1 and cross2 have opposite signs, further determination of the line segment is needed. Whether the line segment AB actually intersects with the target person, if the conditions of being distributed on both sides of the preset detection line and the line segment intersection condition are met, then the vertical distance from the crossing judgment point of the current frame to the preset detection line is further calculated. Only when the vertical distance is greater than 0.2 times the width of the target person's bounding box in the current frame is the target person finally determined to have crossed the line.

[0070] Alarm and Result Output Module: When the boundary crossing event is determined to be true by the boundary crossing determination module, an alarm is triggered and the boundary crossing event is recorded. At the same time, a suppression period is started for the unique identification ID of the target personnel who caused the boundary crossing event. During the suppression period, the same unique identification ID will not be triggered repeatedly.

[0071] The present invention also provides a storage medium in which the method is implemented when the processor executes a program in the storage medium.

[0072] Because the present invention employs the above-mentioned technical means, it has the following beneficial effects:

[0073] 1. By combining YOLO object detection and multi-object tracking (steps S2 and S3), the problem of target identity loss due to occlusion or detection interruption is solved, achieving the effect of maintaining continuous pedestrian tracking and significantly reducing the false detection rate of crossing the line in complex occlusion scenarios.

[0074] 2. By using the midpoint of the width of the pedestrian bounding box and 0.7 of its height as the crossing judgment point (step S4), and performing geometric judgment based on the trajectory of this stable point, this design successfully solves the technical pain point of traditional methods that use the center point or corner points of the bounding box, which leads to deformation of the bounding box and irregular jumping of the centroid due to changes in pedestrian posture, torso swaying, and arm swinging, thus causing frequent false alarms. The stable point close to the ground has a much smaller positional jitter than the traditional frame center, thus ensuring a high recall rate for occluded scenes while greatly improving the detection accuracy for complex poses.

[0075] 3. The dual verification logic (step S5) combines the cross product of vectors to determine the opposite signs of two points with the rapid repulsion experiment and the straddle experiment to determine the intersection conditions of line segments. This avoids misjudgments caused by relying solely on the positional relationship between a single point and a straight line (such as false alarms caused when the target moves parallel to the detection line or when the crossing judgment point is located on the extension of the preset detection line), further improving the accuracy and robustness of the crossing event judgment.

[0076] 4. Through the end-to-end modular design based on the edge lightweight model (steps S1 to S6), relying only on a monocular RGB camera and edge computing hardware, without the need for depth sensors or additional reference objects, it is easy to upgrade and deploy software on existing security systems, and the processing speed can be stably reached 25~30FPS throughout the process, meeting the requirements of real-time video stream analysis.

[0077] 5. This invention creatively combines "YOLO object detection and multi-object tracking (steps S2, S3)" with "crossing judgment based on the midpoint of the bounding box width and a 0.7 height crossing judgment point (steps S4, S5)," resolving the technical contradiction in traditional crossing detection where the two requirements of "maintaining identity continuity" and "stable reference point extraction" are difficult to simultaneously address. The multi-object tracking algorithm ensures that the unique identity ID of the pedestrian remains unchanged after brief occlusion or detection interruption, providing the same historical trajectory of the same target across frames for crossing judgment; while the 0.7 width and height crossing judgment point (close to the position where the leg contacts the ground) provides a posture-robust foot position for judgment. The synergistic effect of the two allows the system to accurately distinguish different pedestrians and accurately judge the crossing behavior of each person when facing complex scenarios such as pedestrians walking side by side, intersecting, or occluding. Combined with the alarm suppression period (step S6) to prevent repeated triggering, the overall performance far exceeds the simple superposition of single technologies, producing a synergistic effect.

[0078] 6. The following is a detailed explanation of how this invention achieves real-time multi-channel high frame rate person crossing detection in edge computing environments, while overcoming false alarms caused by occlusion and false alarms caused by pose changes:

[0079] Under the objective condition of limited computing power at the edge, although using a high-precision heavy-load model can improve detection robustness, it will cause a sharp drop in frame rate and cannot be multi-channel concurrent; although using a lightweight model can meet real-time requirements, it is very easy to lose targets in occluded scenes, resulting in false alarms, and it is sensitive to pose changes, resulting in false alarms.

[0080] First, the front end of this invention outputs a memory buffer via a built-in hardware decoder and transmits it to the neural network processor with zero copy. Combined with the dynamic allocation of multiple video streams to multiple independent computing cores, this eliminates the memory transfer bottleneck without increasing the core inference computing power. This approach enables the system to achieve an extremely high data flow frame rate and multi-channel concurrency capability, thereby providing a high temporal resolution trajectory data foundation and ample processor processing capacity for the mid-end and back-end algorithms.

[0081] Secondly, the mid-range multi-target tracking algorithm fully utilizes the high frame rate data provided by the front end. Through Kalman filter prediction and optimal matching of the bipartite graph, even when the lightweight target detection model fails to obtain a bounding box in a single frame due to occlusion, it can still supplement the boundary judgment point based on the predicted position, maintaining the unique identity of the target and the continuity of the trajectory. This method effectively compensates for the insufficient recall of the lightweight model and solves the false negative problem from a temporal perspective.

[0082] Finally, the backend line-crossing determination module utilizes the continuous cross-frame historical trajectory output from the mid-end, discarding the bounding box center point which is easily affected by upper body posture. Instead, it uses the point at the midpoint of the bounding box width multiplied by an empirical coefficient of 0.7 as the line-crossing judgment point, anchoring it to the contact position between the pedestrian's legs and the ground. Simultaneously, it combines vector cross product judgment with dual geometric verification of the intersection of line segments with rapid rejection and straddle experiments, supplemented by an alarm suppression period. This mechanism does not rely on high-performance depth sensors or complex posture estimation networks; it can completely filter out false alarms caused by bounding box jitter and the extension of preset detection lines using only two-dimensional image coordinates, solving the false alarm problem from a spatial perspective.

[0083] 7. Improved Adaptive Size for Line Crossing Anti-Jitter Detection: This invention, after determining line segment intersection through geometric logic, does not immediately trigger an alarm. Instead, it creatively introduces a dynamic distance anti-jitter mechanism based on the target person's bounding box width. By calculating the projection of the crossed point onto the line and solving for the vertical distance, it ensures that a line crossing is only confirmed when the crossing distance is greater than 0.2 times the object detection box width. This mechanism effectively solves the technical problem of "jumping back and forth in the drawn area and repeatedly triggering alarms" caused by the target person lingering or standing near the warning line, or by slight jitter in the lightweight model detection box at the edge. It greatly eliminates false alarms and improves alarm accuracy in actual working conditions.

[0084] 8. Compared with the traditional "target detection + multi-target tracking + bottom point crossing method", the overall solution of this invention constructs a full-process anti-shake filtering mechanism from "stable point extraction" to "multi-dimensional geometric verification", which significantly reduces the false alarm rate of crossing the line and improves the detection accuracy. Moreover, the advantages of this technology have been confirmed by targeted internal comparative test experiments.

[0085] Traditional bottom-point crossing methods typically extract the midpoint of the bottom edge of the pedestrian detection box as the crossing reference. In actual operation, this bottom point is easily affected by the pedestrian's stride, local foot sway, or slight jitter of the lightweight model's bounding box edge, causing the reference point to jump frequently on both sides of the warning line, resulting in continuous and dense false alarms.

[0086] In comparison, the beneficial effect of this invention lies in the fact that it does not blindly select the bottom midpoint of the frame, but achieves a leap in performance through the following series of multiple anti-shake mechanisms:

[0087] Spatial point stabilization: Abandoning the easily shaken frame bottom center point, it adopts an empirical coefficient point with a height ratio of 0.7 that is extremely insensitive to complex postures and torso sway. This anchors the frame at a stable, approximate projection of the legs and the ground, thereby reducing the point-to-point movement caused by frame deformation at its source.

[0088] Trajectory geometry anti-shake: In cross-frame trajectory determination, this invention not only uses vector cross product to perform preliminary screening of opposite-side symbols of two points, but also combines rapid rejection experiment and cross-intersection experiment to verify the actual intersection of line segments, completely eliminating false alarms caused when the target moves parallel to the detection line or the crossing point is located on the extension of the preset detection line.

[0089] Size-adaptive distance stabilization: After the geometric intersection condition is met, an adaptive size-adaptive anti-shake judgment is creatively added. The vertical distance is calculated and strictly verified to be greater than 0.2 times the width of the current target box, establishing a sufficient safety filtering space and completely blocking false alarms when the target lingers or stops near the warning line.

[0090] The R&D team conducted a simple internal comparative test of the above-mentioned anti-shake mechanism in a real-world scenario. The experimental results show that, while ensuring a high detection rate and high accuracy for routine line-crossing behavior, this solution significantly reduces the false alarm rate compared to the traditional bottom-point line-crossing method for various complex postures and critical lingering interference (the number of interference false alarms was reduced by about 88% in internal testing). It perfectly solves the technical pain point that lightweight edge-end algorithms are difficult to implement in engineering due to frequent false alarms in real-world conditions, and the synergistic effect is significant.

[0091] To verify the effectiveness of this solution in real-world scenarios, the R&D team conducted targeted internal comparative tests in office area corridors and entrances, comprehensively comparing the overall algorithm of this invention with traditional methods.

[0092] 1. Test Conditions and Methods

[0093] Test environment: Under the surveillance camera in the office area (1080P resolution, 25fps frame rate), a single virtual crossing line was drawn.

[0094] Test Samples: Typical test video clips containing different pedestrians were collected, totaling 100 instances of pedestrians crossing the line / borderline behavior.

[0095] Behavioral classification:

[0096] Normal walking crossing the line: 60 times (used to test the baseline detection rate and recall rate).

[0097] Interference behaviors: 40 times (pedestrians stopping, turning around, waving, bending down to pick up bags, etc. near the line, used to test the anti-shake mechanism's ability to filter false alarms).

[0098] Comparison method: The overall performance of this scheme (including the overall algorithm of 0.7 height point, geometric cross product and cross-check, and 0.2 times frame width anti-shake) is compared with that of the traditional method (based on target detection + tracking + bottom edge midpoint determination method) under the same video source.

[0099] 2. Test data recording

[0100] The actual performance data of the two components under the aforementioned internal test samples are shown in the table below:

[0101] Test behavior types Total number of behaviors Traditional method (detection + tracking + bottom edge midpoint) This solution (overall algorithm for end-to-end image stabilization) Overall improvement effect Walking across the line normally (an alarm should be triggered) 60 times Successfully triggered 58 times Successfully triggered 59 times The basic detection rate has steadily increased, effectively reducing underreporting. Critical interference behavior (should not trigger an alarm) 40 times 18 false alarms False alarms 2 The overall number of false alarms due to interference decreased by approximately 88%.

[0102] 3. Test Conclusion

[0103] Internal testing results show that the overall algorithm architecture of this invention is significantly superior to traditional methods in practical applications:

[0104] Under normal traffic conditions, the overall algorithm of this scheme benefits from the smooth maintenance of historical trajectories across frames, enabling it to maintain a more stable and higher detection rate.

[0105] Under various critical interference conditions such as pedestrians stopping, turning around, and swaying, the traditional bottom edge midpoint method, lacking deep anti-shake constraints, is highly susceptible to the effects of footsteps and frame deformation, resulting in up to 18 false alarms (a false alarm rate of 45%). In contrast, this solution, through the synergistic effect of a complete anti-shake mechanism of "spatial point anchoring + cross-frame geometric verification + adaptive size filtering", successfully reduced the number of false alarms to 2 (a false alarm rate of 5%), reducing the overall number of interference false alarms by approximately 88%.

[0106] The experiment strongly demonstrates that the overall solution of this invention achieves a leap in accuracy and a significant reduction in false alarm rate with extremely low code and computational complexity. Its overall performance far exceeds that of simple superposition of traditional methods, and it fully achieves the expected engineering implementation effect. Attached Figure Description

[0107] Figure 1 This is a schematic diagram of the process of the present invention;

[0108] Figure 2 This is a schematic diagram of the data flow of the core algorithm for determining line crossing provided in an embodiment of the present invention.

[0109] Figure 3 This is a schematic diagram of the interaction timing between modules in the pedestrian crossing detection system provided in this embodiment of the invention. Detailed Implementation

[0110] The embodiments of the present invention will be described in detail below. Although the present invention will be described and illustrated in conjunction with some specific embodiments, it should be noted that the present invention is not limited to these embodiments. On the contrary, any modifications or equivalent substitutions made to the present invention should be covered within the scope of the claims of the present invention.

[0111] To address the technical problems of existing personnel crossing detection technologies, such as high end-to-end latency and weak data transmission security in cloud-deployed solutions, low inference frame rate, large memory consumption, and insufficient concurrency support of heavy-duty deep learning models on edge computing devices, as well as poor robustness and weak generalization ability of traditional vision solutions, this invention provides a lightweight personnel crossing detection method and device adapted to edge devices.

[0112] Furthermore, to better illustrate the present invention, numerous specific details are set forth in the following detailed embodiments. Those skilled in the art will understand that the present invention can be practiced without these specific details.

[0113] I. System / Device Implementation Examples

[0114] like Figure 1 As shown, this embodiment provides a pedestrian crossing detection system / device based on target detection and multi-target tracking, which mainly includes the following modules:

[0115] Video stream acquisition module: used to continuously acquire scene video frames from surveillance cameras and output raw image data.

[0116] Pedestrian detection module: Deploys the YOLO object detection network, performs forward inference on each frame of the image, and outputs the bounding box coordinates and confidence scores of all pedestrian targets.

[0117] Multi-target tracking module: Receives bounding boxes output by the pedestrian detection module, combines motion models and appearance features to perform data association on pedestrians in consecutive frames, assigns and maintains a unique identity identifier (ID) for each pedestrian, and outputs a sequence of pedestrian bounding boxes with IDs.

[0118] Crossing point calculation module: For each bounding box with an ID, calculate the coordinates of the crossing judgment point in real time, select the midpoint of the width of the bounding box as x, and select 0.7 of the height of the bounding box as y to form the position point representing the pedestrian's ground projection.

[0119] The boundary crossing determination module maintains the trajectory of each pedestrian's boundary crossing judgment point based on preset boundary crossing detection line parameters. It uses the geometric relationship between the current point and historical points to perform preliminary determination of both sides and intersections. After initially satisfying the crossing and intersection conditions, this module further introduces an anti-shake filtering mechanism: it calculates the vertical distance from the current boundary crossing judgment point to the detection line (i.e., calculates the projection of the boundary crossing point on the line and solves for the distance). Only when this vertical distance is greater than 0.2 times the width of the person's target detection box is the boundary crossing event finally confirmed and the boundary crossing direction determined, thereby completely blocking false alarms caused by the target jumping back and forth in the drawn area due to the shaking of the detection box edge.

[0120] Alarm and Result Output Module: When a cross-line event is established, an alarm is triggered, a log is recorded, or a count is performed, and the same target is temporarily suppressed to prevent repeated triggering.

[0121] Module connection and data flow:

[0122] The video stream acquisition module sequentially feeds image frames into the pedestrian detection module; the pedestrian detection module outputs a list of detection boxes to the multi-target tracking module; the multi-target tracking module sends the matched bounding boxes with IDs to the boundary crossing calculation module; the boundary crossing calculation module outputs the boundary crossing judgment coordinates for each ID to the boundary crossing determination module; the boundary crossing determination module combines historical trajectories and detection line parameters to complete the boundary crossing judgment, and finally, the alarm and result output module executes the response. The entire data flow is a unidirectional pipeline, with structured bounding box coordinates and ID data being passed between modules via memory or message queues.

[0123] II. Method Flow Examples

[0124] like Figure 2 and Figure 3 As shown, this invention provides a real-time person crossing-the-line detection method based on a lightweight edge model. The core innovation lies in combining "YOLO detection + multi-target tracking" with "crossing-the-line determination based on crossing-the-line judgment points," specifically including the following steps:

[0125] Step S1: Real-time video frame acquisition. The surveillance camera continuously collects scene video streams. The computer decodes the video stream and acquires continuous single-frame digital images one by one. The image format is RGB or YUV. The output is a raw image matrix with a fixed resolution, which is used as input for subsequent processing.

[0126] Step S2: Pedestrian Detection

[0127] Each frame of the single-frame digital image is input into the pre-trained and deployed object detection model (such as a lightweight version like YOLOv8n or YOLOv8s, with an input size of, for example, 640×640 or 416×256) for forward inference. The inference process is accelerated in a graphics processing unit (GPU) or neural network processing unit (NPU) using INT8 integer quantization or FP16 half-precision execution (e.g., by calling GPU / CUDA or a dedicated inference engine TensorRT). The model outputs the bounding box parameters and confidence score for each person detected in the current frame. The bounding box parameters are represented as follows: or The confidence scores are filtered according to a preset confidence threshold (such as 0.5 or 0.55), and only bounding box parameters with confidence scores higher than the threshold and the category "target personnel" are retained.

[0128] Step S3: Multi-target tracking

[0129] Based on the bounding box set of the current frame output in step S2, data association is performed with the tracked target person in the previous frame (using DeepSORT, SORT, or a lightweight tracker based on IOU matching), specifically including:

[0130] S3.1: For each tracked target person, predict their bounding box position in the current frame using a Kalman filter. The state vector of the Kalman filter is represented as:

[0131] (Uniform velocity model).

[0132] S3.2: Calculate the similarity between the current frame's bounding box and the predicted bounding box, including motion matching based on Mahalanobis distance and appearance matching based on cosine distance calculated from deep appearance features (such as a 128-dimensional vector extracted by a CNN-based ReID network) (the motion cost weight is, for example, 0.1, and the appearance cost weight is, for example, 0.9 when matching).

[0133] S3.3: Perform optimal matching of the bipartite graph using the Hungarian algorithm to associate the bounding box of the current frame with the tracked target personnel.

[0134] S3.4: Initialize new tracking trajectories for unmatched bounding boxes and assign new unique identifiers (IDs). Delete tracked target personnel that have not been matched for a long time (e.g., trajectory lifespan maximum 30 frames). Output a sequence of target personnel bounding boxes with the unique identifiers (IDs).

[0135] Step S4: Calculation of the crossing point

[0136] For each target person bounding box with the unique identification ID output in step S3, obtain its top-left corner coordinates, width, and height, and calculate the boundary crossing judgment point of the target person bounding box according to the following formula. coordinates :

[0137]

[0138]

[0139] in, The empirical coefficient is preferred. (i.e., height scaling factor). This point is considered as an approximate projection of the target person's standing position onto the image plane, close to the contact point between the target person's legs and the ground, and is extremely insensitive to changes in posture such as torso swaying and arm swinging.

[0140] Step S5: Line Crossing Detection

[0141] Maintaining cross-frame historical trajectories using the target personnel bounding box sequence obtained from multi-target tracking in step S3 specifically includes:

[0142] S5.1: Maintain a fixed-length historical queue of boundary crossing judgment points in memory for each target person with the unique identification ID (e.g., retain the coordinates of the most recent N=10 frames). Whenever a new boundary crossing judgment point is calculated, it is added to the queue; if the current frame does not obtain a bounding box, it is recalculated and supplemented according to the position predicted by the Kalman filter, or the position of the previous frame is maintained to ensure trajectory continuity.

[0143] S5.2: At least one preset detection line is pre-defined in the image coordinate system, each line consisting of endpoints... and endpoints Define and attach a direction attribute. Read the current frame's line-crossing judgment point. And the previous frame Calculate its position sign relative to the preset detection line, and use the following vector cross product formula to judge both sides:

[0144]

[0145]

[0146] like and Different signs (i.e.) If ), it indicates that the two points are distributed on both sides of the preset detection line.

[0147] S5.3: When the signs are opposite, further determine the line segment using the rapid repulsion test and the straddling test. With line segment The system checks whether the lines actually intersect to rule out the possibility that the crossing point is located on the extension of the preset detection line. If both the conditions on both sides and the line segment intersection condition are met, it is determined that the target person has crossed the line.

[0148] S5.4: When the signs are opposite, further determine the line segment using the rapid repulsion test and the straddling test. The system checks whether the line segment AB actually intersects with the target to rule out cases where the crossing point is located on the extension of the preset detection line. If both the conditions on both sides and the line segment intersection condition are met, to prevent false alarms caused by the target jumping back and forth in the drawn area due to detection frame jitter or hesitation, the system will not immediately trigger a crossing event. Instead, it will initiate a crossing anti-jitter judgment based on the target size: First, calculate the crossing point of the current frame. In the endpoint and endpoints Projection points on a defined straight line Subsequently, the point for determining whether a line has been crossed is calculated. Its projection point vertical distance between (That is, the distance from a point to a line), the specific calculation formula is as follows: Finally, the width of the bounding box of the target person in the current frame is obtained. The system verifies the current vertical distance through the decision unit. Does it meet the following stabilization threshold equation: If the above formula is satisfied, it proves that the crossing judgment point has substantially crossed the warning line and reached a sufficient safe filtering distance. Only then is it finally determined that the target person has crossed the line and enters the subsequent alarm process. If the above formula is not satisfied, the current crossing judgment is suspended until the crossing distance of the subsequent frame reaches the requirement, thereby effectively suppressing back-and-forth jumping in the drawn area and frequent false alarms.

[0149] Step S6: Alarm and Result Output

[0150] S6.1: Based on the cross product of vectors arrive The direction of the sign change, combined with the line segment direction attribute of the preset detection line (the direction of the vector from A to B), determines the direction in which the target person crosses the line (such as "inside → outside" or "outside → inside").

[0151] S6.2: When a line-crossing event is established, record the event and trigger an alarm. If a target person with the same ID crosses multiple preset detection lines in the current frame, the line-crossing events will be merged into a single event for reporting.

[0152] S6.3: Start a suppression period (usually set to 30~50 frames, about 1~1.5 seconds) for the unique identifier ID of the target personnel who have crossed the line. During the suppression period, the same ID will not be triggered repeatedly to prevent multiple false alarms caused by frame jitter.

[0153] Key data structure descriptions:

[0154] Bounding box data structure: BBox { , width, height, confidence}

[0155] Tracking target data structure: Track {id, bbox, feature_vector, center_history_queue}

[0156] III. Specific Software and Hardware Implementation Environment and Core Code Examples

[0157] This embodiment is implemented based on the Blaize Xplorer X1600P Q edge AI accelerator card. The specific operating environment and configuration are as follows:

[0158] Accelerator card model: Blaize Xplorer X1600P Q (PN: 900 00014 01)

[0159] Number of SoCs and computing power: 4 × Blaize X1600, 64 TOPS INT8

[0160] Memory: 4GB LPDDR4 per SoC, totaling 16GB

[0161] Video capabilities: Single SoC natively supports 4-channel 1080P H.264 / H.265 hardware encoding and decoding at 30fps.

[0162] Host environment: Ubuntu 24.04.4 LTS (Noble Numbat), SDK version Blaize SDK 26.1.0

[0163] Inference backend: Blaize Codec + GSP NPU

[0164] Cooling method: Passive heat dissipation + forced air cooling (≥500 LFM)

[0165] Overall system call relationships:

[0166] The system adopts the Blaize Analytics Pipeline declarative asynchronous scheduling architecture. The core call chain is: RTSP stream → VideoStreamDecode (Blaize-Codec) → ObjectDetection (YOLOv8s_COCO_amp_416_256) → ObjectTracking (SORT) → ObjectCrossedLine → ReportEvent → Visualizer. All nodes inherit from blaize.analytics.va.Node and are uniformly scheduled by TaskController.

[0167] Core implementation steps (closely related to the SDK source code):

[0168] Step S1 (Device Initialization and Task Loading):

[0169] The main program uses TaskController to complete device binding and resource configuration:

[0170] with TaskController(

[0171] device='BlaizeDevice-0.0',

[0172] integration='blaize.cli',

[0173] config=' / path / to / ShopEntryDetection_config.json'

[0174] as controller:

[0175] controller.run_tasks()

[0176] BlaizeDevice-0.0 uniquely identifies the 0th X1600 SoC on the X1600P Q; the TaskController internally completes GSP core allocation, DMA memory pool initialization, and binding of the codec and NPU inference engine.

[0177] Step S2 Hardware Decoding and Zero-Copy Transmission (Key Technical Points):

[0178] VideoStreamDecode is configured to use the Blaize-Codec implementation:

[0179] decode = VideoStreamDecode(

[0180] path="rtsp: / / 192.168.30.111 / main",

[0181] implementation="Blaize-Codec",

[0182] gsp_device="BlaizeDevice-0.0",

[0183] sync_frame_rate=True,

[0184] read_timeout=10 )

[0186] Technical effects:

[0187] Video decoding is performed directly within the X1600 SoC's internal VPU, and the output is a special buffer in NV12 format (gsp_node.SpecialBuffer). This achieves zero-copy direct access to the NPU inference input without CPU memory copying, significantly reducing memory bandwidth usage and end-to-end latency.

[0188] Step S3: Lightweight Target Detection and Tracking

[0189] The detection and tracking components are configured as follows:

[0190] detector_implementation = "YOLOv8s_COCO_amp_416_256"

[0191] codec_implementation = "Blaize-Codec"

[0192] self.line_cross = ObjectCrossedLine(

[0193] lines=param_mgr.parameters['Lines'],

[0194] classifications=["person"],

[0195] implementation=detector_implementation,

[0196] codec_implementation=codec_implementation,

[0197] conf_threshold=0.55,

[0198] nms_threshold=0.45,

[0199] epsilon=0.05 )

[0201] Internal execution flow: ObjectDetection sends NV12 frames to the GSP NPU to perform INT8 inference; outputs personnel detection boxes; ObjectTracking (SORT algorithm) assigns a unique object_id to each target; maintains the target's historical trajectory for each frame.

[0202] Step S4: Line crossing determination based on trajectory consistency:

[0203] ObjectCrossedLine is determined based on the trajectory state machine:

[0204] if events := await self.line_cross(image):

[0205] await self.report_event(events)

[0206] The decision logic includes:

[0207] Calculate the geometric relationship between the target selection point (e.g., at height 0.7 or the default BottomCenter) and the warning line, and compare the changes in the target's position on the line across multiple consecutive frames. Within the tolerance range of epsilon=0.05, cases where the same target crosses multiple lines within the same frame are merged into a single event for reporting.

[0208] Step S5: Event Reporting and Visualization

[0209] ReportEvent is encapsulated in the Blaize VA standard event format.

[0210] JSON data: trajectory lines and warning lines, supporting local display and remote debugging.

[0211] {

[0212] "Event": {

[0213] "Class": "ObjectMotionDetection",

[0214] "Subclass": "ObjectCrossedLine",

[0215] "Objects": [...],

[0216] "RelationShips": [{

[0217] "PrimaryObjectId": "<person_id> ",

[0218] "Type": "Crossed",

[0219] "SecondaryObjectId": "<line_id> "

[0220] }]

[0221] }

[0222] }

[0223] at the same time:

[0224] Visualizer overlays detection boxes, trajectory lines, and warning lines onto NV12 frames;

[0225] Supports local display and remote debugging.

[0226] Concurrency and Resource Management:

[0227] In an X1600P Q quad-SoC environment, each 1080P video stream is bound to a single X1600 SoC, and the four video streams run in parallel without competing for GPS resources. The system's typical power consumption is ≤50W, with a peak power consumption of ≤75W, and it communicates with the host via PCIe Gen3×4.

[0228] Exception and robust handling:

[0229] RTSP disconnection: max_retries=10 + retry_delay=10s Automatic reconnection.

[0230] Decoding timeout: read_timeout=10s triggers DecoderReadTimeout.

[0231] NPU inference exception: TaskController context manager automatically reclaims resources.

[0232] Fan / Temperature Abnormalities: Onboard temperature sensor + forced air cooling ensures long-term operation.

[0233] IV. Alternative Solutions and Extended Implementations

[0234] Algorithm and model replacements: The detector can be replaced with YOLOv8n, YOLOv10 Nano, NanoDet Plus, or MobileViT SSD; the tracker can be replaced with DeepSORT or ByteTrack; for line crossing determination, a Kalman Filter can be introduced to predict the target position, or the ray casting method can be used instead of the vector cross product method.

[0235] Decoding and hardware implementation alternatives: The decoder can use OpenCV or GStreamer; the pixel format can be converted from NV12 to BGR / RGB (CPU path); the hardware platform can be replaced with the X1600 single SoC module or other GSP architectures.

[0236] System architecture expansion: Supports multi-SoC load balancing (dynamic allocation based on the number of video streams); supports cloud-edge collaboration (structured events are reported at the edge, and key frames are retrieved and reviewed in the cloud); supports multi-model pipelines (detection of cross-line violations in the same video stream, safety helmet recognition, and work clothes recognition).

[0237] Application scenarios expanded: Modifying classifications can be applied to vehicles crossing lines; changing line segments to closed polygons can be applied to area intrusion; and it can be applied to industrial AOI to detect violations of process boundaries.

[0238] Adjustable parameter range: input resolution (256×256 ~ 640×360), confidence threshold (0.3 ~ 0.8), NMS threshold (0.3 ~ 0.7), trajectory window (3 ~ 30 frames), number of concurrent channels (1 ~ 16 channels, limited by the theoretical upper limit of SoC, this embodiment prefers single channel exclusive single SoC to ensure low latency).

[0239] V. Beneficial Effects and Comparison with Existing Technologies

[0240] The embodiments of the present invention have achieved the following significant beneficial effects:

[0241] Reduced false negative rate: Combining YOLO detection with multi-target tracking solves the problem of target identity loss due to occlusion or detection interruptions. Tests show that in densely occluded scenes, the number of ID-Switch operations is reduced by more than 40% compared to traditional inter-frame IOU matching, and the false negative rate for crossing the line is reduced from approximately 18% to below 5%.

[0242] Reduced false alarm rate: Using the midpoint of the bounding box width and a height of 0.7 as the boundary crossing reference point makes it extremely insensitive to attitude changes. Compared to the method using the box center point, the false alarm rate is reduced by about 60%, essentially eliminating false boundary crossing alarms caused by irregular jumps in the box centroid.

[0243] Improved accuracy: The dual verification logic of the two-point opposite sign judgment and the line segment intersection condition avoids false alarms when the target moves parallel to the detection line.

[0244] Easy to deploy and high real-time performance: The end-to-end modular design relies on only a single RGB camera and general computing hardware, and the processing speed can be stably maintained at 25~30FPS throughout the process.

[0245] Significant synergistic effects: "YOLO+ tracking" ensures the same target trajectory across frames, while "0.7 elevation point" provides pose-robust foot position. The synergy between the two resolves the contradiction between "maintaining identity continuity" and "stable reference point extraction," resulting in a synergistic effect where 1+1>2.

[0246] Compared with existing technologies:

[0247] Existing technologies (such as CN111738203B) use general-purpose CPUs to determine single-frame images without dedicated hardware encoding and decoding acceleration. This results in problems such as the need to manually redraw warning lines when the camera is moved, lack of multi-channel concurrent optimization, high false alarms due to jitter detection, and insufficient CPU computing power.

[0248] This invention is based on the Blaize X1600P Q edge card, relying on its built-in independent VPU hardware to complete hardware decoding and output NV12 SpecialBuffer, achieving zero-copy image transmission between the CPU and GSP NPU; it utilizes dedicated GSP NPU operators for INT8 inference; and combines a multi-frame trajectory state machine to suppress false alarms. Four video streams can be allocated to independent computing cores for parallel pipelined processing. The overall real-time processing capability and robustness are significantly superior to existing CPU single-frame solutions, making it widely applicable to weak or no-network environments such as park security, construction site monitoring, and traffic intersections.

[0249] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A real-time person crossing-the-line detection method based on a lightweight edge-end model, characterized in that, Includes the following steps: Step S1: Video stream acquisition: The scene video stream is acquired through the camera connected to the edge computing device, and the scene video stream is decoded by the hardware decoding unit of the edge computing device to obtain continuous single-frame digital images and store them in the hardware buffer. Step S2: Pedestrian detection: The single-frame digital image of each frame is transferred from the hardware buffer to the neural network processor with zero copy, input into the target detection model for forward inference, and outputs the bounding box coordinates and confidence scores of all target persons in the current frame; Step S3: Multi-target tracking: Based on the bounding box set of the current frame output in step S2, data association is performed with the target personnel already tracked in the previous frame. A unique identification ID is assigned to and maintained for each target personnel. When a target personnel is occluded, the position of the target personnel is predicted by the motion model to maintain trajectory continuity. The bounding box sequence of the target personnel with the unique identification ID is output. Step S4: Calculation of the crossing judgment point: For each target personnel bounding box with the unique identification ID output in step S3, calculate the coordinates of the crossing judgment point P of the target personnel bounding box according to the following formula. coordinate: in, and The coordinates of the top left corner of the bounding box of the target person are given, width and height are the width and height of the bounding box of the target person, and K is a preset height ratio empirical coefficient. Step S5: Crossing the Line Determination: Using the target personnel bounding box sequence obtained in Step S3, maintain the cross-frame historical trajectory. Based on the geometric relationship of the crossing judgment points of target personnel with the same unique identification ID in adjacent frames, cross-line determination is performed, specifically including: Calculate the crossing point between adjacent frames and Relative to the endpoints of the preset detection line and endpoints The position symbol is used to determine the crossing point through the cross product of vectors cross1 and cross2. and Whether they are distributed on both sides of the preset detection line; When the cross products of the vectors cross1 and cross2 have opposite signs, further determination of the line segment is needed. Whether the line segment AB actually intersects with the target person, if the conditions of being distributed on both sides of the preset detection line and the line segment intersection condition are met, then the vertical distance from the crossing judgment point of the current frame to the preset detection line is further calculated. Only when the vertical distance is greater than 0.2 times the width of the target person's bounding box in the current frame is the target person finally determined to have crossed the line. Step S6: Alarm and Result Output: When the boundary crossing event determined in step S5 is established, an alarm is triggered and the boundary crossing event is recorded. At the same time, a suppression period is started for the unique identification ID of the target personnel who caused the boundary crossing event. During the suppression period, the same unique identification ID will not be repeatedly triggered with an alarm.

2. The method according to claim 1, characterized in that, Step S1 includes: Step S1.1: Continuously collect scene video streams through monitoring cameras, and dynamically allocate multiple scene video streams to multiple independent computing cores of the edge computing device for parallel processing to support multi-channel high frame rate concurrent detection; Step S1.2: Use the hardware video processing unit (VPU) built into the edge computing device to decode the video stream, acquire consecutive single-frame digital images one by one, the image format is RGB or YUV, and the output is image data in hardware buffer format; Step S1.3: The image data in the hardware buffer format is directly transmitted to the input memory of the neural network processor (NPU) using a zero-copy method to obtain the single-frame digital image for inference, thereby eliminating the latency and bandwidth occupation caused by CPU memory copying.

3. The real-time personnel crossing detection method based on a lightweight edge-end model according to claim 1, characterized in that, Step S2 specifically includes the following sub-steps: Step S2.1: Input each frame of the single-frame digital image into the pre-trained and deployed target detection model for inference. The target detection model is a lightweight YOLO target detection network. The inference process is accelerated by INT8 integer quantization or FP16 half-precision execution in the graphics processing unit (GPU) or neural network processing unit (NPU) of the edge computing device to reduce the computing power consumption of a single inference and meet the real-time requirements in the edge computing power-constrained environment. Step S2.2: Output the bounding box parameters and confidence score for each detected target person in the current frame using the target detection model. The bounding box parameters are represented as follows: or ;in, This represents the x-coordinate of the left edge of the bounding box. The ordinate represents the top edge of the bounding box. Indicates the width of the bounding box. Indicates the height of the bounding box. This represents the x-coordinate of the right edge of the bounding box. Represents the ordinate of the bottom edge of the bounding box; Step S2.3: Filter the confidence scores according to the preset confidence threshold, and retain only the bounding box parameters of the targets whose confidence scores are higher than the preset confidence threshold and whose category is the target personnel, so as to obtain the bounding box coordinates and confidence scores of all target personnel in the current frame.

4. The real-time personnel crossing detection method based on a lightweight edge-end model according to claim 1, characterized in that, Step S3 includes: Step S3.1: For each tracked target person, use a Kalman filter to predict its bounding box position in the current frame, obtaining the predicted bounding box. The state vector of the Kalman filter is represented as: in, Indicates the x-coordinate of the bounding box center. Indicates the y-coordinate of the bounding box center. Indicates the aspect ratio of the bounding box. Indicates the height of the bounding box. Indicates the rate of change of the central x-coordinate. Indicates the rate of change of the central ordinate. Indicates the rate of change of aspect ratio. Indicates the rate of change of altitude; Step S3.2: Calculate the similarity between the bounding box of the current frame output in step S2 and the predicted bounding box. The similarity calculation includes motion matching degree based on Mahalanobis distance and appearance matching degree based on cosine distance calculated from depth appearance features. Step S3.3: Perform optimal matching of the bipartite graph using the Hungarian algorithm to associate the bounding box of the current frame with the tracked target personnel; Step S3.4: Initialize new tracking trajectories for unmatched bounding boxes and assign new unique identifiers (IDs). Delete tracked target personnel that have not been matched for a long time and output a sequence of target personnel bounding boxes with the unique identifiers (IDs). Step S3.5: When the target person is occluded in the current frame and no matching bounding box is obtained, the bounding box position predicted by the Kalman filter is used to maintain the tracking trajectory of the target person, and the unique identity ID is retained to ensure the continuity of identity across frames and overcome tracking loss and missed detection of crossing the line caused by occlusion.

5. The real-time personnel crossing detection method based on a lightweight edge-end model according to claim 1, characterized in that, Step S4: Calculation of the line crossing judgment point, specifically includes the following sub-steps: Step S4.1: For each target personnel bounding box with the unique identification ID output in step S3, obtain the coordinates of the upper left corner, as well as the width and height of the target personnel bounding box; Step S4.2: Calculate the coordinates of the boundary judgment point P of the target person's bounding box according to the following formula. : Wherein, 0.7 is a height ratio coefficient, used to make the crossing judgment point P close to the contact position between the target person's leg and the ground; The crossing point P is considered as an approximate projection point of the target person's standing position on the image plane.

6. The real-time personnel crossing detection method based on a lightweight edge-end model according to claim 1, characterized in that, Step S5: Crossing the line determination, specifically includes the following sub-steps: Step S5.1: Maintain a fixed-length historical queue of crossing judgment points in memory for each target person with the unique identification ID. Whenever a new crossing judgment point is calculated, add it to the corresponding historical queue of crossing judgment points. If the target person in the current frame does not have a bounding box, then according to the bounding box position predicted by the Kalman filter in step S3, the crossing judgment point is recalculated and supplemented according to the formula in step S4, or the position of the previous frame is maintained to ensure the continuity of the trajectory. Step S5.2: Pre-define at least one preset detection line in the image coordinate system. Each preset detection line is defined by two endpoints A and B and has a direction attribute. Step S5.3: For each tracked target person with the unique identification ID, read the line-crossing judgment point of the current frame. The point of crossing the line in the previous frame Calculate the crossing point and The following cross product formula is used to determine the two sides relative to the position sign of the preset detection line: in, and These are the points for determining the crossing line. and The cross product of the vectors relative to the preset detection line, and Let x and y be the x and y coordinates of endpoint A, respectively. and Let x and y be the x and y coordinates of endpoint B, respectively. and These are the points for determining the crossing line. x and y coordinates and These are the points for determining the crossing line. The x and y coordinates; if and A different sign indicates the point where the line was crossed. and Distributed on both sides of the preset detection line; Step S5.4: When cross1 and cross2 have opposite signs, further determine the line segment through rapid repulsion test and straddle test. The system checks whether the line segment AB actually intersects with the target person to rule out false alarms where the crossing point is located on the extension of the preset detection line. If both the conditions of being distributed on both sides of the preset detection line and the line segment intersection condition are met, then a crossing anti-shake determination is performed: the projection point of the crossing point Pt in the current frame onto the straight line determined by endpoints A and B is calculated, and the distance between the crossing point Pt and the projection point is calculated as the crossing vertical distance. When the crossing vertical distance is greater than 0.2 times the width of the target person's bounding box in the current frame, the anti-shake distance requirement is met, and only then is it finally determined that the target person has experienced a crossing event.

7. A real-time person crossing-the-line detection method based on a lightweight edge-end model according to claim 6, characterized in that, Step S6: Alarm and Result Output, specifically includes the following sub-steps: Step S6.1: Based on the sign change direction of the cross product cross1 to cross2 of the vectors, and combined with the line segment direction attribute of the preset detection line, determine the crossing direction of the target person; wherein, the line segment direction attribute is the attribute determined by the vector direction from endpoint A to endpoint B; Step S6.2: When the line crossing event is established, record the line crossing event and trigger an alarm; Step S6.3: Initiate a suppression period for the unique identifier ID of the target personnel who have experienced the crossing event, and do not trigger alarms repeatedly for the same unique identifier ID during the suppression period.

8. The method according to claim 1, characterized in that, The method is applied to an edge computing device containing multiple system-on-a-chip (SoC). The method also includes a multi-channel video concurrent processing step: dynamically binding and allocating multiple scene video streams to different independent SoC computing cores in the edge computing device. Each independent SoC computing core executes the pipeline processing of steps S1 to S6 in parallel, so that the decoding, inference and tracking computing power of each video stream does not compete with each other, realizing multi-channel high frame rate real-time concurrent detection in edge computing power limited environment.

9. A real-time personnel crossing detection module based on a lightweight edge-end model, characterized in that, include: Video stream acquisition module: Acquires scene video streams through cameras connected to edge computing devices, decodes the scene video streams, and obtains continuous single-frame digital images; Pedestrian detection module: Input the single-frame digital image of each frame into the lightweight target detection model for forward inference, and output the bounding box coordinates and confidence scores of all target persons in the current frame; Multi-target tracking module: Based on the set of bounding boxes of the current frame output by the pedestrian detection module, it associates data with the target personnel already tracked in the previous frame, assigns and maintains a unique identity ID for each target personnel, and outputs a sequence of target personnel bounding boxes with the unique identity ID; The line-crossing judgment point calculation module calculates the coordinates of the line-crossing judgment point P of each target person bounding box output by the multi-target tracking module, using the following formula: coordinate: in, and The coordinates of the top left corner of the bounding box of the target person are given, width and height are the width and height of the bounding box of the target person, and K is a preset height ratio empirical coefficient. The line-crossing determination module: Maintains cross-frame historical trajectories using the target personnel bounding box sequence obtained from the multi-target tracking module. Line-crossing determination is performed based on the geometric relationship of line-crossing judgment points of target personnel with the same unique identification ID in adjacent frames. Specifically, this includes: Calculate the crossing point between adjacent frames and Relative to the endpoints of the preset detection line and endpoints The position symbol is used to determine the crossing point through the cross product of vectors cross1 and cross2. and Whether they are distributed on both sides of the preset detection line; When the cross products of the vectors cross1 and cross2 have opposite signs, further determination of the line segment is needed. Whether the line segment AB actually intersects with the target person, if the conditions of being distributed on both sides of the preset detection line and the line segment intersection condition are met, then the vertical distance from the crossing judgment point of the current frame to the preset detection line is further calculated. Only when the vertical distance is greater than 0.2 times the width of the target person's bounding box in the current frame is the target person finally determined to have crossed the line. Alarm and Result Output Module: When the boundary crossing event is determined to be true by the boundary crossing determination module, an alarm is triggered and the boundary crossing event is recorded. At the same time, a suppression period is started for the unique identification ID of the target personnel who caused the boundary crossing event. During the suppression period, the same unique identification ID will not be triggered repeatedly.

10. A storage medium, characterized in that, When the processor executes a program in the storage medium, it implements the method as described in any one of claims 1-8.

Images