Artificial intelligence-based target behavior early warning method and system

Abstract

This disclosure provides an artificial intelligence-based target behavior early warning method and system, electronic device, storage medium, and program product to solve the problems of high false negative detection rate, frequent identity switching during tracking, and high false positive rate in abnormal behavior identification. The method includes: acquiring surveillance video data through a camera and outputting continuous video frames; using a YOLOv5 model with a fusion attention mechanism to detect pedestrian targets in the video frames and outputting the target bounding box coordinates; continuously tracking the detected pedestrian targets based on an improved DeepSort algorithm to generate target motion trajectories; extracting trajectory features from the target motion trajectory, the trajectory features including the positional change of the detection box center point along the vertical axis; determining whether the pedestrian exhibits abnormal behavior based on the trajectory features and a preset threshold; and generating and sending early warning information when abnormal behavior is determined. This disclosure can improve the reliability and practicality of the monitoring and early warning system.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and in particular to a target behavior early warning method and system based on artificial intelligence, electronic device, computer-readable storage medium, and computer program product. Background Technology

[0002] Target behavior detection and early warning refers to a technology that uses intelligent devices to monitor targets in a surveillance scene in real time, identify their behavioral states, determine whether there are any anomalies, and issue timely warnings when potentially dangerous behaviors are detected. This technology is widely used in traffic management, public place security, smart parks, campus monitoring, and other fields, helping managers to identify risks in advance, reduce accidents, and ensure public safety.

[0003] With the rapid development of deep learning technology, object detection and behavior analysis based on artificial intelligence have become a research hotspot in the field of computer vision. Algorithms such as the YOLO series and DeepSort have achieved significant results in object detection and tracking tasks, providing a technological foundation for intelligent video surveillance. However, when applying these technologies to pedestrian abnormal behavior early warning in real-world complex monitoring scenarios, the following technical bottlenecks still exist: First, in the target detection stage, existing detection models lack robustness in complex scenarios. Surveillance scenarios commonly present problems such as pedestrian occlusion, drastic changes in lighting conditions (e.g., backlighting, nighttime, shadows), and small pedestrian scale, leading to a decline in the model's feature extraction ability and a large number of missed detections. While single-stage detectors, such as YOLOv5, offer fast detection speeds, their backbone networks have limited ability to jointly model local features and global contextual information, making it difficult to effectively distinguish occluded targets from the background. This results in decreased detection accuracy and directly impacts the reliability of subsequent tracking and discrimination.

[0004] Second, in the target tracking stage, existing tracking algorithms have weak identity maintenance capabilities in occluded scenarios. Algorithms such as DeepSort predict the target's motion state using Kalman filtering and perform data association using the Hungarian algorithm, but when pedestrians are briefly occluded by other objects or pedestrians, the target's identity identifier tends to switch frequently. This loss of identity identifier causes the movement trajectory of the same pedestrian to be fragmented into multiple segments, making it impossible to form a continuous and stable trajectory. This discontinuity in the trajectory not only results in the loss of the target's motion history information but also makes trajectory-based behavior analysis difficult to conduct effectively.

[0005] Third, in the abnormal behavior identification stage, existing methods have a single modeling dimension for abnormal behavior, resulting in a high false positive rate. Most solutions rely solely on appearance features (such as human posture and contour changes) or simple motion features (such as instantaneous speed) in a single frame image for identification. Whether a pedestrian is abnormal is often reflected in the spatial statistical features of their movement patterns. For example, a pedestrian suddenly crossing the road will produce a sharp displacement along the vertical axis, while loitering behavior manifests as repeated movements in a local area. Single-feature discrimination models struggle to capture these complex patterns, easily misclassifying pedestrians walking normally but with zigzagging paths as abnormal, or missing truly abnormal behavior.

[0006] Fourth, in the early warning and response phase. Existing systems have limited early warning methods and information capacity. Most systems only support local audible and visual alarms or simple pop-up notifications, lacking remote notification capabilities and intuitive presentation of abnormal information. After receiving an alarm, managers often need to review video recordings to pinpoint the specific circumstances of the abnormal event, delaying the optimal time for emergency response.

[0007] In summary, how to achieve accurate detection and stable tracking of pedestrian targets in complex monitoring scenarios, how to extract effective spatial statistical features from trajectory data to reduce the misjudgment rate of abnormal behavior, and how to build a fast, intuitive, and multi-channel early warning and response mechanism are technical problems that urgently need to be solved by those skilled in the art. Summary of the Invention

[0008] To address the problems of high false negative rates in pedestrian detection, frequent identity switching during tracking, and high false positive rates in abnormal behavior identification in existing technologies under complex monitoring scenarios, this disclosure provides an artificial intelligence-based target behavior early warning method and system, electronic device, computer-readable storage medium, and computer program product. Through optimization of the entire process—detection, tracking, feature extraction, discrimination, and early warning—it achieves more accurate pedestrian detection, more stable tracking, lower false positives, and faster early warning in complex scenarios, significantly improving the reliability and practicality of the monitoring and early warning system.

[0009] In a first aspect, this disclosure provides a target behavior early warning method based on artificial intelligence, the method comprising: Acquire surveillance video data through a camera and output continuous video frames; The YOLOv5 model with fusion attention mechanism is used to detect pedestrian targets in the video frame and output the target bounding box coordinates; The improved DeepSort algorithm is used to continuously track detected pedestrian targets and generate target motion trajectories. Extract trajectory features from the target motion trajectory, the trajectory features including the positional change of the detection box center point in the vertical axis direction; Based on the trajectory characteristics and preset thresholds, it is determined whether the pedestrian exhibits abnormal behavior; When abnormal behavior is detected, an alert message is generated and sent.

[0010] Furthermore, the YOLOv5 model with the fused attention mechanism is constructed by replacing the C3 module in the YOLOv5 backbone network with the CBAMC3 module; The CBAMC3 module includes a channel attention submodule and a spatial attention submodule; The channel attention submodule is used to: perform average pooling and max pooling on the input feature map to obtain channel statistics, process the data through a multilayer perceptron to output channel weights, and multiply the channel weights with the input feature map channel by channel to output a channel-weighted feature map. The spatial attention submodule is used to: perform average pooling and max pooling on the channel-weighted feature map in the channel dimension to obtain a first spatial feature map and a second spatial feature map; concatenate the first spatial feature map and the second spatial feature map, process them through a convolutional layer to output spatial weights, and multiply the spatial weights with the channel-weighted feature map pixel by pixel to output the final enhanced feature map.

[0011] Furthermore, the replacement of the C3 module in the YOLOv5 backbone network with the CBAMC3 module specifically includes: Replace the C3 modules in layers 2, 4, and 6 of the YOLOv5s backbone network with CBAMC3 modules; Each of the CBAMC3 modules contains 3 residual units, and a CBAM attention module is inserted after the output of the residual units.

[0012] Furthermore, the improved DeepSort algorithm includes the following steps: A Kalman filter is used to predict the position and velocity of a pedestrian target, and a state equation is established to obtain the target prediction state in the current frame. The state equation is as follows:

[0013] Where a is the state transition matrix, b is the control matrix, and W is the control matrix. k For process noise, Let be the system state vector at time k. This is the control input vector at time k; The Hungarian algorithm is used to associate the predicted state of the target with the current detection box, and the matching cost is the weighted sum of Mahalanobis distance and cosine distance; A trained feature extraction network was used to extract appearance features from the detection boxes. The feature extraction network used ResNet50 as the backbone network, removed the last fully connected layer, and output a 512-dimensional feature vector, which was then L2 normalized. The cross-entropy loss function was used, and the optimizer was SGD (Stochastic Gradient Descent). The network was trained on the pedestrian re-identification dataset Market-1501.

[0014] Furthermore, The matching parameters of the Hungarian algorithm are set as follows: Mahalanobis distance threshold is set to 9.4877, cosine distance threshold is set to 0.2, and maximum cosine distance is set to 0.1. During the continuous tracking of detected pedestrian targets, the minimum detection confidence level was set to 0.3, the non-maximum suppression threshold was set to 0.5, and the maximum trajectory survival time was set to 70 frames.

[0015] Furthermore, the change in position of the center point of the detection frame along the vertical axis is calculated in the following way: After continuously tracking N frames, record the ordinates y1, y2, ..., y3 of the center point of the pedestrian detection box in each frame. N ; Calculate the mean of the ordinate. Calculate the standard deviation σy along the vertical axis using the following formula:

[0016] Where N is the preset number of continuous tracking frames; σy is used as the change in position of the center point of the detection frame along the vertical axis.

[0017] Furthermore, the preset number of continuous tracking frames ranges from 20 to 30 frames; When the vertical coordinate of a frame is missing due to occlusion, the average of the vertical coordinates of the preceding and following frames is used to fill the missing coordinates.

[0018] Furthermore, determining whether a pedestrian exhibits abnormal behavior based on the trajectory features and a preset threshold includes: A preset threshold θ for the standard deviation σy in the vertical direction is set, and the preset threshold θ is determined by statistically analyzing the distribution of σy in the normal behavioral trajectories in the training set. When σy > θ is detected, it is initially determined to be abnormal behavior; The trajectory features are input into a pre-trained behavior prediction model to obtain the probability of abnormal behavior prediction. When the predicted probability of abnormal behavior exceeds a preset probability threshold, the abnormal judgment result is enhanced, and the final abnormal behavior label is output.

[0019] Furthermore, The preset threshold θ is the 95th percentile of the normal behavior trajectory σy value; The preset probability threshold is 0.8; The behavior prediction model is a two-layer fully connected neural network with 128 neurons in the hidden layer and ReLU activation function. The input features of the behavior prediction model include: standard deviation σy in the vertical direction, standard deviation σx in the horizontal direction, average speed, and trajectory length; The behavior prediction model is trained using a confidence-based step loss function.

[0020] Secondly, this disclosure provides a target behavior early warning system based on artificial intelligence, the system comprising: The data acquisition module is configured to acquire surveillance video data through a camera and output continuous video frames. The target detection module is configured to use a YOLOv5 model with a fusion attention mechanism to detect pedestrian targets in the video frame and output the target bounding box coordinates. The target tracking module is configured to continuously track detected pedestrian targets based on an improved DeepSort algorithm and generate the target's motion trajectory; The feature extraction module is configured to extract trajectory features from the target motion trajectory, the trajectory features including the positional change of the detection box center point in the vertical axis direction; An anomaly detection module is configured to determine whether a pedestrian exhibits abnormal behavior based on the trajectory features and a preset threshold. The early warning module is configured to generate and send early warning information when abnormal behavior is detected.

[0021] Thirdly, this disclosure provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores one or more computer programs executable by the at least one processor, the one or more computer programs being executed by the at least one processor to enable the at least one processor to perform the aforementioned artificial intelligence-based target behavior early warning method.

[0022] Fourthly, this disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the aforementioned artificial intelligence-based target behavior early warning method.

[0023] Fifthly, this disclosure provides a computer program product that includes computer-readable code or a non-volatile computer-readable storage medium carrying computer-readable code. When the computer-readable code is run in a processor of an electronic device, the processor in the electronic device executes the aforementioned artificial intelligence-based target behavior early warning method.

[0024] Beneficial effects: This disclosure provides an AI-based target behavior early warning method and system, electronic device, computer-readable storage medium, and computer program product. It can improve detection accuracy by employing a YOLOv5 model with a fusion attention mechanism, enhancing the model's ability to jointly focus on local details and global context of pedestrians in complex scenarios such as occlusion and lighting changes, significantly reducing the false negative rate and providing high-quality target input for subsequent tracking and discrimination. It ensures tracking continuity by continuously tracking the detected target based on an improved DeepSort algorithm. Through dual constraints of Kalman filtering and appearance features, it effectively suppresses the problem of frequent switching of target identity caused by occlusion, generating continuous and stable motion trajectories, providing a reliable data foundation for behavior analysis. It also reduces the risk of false positives by extracting the vertical axis position change from the motion trajectory as a discriminative feature, quantifying the degree of vertical fluctuation of pedestrians and introducing it into anomaly judgment, breaking through the limitations of traditional methods that rely on single appearance features. This feature has a strong correlation with abnormal behavior (such as sudden crossing, climbing, and loitering), which can significantly improve the accuracy of abnormal behavior identification and reduce the false positive rate. Enhance early warning capabilities, help managers locate abnormal events in seconds, transform "post-event tracing" into "in-event response," and truly achieve proactive intelligent security.

[0025] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0026] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the embodiments of the present disclosure to explain the disclosure and do not constitute a limitation thereof. The above and other features and advantages will become more apparent to those skilled in the art from the detailed description of exemplary embodiments with reference to the accompanying drawings, in which: Figure 1 This is a flowchart illustrating a target behavior early warning method based on artificial intelligence, provided in Embodiment 1 of this disclosure. Figure 2 This is a flowchart illustrating a target behavior early warning method based on artificial intelligence, provided in Embodiment 2 of this disclosure. Figure 3This is a block diagram of an artificial intelligence-based target behavior early warning system provided in Embodiment 8 of this disclosure; Figure 4 This is a block diagram of an electronic device provided in Embodiment Nine of this disclosure. Detailed Implementation

[0027] To enable those skilled in the art to better understand the technical solutions of this disclosure, exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments of this disclosure to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0028] Where there is no conflict, the various embodiments of this disclosure and the features thereof in the embodiments may be combined with each other.

[0029] As used herein, the term “and / or” includes any and all combinations of one or more related enumerated entries.

[0030] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. As used herein, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that when the terms “comprising” and / or “made of” are used in this specification, the presence of the stated feature, integral, step, operation, element, and / or component is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof is not excluded. Words such as “connected” or “linked” are not limited to physical or mechanical connections but can include electrical connections, whether direct or indirect.

[0031] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in common dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and this disclosure, and will not be interpreted as having an idealized or overly formal meaning unless expressly so defined herein. Those skilled in the art will understand that the specific order of execution of the steps in the methods described above in the specific embodiments should be determined by their function and possible internal logic.

[0032] The AI-based target behavior warning method according to embodiments of this disclosure can be executed by electronic devices such as terminal devices or servers. Terminal devices can be in-vehicle devices, user equipment (UE), mobile devices, user terminals, terminals, cellular phones, cordless phones, personal digital assistants (PDAs), handheld devices, computing devices, in-vehicle devices, wearable devices, etc. The method can be implemented by a processor calling computer-readable program instructions stored in memory. Alternatively, the method can be executed by a server.

[0033] Example 1

[0034] Figure 1 This is a flowchart illustrating a target behavior early warning method based on artificial intelligence, as provided in Embodiment 1 of this disclosure. Figure 1 As shown, the method includes: Step S100: Acquire surveillance video data through the camera and output continuous video frames; Step S200: Use the YOLOv5 model with fusion attention mechanism to detect pedestrian targets in the video frame and output the target bounding box coordinates; Step S300: Based on the improved DeepSort algorithm, continuously track the detected pedestrian target and generate the target's motion trajectory; Step S400: Extract trajectory features from the target motion trajectory, the trajectory features including the position change of the detection box center point in the vertical axis direction; Step S500: Determine whether the pedestrian exhibits abnormal behavior based on the trajectory features and preset thresholds; Step S600: When abnormal behavior is determined, generate and send an early warning message.

[0035] This disclosed technical solution is applicable to any vertical field requiring behavioral analysis and anomaly warning for moving targets. For example, in urban rail transit platforms, it addresses the difficulty of timely detection by manual monitoring of behaviors such as crossing lines, running, jumping off platforms, and abnormal loitering during periods of high passenger flow in subway / high-speed rail stations; in prisons and detention facilities, it addresses safety hazards posed by behaviors such as fighting, climbing, abnormal gatherings, and unusual nighttime movements among detainees; in school perimeters and key areas, it addresses issues such as wall climbing, illegal intrusion, and unusual nighttime loitering; and in smart factories and industrial safety production, it enables intrusion detection in dangerous areas and monitoring of assembly line worker operating procedures. Its application scenarios can be infinitely extended to any field requiring intelligent determination of whether a target exhibits abnormal spatial displacement. It achieves more accurate detection of pedestrians and other targets in complex scenarios, more stable and continuous tracking, and more accurate and reliable identification of abnormal behavior.

[0036] This disclosure uses the detection and early warning of abnormal pedestrian behavior in a road monitoring scenario as an example to elaborate on the target behavior early warning method based on artificial intelligence. Specifically, the method includes the steps S100-S600 described above.

[0037] S100: Data Acquisition

[0038] The camera continuously captures raw video frames of the monitored scene. Each frame is in RGB format with a resolution of 1920×1080. The video stream is transmitted to the computing node in real time via a wired network. The node preprocesses each frame: it scales the image proportionally (maintaining the aspect ratio) and fills it to the model input size (e.g., 640×640), while normalizing the pixel values ​​to the [0,1] range. The preprocessed consecutive video frames are then sequentially sent to the subsequent processing module.

[0039] S200: Target Detection

[0040] The computing nodes employ a YOLOv5 model with a fusion attention mechanism to detect pedestrian targets in preprocessed video frames. This model uses YOLOv5s as a baseline and embeds convolutional block attention modules into its backbone network. Through the cascading of channel attention and spatial attention, it enhances the model's ability to extract detailed pedestrian features and contextual information, significantly reducing the false negative rate, especially in complex scenes such as pedestrian occlusion and drastic lighting changes.

[0041] The model has been trained offline: the training dataset contains labeled pedestrian images covering various weather conditions, time periods, and degrees of occlusion, and data augmentation strategies (such as random flipping, color jittering, and scale scaling) are used to improve generalization ability. Average precision (AP) is used as the evaluation metric during training, ultimately resulting in a detection model that meets real-time requirements.

[0042] During inference, the model outputs the bounding box coordinates (represented by the coordinates of the top left and bottom right corners of a rectangle) and its confidence score for each pedestrian target in each frame. In this embodiment, a confidence threshold is set, and only detection boxes with a confidence score higher than the threshold are retained for subsequent tracking.

[0043] S300: Target Tracking

[0044] The computing nodes perform cross-frame data association on pedestrian targets detected in the current frame based on the improved DeepSort algorithm, assign a unique identifier to each pedestrian and generate a continuous motion trajectory.

[0045] Specifically, for each established trajectory, a Kalman filter is used to predict its position and motion state in the current frame. The predicted state is then correlated with the detection box in the current frame. The correlation cost is a comprehensive measure of motion similarity (Mahaviron distance) and appearance similarity (cosine distance), and the optimal match is solved using the Hungarian algorithm. Appearance features are extracted by a lightweight convolutional neural network, which is pre-trained on a pedestrian re-identification dataset and can output discriminative feature vectors that distinguish different pedestrians.

[0046] Successfully matched detection boxes are used to update the state and appearance features of the corresponding trajectory; unmatched detection boxes are initialized as new trajectories; trajectories that are unmatched for multiple consecutive frames are considered to have left the scene and are deleted. Finally, the system generates motion trajectory data for each target, including trajectory ID, bounding box sequence, and center point coordinate sequence.

[0047] For pedestrian movement trajectories, trajectory features for abnormal behavior detection are extracted. In this embodiment, the core feature is the change in position of the center point of the detection box along the vertical axis. This feature is used to quantify the degree of fluctuation in the pedestrian's movement in the vertical direction. The standard deviation of the sequence formed by statistically analyzing the vertical coordinates of the center point of the detection box in each frame can be used as the quantification value of the position change. The larger the standard deviation, the more intense the pedestrian's movement in the vertical direction, and the more correlated it is with abnormal behaviors such as sudden crossing, straddling, or running.

[0048] S500: Anomaly Detection

[0049] Based on the extracted vertical axis position change σy and the preset threshold θ, it is determined whether the current pedestrian is exhibiting abnormal behavior.

[0050] The threshold θ is determined through offline statistical analysis: a large amount of trajectory data from normal pedestrians is collected, their σy values ​​are calculated and plotted, and the high quantile (e.g., the 95th percentile) of the distribution is used as the discrimination threshold. When σy > θ is detected for a certain trajectory, the pedestrian's current behavior is determined to be abnormal.

[0051] To improve the robustness of the judgment, this embodiment can further combine the characteristics of other dimensions of the trajectory (such as the standard deviation of the horizontal axis, the average speed, and the degree of trajectory tortuosity) for comprehensive judgment, but the change in the position of the vertical axis is always the most core judgment criterion.

[0052] S600: Warning Triggered

[0053] Once abnormal pedestrian behavior is detected, the system immediately triggers an alert. The alert information includes the following: Time of anomaly occurrence: System timestamp accurate to the second; Location of the anomaly: Camera number, installation location description, or GPS coordinates; Pedestrian trajectory screenshot: Capture several frames of images during the period when the anomaly occurred, and overlay the trajectory path to intuitively show the anomaly process; Anomaly type: Based on the degree of σy exceeding the limit, it can be preliminarily classified (such as "severe lateral displacement", "sudden standing up", etc.).

[0054] Warning information is sent to administrators through at least one of the following methods: Local audible and visual alarm: Controls the local alarm via GPIO interface, emitting flashing lights and a buzzer sound; Remote message push: The system sends a warning notification containing a detailed link to the mobile phone of the on-duty personnel via 4G / 5G module or WeChat Work and SMS gateway; Monitoring platform pop-up: An abnormal image will automatically pop up on the central monitoring screen, and the target trajectory will be highlighted.

[0055] This disclosure achieves fully automated early warning of abnormal pedestrian behavior in complex monitoring scenarios, improving detection accuracy: the YOLOv5 model, incorporating an attention mechanism, significantly enhances the ability to identify occluded, small-scale, and low-light targets, reducing the pedestrian false detection rate by more than 30% compared to the baseline model. Enhanced tracking continuity: the improved DeepSort algorithm, through dual constraints of motion and appearance, effectively suppresses target identity switching caused by occlusion, improving trajectory completeness and providing high-quality data for subsequent behavior analysis. Reduced false positive rate: using the change in position along the vertical axis as the core discriminative feature, it directly addresses the spatial fluctuations of abnormal behavior. Combined with statistical threshold judgment, the false positive rate is controlled below 5%, far exceeding traditional single appearance feature methods. Timely early warning response: from the occurrence of an anomaly to the delivery of the early warning information to management personnel, the end-to-end delay is less than 2 seconds, achieving "in-process response" rather than "post-event tracing," greatly improving the proactive protection capabilities of the security system.

[0056] Furthermore, the YOLOv5 model with the fused attention mechanism is constructed by replacing the C3 module in the YOLOv5 backbone network with the CBAMC3 module; The CBAMC3 module includes a channel attention submodule and a spatial attention submodule; The channel attention submodule is used to: perform average pooling and max pooling on the input feature map to obtain channel statistics, process the data through a multilayer perceptron to output channel weights, and multiply the channel weights with the input feature map channel by channel to output a channel-weighted feature map. The spatial attention submodule is used to: perform average pooling and max pooling on the channel-weighted feature map in the channel dimension to obtain a first spatial feature map and a second spatial feature map; concatenate the first spatial feature map and the second spatial feature map, process them through a convolutional layer to output spatial weights, and multiply the spatial weights with the channel-weighted feature map pixel by pixel to output the final enhanced feature map.

[0057] This embodiment uses YOLOv5s as the baseline model. YOLOv5s consists of four parts: the input network, the backbone, the neck network, and the head. The backbone network adopts the CSPDarknet structure, which includes a focus module, multiple C3 modules, and an SPP module. The C3 module is the core feature extraction unit of the backbone network, consisting of three standard convolutional layers and multiple residual units.

[0058] Construction of CBAMC3 module

[0059] In this embodiment, the C3 modules in layers 2, 4, and 6 of the YOLOv5s backbone network are replaced with CBAMC3 modules. While retaining the residual structure of the original C3 modules, the CBAMC3 modules insert a CBAM attention module after the output of each residual unit, forming a feature extraction paradigm of "convolution—residual connection—CBAM enhancement".

[0060] The specific structure of each CBAMC3 module is as follows: Input layer: Receives the feature map F∈R output from the previous network layer. C×H×W .

[0061] Three residual units: Each residual unit contains two 1×1 convolutions and one 3×3 convolution, with residual edges connecting the unit input directly to the unit output.

[0062] CBAM embedding location: Insert the CBAM attention module after the output feature map of each residual unit and before the residual summation.

[0063] Output layer: The feature map enhanced by CBAM is processed by 1×1 convolution to adjust the number of channels, and then added to the shorted branch of the module input before being output.

[0064] Channel attention submodule details

[0065] The input to the channel attention submodule is the feature map F∈R output by the residual unit. C×H×W The processing flow is as follows: Spatial information compression: Global average pooling and global max pooling are used simultaneously to compress the feature map in the H×W spatial dimension, resulting in two C×1×1 channel descriptors F. avg and F max Average pooling preserves the overall response level, while max pooling preserves the most significant response; the two are complementary.

[0066] Channel weight learning: F avg and F maxThe signals are fed into a shared multilayer perceptron. This perceptron is a two-layer fully connected network: the first layer has C / r neurons (r is the reduction rate, which is 16 in this embodiment) and uses ReLU as the activation function; the second layer has C neurons. The weight parameters of the two perceptrons are shared between the two branches.

[0067] Weight fusion and activation: The two channel weight vectors output by the perceptron are added element-wise, and then mapped to the [0,1] interval using the Sigmoid function to obtain the final channel attention weight Mc∈R. C×1×1 .

[0068] Feature recalibration: Multiply the input feature map F with the channel weights Mc channel by channel to output a channel-weighted feature map F′=Mc. F.

[0069] Spatial Attention Submodule Details

[0070] The input to the spatial attention submodule is the channel-weighted feature map F′∈R output by the channel attention submodule. C ×H×W The processing flow is as follows: Channel information compression: In the channel dimension, global average pooling and global max pooling are performed on F′ respectively to obtain two two-dimensional spatial feature maps F. avg ′∈R 1×H×W and F max ′∈R 1×H×W The former encodes the average response at each spatial location, while the latter encodes the maximum response at each spatial location.

[0071] Feature map stitching: F avg ′ and F max The spatial feature descriptor F is obtained by concatenating the features along the channel dimension. cat ∈R 2×H×W This tensor fully preserves the mean and extreme value information after channel compression.

[0072] Spatial weight generation: for F cat A convolution operation is performed with a 7×7 kernel (a 3×3 kernel can also be used to balance efficiency and performance), a stride of 1, and the same padding. The output channel is 1. The convolutional layer is followed by a Sigmoid activation function to generate spatial attention weights Ms∈R. 1×H×W Each pixel value represents the importance of that location.

[0073] Feature Adaptive Weighting: Multiply the spatial weight Ms by the channel-weighted feature map F′ pixel by pixel (the spatial weight is broadcast along the channel dimension), and output the final enhanced feature map F′′=Ms. F′.

[0074] Network integration location optimization

[0075] This embodiment uses ablation experiments to determine the optimal insertion position of the CBAMC3 module. The experiments show that: Inserted only in the deep layers of the backbone network (after layer 6), it has limited gain for small target detection; Inserting only in the shallow layers of the backbone network (before layer 2) significantly increases computational cost and results in insufficient feature semantics; The best strategy is to replace the CBAMC3 module with the second layer (shallow details), the fourth layer (middle semantics), and the sixth layer (deep semantics) to form a multi-level attention feature pyramid, which takes into account both detail sensitivity and semantic abstraction ability.

[0076] Dataset Construction

[0077] The training dataset consists of 2000 manually annotated images, covering the following complex scenarios: Obstruction scenarios: Pedestrians are obstructed by trees, vehicles, and other pedestrians (accounting for 30%). Scenes with changing lighting: backlight, strong light, nighttime infrared, and shadow areas (accounting for 35%); Scale-changing scenarios: small targets at a distance and large targets in the foreground (accounting for 20%). Dense scenarios: large gatherings of many people, localized congestion (accounting for 15%).

[0078] The annotation format adopts YOLO format, and each target record includes the category ID (pedestrian = 0) and the normalized bounding box center coordinates, width, and height.

[0079] Data augmentation strategy

[0080] To improve the model's generalization ability, online data augmentation was used during training: Geometric transformations: random horizontal flip (probability 0.5), random scaling (0.8~1.2 times), random translation (±10%). Color adjustment: HSV color dithering, brightness adjustment ±10%, saturation adjustment ±15%, hue adjustment ±10%; Occlusion simulation: Random Erasing generates random rectangular blocks in the image and fills them with noise to simulate local occlusion; Stitching Enhancement: Mosaic data enhancement randomly stitches four images together into one, enriching the small target samples.

[0081] Training parameter configuration

[0082] Pre-trained weights: Load pre-trained YOLOv5s weights from the COCO dataset; Optimizer: SGD, initial learning rate 0.001, momentum 0.937, weight decay 0.0005; Learning rate scheduling: The first 5 rounds are the warm-up phase, with the learning rate increasing linearly to 0.001; from rounds 5 to 150, it remains at 0.001; from rounds 151 to 200, cosine annealing decays it to 0.0001. Batch size: 8; Training rounds: 200 rounds; Loss function: The native loss function of YOLOv5 is adopted, which includes bounding box regression loss (CIoU), confidence loss (BCE), and classification loss (BCE), which are weighted and summed. Early stopping strategy: Terminate training early when the AP on the validation set shows no improvement for 10 consecutive rounds.

[0083] Evaluation indicators

[0084] Average precision (AP) was used as the primary evaluation metric, and the Intersection over Union (IoU) threshold was set to 0.5. Bounding box loss, confidence loss, and object detection speed (FPS) were also monitored.

[0085] After training, the model achieved an AP@0.5 of 94.2% on the validation set, a 4.5 percentage point improvement over the original YOLOv5s (AP@0.5=89.7%). The inference speed on the NVIDIA Jetson Xavier NX platform was 32ms / frame (approximately 31FPS), meeting the requirements for real-time monitoring.

[0086] Through the cascaded attention mechanism of the CBAMC3 module, the model performs well in the following challenging scenarios: Occlusion scenarios: when pedestrians are occluded by more than 50%, the recall rate is improved by 23% compared to the baseline; Small target scenarios: for pedestrians with a pixel height of <40px, the AP is improved by 31%; Dramatic lighting scenarios: under backlight / strong shadow, the false positive rate is reduced by 42%.

[0087] The attention heatmap output by the CBAM module reveals the following: Channel attention: automatically enhances the response to discriminative parts of the pedestrian, such as the head, shoulders, and hands, while suppressing irrelevant channels such as background textures and ground markings; Spatial attention: focuses on the pedestrian outline area, weakens occlusions and noise interference, and enables the model to "know where to look"; Cascade effect: channel attention first filters "what to look at", and spatial attention then locates "where to look", the two work together to make the features more discriminative.

[0088] The CBAM module is a lightweight, plug-and-play module that adds only a small amount of parameters (approximately 2.1%) and computation (approximately 3.5% FLOPs), without altering the parallel computing structure of YOLOv5, and has a negligible impact on inference speed. Compared to stacking more convolutional layers in the backbone network or introducing Transformer modules, this approach offers a significant advantage in terms of accuracy / efficiency balance.

[0089] The model trained in this embodiment can be directly transferred to other monitoring scenarios (such as campuses, parks, and shopping malls) without retraining or requiring only minor fine-tuning. Testing on the CityPersons public dataset (which was not used in the training) yielded an AP@0.5 score of 87.6%, demonstrating its excellent scene generalization ability.

[0090] Furthermore, the replacement of the C3 module in the YOLOv5 backbone network with the CBAMC3 module specifically includes: Replace the C3 modules in layers 2, 4, and 6 of the YOLOv5s backbone network with CBAMC3 modules; Each of the CBAMC3 modules contains 3 residual units, and a CBAM attention module is inserted after the output of the residual units.

[0091] This embodiment does not indiscriminately replace all C3 modules, but determines the optimal replacement set through theoretical analysis and ablation experiments, following the following three core principles: Principle 1: Focus on superficial details and avoid excessive intervention. The first layer (Focus) and the second layer (C3 module) are responsible for extracting low-level visual features such as edges, corners, and colors. Introducing CBAM attention in the second layer allows for the establishment of perceptual biases towards discriminative regions such as pedestrian contours, heads, and shoulders early in feature extraction, enabling subsequent layers to focus on effective features. However, replacing the first layer (Focus) would compromise the integrity of the downsampling structure and drastically increase computational costs; replacing the third layer and subsequent shallower layers would miss the opportunity for early feature guidance.

[0092] Principle Two: The middle layer serves as a bridge between the upper and lower levels, requiring dual enhancement.

[0093] Layers 4 and 6 are central to the feature pyramid, corresponding to 80×80 (sensitive to small targets) and 40×40 (sensitive to medium targets) feature maps, respectively. The outputs of these two layers are directly fed into the neck network for multi-scale feature fusion. Embedding CBAM here accomplishes two things simultaneously: first, it refines the shallow features passed from upstream with attention; second, it provides cleaner feature input for downstream FPN fusion.

[0094] Principle 3: Deep parameters are sparse and should not be excessively disturbed.

[0095] Layers 7-10 are deep semantic layers with a large number of channels and parameters. Experiments show that replacing the CBAMC3 module at these layers results in a marginal decrease in accuracy gain (<0.3% AP) while significantly increasing inference latency (>15%). Furthermore, deep features already possess strong semantic abstraction capabilities, and the gain of CBAM is diluted by the residual structure. Therefore, this embodiment forgoes replacing layers 7-10 to achieve a better speed-accuracy balance.

[0096] In this embodiment, the CBAM attention module is inserted inside each residual unit.

[0097] Each CBAMC3 module contains three identical residual units, which are connected in series and share the same design paradigm.

[0098] Each residual unit is independently configured with a CBAM module, and the attention weights are specifically adapted to the feature distribution of the current residual branch. The first residual unit focuses on the basic contour, the second residual unit enhances local components, and the third residual unit refines the global structure, forming a "three-level enhancement" attention pyramid.

[0099] The CBAM module is located inside the residual connection, and its output augmented features are directly added to the residual input. Gradients can be backpropagated through two paths (the main path with CBAM + the identity mapping path), avoiding the attention module becoming a bottleneck for gradient propagation, resulting in more stable training and faster convergence.

[0100] Each CBAM module increases the number of parameters by only about 2.1% (based on the C3 module). 3 residual units × 3 replacement layers = 9 CBAM modules, with the total parameter increase controlled within 0.3M, which is far lower than the design of stacking convolutional layers or Transformers outside the modules.

[0101] Furthermore, the improved DeepSort algorithm includes the following steps: A Kalman filter is used to predict the position and velocity of a pedestrian target, and a state equation is established to obtain the target prediction state in the current frame. The state equation is as follows:

[0102] Where a is the state transition matrix, b is the control matrix, and W is the control matrix. k For process noise, Let be the system state vector at time k. This is the control input vector at time k; The Hungarian algorithm is used to associate the predicted state of the target with the current detection box, and the matching cost is the weighted sum of Mahalanobis distance and cosine distance; A trained feature extraction network is used to extract appearance features from the detection boxes. The feature extraction network uses ResNet50 as the backbone, removes the last fully connected layer, outputs a 512-dimensional feature vector and performs L2 normalization. The cross-entropy loss function is used and the optimizer is SGD. The network is trained on the pedestrian re-identification dataset Market-1501.

[0103] The target tracking is based on the improved DeepSort algorithm, which first predicts the pedestrian's position and velocity using a Kalman filter. The state equation is:

[0104] Where a is an 8×8 state transition matrix, b is the control matrix (set to 0), and Wk is the process noise; State vector definition The motion state of the pedestrian target is described using an eight-dimensional state vector: X k =[x,y,a,h,vx,vy,va,vh] T Where (x,y) are the coordinates of the center point of the detection box, a is the aspect ratio, h is the height, and (vx,vy,va,vh) are the corresponding velocity components.

[0105] The state transition matrix a adopts a constant velocity model, mapping the position and velocity of the previous frame to the current frame; The control matrix b is set to zero, and the control input U k The value is always zero (in surveillance scenarios, it is impossible to control pedestrians). Process noise W k The covariance matrix was set as a diagonal matrix and its values ​​were determined experimentally.

[0106] Predicted output

[0107] First, Kalman filtering is performed on all confirmed trajectories to predict the target prediction state (predicted box position) for the current frame, which serves as the input for subsequent data association.

[0108] The target state predicted by the Kalman filter is associated with the detection box output by the detector in the current frame. The matching cost is a weighted sum of Mahalanobis distance and cosine distance. c=λ d mahalanobis +(1-λ) d cosine

[0109] The value of λ is set to 0.5 to balance the contributions of motion similarity and appearance similarity.

[0110] Mahalanobis distance measures the statistical distance between the predicted bounding box and the detected bounding box in motion space. It is calculated using the innovation covariance matrix of the Kalman filter and can effectively measure motion consistency.

[0111] The cosine distance measures the similarity between the historical appearance features of the trajectory and the appearance features of the current detection box. The trajectory appearance features are maintained as the exponential moving average of the most recently successfully matched frames.

[0112] The Hungarian algorithm is used to perform global optimal matching of the cost matrix to obtain a one-to-one correspondence between the detection box and the trajectory.

[0113] The feature extraction network uses ResNet50 as its backbone and undergoes the following modifications: Remove the global average pooling layer and the 1000-dimensional classification layer from the original network; An adaptive average pooling layer is added after the convolutional output to output 2048-dimensional features. By connecting a fully connected layer, the features are compressed to 512 dimensions; Finally, an L2 normalization layer is added to constrain the output features to a unit hypersphere, which facilitates the calculation of cosine distance.

[0114] Training configuration: The network was trained on the Market-1501 person re-identification dataset. Loss function: Cross-entropy loss; Optimizer: SGD, initial learning rate 0.1, momentum 0.9, weight decay 0.0005; Training rounds: 120 rounds; Data augmentation: random flipping, random erasure, and normalization.

[0115] The trained network extracts a 512-dimensional normalized feature vector for each detection box, which is used to calculate the cosine distance. On edge devices, a single feature extraction takes approximately 3ms, meeting real-time requirements.

[0116] This disclosure achieves improved tracking stability through an improved DeepSort algorithm: the synergistic effect of Kalman filtering and weighted cost association strategy reduces the number of identity switching events by more than 50% in pedestrian occlusion scenarios, ensuring continuous and complete trajectories. Discriminative feature assurance is also achieved: the feature extraction network trained on Market-1501 based on ResNet50 outputs features with significantly better discriminative power than the simple CNN native to DeepSort, and the cosine distance threshold can be tightened to 0.2, greatly reducing false matches.

[0117] Furthermore, The matching parameters of the Hungarian algorithm are set as follows: Mahalanobis distance threshold is set to 9.4877, cosine distance threshold is set to 0.2, and maximum cosine distance is set to 0.1. During the continuous tracking of detected pedestrian targets, the minimum detection confidence level was set to 0.3, the non-maximum suppression threshold was set to 0.5, and the maximum trajectory survival time was set to 70 frames.

[0118] The Hungarian algorithm matching parameters are set as follows: The Mahalanobis distance threshold is 9.4877, derived from the chi-square distribution statistical principle. With an observation vector dimension of 4 (detection box center point x, y, aspect ratio, height), a chi-square distribution quantile of 4 degrees of freedom and 95% confidence is 9.4877. This threshold is used to filter motion consistency: when the Mahalanobis distance between the predicted box and the detected box exceeds this value, the motion characteristics of the two are considered too different, and association is directly rejected.

[0119] The cosine distance threshold is 0.2, determined based on feature distribution analysis of the Market-1501 dataset. Statistics show that the mean cosine distance for the same pedestrian is 0.12 with a standard deviation of 0.06; the mean cosine distance for different pedestrians is 0.78 with a standard deviation of 0.11. Using 0.2 as the threshold achieves the best balance between high recall and low false positives; only detection boxes with a cosine distance below 0.2 are considered visually similar.

[0120] Maximum cosine distance: 0.1. This parameter is used for numerical truncation of the cost matrix. Any cosine distance greater than 0.1 is directly assigned a value of 0.1 when calculating the matching cost. This avoids abnormal perturbations to the Hungarian algorithm's assignment results caused by extremely dissimilar samples, making the association decision more stable. Experimental results show that this setting can further reduce the number of identity switching by approximately 12%.

[0121] The parameter settings for the entire target tracking process are as follows: The minimum detection confidence threshold was set to 0.3. This lowers the detection confidence threshold of the YOLOv5 model from the default 0.5 to 0.3, aiming to recall more partially occluded and low-confidence pedestrian targets. Experimental data shows that this adjustment improved the pedestrian recall rate by 11.3%, while the false positive rate only increased by 1.7%. Downstream tracking algorithms can effectively filter false positives through motion constraints and appearance features, resulting in significant overall benefits.

[0122] Non-maximum suppression threshold: 0.5. The standard NMS algorithm is used to merge highly overlapping detection boxes of the same type. The IoU threshold is set to 0.5, which achieves a good balance between recall and redundancy suppression.

[0123] Maximum trajectory survival time: 70 frames. When a trajectory fails to match any detection box for 70 consecutive frames, the target is determined to have left the scene and the trajectory is deleted. This parameter is based on actual measured statistics of the maximum occlusion duration in the monitored scene: typical occlusion (such as a pedestrian being briefly occluded by a vehicle) usually does not exceed 3 seconds (75 frames @ 25fps). 70 frames can preserve the trajectory in most occluded scenarios, while avoiding deadlocking of long-term disconnected trajectories.

[0124] Furthermore, the change in position of the center point of the detection frame along the vertical axis is calculated in the following way: After continuously tracking N frames, record the ordinates y1, y2, ..., y3 of the center point of the pedestrian detection box in each frame. N ; Calculate the mean of the ordinate. Calculate the standard deviation σy along the vertical axis using the following formula:

[0125] Where N is the preset number of continuous tracking frames; σy is used as the change in position of the center point of the detection frame along the vertical axis.

[0126] During target tracking, once a pedestrian's trajectory has been consistently and stably tracked for more than a preset number of frames, the feature extraction process is initiated. This embodiment uses continuous tracking of 30 frames as an example for illustration.

[0127] Vertical coordinate sequence recording: For the pedestrian trajectory with target ID ID001, backtrack 30 consecutive frames from the current frame, and record the vertical coordinate (y-value in the image coordinate system) of the center point of the pedestrian detection box in each frame, resulting in the sequence: y1, y2, ..., y 30

[0128] Calculate the standard deviation σy in the vertical direction according to the σy calculation formula, i=30.

[0129] This formula measures the degree of dispersion of the ordinate sequence relative to the mean. If the calculated value is σy≈1.2, it indicates that the pedestrian's movement along the ordinate is smooth.

[0130] The degree of vertical axis fluctuation is quantified by standard deviation, transforming the intensity of pedestrian vertical movement into a single numerical indicator. This indicator is strongly correlated with abnormal behaviors (such as sudden crossing, jumping, and falling), providing a quantitative basis for subsequent threshold determination.

[0131] Furthermore, the preset number of continuous tracking frames ranges from 20 to 30 frames; When the vertical coordinate of a frame is missing due to occlusion, the average of the vertical coordinates of the preceding and following frames is used to fill the missing coordinates.

[0132] The value of the number of continuous tracking frames N is dynamically adjusted according to the characteristics of the monitoring scenario and real-time requirements: When N=20 frames: It is suitable for scenes with rapid motion changes (such as intersections and shopping mall entrances), and can respond to behavior changes faster, with anomaly detection latency of about 0.8 seconds (@25fps). When N=30 frames: It is suitable for scenes with relatively smooth motion (such as passages and corridors), with more sufficient statistical samples, more stable anomaly detection, and a delay of about 1.2 seconds.

[0133] This embodiment uses N=30 frames by default to achieve a balance between statistical significance and response speed.

[0134] During pedestrian tracking, brief occlusion (such as being obscured by vehicles or other pedestrians) may cause missing detection boxes in a few frames, resulting in gaps in the corresponding y-coordinate records. For example, a trajectory should record 30 frames, but due to occlusion, detection boxes in frames 15 and 16 are missing. The missing data is handled using the average of preceding and following frames: y i =(y i-1 +y i+1 ) / 2.

[0135] Furthermore, determining whether a pedestrian exhibits abnormal behavior based on the trajectory features and a preset threshold includes: A preset threshold θ for the standard deviation σy in the vertical direction is set, and the preset threshold θ is determined by statistically analyzing the distribution of σy in the normal behavioral trajectories in the training set. When σy > θ is detected, it is initially determined to be abnormal behavior; The trajectory features are input into a pre-trained behavior prediction model to obtain the probability of abnormal behavior prediction. When the predicted probability of abnormal behavior exceeds a preset probability threshold, the abnormal judgment result is enhanced, and the final abnormal behavior label is output.

[0136] Determining the preset threshold θ: During the offline phase, at least 1000 trajectory samples of normal pedestrians in the monitored scene are collected. The standard deviation σy along the vertical axis is calculated for each trajectory, and the distribution of σy values ​​for all normal trajectories is statistically analyzed. The 95th percentile of this distribution is used as a preset threshold θ. For example, if the measured normal trajectory σy values ​​in a certain scene are concentrated between 1 and 15, and the 95th percentile is 32, then θ = 32.

[0137] Preliminary assessment (threshold filtering)

[0138] During the real-time processing phase, for each pedestrian trajectory that has been continuously tracked for more than N frames, its current σy value is calculated. If σy > θ, the trajectory is marked as "suspected abnormal" and proceeds to the next level of judgment; otherwise, it is directly judged as "normal" and no further processing is triggered.

[0139] This threshold filtering can quickly exclude the vast majority of normal walking behaviors, significantly reducing the frequency of subsequent behavior prediction model calls and saving computing resources.

[0140] Enhanced decision-making in behavior prediction models

[0141] For trajectories suspected of being abnormal, their multidimensional feature vectors are extracted and input into a pre-trained behavior prediction model. The model outputs an abnormal probability value between 0 and 1. If the probability value exceeds a preset probability threshold (e.g., 0.8), the abnormality judgment result is strengthened, and a "normal behavior" label is finally output; if the probability value is below the threshold, the "normal" judgment is maintained.

[0142] The relationship of dual determination: First hurdle (threshold): High recall and low overhead, ensuring that no abnormal behavior is missed; The second stage (model): high accuracy and low false alarms, to finely identify suspected samples.

[0143] The two are linked together to form a progressive discrimination process of "coarse screening + fine judgment".

[0144] This disclosed embodiment achieves a significant reduction in false alarm rate through dual-judgment. A single threshold method easily misclassifies normal behaviors with large trajectory fluctuations (such as obstacle avoidance) as abnormal. After introducing a behavior prediction model, secondary screening using multi-dimensional features reduces the false alarm rate by over 60%. Computational efficiency is optimized: the behavior prediction model is a lightweight neural network, but still incurs fixed computational overhead. Threshold filtering can intercept over 90% of normal trajectories, reducing the model call frequency to only 1 / 10 of the original scheme, and lowering the overall CPU load by approximately 35%. Decision interpretability is enhanced: the dual-judgment result can be broken down into two sub-reasons: "σy exceeds the limit" and "model probability exceeds the limit," facilitating management personnel's understanding of the alarm logic and enabling subsequent algorithm iteration and optimization.

[0145] Furthermore, The preset threshold θ is the 95th percentile of the normal behavior trajectory σy value; The preset probability threshold is 0.8; The behavior prediction model is a two-layer fully connected neural network with 128 neurons in the hidden layer and ReLU activation function. The input features of the behavior prediction model include: standard deviation σy in the vertical direction, standard deviation σx in the horizontal direction, average speed, and trajectory length; The behavior prediction model is trained using a confidence-based step loss function.

[0146] Statistical determination of the preset threshold θ

[0147] Collect surveillance video continuously for 7 days in the target application scenario (such as an urban road intersection) and extract no fewer than 1,000 complete trajectories of normal pedestrians. "Normal" means without abnormal actions such as running, crossing, loitering, or jumping.

[0148] Calculate the σy value for each trajectory and plot its histogram. The σy distribution of normal trajectories is right-skewed, with the vast majority concentrated in the lower interval. Using the 95th percentile as the threshold θ, only 5% of normal trajectories will be misjudged as suspected anomalies due to random fluctuations.

[0149] The threshold θ can be periodically recalculated as the season and scene change to maintain scene adaptability.

[0150] Structure and Training of Behavioral Prediction Models

[0151] The model architecture uses a two-layer fully connected neural network: Input layer: 4 neurons, corresponding to 4-dimensional input features; Hidden layer: 128 neurons, activation function is ReLU; Output layer: 1 neuron, activation function is sigmoid, outputs the probability of anomalies.

[0152] The total number of parameters is approximately (4×128 + 128) + (128×1 + 1) = 640 + 129 = 769, making it an extremely lightweight model with a single inference time of <0.1ms.

[0153] Eighty hundred normal trajectories and 160 abnormal trajectories (including traversing, wandering, running, jumping, etc.) were collected and divided into training and validation sets in a 7:3 ratio. The above four-dimensional features were calculated for each trajectory and labeled (0 normal / 1 abnormal).

[0154] Loss function: Credibility-based step loss function

[0155] To address the issues of blurred boundaries between normal and abnormal samples and the milder manifestation of some abnormal behaviors, this embodiment employs a custom step loss function: L = -log(p) e -δ (p-0.5) Real label = 1; L = -log (1-p) e δ (p-0.5) , Real label = 0.

[0156] Where p is the model prediction probability, and δ is the step intensity coefficient (0.3 in this embodiment).

[0157] The characteristics of this loss function are: it imposes a greater penalty on high-confidence error samples, forcing the model to be cautious about boundary samples; it encourages the model to output clear decisions close to 0 or 1, avoiding ambiguous probability outputs; according to experimental comparison, the validation set accuracy improved by 3.2% after adopting this loss function, reaching 89.5%.

[0158] Training configuration: Optimizer: Adam, learning rate 0.001; batch size: 32; training epochs: 100 epochs; early stopping policy: terminate if the validation set loss does not decrease for 10 consecutive epochs.

[0159] The preset probability threshold is 0.8. This value is determined by the precision-recall curve (PR curve) of the validation set: the threshold point that maximizes precision while ensuring a recall rate ≥ 90% is selected. In this embodiment, 0.8 is the optimal balance point.

[0160] The threshold θ in this embodiment is based on actual scene measurements and statistical analysis, while the probability threshold is optimized based on the PR curve. The model structure is lightweight and standard, the loss function is clearly defined, there is no black-box parameter tuning, and it has complete technical reproducibility. On the validation set, the combined application of the dual-judgment mechanism achieves an accuracy of 89.5%, a recall of 91.2%, and a false positive rate of 4.3%, significantly outperforming the single feature threshold method (accuracy 76.8%, false positive rate 18.7%). The behavior prediction model has fewer than 800 parameters, and the time consumed by a single inference on the CPU is negligible, having no impact on the overall processing frame rate of the early warning system. It has strong transferability: the model only relies on trajectory statistical features and does not depend on specific scene visual features, allowing it to be quickly adapted to other monitoring scenarios. Only re-statistical threshold θ (scene adaptation) and fine-tuning of the model (with a small number of samples) are required.

[0161] This disclosure presents a complete technical solution encompassing "precise detection, stable tracking, multi-dimensional features, dual discrimination, and rapid early warning," achieving efficient identification and proactive early warning of abnormal target behavior in complex scenarios. In the target detection stage, a YOLOv5 model integrating the CBAM attention mechanism is employed, significantly improving target detection rates under complex conditions such as occlusion and lighting changes through cascaded enhancement of channel and spatial attention. In the target tracking stage, an improved DeepSort algorithm introduces dual constraints of motion and appearance, effectively suppressing frequent identity switching, significantly improving trajectory continuity and completeness, and drastically reducing the number of identity switching events. In the feature extraction stage, the standard deviation of the vertical axis is extracted from the motion trajectory as the core feature, quantifying the degree of vertical fluctuation into a calculable index, providing a more discriminative basis for anomaly discrimination. In the anomaly discrimination stage, a dual mechanism of "threshold coarse screening + behavior prediction model fine judgment" is adopted, combining statistical thresholds and lightweight neural networks, reducing the false positive rate to below 5% and achieving an accuracy of 89.5%. In the early warning response stage, a combination of local audible and visual alarms and remote text and image push notifications achieves second-level response and precise positioning. This solution systematically addresses the pain points of existing technologies, such as "missed detection, incomplete tracking, inaccurate judgment, and slow response," and significantly improves the proactive protection capabilities and reliability of intelligent monitoring systems.

[0162] Example 2

[0163] Embodiment 2 of this disclosure provides a target behavior early warning method based on artificial intelligence, the process of which is as follows: Figure 2 As shown, it includes: Data acquisition phase: A high-definition camera with a resolution of 1920×1080 and a frame rate of 25fps is selected and installed at a height of 5 meters at the road intersection with a horizontal downward angle of 30° to ensure coverage of pedestrian activity areas within a 50-meter range. The camera continuously acquires road monitoring video and outputs video frames, with each frame stored in JPEG format and a pixel value range of 0-255.

[0164] In the object detection stage, a YOLOv5 model with CBAM attention mechanism is adopted. This model replaces the C3 module in the YOLOv5 backbone network with the CBAMC3 module. The channel attention submodule first performs average pooling and max pooling on the input feature map to obtain two channel statistical vectors. After processing by a multilayer perceptron with 64 neurons, the output channel weights are multiplied with the original feature map channel by channel. The spatial attention submodule performs average pooling and max pooling on the channel-weighted feature map in the channel dimension to obtain two spatial feature maps. After concatenation, the spatial weights are output by 3×3 convolutional layers to reduce the dimensionality, and then multiplied with the feature map pixel by pixel.

[0165] 2000 labeled images were divided into training and validation sets in an 8:2 ratio. The training was performed 200 times with a batch size of 8 and an initial learning rate of 0.001. The average accuracy was used as the evaluation metric. The final output was the coordinates of the top left and bottom right corners of the pedestrian target bounding box.

[0166] Target tracking phase: Based on the improved DeepSort algorithm, the pedestrian's position and velocity are first predicted using Kalman filtering. The state equation is:

[0167] in It is an 8×8 state transition matrix. This is the control matrix (set to 0). This is process noise; The Hungarian algorithm is then used to match the predicted trajectory with the current detection box. The matching cost is the weighted sum of Mahalanobis distance and cosine distance. The feature extraction network is trained using the Market-1501 dataset with 120 training epochs and an initial learning rate of 0.1. The tracking success rate is used as the evaluation metric to generate a motion trajectory containing frame_id, target_id and center point coordinates.

[0168] Feature extraction stage: After continuously tracking for 30 frames, record the center point of the pedestrian detection box in each frame. coordinate arrive Calculate the mean Standard deviation along the vertical axis:

[0169] Anomaly detection stage: The preset threshold θ=32 for σy (determined by statistically analyzing the 95th percentile of 1000 normal trajectories). When σy > 32, combined with the behavior prediction model trained by the confidence-based step loss function, if the abnormal probability output by the model exceeds 0.8, it is judged as abnormal behavior.

[0170] Warning Triggering Phase: After the warning is triggered, a warning message is generated that includes the time of the abnormality (accurate to the second), the location (camera number and latitude and longitude), and a screenshot of the pedestrian's trajectory. The message is sent via local audible and visual alarms (alarm flashing frequency 2Hz, sound pressure level 85dB) and remote SMS notification (sending an SMS with details to the mobile phone of the management personnel).

[0171] Example 3

[0172] The construction details of the YOLOv5 model incorporating the CBAM attention mechanism for object detection are as follows: The initial YOLOv5s network includes Mosaic data augmentation at the input, a backbone network consisting of Focus, three C3 modules, and SPP, a neck network of FPN+PAN, and an output of YOLOHead. The C3 modules in layers 2, 4, and 6 of the backbone network are replaced with CBAMC3 modules. Each CBAMC3 module contains three residual units, and a CBAM attention module is inserted after the output of the residual units.

[0173] During model training, 2000 labeled images were preprocessed, including random horizontal flipping with a probability of 0.5, random cropping with scaling range of 0.8-1.2, HSV color jitter adjustment of brightness ±10%, saturation adjustment ±15%, and hue adjustment ±10°. The labeled files were converted from VOC format to YOLO format, and each target label was "0 x_center y_centerwidth height" with coordinates normalized to 0-1.

[0174] The training process initially loads YOLOv5s pre-trained weights. The first 5 rounds are warm-up training with the learning rate increasing linearly to 0.001. From rounds 5 to 150, the learning rate is kept at 0.001. From rounds 151 to 200, it is reduced to 0.0001. After each round of training, AP is calculated on the validation set with IoU of 0.5. When AP does not improve for 10 consecutive rounds, the learning rate is decayed. The model converges when the Box_loss is less than 0.05 and the Objectness is less than 0.03, achieving a detection speed of 30fps.

[0175] Example 4

[0176] The improved DeepSort algorithm for target tracking is implemented as follows: The feature extraction network uses ResNet50 as the backbone, removes the last fully connected layer, outputs a 512-dimensional feature vector, and performs L2 normalization. It is trained using the Market-1501 dataset, with 12,936 images from 751 people in the training set and 19,732 images from 750 people in the test set. Image preprocessing includes resizing to 128×64, random horizontal flipping, and standardization with mean [0.485, 0.456, 0.406] and standard deviation [0.229, 0.224, 0.225]. The cross-entropy loss function is used, and the optimizer is SGD with a learning rate of 0.1, momentum of 0.9, and weight decay of 0.0005. After 120 training rounds, the mAP on the test set reaches 83.5%.

[0177] In Kalman filtering, the observation equation is: , The observation matrix is ​​4×8, retaining the center point x, y, aspect ratio, and height corresponding rows, with measurement noise. The covariance matrix R is a diagonal matrix with elements [1, 1, 0.01, 0.01]. The Kalman gain during state update... pass:

[0178] calculate.

[0179] When using the Hungarian algorithm for matching, the Mahalanobis distance threshold is set to 9.4877, the cosine distance threshold is set to 0.2, the maximum survival time is 70 frames, and the number of frames lost is 3 frames.

[0180] Example 5: In the feature extraction step, the number of frames N for continuous tracking can be adjusted for different scenarios. When the pedestrian's movement is relatively stable, N is set to 30, and when the movement is more intense, N is set to 20.

[0181] When calculating the standard deviation of the vertical axis, if a frame is affected by occlusion... Coordinates are missing; use the coordinates from the preceding and following frames. Coordinate mean fill is used to ensure data integrity.

[0182] For example, the trajectory of a normal pedestrian The coordinates are [130, 131, 129, ..., 132]. The calculation yields... =130.5, σy=1.2; a certain abnormal pedestrian trajectory The coordinates are [130, 150, 120, ..., 160]. The calculation yields... =135, σy=35.6, and the x-axis coordinate sequence and trajectory length, i.e. the sum of Euclidean distances within 30 frames, are recorded as auxiliary features.

[0183] Example 6: Threshold in anomaly detection The determination process involved collecting 1000 normal trajectories (including straight-line and normal turns) and 200 abnormal trajectories (including sudden crossings and hesitations). The σy value was calculated for each trajectory and a histogram was plotted. The σy values ​​for normal trajectories were concentrated between 1 and 15, while those for abnormal trajectories were concentrated between 30 and 60. The 95th percentile of the σy value for normal trajectories, 32, was taken as the threshold value. .

[0184] The behavior prediction model employs a two-layer fully connected network with 128 hidden neurons and ReLU activation function. The inputs are [σx, σy, average velocity, trajectory length], and the output is the anomaly probability. The training set contains 800 normal trajectories and 160 anomalous trajectories, split in a 7:3 ratio. A confidence-based step loss function is used. =0.3, trained for 100 rounds, batch_size=32, initial learning rate 0.001, validation set accuracy reached 89.5%, and when σy>32 and prediction probability>0.8, it was judged as an anomaly.

[0185] Example 7

[0186] The early warning system hardware includes a front-end 1080PPoE camera, an edge computing box equipped with NVIDIA Jetson Nano for detection and tracking, an Intel Xeon E5 back-end server for feature extraction and anomaly detection, an audible and visual alarm, and a GSM module.

[0187] When an early warning message is generated, the server extracts the target's trajectory from the trajectory database for the past 10 frames, draws a trajectory overlay map using OpenCV, obtains location information from the camera's GPS, and obtains the time of the anomaly from the system time.

[0188] After the warning is triggered, the edge computing box controls the sound and light alarm through the GPIO interface and triggers it with a high level for 5 seconds. The backend server sends a text message through the Huawei ME909s-821 GSM module, sets the text mode using AT commands, and sends a warning message containing time, location, and trajectory screenshot. The server also records the warning log, including time, location, and processing status.

[0189] Example 8

[0190] This disclosure provides a target behavior early warning system based on artificial intelligence, such as Figure 3 As shown, the system includes: Data acquisition module 11 is configured to acquire surveillance video data through a camera and output continuous video frames; The target detection module 12 is configured to use a YOLOv5 model with a fusion attention mechanism to detect pedestrian targets in the video frame and output the target bounding box coordinates. The target tracking module 13 is configured to continuously track detected pedestrian targets based on the improved DeepSort algorithm and generate target motion trajectories; Feature extraction module 14 is configured to extract trajectory features from the target motion trajectory, the trajectory features including the position change of the detection box center point in the vertical axis direction; The anomaly detection module 15 is configured to determine whether a pedestrian exhibits abnormal behavior based on the trajectory features and a preset threshold. The early warning module 16 is configured to generate and send early warning information when abnormal behavior is detected.

[0191] Furthermore, the YOLOv5 model with the fused attention mechanism is constructed by replacing the C3 module in the YOLOv5 backbone network with the CBAMC3 module; The CBAMC3 module includes a channel attention submodule and a spatial attention submodule; The channel attention submodule is used to: perform average pooling and max pooling on the input feature map to obtain channel statistics, process the data through a multilayer perceptron to output channel weights, and multiply the channel weights with the input feature map channel by channel to output a channel-weighted feature map. The spatial attention submodule is used to: perform average pooling and max pooling on the channel-weighted feature map in the channel dimension to obtain a first spatial feature map and a second spatial feature map; concatenate the first spatial feature map and the second spatial feature map, process them through a convolutional layer to output spatial weights, and multiply the spatial weights with the channel-weighted feature map pixel by pixel to output the final enhanced feature map.

[0192] Furthermore, the replacement of the C3 module in the YOLOv5 backbone network with the CBAMC3 module specifically includes: Replace the C3 modules in layers 2, 4, and 6 of the YOLOv5s backbone network with CBAMC3 modules; Each of the CBAMC3 modules contains 3 residual units, and a CBAM attention module is inserted after the output of the residual units.

[0193] Furthermore, the improved DeepSort algorithm includes the following steps: A Kalman filter is used to predict the position and velocity of a pedestrian target, and a state equation is established to obtain the target prediction state in the current frame. The state equation is as follows:

[0194] Where a is the state transition matrix, b is the control matrix, and W is the control matrix. k For process noise, Let be the system state vector at time k. This is the control input vector at time k; The Hungarian algorithm is used to associate the predicted state of the target with the current detection box, and the matching cost is the weighted sum of Mahalanobis distance and cosine distance; A trained feature extraction network is used to extract appearance features from the detection boxes. The feature extraction network uses ResNet50 as the backbone, removes the last fully connected layer, outputs a 512-dimensional feature vector and performs L2 normalization. The cross-entropy loss function is used and the optimizer is SGD. The network is trained on the pedestrian re-identification dataset Market-1501.

[0195] Furthermore, The matching parameters of the Hungarian algorithm are set as follows: Mahalanobis distance threshold is set to 9.4877, cosine distance threshold is set to 0.2, and maximum cosine distance is set to 0.1. During the continuous tracking of detected pedestrian targets, the minimum detection confidence level was set to 0.3, the non-maximum suppression threshold was set to 0.5, and the maximum trajectory survival time was set to 70 frames.

[0196] Furthermore, the feature extraction module 14 is specifically configured as follows: After continuously tracking N frames, record the ordinates y1, y2, ..., y3 of the center point of the pedestrian detection box in each frame. N ; Calculate the mean of the ordinate. Calculate the standard deviation σy along the vertical axis using the following formula:

[0197] Where N is the preset number of continuous tracking frames; σy is used as the change in position of the center point of the detection frame along the vertical axis.

[0198] Furthermore, the preset number of continuous tracking frames ranges from 20 to 30 frames; When the vertical coordinate of a frame is missing due to occlusion, the average of the vertical coordinates of the preceding and following frames is used to fill the missing coordinates.

[0199] Furthermore, the anomaly detection module 15 is specifically configured as follows: A preset threshold θ for the standard deviation σy in the vertical direction is set, and the preset threshold θ is determined by statistically analyzing the distribution of σy in the normal behavioral trajectories in the training set. When σy > θ is detected, it is initially determined to be abnormal behavior; The trajectory features are input into a pre-trained behavior prediction model to obtain the probability of abnormal behavior prediction. When the predicted probability of abnormal behavior exceeds a preset probability threshold, the abnormal judgment result is enhanced, and the final abnormal behavior label is output.

[0200] Furthermore, the preset threshold θ is the 95th percentile of the normal behavioral trajectory σy value; The preset probability threshold is 0.8; The behavior prediction model is a two-layer fully connected neural network with 128 neurons in the hidden layer and ReLU activation function. The input features of the behavior prediction model include: standard deviation σy in the vertical direction, standard deviation σx in the horizontal direction, average speed, and trajectory length; The behavior prediction model is trained using a confidence-based step loss function.

[0201] The AI-based target behavior early warning system of this disclosure is used to implement the AI-based target behavior early warning method in embodiments one to seven, so the description is relatively simple. For details, please refer to the relevant descriptions in the previous method embodiments, which will not be repeated here.

[0202] This disclosure presents an AI-based target behavior early warning method and system. It solves the target detection problem by integrating a YOLOv5 model with a CBAM attention mechanism. The system is constructed by replacing the C3 module in the YOLOv5 backbone network with a CBAMC3 module containing channel attention and spatial attention submodules. The channel attention submodule obtains channel statistics of the feature map through average pooling and max pooling, and outputs channel weights via a multilayer perceptron. The spatial attention submodule obtains spatial statistics through average pooling and max pooling in the channel dimension, and outputs spatial weights via a convolutional layer. This enhances the capture of pedestrian features in complex scenes and reduces missed detections.

[0203] Furthermore, in terms of target tracking, this AI-based target behavior early warning method and system first uses the improved DeepSort algorithm to predict the position and speed of pedestrian targets using Kalman filtering, and then uses the Hungarian algorithm to match the predicted trajectory with the current detection box. The matching cost is the weighted sum of Mahalanobis distance and cosine distance. In addition, the trajectory feature extraction uses a feature network trained on the Market-1501 dataset to improve the matching accuracy under occlusion conditions, reduce identity switching, and ensure the continuity and stability of the trajectory.

[0204] Furthermore, this AI-based target behavior early warning method and system features a feature extraction module that extracts trajectory features, including the standard deviation along the vertical axis, from the target's motion trajectory. An anomaly detection module determines anomalies based on these features and a preset threshold. Simultaneously, it incorporates a behavior prediction model trained using a confidence-based step loss function. When the standard deviation along the vertical axis exceeds the threshold and the anomaly probability exceeds 0.8, the detection is strengthened, avoiding the limitations of single-feature judgment and reducing the false positive rate. The early warning module sends information including the time, location, and trajectory screenshot of the anomaly via local audible and visual alarms and remote SMS notifications, improving the reliability and timeliness of the early warning. This addresses the shortcomings of existing technologies in target detection, where current models perform poorly in complex scenarios, often missing detections when pedestrians occlude each other or lighting conditions change, making accurate target capture difficult. In target tracking, due to pedestrian occlusion and other issues, tracking algorithms frequently switch target identifiers, failing to form continuous and stable motion trajectories. Furthermore, in anomaly behavior detection, most rely solely on single features without considering the spatial distribution characteristics of pedestrian motion trajectories, leading to a high false positive rate and affecting the reliability of the early warning.

[0205] Figure 4This is a block diagram of an electronic device provided in Embodiment Nine of this disclosure.

[0206] Reference Figure 4 This disclosure provides an electronic device, which includes: at least one processor 701; at least one memory 702; and one or more I / O interfaces 703 connected between the processor 701 and the memory 702; wherein the memory 702 stores one or more computer programs that can be executed by the at least one processor 701, and the one or more computer programs are executed by the at least one processor 701 to enable the at least one processor 701 to execute the above-described artificial intelligence-based target behavior early warning method.

[0207] This disclosure also provides a computer-readable storage medium storing a computer program thereon, wherein the computer program, when executed by a processor, implements the aforementioned artificial intelligence-based target behavior early warning method. The computer-readable storage medium may be volatile or non-volatile.

[0208] This disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code. When the computer-readable code is run in the processor of an electronic device, the processor in the electronic device executes the above-described artificial intelligence-based target behavior early warning method.

[0209] Those skilled in the art will understand that all or some of the steps, systems, and apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software can be distributed on a computer-readable storage medium, which may include computer storage media (or non-transitory media) and communication media (or transient media).

[0210] As is known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable program instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technologies, portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, it is known to those skilled in the art that communication media typically contain computer-readable program instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0211] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0212] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0213] The computer program product described herein can be implemented specifically through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0214] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0215] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0216] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0217] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0218] Example embodiments have been disclosed herein, and while specific terminology has been used, it is for illustrative purposes only and should be construed as such, and is not intended to be limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in connection with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in connection with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of this disclosure as set forth by the appended claims.

Claims

1. A target behavior early warning method based on artificial intelligence, characterized in that, The method includes: Acquire surveillance video data through a camera and output continuous video frames; The YOLOv5 model with fusion attention mechanism is used to detect pedestrian targets in the video frame and output the target bounding box coordinates; The improved DeepSort algorithm is used to continuously track detected pedestrian targets and generate target motion trajectories. Extract trajectory features from the target motion trajectory, the trajectory features including the positional change of the detection box center point in the vertical axis direction; Based on the trajectory characteristics and preset thresholds, it is determined whether the pedestrian exhibits abnormal behavior; When abnormal behavior is detected, an alert message is generated and sent.

2. The target behavior early warning method according to claim 1, characterized in that, The YOLOv5 model with the fused attention mechanism is constructed by replacing the C3 module in the YOLOv5 backbone network with the CBAMC3 module. The CBAMC3 module includes a channel attention submodule and a spatial attention submodule; The channel attention submodule is used to: perform average pooling and max pooling on the input feature map to obtain channel statistics, process the data through a multilayer perceptron to output channel weights, and multiply the channel weights with the input feature map channel by channel to output a channel-weighted feature map. The spatial attention submodule is used to: perform average pooling and max pooling on the channel-weighted feature map in the channel dimension to obtain a first spatial feature map and a second spatial feature map; concatenate the first spatial feature map and the second spatial feature map, process them through a convolutional layer to output spatial weights, and multiply the spatial weights with the channel-weighted feature map pixel by pixel to output the final enhanced feature map.

3. The target behavior early warning method according to claim 2, characterized in that, The replacement of the C3 module in the YOLOv5 backbone network with the CBAMC3 module specifically includes: Replace the C3 modules in layers 2, 4, and 6 of the YOLOv5s backbone network with CBAMC3 modules; Each of the CBAMC3 modules contains 3 residual units, and a CBAM attention module is inserted after the output of the residual units.

4. The target behavior early warning method according to claim 1, characterized in that, The improved DeepSort algorithm includes the following steps: A Kalman filter is used to predict the position and velocity of a pedestrian target, and a state equation is established to obtain the target prediction state in the current frame. The state equation is as follows: Where a is the state transition matrix, b is the control matrix, and W is the control matrix. k For process noise, Let be the system state vector at time k. This is the control input vector at time k; The Hungarian algorithm is used to associate the predicted state of the target with the current detection box, and the matching cost is the weighted sum of Mahalanobis distance and cosine distance; A trained feature extraction network is used to extract appearance features from the detection boxes. The feature extraction network uses ResNet50 as the backbone, removes the last fully connected layer, outputs a 512-dimensional feature vector and performs L2 normalization. The cross-entropy loss function is used and the optimizer is the stochastic gradient descent optimizer SGD. The network is trained on the pedestrian re-identification dataset Market-1501.

5. The target behavior early warning method according to claim 4, characterized in that, The matching parameters of the Hungarian algorithm are set as follows: Mahalanobis distance threshold is set to 9.4877, cosine distance threshold is set to 0.2, and maximum cosine distance is set to 0.

1. During the continuous tracking of detected pedestrian targets, the minimum detection confidence level was set to 0.3, the non-maximum suppression threshold was set to 0.5, and the maximum trajectory survival time was set to 70 frames.

6. The target behavior early warning method according to claim 1, characterized in that, The change in position of the center point of the detection frame along the vertical axis is calculated in the following way: After continuously tracking N frames, record the ordinates y1, y2, ..., y3 of the center point of the pedestrian detection box in each frame. N ; Calculate the mean of the ordinate. Calculate the standard deviation σy along the vertical axis using the following formula: Where N is the preset number of continuous tracking frames; σy is used as the change in position of the center point of the detection frame along the vertical axis.

7. The target behavior early warning method according to claim 6, characterized in that, The preset number of continuous tracking frames ranges from 20 to 30 frames. When the vertical coordinate of a frame is missing due to occlusion, the average of the vertical coordinates of the preceding and following frames is used to fill the missing coordinates.

8. The target behavior early warning method according to claim 6, characterized in that, The step of determining whether a pedestrian exhibits abnormal behavior based on the trajectory features and a preset threshold includes: A preset threshold θ for the standard deviation σy in the vertical direction is set, and the preset threshold θ is determined by statistically analyzing the distribution of σy in the normal behavioral trajectories in the training set. When σy > θ is detected, it is initially determined to be abnormal behavior; The trajectory features are input into a pre-trained behavior prediction model to obtain the probability of abnormal behavior prediction. When the predicted probability of abnormal behavior exceeds a preset probability threshold, the abnormal judgment result is enhanced, and the final abnormal behavior label is output.

9. The target behavior early warning method according to claim 8, characterized in that, The preset threshold θ is the 95th percentile of the normal behavior trajectory σy value; The preset probability threshold is 0.8; The behavior prediction model is a two-layer fully connected neural network with 128 neurons in the hidden layer and ReLU activation function. The input features of the behavior prediction model include: standard deviation σy in the vertical direction, standard deviation σx in the horizontal direction, average speed, and trajectory length; The behavior prediction model is trained using a confidence-based step loss function.

10. A target behavior early warning system based on artificial intelligence, characterized in that, The system includes: The data acquisition module is configured to acquire surveillance video data through a camera and output continuous video frames. The target detection module is configured to use a YOLOv5 model with a fusion attention mechanism to detect pedestrian targets in the video frame and output the target bounding box coordinates. The target tracking module is configured to continuously track detected pedestrian targets based on an improved DeepSort algorithm and generate the target's motion trajectory; The feature extraction module is configured to extract trajectory features from the target motion trajectory, the trajectory features including the positional change of the detection box center point in the vertical axis direction; An anomaly detection module is configured to determine whether a pedestrian exhibits abnormal behavior based on the trajectory features and a preset threshold. The early warning module is configured to generate and send early warning information when abnormal behavior is detected.

11. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores one or more computer programs that can be executed by the at least one processor, and the one or more computer programs are executed by the at least one processor to enable the at least one processor to perform the AI-based target behavior early warning method as described in any one of claims 1-9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the target behavior early warning method based on artificial intelligence as described in any one of claims 1-9.

13. A computer program product, characterized in that, Includes computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device executes the AI-based target behavior early warning method as described in any one of claims 1-9.

Images