Training method of secondary unique row detection model, secondary unique row detection method and system

Abstract

The application provides a training method of a secondary unique walking detection model, a secondary unique walking detection method and a system. The training method comprises the following steps: constructing an initial secondary unique walking detection model, the initial secondary unique walking detection model comprising an initial YOLO fine-tuning model and an initial MobileViT+ space-time model, the initial YOLO fine-tuning model outputting a position fine-tuning result and a unique walking score prediction result according to an input first detection result; the initial MobileViT+ space-time model outputting a region risk prediction result and a time sequence behavior prediction result according to the output of the initial YOLO fine-tuning model; constructing a fine-tuning data set, training the initial YOLO fine-tuning model to convergence to obtain a YOLO fine-tuning model; and constructing a unique walking data set, training the initial MobileViT+ space-time model to convergence by using the COCO-Person data set and the unique walking data set to obtain a MobileViT+ space-time model. The application can further refine target behavior analysis on the basis of basic behavior detection of human target behavior, and effectively improve the accuracy of identifying potential dangerous behavior.

Description

Technical Field

[0001] This invention relates to the field of smart city security management technology, and in particular to a training method, a secondary lone-walking detection method and system for a secondary lone-walking detection model. Background Technology

[0002] With the development of intelligent monitoring and security technologies, human target behavior analysis has become one of the important technologies in monitoring systems. Although artificial intelligence-driven monitoring systems are now widely used, there are still significant technical bottlenecks in the field of human target behavior analysis.

[0003] Mainstream solutions mostly use single-modal deep learning models (such as YOLO, Faster R-CNN, etc.) to complete basic behavior detection (such as boundary crossing, intrusion), but these models have two major defects: (1) fragmented temporal information: existing models rely on short temporal segments (3-5 frames) for isolated judgment, lacking the ability to track target behavior over a long period of time. For example, after detecting boundary crossing behavior, the system cannot analyze whether the target exhibits isolated characteristics through subsequent frames, which can easily lead to misjudgment of risk level. (2) lack of multi-level analysis: most systems adopt a flat architecture of "detection-output" and do not build a hierarchical analysis model for behavior features. For example, intrusion detection and isolated behavior judgment belong to independent algorithm modules, making it difficult to achieve dynamic risk assessment through contextual association.

[0004] Taking a typical algorithm box solution as an example, its closed architecture solidifies the algorithm logic, making it impossible to coordinate with the detection results of other manufacturers, and it is also difficult to optimize the model through incremental learning. Although some solutions attempt to introduce multi-sensor fusion, they are still limited by the one-time output mechanism and cannot form a complete decision chain of "basic behavior detection → derived feature analysis → risk classification", resulting in the inability to accurately identify potential dangerous behaviors. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a training method, a secondary lone-walking detection method, and a system for a secondary lone-walking detection model. This solves the problem that existing technologies often use single-modal deep learning models to complete basic behavior detection in human target behavior analysis, lacking effective secondary analysis models, which cannot further refine the target behavior analysis and thus cannot accurately identify potential dangerous behaviors.

[0006] According to an embodiment of the present invention, a training method for a quadratic lone-walk detection model is provided, comprising the following steps:

[0007] An initial secondary lone human detection model is constructed, comprising an initial YOLO refinement model and an initial MobileViT+ spatiotemporal model. The initial YOLO refinement model refines the location and predicts the lone human score from the input primary detection result, outputting the refined location result and the predicted lone human score. The primary detection result contains one human target. The initial MobileViT+ spatiotemporal model performs regional risk prediction and temporal behavior prediction on the output of the initial YOLO refinement model, outputting the regional risk prediction result and the temporal behavior prediction result.

[0008] A refined dataset is constructed, and the initial YOLO refined model is trained to convergence to obtain the refined YOLO model. The refined dataset contains several bounding box regions extracted from a single detection result and their corresponding first labels. The first labels include a refined bounding box label and a single-line label. A single-line dataset is also constructed, and the initial MobileViT+ spatiotemporal model is trained to convergence using the COCO-Person dataset and the single-line dataset to obtain the MobileViT+ spatiotemporal model. The COCO-Person dataset contains several full-body human images and their corresponding second labels. The second labels include bounding box labels, pose labels, and scene labels. The single-line dataset contains several single-line positive samples, group negative samples, special scene samples, and adversarial samples, as well as the third labels corresponding to the samples. The third labels include spatial feature labels and temporal feature labels.

[0009] According to another embodiment of the present invention, a secondary independent detection method is also provided, comprising the following steps:

[0010] Acquire a human target video image, perform a detection based on the human target video image, and output a detection result, wherein the detection result contains a human target;

[0011] The YOLO refinement model obtained using the training method described above is used to analyze and predict the detection result of the first detection, thereby obtaining the position refinement result and the lone target score prediction result of the human target; and candidate lone targets are determined based on the position refinement result and the lone target score prediction result.

[0012] The MobileViT+ spatiotemporal model obtained by the training method described above is used to analyze and predict the candidate lone target, and the regional risk prediction result and the temporal behavior prediction result of the human target are obtained; and the determination result of whether the human target is a lone person is obtained based on the regional risk prediction result and the temporal behavior prediction result.

[0013] Based on the determination result, it is determined whether an alarm should be triggered.

[0014] The determination of whether a human target is a lone traveler based on the regional risk prediction result and the temporal behavior prediction result involves the following steps:

[0015] By integrating regional risk prediction results and time-series behavior prediction results, a comprehensive risk score is calculated.

[0016] The comprehensive risk score is compared with a preset safety threshold. If the comprehensive risk score exceeds the preset safety threshold, the human target is determined to be a lone pedestrian, and an alarm is triggered.

[0017] According to another embodiment of the present invention, a secondary lone-walking detection system is also provided, the secondary lone-walking detection system comprising:

[0018] A smart camera is used to capture video images of human targets and perform a single detection using a built-in deep learning model, outputting a single detection result containing a human target.

[0019] An edge computing device is used to deploy the training method described above to obtain a YOLO refinement model and a MobileViT+ spatiotemporal model, and then obtain a detection result again. The YOLO refinement model and the MobileViT+ spatiotemporal model are used to determine whether the human target is a lone pedestrian.

[0020] Alarm devices are used to determine whether to trigger an alarm based on the received judgment results.

[0021] Compared with the prior art, the present invention has the following beneficial effects:

[0022] By adding a second independent detection of human target behavior on the basis of a single detection, the target behavior can be further analyzed. Compared with the traditional single detection of human target behavior, it can identify the potential dangerous behaviors of human targets.

[0023] The YOLO refinement model refines the location and predicts the isolated score of the input detection result, and outputs the location refinement result and the isolated score prediction result. This allows the YOLO refinement model to verify the detection result and determine whether the human target in the detection result and the location refinement result is the same target, thereby improving the detection accuracy. At the same time, it performs preliminary isolated score prediction on the detection result to filter out non-isolated targets.

[0024] By using the MobileViT+ spatiotemporal model to perform regional risk prediction and temporal behavior prediction on the output of the YOLO refined model, the MobileViT+ spatiotemporal model can perform spatial density analysis on the monitoring scene and temporal persistence verification of human targets. This dual verification of spatial density analysis and temporal persistence effectively distinguishes between real solitary behavior and occasional abnormal events. Compared with traditional detection methods that rely on single-frame image analysis or simple trajectory tracking, it can reduce false positives and effectively improve the accuracy of identifying potential dangerous behaviors. Attached Figure Description

[0025] Figure 1 This is a flowchart illustrating the training steps of a secondary lone detection model according to an embodiment of the present invention.

[0026] Figure 2 The diagram illustrates the specific steps of training the YOLO refinement model for a training method of a secondary independent detection model according to an embodiment of the present invention.

[0027] Figure 3 The diagram illustrates the specific steps of training the MobileViT+ spatiotemporal model, which is a training method for a secondary lone detection model according to an embodiment of the present invention.

[0028] Figure 4 This is a flowchart illustrating the steps of a secondary independent detection method according to another embodiment of the present invention. Detailed Implementation

[0029] The technical solutions of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0030] like Figures 1-3 As shown in the figure, this embodiment of the invention proposes a training method for a quadratic lone-walk detection model, including the following steps:

[0031] S1. Construct an initial secondary lone human detection model, which includes an initial YOLO refinement model and an initial MobileViT+ spatiotemporal model. The initial YOLO refinement model is used to refine the location and predict the lone human score of the input primary detection result, and outputs the location refinement result and the lone human score prediction result. The primary detection result contains one human target. The initial MobileViT+ spatiotemporal model is used to perform regional risk prediction and temporal behavior prediction on the output of the initial YOLO refinement model, and outputs the regional risk prediction result and the temporal behavior prediction result.

[0032] S2. Construct a refined dataset and train the initial YOLO refined model until convergence to obtain the refined YOLO model. The refined dataset contains several bounding box regions extracted from a single detection result and their corresponding first labels. The first labels include a refined bounding box label and a single-line label. Construct a single-line dataset and train the initial MobileViT+ spatiotemporal model until convergence using the COCO-Person dataset and the single-line dataset to obtain the MobileViT+ spatiotemporal model. The COCO-Person dataset contains several full-body human images and their corresponding second labels. The second labels include bounding box labels, pose labels, and scene labels. The single-line dataset contains several single-line positive samples, group negative samples, special scene samples, and adversarial samples, as well as the third labels corresponding to the samples. The third labels include spatial feature labels and temporal feature labels.

[0033] Specifically, the YOLO refinement model is an improvement on YOLOv5s. Its core innovation lies in the addition of a single-line score prediction branch, forming a dual-task parallel output architecture. The YOLO refinement model includes a backbone network, a feature fusion layer, and a detection head. The backbone network adopts the CSPDarknet53 structure of YOLOv5s, the feature fusion layer adopts the FPN+PANet structure, and the detection head includes a position refinement branch and a single-line score branch. The two branches are processed in parallel, outputting position refinement results and single-line score prediction results.

[0034] The MobileViT+ spatiotemporal model is a composite model that combines the lightweight MobileViT backbone network with spatiotemporal dual branches (spatial and temporal branches). The MobileViT+ spatiotemporal model includes a backbone network, spatial branches, and temporal branches. The backbone network adopts the MobileViT-S architecture (a lightweight Transformer architecture; MobileViT combines the advantages of CNN and Transformer, making it suitable for edge device deployment). The MobileViT-S architecture includes multiple MobileViT blocks, each of which is a hybrid framework composed of MobileNetV2 blocks and Transformer modules. The structure consists of convolutional layers in each MobileNetV2 block. The convolutional layers are responsible for basic feature extraction, and the Transformer layers capture the global context through a self-attention mechanism. The input channels are 3 (RGB image), and the layer configuration is [2, 4, 3] (representing the number of blocks in different stages, i.e., the first stage contains 2 MobileViT blocks, the second stage contains 4 MobileViT blocks, and the third stage contains 3 MobileViT blocks). The spatial branch contains a Voronoi grid generator and an MLP classifier (multilayer perceptron), and the temporal branch contains a Bi-LSTM (bidirectional long short-term memory network). The two branches are processed in parallel, and the output is the regional risk prediction result and the temporal behavior prediction result.

[0035] The training process of the YOLO refinement model specifically includes:

[0036] S21. Input the refined dataset into the backbone network of the YOLO refined model, extract multi-scale features, and output multi-scale feature maps.

[0037] Specifically, the backbone network of the YOLO refinement model adopts the CSPDarknet53 structure of YOLOv5s. It inputs the image patch (the image patch region must contain a human target) extracted from a detection result into the backbone network. The backbone network can extract multi-scale features from the image patch and output multi-scale feature maps.

[0038] S22. The multi-scale feature map is received through the feature fusion layer of the YOLO refinement model, and the multi-scale feature map is fused to form a multi-scale fused feature map and output.

[0039] Specifically, the feature fusion layer of the YOLO refinement model adopts the FPN+PANet structure. The feature fusion layer can receive multi-scale feature maps output by the backbone network, fuse the received multi-scale feature maps, and finally form a multi-scale fused feature map and output it.

[0040] S23. The multi-scale fusion feature map is received by the detection head of the YOLO refinement model, and output to the position refinement branch and the independent score branch respectively. The position refinement branch predicts the position and category of the target based on the received multi-scale fusion feature map, and outputs the position refinement result of the target. The independent score branch predicts the independent score of the target based on the received multi-scale fusion feature map, and outputs the independent score prediction result of the target.

[0041] Specifically, each scale of the fused feature map output by the feature fusion layer of the YOLO refinement model corresponds to a detector head. Each detector head contains a location refinement branch and a solo score branch, which are processed in parallel. The location refinement branch receives the multi-scale fused feature map output by the feature fusion layer, predicts the target's location and category respectively, and outputs the location refinement result, i.e., the target's location coordinates (x1, y1, x2, y2). The solo score branch receives the fused feature maps of different scales output by the feature fusion layer, predicts the target's solo score (solo_score), and outputs the solo score prediction result. The predicted solo score of the target is determined by local spatial density analysis to determine whether there are other targets around the target (a score between 0 and 1, where the higher the score, the greater the probability of soloing).

[0042] Specifically, the isolated score branch predicts the isolated score of the target based on the received multi-scale fused feature map and outputs the isolated score prediction result of the target, including:

[0043] The multi-scale fused feature map is compressed, and then the compressed feature map is spatially pooled to obtain the feature vector;

[0044] The feature vector is input into a fully connected layer, and density-sensitive features are learned through a weight matrix W and a bias b. The column vectors of the weight matrix W are interpreted as the response strength to the density features of different local regions. The fully connected layer optimizes the weight b through backpropagation, enabling the YOLO refinement model to distinguish between sparse / dense target distributions in order to learn the density features of local regions.

[0045] The output of the fully connected layer is mapped to a probability value in the range [0,1], and the solitary probability value is output, where 0 indicates that there are other targets around and 1 indicates that the solitary travel is completely isolated.

[0046] Specifically, the multi-scale fused feature map is compressed to a fixed size (7×7, using SPP or SPPF structure) to eliminate the influence of target size differences. Then, spatial pooling (global average pooling or max pooling) is performed on the compressed 7×7 feature map to obtain a 1×1×512 feature vector. This 512-dimensional feature vector is then input into a fully connected layer, where density-sensitive features are learned through the weight matrix W and bias b. The column vectors of the weight matrix can be interpreted as the response strength to density features in different local regions (e.g., higher density regions correspond to higher weight activation). The fully connected layer optimizes the weights through backpropagation, enabling the network to distinguish between sparse and dense target distributions (e.g., lone targets correspond to low-density feature combinations, and clustered targets correspond to high-density feature combinations) to learn the density features of local regions. Finally, the Sigmoid activation function is used to map the output of the fully connected layer to probability values ​​in the [0,1] interval, outputting the lone probability value (0 = other targets are present, 1 = completely lone). The specific calculation formula is as follows:

[0047]

[0048] in, , represents the probability value of walking alone; z is the output of the fully connected layer.

[0049] S24. Construct a first loss using the position refinement result, the isolated score prediction result, and the corresponding first label, and update the YOLO refinement model parameters based on the first loss.

[0050] Specifically, the training process of the YOLO refinement model is as follows: (1) Construct and preprocess the refinement dataset: The refinement dataset contains several bounding box regions extracted from a single detection result, as well as corresponding labels. These labels include the refined bounding box label and the single-line label. After construction, the bounding box regions are standardized (e.g., resized to 640x640 resolution) and enhanced (e.g., random rotation, scaling, random occlusion, lighting jitter, etc.) to improve the generalization ability. Among them, the single-line label is a density label that relies on manual annotation. The specific annotation rule is: take the center of the target as the center and count the number of other targets within a radius of 50 pixels.

[0051] No other targets → Tag value 1.0

[0052] 1 other target → Tag value 0.5

[0053] ≥2 other targets → Label value 0.1.

[0054] (2) The YOLO refinement model is trained using the constructed refinement dataset. Specifically, the constructed refinement dataset is input into the backbone network of the YOLO refinement model, multi-scale features are extracted, multi-scale feature maps are output, and then the multi-scale feature maps are input into the feature fusion layer to fuse the multi-scale feature maps, form a multi-scale fused feature map and output it. Then, the multi-scale fused feature maps are input into the position refinement branch and the independent score branch respectively. The position refinement branch predicts the position and category of the target based on the received multi-scale fused feature maps and outputs the position refinement result of the target. The independent score branch predicts the independent score of the target based on the received multi-scale fused feature maps and outputs the independent score prediction result of the target.

[0055] The training process continues until the YOLO refinement model converges. During training, the AdamW optimizer is used with an initial learning rate of 0.001. The learning rate is gradually reduced with each training cycle according to a cosine decay strategy. Each batch consists of 32 samples. The model converges after 300 training cycles, or early stopping is triggered when the loss of the same sample does not decrease for 30 consecutive training cycles (i.e., training is stopped). Finally, a deployable TensorRT engine file (yolo_refinement.pt) is generated.

[0056] (3) During training, a loss function is used for supervision, namely: the first loss is constructed using the location refinement results, the single-line prediction results, and the corresponding labels, and the YOLO refinement model parameters are updated based on the first loss. The first loss consists of bounding box regression loss, single-line score prediction loss, and original classification loss. The bounding box regression loss is constructed based on the difference between the location refinement branch and the true refined box label, and is calculated using CIoU loss; the single-line score prediction loss is constructed based on the difference between the single-line score branch and the true single-line label, and is calculated using binary cross-entropy loss with sample weights (the weight of semi-single samples is 1.5, and the weight of non-single samples is 0.8); the original classification loss is the classification loss of the original YOLOv5s model, and is calculated using binary cross-entropy loss. The specific calculation formula for the first loss is:

[0057]

[0058] in, The first loss, , and These are the bounding box regression loss, the single-line score prediction loss, and the original classification loss, respectively. , and These are the weight parameters for the bounding box regression loss, the individual score prediction loss, and the original classification loss, respectively.

[0059] The training process of the MobileViT+ spatiotemporal model specifically includes:

[0060] S25. First, freeze the parameters of the backbone network of the MobileViT+ spatiotemporal model, input the COCO-Person dataset into the MobileViT+ spatiotemporal model, and train the head layers of the spatial and temporal branches of the MobileViT+ spatiotemporal model to achieve pre-training of the MobileViT+ spatiotemporal model.

[0061] The pre-training of the MobileViT+ spatiotemporal model specifically includes:

[0062] Each frame of the COCO-Person dataset is cropped into a 128×128 pixel human target image patch, and five consecutive frames are extracted according to the target ID to form a time sequence, resulting in several sets of five consecutive frames of the same target image patch sequence.

[0063] Specifically, the COCO-Person dataset (a general human detection dataset) is a static image set. To obtain a sequence of five consecutive frames of the same target, it is necessary to simulate the micro-motion of the target (equivalent to natural movement within 0.5 seconds). That is, before inputting the COCO-Person dataset into the MobileViT+ spatiotemporal model, each image needs to be subjected to random transformations (translation ±10 pixels, rotation ±5 degrees, scaling ±3%, and brightness adjustment ±10%). By translating each image by ±10 pixels, a sequence of five "consecutive frames" of the same target is generated, thus obtaining several sets of five consecutive frames of the same target.

[0064] The COCO-Person dataset specifically includes:

[0065]

[0066] The pose labels in the COCO-Person dataset cover the diversity of human poses, including 20+ common poses (walking / running / squatting, etc.); the scene labels need to cover daylight / dusk / night / backlight scenes.

[0067] The image patch sequence is input into the backbone network of the MobileViT+ spatiotemporal model to extract image features and output a three-layer feature pyramid containing P3 feature maps, P4 feature maps and P5 feature maps.

[0068] Specifically, the step of inputting the image patch sequence into the backbone network of the MobileViT+ spatiotemporal model to extract image features and output a three-layer feature pyramid containing P3, P4, and P5 feature maps includes:

[0069] When the image patch sequence is input into the backbone network of the MobileViT+ spatiotemporal model, each frame of image independently enters a hybrid architecture composed of MobileNetV2 blocks and Transformer modules. The first three stages use convolution operations to gradually reduce the resolution from 128×128 to 16×16 and output the local features of each stage. The fourth stage uses the Transformer module to perform global context encoding on the 16×16 features and output global features.

[0070] Subsequently, a three-layer feature pyramid is constructed and output. The P3 feature map directly integrates the local features of the second stage and the global features of the fourth stage. The P4 feature map directly integrates the local features of the third stage and the global features of the fourth stage. The P5 feature map downsamples the P4 feature map and integrates it with the global features of the fourth stage. The P5 feature map is a high-order semantic feature with an 8×8 resolution.

[0071] Specifically, when a sequence of 5 consecutive frames (each frame cropped to 128×128 pixels by YOLO retouching boxes) of the same target image patch is input into the backbone network (MobileViT-S) of the MobileViT+ spatiotemporal model, each frame of the image independently enters a hybrid architecture composed of MobileNetV2 blocks and Transformer modules:

[0072] (a) The first three stages employ convolution operations (gradually downsampling the resolution through convolution with a stride of 2), gradually reducing the resolution from 128×128 to 16×16 (128×128→64×64→32×32→16×16); the output of each stage contains a mixture of local convolutional features and lightweight Transformer information, where the 16×16 features output from the third stage will be used as the input to the Transformer module;

[0073] (b) In the fourth stage, a lightweight Transformer module is introduced. The Transformer module is used to perform global context encoding on the 16×16 features. Global dependencies are established through a multi-head self-attention mechanism, and the enhanced high-order semantic features (i.e. global features) are output. The features in this stage have the same spatial resolution (16×16) as the output of the third stage, but contain richer global information.

[0074] (c) Construct and output a three-layer feature pyramid, which includes P3 feature map, P4 feature map and P5 feature map. The P3 feature map directly fuses the local features of the second stage and the global features of the fourth stage. The P4 feature map directly fuses the local features of the third stage and the global features of the fourth stage. The P5 feature map downsamples the P4 feature map and fuses it with the global features of the fourth stage (the fusion method usually adopts element-wise addition or channel concatenation, followed by a convolutional layer to eliminate feature conflicts). The P5 feature map is a high-order semantic feature with an 8×8 resolution.

[0075] The P5 feature map of the last frame is input into the spatial branch of the MobileViT+ spatiotemporal model. The P5 feature map is divided into several spatial regions by the Voronoi grid generator. Then, the risk score of each region is predicted by the MLP classifier, and the regional risk prediction result is output.

[0076] Specifically, the spatial branch of the MobileViT+ spatiotemporal model includes a Voronoi grid generator and an MLP classifier (multilayer perceptron). When the P5 feature map of the last frame is input into the spatial branch, the spatial region is first segmented by the Voronoi grid generator. Specifically, spatial seed points are generated based on the keypoint coordinates predicted by the P5 feature map, and the image plane is segmented into irregular regions using the Thiessen polygon algorithm, with each region corresponding to a potential risk unit. For example, in a restricted area monitoring scenario, the generated polygonal region may cover the area surrounding sensitive facilities. The segmented spatial region features are then compressed into a 256-dimensional vector using mean pooling and input into the MLP classifier (multilayer perceptron) to predict the risk score of each region (this process relies on manually labeled Voronoi region risk levels as supervision signals, such as labeling the core restricted area as 1.0 and the safe area as 0, and optimizing network parameters using binary cross-entropy loss). Finally, the regional risk prediction result, i.e., the spatial risk heatmap, is output to quantitatively display the risk level of different regions.

[0077] The Voronoi grid generation logic is as follows:

[0078] Design principle: Use 1×1 convolution to compress the 256-channel feature map to 32 channels, extract the feature points with the strongest spatial response (such as key parts of the human body), and use the ReLU function to filter out negative responses and retain the effective feature activation regions.

[0079] Physical meaning: Each output channel corresponds to a potential key point heatmap, and high response locations represent target joints / environmental landmarks.

[0080] MLP classifiers (compared to direct regression, MLP structures can learn complex spatial patterns through nonlinear transformations, such as the gradual risk of curved restricted areas). Regional risk scores. The calculation logic is as follows:

[0081]

[0082] in, Compress the output to [0,1], such as 0.3 representing a low-risk area and 0.8 representing a high-risk area; Dimensionality is reduced from 256 to 128, and semantic features of the region (such as whether it is close to the boundary of the restricted area) are extracted. This represents a 128-to-1 dimension mapping to quantify the degree of risk. The input feature vector; and These are the bias terms, Corresponding to and Bias after linear transformation Corresponding to the output of the ReLU function The transformed bias.

[0083] The P5 feature maps of 5 consecutive frames are input into the temporal branch of the MobileViT+ spatiotemporal model, and the temporal behavior is analyzed by Bi-LSTM to output the temporal behavior prediction results.

[0084] Specifically, the temporal branch of the MobileViT+ spatiotemporal model includes Bi-LSTM (Bidirectional Long Short-Term Memory Network). When the P5 feature maps of 5 consecutive frames are input into the temporal branch, firstly, global average pooling is performed on the P5 feature maps of each frame to compress the 8×8×256 feature maps into 256-dimensional temporal vectors to eliminate spatial dimensional redundancy.

[0085] Next, the compressed 256-dimensional time-series vector is input into a Bi-LSTM. The Bi-LSTM performs forward propagation (analyzing the behavioral evolution from t to t+4) and backward propagation (tracing back the behavioral triggers from t+4 to t) to capture key behavioral features such as wandering and turning back (forward propagation analyzes the behavioral evolution frame by frame and outputs the hidden states). It captures temporal patterns such as acceleration and turning; it backpropagates back to trace the behavioral triggers and outputs the hidden states. (Identifying reverse features such as the return point), the final hidden state vector at each time step is a concatenation of bidirectional results: This vector contains both future trend and historical motivation information;

[0086] The hidden state vector is then mapped to a behavior probability distribution (e.g., 0.8 for wandering probability and 0.1 for straight-line movement probability) through a fully connected layer. The hidden state vector is formed by concatenating the hidden layer states at each time step during the forward and backward propagation of the Bi-LSTM, with each time step corresponding to a 256-dimensional feature.

[0087] Then, the hidden state vector is used to predict the coordinates at the next time step. And use the current prediction result as the input for the next moment (the predicted result). The point is used as new input, concatenated with historical trajectories, and then used for prediction again. (By generating points in a chain-like recursive manner), the predicted trajectory is gradually generated, and the temporal behavior prediction results, namely the behavior probability distribution and the predicted trajectory, are output. Simultaneously, a temporal continuity verification module is used to verify the prediction based on a displacement threshold. To filter out momentary interference, this threshold is set based on normal walking speed statistics (1.2-1.5 meters per second for adults) to ensure the physical rationality of behavior judgment.

[0088] A second loss is constructed using the regional risk prediction results, the temporal behavior prediction results, and the corresponding second labels, and the parameters of the spatial branch and the temporal branch of the MobileViT+ spatiotemporal model are updated based on the second loss.

[0089] Specifically, the pre-training process of the MobileViT+ spatiotemporal model is as follows: (1) Freeze all parameters of the backbone network of the MobileViT+ spatiotemporal model, and use the COCO-Person dataset to train the spatial and temporal branches of the MobileViT+ spatiotemporal model. Specifically, input the COCO-Person dataset into the backbone network of the MobileViT+ spatiotemporal model, extract image features, and output a three-layer feature pyramid of P3 feature map, P4 feature map and P5 feature map; then input the P5 feature map of the last frame into the spatial branch of the MobileViT+ spatiotemporal model, divide the P5 feature map into several spatial regions through the Voronoi grid generator, and then predict the risk score of each region through the MLP classifier, and output the regional risk prediction result; at the same time, input the P5 feature map of 5 consecutive frames into the temporal branch of the MobileViT+ spatiotemporal model, analyze the temporal behavior through Bi-LSTM, and output the temporal behavior prediction result.

[0090] The MobileViT+ spatiotemporal model was trained until convergence, thus achieving pre-training of the MobileViT+ spatiotemporal model (and updating only the head parameters of the spatial and temporal branches through backpropagation). During training, the AdamW optimizer was used with a weight decay of 0.05, and convergence was achieved after 100 training epochs, completing the pre-training of the MobileViT+ spatiotemporal model. In the COCO-Person dataset, enhancement strategies such as random occlusion (20%-50% region), color jitter (brightness ±30%, contrast ±20%), and motion blur (maximum 15px displacement) can be adopted to effectively improve the robustness of the MobileViT+ spatiotemporal model.

[0091] (2) During pre-training, a loss function is used for supervision, namely: a second loss is constructed using the regional risk prediction results, the temporal behavior prediction results, and the corresponding second labels, and the parameters of the spatial and temporal branches of the MobileViT+ spatiotemporal model are updated based on the second loss. The second loss consists of a first spatial loss and a first temporal loss. The first spatial loss is constructed based on the difference between the spatial branch and the real scene labels and is used to handle the imbalanced classification of spatial risk scores; the first temporal loss is constructed based on the difference between the temporal branch and the real pose labels and is used for temporal behavior prediction (such as MAE or cross-entropy). The specific calculation formula for the second loss is as follows:

[0092]

[0093]

[0094]

[0095] in, This is the second loss; This is the first space loss; This is the first-series loss; Output for spatial branching; For spatial labels (e.g., 0.8 for the core restricted area); Output for time branch; pt represents the time label (e.g., the probability of wandering is 1.0); pt represents the model's predicted probability, which is the predicted value for the true class (spatial branch output). Class weights are used to balance positive and negative samples. For focusing parameters, usually =2; This is a real label; Output for time branch.

[0096] In the formula for calculating the second loss, the fixed weight coefficients of both spatial loss (FocalLoss) and temporal loss (TemporalLoss) are set to 0.5, which means that the two tasks are given equal importance during training. By pre-training the MobileViT+ spatiotemporal model, the spatial branch learns to generate risk region division from P5 features, the temporal branch masters basic human behavior recognition capabilities, and provides an adapted head structure for the feature pyramid output by the backbone network.

[0097] S26. Then, gradually unfreeze the parameters of the last two layers of the backbone network, input the isolated dataset into the MobileViT+ spatiotemporal model, train the MobileViT+ spatiotemporal model, and achieve fine-tuning of the MobileViT+ spatiotemporal model.

[0098] Specifically, the fine-tuning of the MobileViT+ spatiotemporal model includes:

[0099] Each frame of the isolated dataset is cropped into a 128×128 pixel human target image block, and five consecutive frames are extracted according to the target ID to form a time sequence, resulting in several sets of five consecutive frames of the same target image block sequence.

[0100] Specifically, the lone target dataset is a video surveillance dataset. It is generated by capturing video streams and inspecting video images frame by frame (one frame is captured every 0.4 seconds, meaning five consecutive frames need to cover 2 seconds), thus obtaining several sets of five consecutive frames of image patches of the same target. The lone target dataset contains several lone positive samples, group negative samples, special scene samples, and adversarial samples, specifically:

[0101]

[0102] Among them, the positive sample of lone walking refers to the surveillance video clip that clearly contains a lone walking person. It usually occurs in sensitive areas (such as restricted areas or around important facilities). The construction method is as follows: surveillance cameras are installed in restricted areas and sensitive areas to record for a long time. The duration of each video clip is usually between 15 and 30 seconds to ensure that the target is clearly visible in the picture. Then, the clips in which the lone target appears in the scene are manually selected and the selected clips are labeled. The labeling content includes the target's precise location (image block sequence), appearance time, target trajectory (position change in consecutive frames) and risk score (classified according to the importance of the area).

[0103] Group negative samples refer to surveillance videos containing multiple people appearing in a scene simultaneously, used to provide examples of non-solo behavior. The construction method is as follows: record video during peak hours in public places (such as squares, shopping mall entrances and exits, subway stations) to collect scenes where multiple people appear at the same time; then manually filter out the segments in the scene where group targets appear, and label the selected segments. The labeling content includes: group density heat map (generating a heat distribution map by marking the location of each target), number of targets and distribution area, with special emphasis on marking areas where people gather, to avoid misjudging the group as multiple solo targets.

[0104] Special scenarios refer to videos of lone individuals taken under harsh environmental conditions (rain, fog, night, low light) to improve the robustness of the model in complex environments. The construction method is as follows: continuously collect surveillance videos under low light conditions such as rain, fog, night, and dusk. After collection, the required segments are manually selected, labeled, and visibility enhancement techniques (such as infrared imaging and low light enhancement) are used to assist in the labeling. The labeling content includes: in addition to the regular target location, environmental condition labels (such as light level and weather category) and image quality scores (used for weighting during training) are also required. At the same time, image enhancement algorithms (such as CLAHE) may be used to improve image quality, but the labeling process is still based on the original image.

[0105] Adversarial examples refer to challenging examples created using generative adversarial networks (GANs), such as partially occluded or disguised targets, to train models to withstand challenges in real-world scenarios. The construction method involves using generative adversarial networks such as StyleGAN2 to generate occluded or disguised human targets, inserting these generated targets into different backgrounds to simulate real surveillance scenarios, manually selecting the required segments, and annotating the selected segments. The annotation content includes, in addition to regular annotations, marking the location of the occlusion area, the type of disguise, and the difficulty level.

[0106] The image patch sequence is input into the backbone network of the MobileViT+ spatiotemporal model to extract image features and output a three-layer feature pyramid containing P3 feature maps, P4 feature maps and P5 feature maps.

[0107] The P5 feature map of the last frame is input into the spatial branch of the MobileViT+ spatiotemporal model. The P5 feature map is divided into several spatial regions by the Voronoi grid generator. Then, the risk score of each region is predicted by the MLP classifier, and the regional risk prediction result is output.

[0108] The P5 feature maps of 5 consecutive frames are input into the temporal branch of the MobileViT+ spatiotemporal model, and the temporal behavior is analyzed by Bi-LSTM to output the temporal behavior prediction results.

[0109] A third loss is constructed using the regional risk prediction results, the temporal behavior prediction results, and the corresponding third labels. Based on the third loss, the parameters of the third and fourth stages of the backbone network of the MobileViT+ spatiotemporal model, as well as the parameters of the spatial branch and the temporal branch, are updated.

[0110] Specifically, the fine-tuning process of the MobileViT+ spatiotemporal model is as follows: (1) Gradually unfreeze the parameters of the last two layers of the backbone network (e.g., unfreeze one layer every 10 training processes), and use the standalone dataset to train the last two stages of the backbone network of the MobileViT+ spatiotemporal model, as well as the spatial and temporal branches. Specifically, input the standalone dataset into the backbone network of the MobileViT+ spatiotemporal model, extract image features, and output a three-layer feature pyramid of P3 feature map, P4 feature map and P5 feature map; then input the P5 feature map of the last frame into the spatial branch of the MobileViT+ spatiotemporal model, divide the P5 feature map into several spatial regions through the Voronoi grid generator, and then predict the risk score of each region through the MLP classifier, and output the regional risk prediction result; at the same time, input the P5 feature map of 5 consecutive frames into the temporal branch of the MobileViT+ spatiotemporal model, analyze the temporal behavior through Bi-LSTM, and output the temporal behavior prediction result.

[0111] The MobileViT+ spatiotemporal model was trained until convergence, enabling fine-tuning of the model. During training, the AdamW optimizer was used, employing a cosine annealing strategy with hot restart (minimum learning rate 1e-6). Each round consisted of 10 epochs, with one layer of the backbone network unfrozen every 10 epochs. The model converged after 50 training epochs. Adversarial examples were introduced during training to create camouflaged targets (such as umbrella occlusion) generated by GANs, forcing the model to learn robust features. When fine-tuning the MobileViT+ spatiotemporal model, the learning rate for the spatial branch was 1e-4, the learning rate for the temporal branch was 1e-4, and the learning rate for the backbone network was 5e-5.

[0112] (2) During the fine-tuning process, a loss function is used for supervision, namely: the third loss is constructed using the regional risk prediction results, the temporal behavior prediction results, and the corresponding third labels, and the parameters of the third and fourth stages of the backbone network of the MobileViT+ spatiotemporal model, as well as the parameters of the spatial and temporal branches, are updated based on the third loss. The third loss consists of the second spatial loss and the second temporal loss. The specific calculation formula for the third loss is as follows:

[0113]

[0114] in, This is the third loss; The second spatial loss is constructed based on the difference between the spatial branch and the true spatial feature label. It is calculated using the standard binary cross-entropy loss (including Sigmoid activation, i.e., the BCEWithLogitsLoss function) and is suitable for risk region classification. The second temporal loss is constructed based on the difference between the temporal loss and the true temporal feature label. It is calculated using the causal contrastive loss (i.e., the CausalContrastiveLoss function) to enhance the consistency of behavior in consecutive frames (to ensure temporal consistency, the same transformation, such as random rotation, is applied to consecutive frames). Spatial weights are optimized through Voronoi regional risk prediction and dynamically weighted according to risk areas (e.g., high-risk areas have a weight of 3.0). The weights are calculated based on the continuity of motion (only applicable to the model training phase), i.e., dynamically adjusted based on the target displacement (the larger the displacement, the higher the weight).

[0115] It is worth noting that when applying the MobileViT+ spatiotemporal model, The adjustment strategy is as follows:

[0116]

[0117] The 2-meter threshold is the upper limit of a normal adult stride of 2.5 seconds (1.5m / s × 1.67s). This indicates abnormally rapid movement (such as running or fleeing). In this case, the time sequence weight is increased to 0.7 to enhance sensitivity to sudden behavior.

[0118] The logic for determining the spatial weights and temporal weights is as follows:

[0119]

[0120] In the formula for calculating the third loss, 0.4 and 0.6 are hyperparameters, with the temporal branch hyperparameter 0.6 > the spatial branch hyperparameter 0.4, to emphasize the dominant role of temporal continuity analysis in behavioral verification (behavior reflects risk better than static location), because isolated behavior can only be accurately determined through continuous tracking (not single-frame spatial location).

[0121] like Figure 4 As shown, according to another embodiment of the present invention, a secondary independent detection method is also proposed, comprising the following steps:

[0122] S3. Acquire a human target video image, perform a detection based on the human target video image, and output a detection result, wherein the detection result contains a human target;

[0123] Specifically, in campus security scenarios, several smart cameras are deployed in the areas within the campus that need to be monitored. These cameras monitor the areas and collect video images in real time. The collected video images are then subjected to an initial inspection, and a result is output. This initial inspection includes three scenarios:

[0124] (1) Camera with computing power → generates an alarm event → ONVIFCameraManager accesses and converts it into an RTSP stream → secondary detection (video of a specific event segment + alarm information);

[0125] (2) Camera without computing power → ONVIFCameraManager access and conversion into RTSP stream → Configure third-party algorithm box (this type of device from third-party manufacturers also needs to access RTSP stream for real-time analysis) → Generate an alarm event → Secondary detection (video of specific event segment + alarm information);

[0126] (3) Camera without computing power → ONVIFCameraManager access and conversion into RTSP stream → Use Yolov5 model for structured analysis → Generate a structured parsing event → Secondary detection (video of specific event segment + structured information).

[0127] Since scenarios 1 and 2 above utilize the built-in computing power of third-party devices, only the process will be described. The focus will be on describing how to perform structured primary detection using the Yolov5 model:

[0128] (a) A camera is used to capture video images of the human target, and the images are input into the YOLOv5 model after capture. The input images are then uniformly scaled to a resolution of 640×640 to adapt to the YOLOv5 model architecture. The input data of the image is represented as M0(x0,y0), where M0(x0,y0) is the initial spatial coordinates (center coordinates) of the human target in the image plane.

[0129] (b) After the input image is standardized and scaled, basic features are first extracted through the convolutional layer of the YOLOv5 model (the YOLOv5 model includes convolutional layer, residual layer, downsampling layer, upsampling layer and routing layer), and the sliding window mechanism + SiLU activation function is used to initially capture local texture and contour information;

[0130] Subsequently, the extracted feature data flows into the residual layer of the YOLOv5 model. Relying on the cross-layer connection path unique to the CSPDarknet53 architecture, the gradient degradation problem of deep networks is solved through identity mapping and feature addition operations, so as to preserve key human morphological features.

[0131] Subsequently, spatial dimension compression is performed through the downsampling layer of the Yolov5 model. By using a convolutional kernel with a stride of 2, the feature map size is gradually reduced (e.g., 320×320→160×160) while maintaining the increasing number of channels, thus constructing a multi-scale abstract representation.

[0132] Subsequently, the spatial resolution is restored by the upsampling layer of the Yolov5 model through bilinear interpolation, aligning the deep semantic features with the shallow detail information;

[0133] Finally, multi-scale feature fusion is completed in the routing layer of the Yolov5 model. The channel concat technique is used to integrate feature maps of different abstraction levels in the channel dimension to form a pyramid feature structure that combines spatial details and semantic information, providing a comprehensive basis for subsequent gridded prediction.

[0134] (c) After the feature pyramid is constructed, the core mechanism for performing gridded prediction is as follows: First, the feature maps of each level are divided into an S×S grid cell array in the spatial dimension, where the S value is dynamically adjusted according to the feature level (e.g., P3 high-level features correspond to 80×80 grids, and P5 deep features are reduced to 20×20 grids).

[0135] Then, potential targets within the physical coverage area of ​​each grid cell are predicted. In the cell prediction stage, three anchor boxes of different scales are preset for each grid cell, and four key parameters are output simultaneously: the bounding box coordinate offset is used to adjust the preset anchor box position; the existence probability value P(object) is compressed to the (0,1) interval through the Sigmoid function; the category probability distribution vector covers 80 object types in the COCO dataset, as well as the implicit intersection-union ratio (IoU_pred); the confidence calculation serves as a quantitative indicator of prediction reliability, and its principle follows the joint evaluation system of probability and positioning accuracy—the existence probability P(object) is used as the criterion for target existence, while IoU_pred represents the positioning accuracy. Finally, the confidence = P(object) × IoU_pred constitutes a composite index. This calculation mode provides supervision signals through real labeled boxes during the training stage (the loss function forces the predicted confidence to approach the real IoU value), and directly outputs the confidence estimate implicitly learned by the network during the inference stage, forming a complete prediction closed loop that takes into account both target existence determination and positioning quality assessment.

[0136] (d) Perform bounding box optimization and filtering. First, prioritize the confidence scores and sort all initial prediction boxes in descending order of confidence scores to ensure that the detection boxes with the highest confidence scores participate in the filtering process first. Then, perform an iterative filtering main loop, which is as follows: extract the detection box with the highest confidence score in the current list as the reference box, move the reference box into the final output result set, and then traverse the remaining detection box list to calculate the intersection-union ratio (IoU) between each box and the reference box. Discard all detection boxes with IoU values ​​exceeding the preset threshold (0.45). Repeat the iterative filtering main loop steps until all detection boxes have been processed. The final output is the optimized result with highly overlapping redundant boxes removed.

[0137] (e) After completing object detection and bounding box optimization, the system enters the human body structured output stage. Specifically, the system first traverses the final detection result set and performs category determination for each target that meets the confidence standard. When the detected object category identifier (class_id) matches the predefined "pedestrian" category number 0 in the COCO dataset, the structured data processing flow is triggered. At this time, the system performs the core transformation operation: converts the center coordinate format (x_center, y_center, width, height) of the original bounding box into the corner coordinates (x_min, y_min, x_max, y_max) that can be directly located. At the same time, the system calls the preset world coordinate system conversion function to map the two-dimensional pixel position to the three-dimensional physical space based on the camera calibration parameters.

[0138] (f) The structured data is encapsulated in dictionary form for key attributes: the "type" field is fixedly marked as "pedestrian", "bbox" stores the positioning rectangle in the pixel coordinate system, "confidence" retains the detection confidence value, and "position" records the three-dimensional spatial coordinates. This data encapsulation is written to the persistent storage system through the database storage interface to form a traceable detection record. The output fields adopt the standardized JSON format and contain six core elements: a unique detection identifier (detection_id) generated according to the rule of "camera number-frame number-target number", a timestamp accurate to the millisecond level, a target type (target_type) clearly classified, a bbox_pixels quadruple to define the pixel area, world_coord a three-dimensional floating-point number to locate the physical position, and a reserved pose_attributes field to provide an extended interface for subsequent skeletal keypoint analysis.

[0139] S4. Using the YOLO refinement model obtained by the training method described above, analyze and predict the detection result of the first detection to obtain the position refinement result and the lone score prediction result of the human target; and determine the candidate lone target based on the position refinement result and the lone score prediction result.

[0140] The step of determining candidate lone targets based on the location refinement results and the lone-walking score prediction results involves the following steps:

[0141] The overlap between the position refinement result and the first detection result is determined, and the determination result is output. If the determination result exceeds the preset refinement threshold, the human target is confirmed to be the same target.

[0142] The predicted lone target score is then compared with a preset lone target threshold. If the predicted lone target score exceeds the preset lone target threshold, the human target is confirmed as a candidate lone target and added to the subsequent time series analysis queue.

[0143] Specifically, the overlap between the position refinement result and the first detection result is determined using the IoU (Intersection over Union) ratio. The specific calculation formula is as follows:

[0144]

[0145]

[0146]

[0147]

[0148]

[0149] The coordinates of the refinement box corresponding to the position refinement result are: The coordinates of the detection box corresponding to a single detection result are: , The area of ​​the overlapping region. The area of ​​the union region.

[0150] If the calculated IoU is greater than the preset refinement threshold (e.g., 0.8), the refinement result is considered to correspond to the same target, and the target is retained. Otherwise, it indicates that the refinement box has deviated from the original target (e.g., occlusion or mismatch), and it is discarded directly to avoid interfering with subsequent analysis. Then, the single-line score prediction result is compared with the preset single-line threshold. If the single-line score prediction result is greater than the preset single-line threshold (e.g., 0.5), the target is marked as a candidate single-line target and added to the subsequent time series analysis queue.

[0151] S5. Using the MobileViT+ spatiotemporal model obtained by the training method described above, the candidate lone target is analyzed and predicted to obtain the regional risk prediction result and the temporal behavior prediction result of the human target; and based on the regional risk prediction result and the temporal behavior prediction result, a determination result is obtained as to whether the human target is a lone person.

[0152] The determination of whether a human target is a lone traveler based on the regional risk prediction result and the temporal behavior prediction result involves the following steps:

[0153] By integrating regional risk prediction results and time-series behavior prediction results, a comprehensive risk score is calculated.

[0154] The comprehensive risk score is compared with a preset safety threshold. If the comprehensive risk score exceeds the preset safety threshold, the human target is determined to be a lone pedestrian, and an alarm is triggered.

[0155] Specifically, the formula for calculating the comprehensive risk score is as follows:

[0156]

[0157] in, To calculate the overall risk score, and The results are the regional risk prediction results and the time-series behavior prediction results, respectively. 0.4 and 0.6 are the weights of the regional risk prediction results and the time-series behavior prediction results, respectively. The time weight (0.6) is greater than the spatial weight (0.4) to emphasize that behavioral anomalies are more indicative of risk than static locations.

[0158] S6. Determine whether to trigger an alarm based on the determination result.

[0159] Specifically, the system determines whether to trigger an alarm based on the judgment result and ultimately outputs alarm information, including: the target's unique ID (generated by the SORT tracker), the target's location (the coordinates of the image block after refinement in the current frame), the comprehensive risk score, and the timestamp. The structure of the target's unique ID is as follows: [Device ID]-[T Tracking Number]-[P Target Sequence Number] (e.g., CAM01-T0073-P019, where CAM01 is the camera device number, such as camera number 1 on the east side of the lobby; T0073 is the unique trajectory number assigned by the system, which needs to be continuously tracked until the target disappears; and P019 is the target's sequence number in the current frame, which needs to be recalculated for each frame).

[0160] The specific alarm strategy is as follows:

[0161]

[0162] The detailed working process of this embodiment is as follows:

[0163] Intelligent cameras are used to monitor the scene area that needs to be monitored and to collect video images of the scene area in real time. The Yolov5 model is used to perform a detection on the collected video images and output a detection result (the detection result must contain a human target).

[0164] Next, the detection result is input into the YOLO refinement model. Multi-scale features are extracted through the backbone network of the YOLO refinement model, and a multi-scale feature map is output. This multi-scale feature map is then input into the feature fusion layer of the YOLO refinement model to fuse the multi-scale feature maps, forming a multi-scale fused feature map, which is then output. The multi-scale fused feature map is then used in parallel with the detection head of the YOLO refinement model, and input into the position refinement branch and the independent score branch. The position refinement branch uses the received multi-scale fused feature map to predict the target's position and category, outputting the target's position refinement result. The independent score branch uses the received multi-scale fused feature map to predict the target's independent score, outputting the target's independent score prediction result.

[0165] Subsequently, the overlap between the output position refinement result and the first detection result is determined, and the determination result is output. If the determination result exceeds the preset refinement threshold, the human target is confirmed to be the same target. Then, the lone score prediction result is compared with the preset lone threshold. If the lone score prediction result exceeds the preset lone threshold, the human target is confirmed as a candidate lone target and added to the subsequent time series analysis queue.

[0166] For each candidate lone target, the refined image patch positions of the current target in 5 consecutive frames are obtained (the tracking sequence of each target needs to be maintained). Target image patches are then cropped from each of these 5 frames based on their respective positions, forming a 5-frame image sequence. This image sequence is input into the MobileViT+ spatiotemporal model. Image features are extracted through the backbone network of the MobileViT+ spatiotemporal model, outputting a three-layer feature pyramid of P3, P4, and P5 feature maps. The P5 feature map of the last frame is input into the spatial branch of the MobileViT+ spatiotemporal model. A Voronoi grid generator divides the P5 feature map into several spatial regions, and an MLP classifier predicts the risk score for each region, outputting the region risk prediction result. The P5 feature maps of the 5 consecutive frames are input into the temporal branch of the MobileViT+ spatiotemporal model, and Bi-LSTM is used to analyze temporal behavior, outputting the temporal behavior prediction result.

[0167] Subsequently, the regional risk prediction results and temporal behavior prediction results are merged to calculate a comprehensive risk score. This comprehensive risk score is then compared with a preset safety threshold. If the comprehensive risk score exceeds the preset safety threshold, the human target is determined to be a lone pedestrian, triggering an alarm and completing the entire detection process. Throughout the detection process, by adding a second lone pedestrian detection on top of the first detection, the problem of existing behavior detection systems lacking an effective secondary analysis model and being unable to further refine the target behavior analysis, thus failing to accurately identify potential dangerous behaviors, is solved. This allows for further analysis of target behavior to identify potential dangerous behaviors. During the second detection, the YOLO refinement model is used to analyze the target's lone pedestrian behavior, and the MobileViT+ spatiotemporal model is used to perform spatial density analysis of the monitoring scene. The dual verification of the human target's temporal continuity is also performed, effectively distinguishing between real lone pedestrian behavior and occasional abnormal events, thus effectively improving the accuracy of identifying potential dangerous behaviors.

[0168] According to another embodiment of the present invention, a secondary lone-walking detection system is also proposed, the secondary lone-walking detection system comprising:

[0169] A smart camera is used to capture video images of human targets and perform a single detection using a built-in deep learning model, outputting a single detection result containing a human target.

[0170] An edge computing device is used to deploy the training method described above to obtain a YOLO refinement model and a MobileViT+ spatiotemporal model, and then obtain a detection result again. The YOLO refinement model and the MobileViT+ spatiotemporal model are used to determine whether the human target is a lone pedestrian.

[0171] Alarm devices are used to determine whether to trigger an alarm based on the received judgment results.

[0172] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A training method for a quadratic independent detection model, characterized in that: The method comprises the following steps: constructing an initial secondary unique row detection model, the initial secondary unique row detection model comprising an initial YOLO fine-tuning model and an initial MobileViT+ space-time model, wherein the initial YOLO fine-tuning model is used for position fine-tuning and unique row score prediction on an input primary detection result, and outputs a position fine-tuning result and a unique row score prediction result, the primary detection result comprising a human body target; the initial MobileViT+ space-time model is used for region risk prediction and time sequence behavior prediction on the output of the initial YOLO fine-tuning model, and outputs a region risk prediction result and a time sequence behavior prediction result; constructing a fine-tuning data set, training the initial YOLO fine-tuning model to convergence to obtain a YOLO fine-tuning model, the fine-tuning data set comprising a plurality of boundary box regions extracted from a primary detection result and corresponding first labels, the first labels comprising fine bounding box labels and unique row labels; and constructing a unique row data set, training the initial MobileViT+ space-time model to convergence by using a COCO-Person data set and the unique row data set to obtain a MobileViT+ space-time model, the COCO-Person data set comprising a plurality of full-body human images and corresponding second labels, the second labels comprising boundary box labels, posture labels and scene labels, the unique row data set comprising a plurality of unique row positive samples, group negative samples, special scene samples and adversarial samples, and corresponding third labels of the samples, the third labels comprising spatial feature labels and temporal feature labels; wherein the YOLO fine-tuning model is improved based on YOLOv5s and comprises a backbone network, a feature fusion layer and a detection head, the backbone network adopts the CSPDarknet53 structure of YOLOv5s, the feature fusion layer adopts the FPN+PANet structure, the detection head comprises a position fine-tuning branch and a unique row score branch, and the two branches are processed in parallel to output the position fine-tuning result and the unique row score prediction result; the MobileViT+ space-time model combines a MobileViT lightweight backbone network and a space-time double branch, and comprises a backbone network, a spatial branch and a temporal branch, the backbone network adopts the MobileViT-S structure, the spatial branch comprises a Voronoi grid generator and an MLP classifier, the temporal branch comprises a Bi-LSTM, the two branches are processed in parallel to output the region risk prediction result and the time sequence behavior prediction result. 2.The method of claim 1, wherein: The training process of the YOLO fine-tuning model comprises the following steps: inputting the fine-tuning data set into the backbone network of the YOLO fine-tuning model to extract multi-scale features and output multi-scale feature maps; receiving the multi-scale feature maps by the feature fusion layer of the YOLO fine-tuning model, fusing the multi-scale feature maps, finally forming multi-scale fusion feature maps and outputting the multi-scale fusion feature maps; The detection head of the YOLO fine-tuning model receives the multi-scale fusion feature map and outputs to a position fine-tuning branch and a single-row score branch respectively, the position fine-tuning branch predicts the position and category of the target based on the received multi-scale fusion feature map, and outputs the position fine-tuning result of the target; the single-row score branch predicts the single-row score of the target based on the received multi-scale fusion feature map, and outputs the single-row score prediction result of the target. The position fine-tuning result, the single-row score prediction result and the corresponding first label are used to construct a first loss, and the first loss is used to update the parameters of the YOLO fine-tuning model. 3.The method of claim 2, wherein: The single-row score branch predicts the single-row score of the target based on the received multi-scale fusion feature map, and outputs the single-row score prediction result of the target, specifically including: The multi-scale fusion feature map is compressed, and the compressed feature map is subjected to spatial pooling to obtain a feature vector; The feature vector is input into a fully connected layer to learn the density-sensitive feature through a weight matrix W and a bias b, wherein the column vector of the weight matrix W is interpreted as the response intensity of the density feature of different local regions, the fully connected layer optimizes the weight b through back propagation, so that the YOLO fine-tuning model can distinguish sparse / dense target distribution to learn the density feature of the local region; The output of the fully connected layer is mapped to a probability value in the interval [0, 1], and the single-row probability value is output, wherein 0 represents the existence of other targets around, and 1 represents complete single row. 4.The method of claim 1, wherein: The training process of the MobileViT+ space-time model specifically includes: First, freeze the parameters of the backbone network of the MobileViT+ space-time model, input the COCO-Person dataset into the MobileViT+ space-time model, train the head layers of the spatial branch and the temporal branch of the MobileViT+ space-time model, and realize the pre-training of the MobileViT+ space-time model; Then, gradually unfreeze the last two layers of the backbone network, input the single-row dataset into the MobileViT+ space-time model, train the MobileViT+ space-time model, and realize the fine-tuning of the MobileViT+ space-time model. 5.The method of claim 4, wherein: The pre-training of the MobileViT+ space-time model specifically includes: Crop each frame of image in the COCO-Person dataset into a human target image block of 128x128 pixels, and extract a time sequence of 5 consecutive frames according to the target ID to obtain a plurality of image block sequences of the same target of 5 consecutive frames; Input the image block sequence into the backbone network of the MobileViT+ space-time model to extract image features, and output a three-layer feature pyramid including P3 feature map, P4 feature map and P5 feature map; Input the P5 feature map of the last frame into the spatial branch of the MobileViT+ space-time model, divide the P5 feature map into a plurality of spatial regions through a Voronoi grid generator, and predict the risk score of each region through an MLP classifier to output the region risk prediction result; The P5 feature map of the continuous 5 frames is input into the time branch of the MobileViT+ space-time model, the time sequence behavior is analyzed by Bi-LSTM, and the time sequence behavior prediction result is output; The second loss is constructed by using the region risk prediction result, the time sequence behavior prediction result and the corresponding second label, and the parameters of the space branch and the time branch of the MobileViT+ space-time model are updated based on the second loss. 6.The method of claim 5, wherein: The image block sequence is input into the backbone network of the MobileViT+ space-time model, image features are extracted, and a three-layer feature pyramid including P3 feature map, P4 feature map and P5 feature map is output, specifically including: After the image block sequence is input into the backbone network of the MobileViT+ space-time model, each frame of image independently enters the mixed architecture composed of MobileNetV2 block and Transformer module, wherein the first three stages adopt convolution operation, and the resolution is gradually reduced from 128*128 to 16*16, and the local features of each stage are output, and the fourth stage adopts Transformer module to encode the global context of 16*16 features, and outputs the global features; Then, a three-layer feature pyramid is constructed and output, wherein the P3 feature map directly fuses the second stage local feature and the fourth stage global feature, the P4 feature map directly fuses the third stage local feature and the fourth stage global feature, the P5 feature map is down-sampled from the P4 feature map and fused with the fourth stage global feature, and the P5 feature map is a high-order semantic feature with 8*8 resolution.

7. The method of claim 4, wherein: The fine-tuning of the MobileViT+ space-time model specifically includes: Each frame of image in the single-person dataset is cropped into a human target image block with 128*128 pixels, and continuous 5 frames are extracted according to the target ID to form a time sequence, so as to obtain a plurality of groups of image block sequences of the same target with continuous 5 frames; The image block sequence is input into the backbone network of the MobileViT+ space-time model, image features are extracted, and a three-layer feature pyramid including P3 feature map, P4 feature map and P5 feature map is output; The P5 feature map of the last frame is input into the space branch of the MobileViT+ space-time model, the P5 feature map is divided into a plurality of spatial regions by the Voronoi grid generator, and then the risk score of each region is predicted by the MLP classifier, and the region risk prediction result is output; The P5 feature map of the continuous 5 frames is input into the time branch of the MobileViT+ space-time model, the time sequence behavior is analyzed by Bi-LSTM, and the time sequence behavior prediction result is output; The third loss is constructed by using the region risk prediction result, the time sequence behavior prediction result and the corresponding third label, and the parameters of the third stage and the fourth stage of the backbone network of the MobileViT+ space-time model, and the parameters of the space branch and the time branch are updated based on the third loss.

8. A method of quadratic unambiguous detection, characterized in that: The method comprises the following steps: The method comprises the following steps: Acquiring a human target video image, performing a first detection based on the human target video image, and outputting a first detection result, wherein the first detection result includes a human target; analyzing and predicting the first detection result by using the YOLO refining model obtained by the training method in any one of claims 1-7 to obtain a position refining result and a solo score prediction result of the human target; and determining a candidate solo target based on the position refining result and the solo score prediction result; analyzing and predicting the candidate solo target by using the MobileViT+ space-time model obtained by the training method in any one of claims 1-7 to obtain a region risk prediction result and a time sequence behavior prediction result of the human target; determining a judgment result of whether the human target is a solo person based on the region risk prediction result and the time sequence behavior prediction result; judging whether to trigger an alarm based on the judgment result; wherein the judgment result of whether the human target is a solo person based on the region risk prediction result and the time sequence behavior prediction result is determined by the following steps: fusing the region risk prediction result and the time sequence behavior prediction result to calculate a comprehensive risk score; comparing the comprehensive risk score with a preset safety threshold, and if the comprehensive risk score exceeds the preset safety threshold, determining that the human target is a solo person and triggering an alarm.

9. The method of Claim 8, wherein: The determination of the candidate solo target based on the position refining result and the solo score prediction result is performed by the following steps: determining the overlap degree between the position refining result and the first detection result to output a judgment result, and if the judgment result exceeds a preset refining threshold, confirming that the human target is the same target; comparing the solo score prediction result with a preset solo threshold, and if the solo score prediction result exceeds the preset solo threshold, confirming that the human target is a candidate solo target and adding it to a subsequent space-time analysis queue.

10. A quadratic unambiguous detection system characterized by: The secondary solo detection system includes: an intelligent camera for capturing a human target video image and performing a first detection by using an embedded deep learning model to output a first detection result, wherein the first detection result includes a human target; an edge computing device for deploying the YOLO refining model and the MobileViT+ space-time model obtained by the training method in any one of claims 1-7, and then obtaining the first detection result, and determining a judgment result of whether the human target is a solo person by using the YOLO refining model and the MobileViT+ space-time model; an alarm device for determining whether to trigger an alarm based on the received judgment result.

Images