An artificial intelligence-based monitoring method and system

Abstract

The application discloses a kind of based on artificial intelligence monitoring method and system, method includes: receiving real-time monitoring data stream from multimodal sensor array, based on real-time monitoring data stream generates and fuses data sequence containing visual feature and behavior feature;Fusion data sequence is extracted in space-time feature, constructs behavior feature vector set, feature vector in behavior feature vector set is decomposed into short-term dynamic feature and long-term trend feature;Short-term dynamic feature and long-term trend feature are quantified by using twin network trained by confrontation, and the abnormal probability distribution graph of monitoring scene is generated;According to abnormal probability distribution graph, multi-threshold grading early warning and disposal strategy matching are carried out, and the final disposal instruction is determined as the response output of monitoring system.Utilize the embodiment of the application, can improve early discovery and accurate early warning capability to potential risk, effectively enhance the reliability and practicality of monitoring system.

Description

Technical Field

[0001] This invention belongs to the field of monitoring technology, specifically a monitoring method and system based on artificial intelligence. Background Technology

[0002] In the field of security monitoring, AI-based behavior analysis technology has become a research hotspot. Existing intelligent monitoring methods typically rely on real-time analysis of video streams to detect predefined abnormal behaviors. However, most of these methods have significant limitations: First, they often focus on short-term, transient visual features, lacking multi-scale modeling of long-term behavioral trends, resulting in insensitivity to latent, progressive anomalies; second, traditional models are usually trained on closed, static datasets, and their generalization ability and robustness are insufficient when facing the complex and ever-changing environments and adversarial interference in real-world monitoring scenarios, easily leading to false alarms; furthermore, most systems use a single warning threshold, making it difficult to provide differentiated handling strategies based on the risk level of events, failing to meet the needs of modern security systems for accurate early warning and efficient response. Summary of the Invention

[0003] The purpose of this invention is to provide an artificial intelligence-based monitoring method and system to address the shortcomings of existing technologies, improve the ability to detect potential risks early and provide accurate warnings, and effectively enhance the reliability and practicality of the monitoring system.

[0004] One embodiment of this application provides an artificial intelligence-based monitoring method, the method comprising: Receive real-time monitoring data streams from a multimodal sensor array, and generate a fused data sequence containing visual and behavioral features based on the real-time monitoring data streams; Spatiotemporal features are extracted from the fused data sequence to construct a set of behavioral feature vectors. The feature vectors in the set of behavioral feature vectors are decomposed into short-term dynamic features and long-term trend features to capture the multi-scale behavioral patterns of the monitored object. An anomaly metric is performed on the short-term dynamic features and long-term trend features using an adversarial-trained Siamese network to generate an anomaly probability distribution map of the monitoring scene. Based on the aforementioned anomaly probability distribution map, multi-threshold hierarchical early warning and handling strategies are matched to determine the final handling instruction, which serves as the response output of the monitoring system.

[0005] Another embodiment of this application provides an artificial intelligence-based monitoring system, the system comprising: The receiving module is used to receive real-time monitoring data streams from a multimodal sensor array and generate a fused data sequence containing visual and behavioral features based on the real-time monitoring data streams. The extraction module is used to extract spatiotemporal features from the fused data sequence, construct a set of behavioral feature vectors, and decompose the feature vectors in the set of behavioral feature vectors into short-term dynamic features and long-term trend features to capture the multi-scale behavioral patterns of the monitored object. The quantization module is used to measure the anomalies of the short-term dynamic features and long-term trend features using an adversarially trained Siamese network, and generate an anomaly probability distribution map of the monitoring scene. The handling module is used to perform multi-threshold hierarchical early warning and handling strategy matching based on the anomaly probability distribution map, determine the final handling instruction, and serve as the response output of the monitoring system.

[0006] Another embodiment of this application provides a storage medium storing a computer program, wherein the computer program is configured to execute the method described in any of the preceding claims when running.

[0007] Another embodiment of this application provides an electronic device including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the method described in any of the preceding claims.

[0008] Compared with existing technologies, this invention provides an artificial intelligence-based monitoring method that receives real-time monitoring data streams from a multimodal sensor array and generates a fused data sequence containing visual and behavioral features based on the real-time monitoring data streams. The fused data sequence undergoes spatiotemporal feature extraction to construct a set of behavioral feature vectors, which are then decomposed into short-term dynamic features and long-term trend features. An adversarially trained Siamese network is used to quantify anomalies in the short-term dynamic features and long-term trend features, generating an anomaly probability distribution map of the monitoring scene. Based on the anomaly probability distribution map, multi-threshold hierarchical early warning and response strategy matching are performed to determine the final response instruction, which serves as the response output of the monitoring system. This improves the early detection and accurate early warning capabilities for potential risks, effectively enhancing the reliability and practicality of the monitoring system. Attached Figure Description

[0009] Figure 1 A hardware structure block diagram of a computer terminal for an artificial intelligence-based monitoring method provided in an embodiment of the present invention; Figure 2 A flowchart illustrating an artificial intelligence-based monitoring method provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of an artificial intelligence-based monitoring system provided in an embodiment of the present invention. Detailed Implementation

[0010] The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0011] This invention first provides an artificial intelligence-based monitoring method, which can be applied to electronic devices, such as computer terminals, specifically ordinary computers.

[0012] The following detailed explanation uses a computer terminal as an example. Figure 1 This is a hardware structure block diagram of a computer terminal for an artificial intelligence-based monitoring method provided in an embodiment of the present invention. (See diagram below.) Figure 1 As shown, the computer device includes a processor, memory, and network interface connected via a system bus, wherein the memory may include non-volatile storage media and internal memory.

[0013] See Figure 2 The present invention provides an artificial intelligence-based monitoring method, which may include the following steps: S201, Receive real-time monitoring data stream from multimodal sensor array, and generate a fused data sequence containing visual features and behavioral features based on the real-time monitoring data stream; Specifically, video surveillance data, infrared thermal imaging data, and depth image data can be acquired simultaneously through a multimodal sensor array, and a timestamp alignment algorithm can be used to ensure the temporal consistency of multi-source data and generate a synchronous multimodal data stream. The multimodal sensor array needs to be adapted to the monitoring scenario (taking school gate monitoring as an example), and consists of three types of core sensors to ensure that the data covers visual, temperature, and spatial dimensions: Video surveillance data acquisition: A 2-megapixel high-definition network camera (model Hikvision DS-2CD3T26WD-I5) with a resolution of 1920×1080, a frame rate of 30fps, and a lens focal length of 4mm (covering a 10-meter monitoring range) is used. It supports H.265 encoding (high compression ratio, saving bandwidth). The acquired content is dynamic video of pedestrians at the school gate (such as students entering and exiting, and vehicles passing by). Each frame of video data is approximately 50KB in size. The timestamp is generated by the camera's built-in real-time clock (RTC) with millisecond-level accuracy (format "YYYYMMDDHHMMSSmmm", such as "20251009143000123").

[0014] Infrared thermal imaging data acquisition: Equipped with an infrared thermal imaging sensor (model FLIR AX8), with a resolution of 320×240, a temperature measurement range of -20℃ to 150℃, a temperature measurement accuracy of ±2℃, and a frame rate of 15fps (adapted to the video frame rate to avoid data redundancy). The acquired content is the temperature distribution image of the monitored area (such as human body temperature 36~37℃, vehicle engine temperature 60~80℃, ambient temperature 25℃). Each frame of infrared data is accompanied by a temperature matrix (320×240 temperature values), and the timestamp is also accurate to milliseconds.

[0015] Depth image data acquisition: A TOF (Time-of-Flight) depth camera (Microsoft Azure KinectDK) was used, with a resolution of 640×576, a depth measurement range of 0.5~8 meters, an accuracy of ±1% (error ≤2cm within 2 meters), a frame rate of 10fps, and the acquired content was 3D point cloud data of the monitored area (reflecting the spatial position of pedestrians and objects, such as a student's height of 1.6~1.8 meters and a doorway pillar diameter of 0.3 meters). Each frame of depth data contained 640×576 depth values, with timestamp accuracy at the millisecond level.

[0016] The timestamp alignment algorithm uses a combination of hardware triggering and software calibration to ensure the consistency of the timing of the three types of data. Hardware triggering: The main controller (such as Raspberry Pi 4B) sends a synchronization signal once per second via the synchronization trigger line (RS485 bus) of the sensor array. After all sensors receive the signal, they immediately generate the same reference timestamp (such as "20251009143000000") to avoid deviations in independent timing by the sensors. Software calibration: For the collected multi-source data, calculate the difference between the timestamp of each sensor and the reference timestamp (e.g., 12ms difference for video, 8ms difference for infrared, and 5ms difference for depth). Adjust the timing position of the data frames through linear interpolation (e.g., delay the infrared data frame by 8ms before inserting it into the data stream). Finally, ensure that the monitored scene at the same time corresponds to 1 frame of video, 1 frame of infrared, and 1 frame of depth data, with a timing deviation ≤10ms. Generate a synchronous multimodal data stream (containing 30 frames of video, 15 frames of infrared, and 10 frames of depth per second, arranged in ascending order of timestamp).

[0017] The synchronous multimodal data stream is preprocessed, video data is processed using wavelet denoising algorithm, infrared data is processed using non-uniformity correction algorithm, and depth data is processed using point cloud registration algorithm to generate preprocessed multimodal data. Preprocessing addresses noise and bias issues in multi-source data to ensure the accuracy of subsequent feature extraction. The specific implementation is as follows: Video data wavelet denoising: Video data is susceptible to changes in lighting and electronic noise (such as darker images on cloudy days and snow noise generated by camera circuitry). A 3-level wavelet decomposition is performed using the db4 wavelet basis (Daubechies wavelet, which has good temporal locality and is suitable for dynamic video denoising). The YUV channels (Y for luminance and UV for chrominance) of each video frame are decomposed separately, and the image signal is decomposed into low-frequency approximation coefficients (preserving the main visual information) and high-frequency detail coefficients (including noise). Soft thresholding is applied to high-frequency coefficients (the threshold is set to 0.02 × the maximum value of the coefficient to balance noise reduction and detail preservation), and coefficients with absolute values ​​less than the threshold are set to 0 to suppress noise; The image is reconstructed by inverse wavelet transform to obtain the denoised video frame (such as reducing the snow noise of the original video by 80% and making the pedestrian outline clearer).

[0018] Infrared data non-uniformity correction: Infrared sensors are prone to "fixed pattern noise" (such as some pixels always displaying high temperature) due to inconsistent pixel responses. A two-point correction method is used. Infrared data of two standard blackbodies were collected: the blackbody temperatures were set to 30℃ (ambient temperature) and 50℃ (close to human body temperature), respectively, and the sensor was placed directly in front of the blackbody to collect two frames of infrared images, thus obtaining reference temperature matrices T1 (30℃) and T2 (50℃). Calculate the correction coefficient for each pixel: For each pixel (i,j) in the infrared image, according to the formulas K (i,j)=(T2-T1) / (V2 (i,j)-V1 (i,j)) and B (i,j)=T1-K (i,j)×V1 (i,j), where V1 (i,j) and V2 (i,j) are the gray values ​​of pixel (i,j) in the two reference images, respectively, K (i,j) is the gain coefficient, and B (i,j) is the offset coefficient; For each frame of infrared image in the synchronous data stream, the temperature of each pixel is corrected using the formula T(i,j)=K(i,j)×V(i,j)+B(i,j). After correction, the non-uniformity error of the infrared image is reduced from 10% to 2% (e.g., an abnormal pixel that originally displayed 35℃ will display 30℃ after correction, which is consistent with the ambient temperature).

[0019] Deep data point cloud registration: Point clouds acquired by depth cameras are prone to spatial shifts due to device vibration (such as slight camera shaking caused by wind, resulting in misalignment between the point cloud and the actual scene). The ICP (Iterative Closest Point) algorithm is used to address this. The point cloud of the first frame of depth data is selected as the "target point cloud" (containing 100,000 three-dimensional points, coordinate format (x,y,z), unit meters), and the point clouds of subsequent frames are selected as the "source point cloud". Calculate the nearest point from each point in the source point cloud to the target point cloud, construct corresponding point pairs, and solve the rotation matrix R (3×3) and translation vector t (3×1) by the least squares method to minimize the Euclidean distance error between the source point cloud and the target point cloud; The process of "finding the nearest point - solving the transformation matrix - updating the source point cloud" is executed iteratively. The maximum number of iterations is set to 20, and the error threshold is 0.01 meters (stop when the error difference between two iterations is less than 0.01 meters). After registration, the spatial deviation between the depth point cloud and the actual scene is ≤3cm (e.g., the point cloud position of the door pillar basically coincides with the actual position).

[0020] The final preprocessed multimodal data includes denoised 1080P video frames, corrected 320×240 infrared temperature frames, and registered 640×576 depth point cloud frames. The three types of data remain synchronized in time.

[0021] For the preprocessed multimodal data, key point features of the monitoring target are extracted from the video data, temperature distribution features are extracted from the infrared data, and spatial motion trajectory features are extracted from the depth data to generate a multimodal feature set. The extracted features should focus on the core attributes of the monitored targets (such as pedestrians and vehicles) to provide effective information for subsequent fusion. Video target key point feature extraction: For pedestrian targets in the video, the YOLOv8-Pose algorithm (high real-time performance, supports 17 human key point detections) is used: For each preprocessed video frame, the YOLOv8-Pose detection head identifies pedestrian targets and outputs the target bounding box (e.g., coordinates (x1=300, y1=200, x2=400, y2=450), corresponding to a pedestrian area of ​​100×250 pixels). For each pedestrian area, 17 keypoints were extracted (e.g., head keypoint (x=350, y=220), left shoulder keypoint (x=330, y=280), right knee keypoint (x=380, y=400)). The relative coordinates (with the top left corner of the bounding box as the origin) and confidence scores of each keypoint were calculated (e.g., 0.98 confidence score for the head keypoint and 0.92 confidence score for the right knee keypoint). The relative coordinates of the 17 key points are normalized (mapped to the range of 0~1) to generate a 17×2=34-dimensional key point feature vector (each key point contains two coordinate values, x and y). For example, the feature vector of a pedestrian is [0.5,0.2,0.3,0.8,...,0.8,0.6].

[0022] Infrared temperature distribution feature extraction: For temperature anomaly regions (such as high-temperature objects or the human body) in infrared frames, a sliding window statistical method is used. Set a 3×3 pixel sliding window (to balance local temperature and regional range), slide it window by window on the infrared temperature frame, and calculate the temperature mean (reflecting the average temperature of the region) and variance (reflecting the temperature uniformity) of each window. Filter windows with an average temperature more than 5°C higher than the ambient temperature (25°C) (e.g., average temperature of 36°C in the human body area and 60°C in the vehicle engine area), and record the coordinates and temperature statistics of these windows. For the filtered windows, four features are extracted: "mean temperature, temperature variance, number of windows, and maximum temperature value" to generate a 4-dimensional temperature distribution feature vector. For example, the feature vector of a frame of infrared data is [36.5, 1.2, 20, 37.8] (mean 36.5℃, variance 1.2, 20 high-temperature windows, maximum temperature 37.8℃).

[0023] Deep spatial motion trajectory feature extraction: For moving targets (such as pedestrians) in deep point clouds, Kalman filtering is used for tracking. For each frame of registered depth point cloud, the target is segmented using Euclidean clustering algorithm (with a clustering distance threshold of 0.3 meters, points with a distance of less than 0.3 meters are grouped into the same target), and the target's 3D bounding box is obtained (e.g., pedestrian bounding box x range 1.5~2.0 meters, y range 3.0~3.5 meters, z range 0~1.7 meters). Calculate the center coordinates (xc, yc, zc) of each target bounding box (e.g., (1.75, 3.25, 0.85)) as the target's position feature; Kalman filtering is used to predict the target's position in the next frame, and the trajectory is updated by combining the actual detection position. The sampling frequency is 10Hz (consistent with the depth data frame rate). The position coordinates of the target are recorded for 5 consecutive frames, generating a 5×3=15-dimensional motion trajectory feature vector (e.g., [(1.75,3.25,0.85),(1.80,3.30,0.85),...,(1.90,3.40,0.86)]).

[0024] The three types of features mentioned above are associated by timestamp to generate a multimodal feature set (containing 30 video key point features, 15 infrared temperature features, and 10 depth trajectory features per second, with each feature corresponding to a unique timestamp).

[0025] A feature-level fusion algorithm is used to deeply fuse visual and behavioral features in a multimodal feature set. The feature representations of different modalities are weighted and fused through an attention mechanism, and finally a fused data sequence containing visual and behavioral features is generated.

[0026] Feature-level fusion needs to highlight the advantages of different modalities (video reflects target shape, infrared reflects temperature attributes, and depth reflects spatial motion). An attention mechanism is used to dynamically allocate weights, as specifically implemented below: Unifying Feature Dimensions: First, the three types of features are converted to the same dimension to facilitate fusion calculation. Video keypoint features (34-dimensional) are mapped to 64-dimensional data through a fully connected layer (34-dimensional input, 64-dimensional output). Infrared temperature features (4-dimensional) are mapped to 64-dimensional data through a fully connected layer (4-dimensional input, 64-dimensional output). The depth trajectory features (15-dimensional) are mapped to 64-dimensional features through a fully connected layer (15-dimensional input, 64-dimensional output); The ReLU activation function is used during the mapping process to enhance the nonlinear representation of features and ensure that the feature dimensions of each modality are consistent (all are 64-dimensional).

[0027] Attention weight calculation: Attention weights are calculated based on the information entropy of each modality's features (the lower the information entropy, the higher the feature certainty, and the greater the weight). Calculate the information entropy of each modal feature: Information entropy formula H=-Σp (x) log2p (x), where p (x) is the normalized probability of each dimension value of the feature (e.g., if the value of a certain dimension of the video key point feature is 0.5, the normalized probability is 0.5 / Σ all dimension values); Example calculations: Video feature information entropy H1=2.1 (key point location is determined, low entropy), infrared feature H2=3.5 (temperature fluctuation is large, high entropy), depth feature H3=2.8 (trajectory is relatively stable, medium entropy); Weight normalization: The entropy value is converted into a weight by the softmax function. The weight is inversely proportional to the entropy value (the lower the entropy, the higher the weight). The calculated weights are: video weight w1 = 0.4, infrared weight w2 = 0.2, and depth weight w3 = 0.4 (the sum of the weights is 1). This weight reflects the fusion logic of "video features are the most critical, followed by depth, and infrared is the auxiliary feature".

[0028] Weighted fusion and sequence generation: The fusion feature is calculated using the formula "fusion feature = w1 × video feature + w2 × infrared feature + w3 × depth feature". Element-wise weighted summation is performed to generate a 64-dimensional single-frame fusion feature (e.g., the fusion feature of a certain frame is [0.4 × 0.5 + 0.2 × 36.5 + 0.4 × 1.75, ...], and the values ​​have been normalized to the range of 0 to 1). The fusion features of consecutive timestamps are arranged in order to generate a fusion data sequence: 10 fusion features are generated per second in the monitoring scene (consistent with the deep data frame rate, balancing real-time performance and data volume), and the sequence length is dynamically adjusted according to the monitoring duration (e.g., 600 fusion features are generated for 1 minute of monitoring, forming a 600×64 fusion data sequence).

[0029] The final fused data sequence contains pedestrian visual keypoints, temperature attributes, and spatial trajectory information, laying the foundation for subsequent spatiotemporal feature extraction.

[0030] S202, perform spatiotemporal feature extraction on the fused data sequence, construct a set of behavioral feature vectors, and decompose the feature vectors in the set of behavioral feature vectors into short-term dynamic features and long-term trend features to capture the multi-scale behavioral patterns of the monitored object. Specifically, the fused data sequence can be segmented into time series segments by using a sliding window algorithm to divide the continuous data stream into time series segments of equal length, thereby generating a set of time series data segments; The initially generated fused data sequence is a continuous feature stream, the specific form of which needs to be determined based on the monitoring scenario (taking pedestrian monitoring at a school gate as an example): This sequence contains 64-dimensional feature vectors (fused video key points, infrared temperature, and depth trajectory information), with a sampling frequency of 10 vectors / second (i.e., generating 10 64-dimensional vectors per second). If the monitoring duration is 1 minute, the complete fused data sequence is a 600×64 matrix (600 time steps × 64-dimensional features). The core of time series segmentation is to extract equal-length segments through a sliding window, ensuring that each segment covers a sufficient time dimension to capture local behavior, while balancing real-time performance and data overlap through a reasonable step size.

[0031] The key parameters of the sliding window algorithm need to be set in conjunction with the behavior cycle of the monitored target: Window size: set to 5 seconds, corresponding to 50 feature vectors (10 vectors / second × 5 seconds). The reason for choosing 5 seconds is that typical short-term behaviors of pedestrians at the campus gate (such as stopping, turning around, and picking up objects) are usually completed within 3-7 seconds. A 5-second window can fully cover these behaviors, avoiding missing behavioral details if the window is too short (such as 2 seconds) or causing data redundancy if the window is too long (such as 10 seconds).

[0032] Sliding step size: set to 2 seconds, corresponding to 20 feature vectors. A step size smaller than the window size can preserve data overlap (the overlap rate between a 5-second window and a 2-second step size is 60%), ensuring the continuity of continuous behavior (such as the transition of a pedestrian from "walking towards the door" to "staying" can be captured simultaneously by adjacent windows). At the same time, a 2-second step size can ensure that the system generates a new segment every 2 seconds, meeting the response requirements of real-time monitoring (delay ≤ 2 seconds).

[0033] The specific implementation of the segmentation process: Taking a 1-minute fused data sequence as an example, starting from time step 0, the first window captures time steps 0-49 (0-5 seconds), generating time segment 1 (dimension 50×64); the second window starts from time step 20 (delayed by 2 seconds), capturing time steps 20-69 (2-7 seconds), generating time segment 2; and so on, until all time steps are covered. If the last window captures less than 50 time steps at the end of the sequence (e.g., 35 time steps remain), zero-padding is used to add 15 64-dimensional zero vectors at the end of the segment, ensuring that all segments have a uniform dimension of 50×64. Ultimately, the 1-minute monitoring data can generate 26 time segments (calculated as (600-50) / 20 + 1 = 26.5, rounded up to 26). All segments are integrated into a time-series data segment set, with each segment corresponding to a continuous local behavioral feature.

[0034] Spatiotemporal features are extracted from each time series data segment, spatial features are extracted using a three-dimensional convolutional neural network, and time-dependent features are extracted using a recurrent neural network to generate a spatiotemporal feature vector. Each time-series data segment (50×64) contains both a time dimension (50 time steps) and a spatial dimension (64-dimensional fused features). Spatial features (internal correlations of the 64-dimensional features) and time features (behavioral dependencies of the 50 time steps) need to be extracted separately and then fused into a unified spatiotemporal feature vector.

[0035] 1. Extracting spatial features using 3D convolutional neural networks (3D CNN): The core advantage of 3D CNNs is their ability to handle both temporal and spatial dimensions simultaneously. Here, the "time step-feature dimension" structure of the temporal segment needs to be adapted to a 3D input format. First, the 50×64 segment is reshaped into a 5-dimensional tensor (1, 50, 1, 64, 1) (format: batch size × temporal depth × spatial height × spatial width × number of channels). Here, "spatial height = 1" and "spatial width = 64" correspond to the spatial distribution of 64 features, "temporal depth = 50" corresponds to 50 time steps, and "number of channels = 1" represents the feature channels.

[0036] A lightweight 3D CNN architecture was selected (to accommodate the computing resources of the monitoring terminal), and its specific structure and parameters are as follows: The first convolutional layer has a kernel size of (3, 1, 4) (temporal depth × spatial height × spatial width), a stride of (1, 1, 1), and padding of "same" (ensuring the output dimension matches the input). It has 32 output channels. This kernel captures the spatial relationships across 4 adjacent feature dimensions within 3 consecutive time steps (e.g., the local relationship between video keypoint features and depth trajectory features), and the output tensor dimension is (1, 50, 1, 64, 32).

[0037] First-level pooling: 3D max pooling is used with a pooling kernel size of (2, 1, 2) and a stride of (2, 1, 2). Its purpose is to compress the dimensionality and retain key features. The output tensor dimension becomes (1, 25, 1, 32, 32) (the temporal depth is halved to 25 and the spatial width is halved to 32).

[0038] The second convolutional layer has a kernel size of (3, 1, 3), a stride of (1, 1, 1), padding with "same", and 64 output channels. It further extracts complex spatial relationships and outputs a tensor of (1, 25, 1, 32, 64).

[0039] Second-layer pooling: 3D max pooling kernel (2, 1, 2), stride (2, 1, 2), output tensor (1, 13, 1, 16, 64) (temporal depth 13, spatial width 16).

[0040] The final pooled tensor is flattened into a two-dimensional structure of "time step × number of spatial features": time step 13, number of spatial features = 1 × 16 × 64 = 1024, that is, after flattening it is a 13 × 1024 matrix, which contains the spatial feature information of each time step.

[0041] 2. Recurrent Neural Networks (RNNs) extract time-dependent features: Long Short-Term Memory (LSTM) network was selected to process the flattened 13×1024 matrix. LSTM can effectively capture long-term dependencies in time series (such as the "walk-stop-walk" behavioral correlation of a pedestrian in the first 5 seconds). The specific parameters and structure are as follows: LSTM layer: The hidden layer dimension is set to 128, using a bidirectional structure (forward LSTM captures future time dependencies, backward LSTM captures past time dependencies), with a dropout rate of 0.2 (to prevent overfitting). The input is a 13×1024 matrix. The LSTM layer outputs 128-dimensional hidden states at each time step, and the bidirectional structure ultimately outputs 256-dimensional hidden states at each time step (forward 128 + backward 128).

[0042] Feature aggregation: Take the 256-dimensional hidden state of the last time step of the LSTM layer (containing the temporal dependency information of the entire 13 time steps), input it into the fully connected layer (input dimension 256, output dimension 128), and use the ReLU activation function to enhance the nonlinear expression, finally obtaining a 128-dimensional spatiotemporal feature vector.

[0043] Taking time segment 1 (0-5 seconds of normal pedestrian behavior) as an example, its spatiotemporal feature vector is [0.23, -0.15, 0.42, ..., 0.08] (128 dimensions, numerical range -1 to 1). Among them, the dimension values ​​related to "walking trajectory" (such as the 15th and 32nd dimensions) are significantly higher than the dimension values ​​related to "abnormal stopping" (such as the 78th and 95th dimensions). Each time segment corresponds to a 128-dimensional spatiotemporal feature vector, laying the foundation for the subsequent construction of the feature set.

[0044] A set of behavioral feature vectors is constructed, and the spatiotemporal feature vectors are arranged in chronological order to form a feature matrix. The dimensionality is reduced by principal component analysis to generate an optimized set of behavioral feature vectors. First, construct a set of behavioral feature vectors: Arrange the 128-dimensional spatiotemporal feature vectors of each time segment generated in step two in chronological order to form a feature matrix. Taking 26 time segments generated from 1 minute of monitoring data as an example, the feature matrix has a dimension of 26×128 (26 vectors × 128 dimensions). Each row of the matrix corresponds to a behavioral feature of a time window (e.g., the first row corresponds to 0-5 seconds, the second row corresponds to 2-7 seconds), and each column corresponds to a feature dimension (e.g., the first column corresponds to the "walking speed" feature, the 50th column corresponds to the "temperature anomaly" feature).

[0045] The core of Principal Component Analysis (PCA) dimensionality reduction is to reduce the feature dimension while preserving key information, thereby reducing the computational complexity of subsequent operations. The specific implementation steps are as follows: 1. Data standardization: Because the numerical ranges of different feature dimensions may differ (e.g., the value for the "temperature feature" dimension is 0.1~0.3, and the value for the "trajectory feature" dimension is 0.5~0.9), the 26×128 feature matrix needs to be standardized first. The formula is: x_norm = (x - μ) / σ, where μ is the mean of a certain feature dimension, and σ is the standard deviation of that dimension. After standardization, the mean of each feature dimension is 0, and the standard deviation is 1, thus avoiding the influence of differences in numerical ranges on the feature selection of PCA.

[0046] 2. Calculate the covariance matrix: The covariance matrix is ​​used to measure the linear correlation between different feature dimensions. It has a dimension of 128×128, and the element C_ij represents the covariance between the i-th feature and the j-th feature. The formula is: C_ij = (1 / (n-1))×Σ(x_i - μ_i)(x_j - μ_j) (n=26 is the number of samples). If C_ij is close to 0, it indicates a weak correlation between the i-th and j-th features; if C_ij > 0.7, it indicates a strong correlation and information redundancy (e.g., the correlation between the features "walking trajectory length" and "walking time" may reach 0.85).

[0047] 3. Solving for eigenvalues ​​and eigenvectors: Eigenvalue decomposition is performed on the 128×128 covariance matrix to obtain 128 eigenvalues ​​and corresponding 128 eigenvectors. The magnitude of the eigenvalue represents the proportion of information carried by the corresponding eigenvector; the larger the eigenvalue, the more original data information the eigenvector (principal component) contains. For example, the first 10 eigenvalues ​​are 56.2, 48.5, 39.1, ..., 0.8, and their sum accounts for the proportion of the sum of all eigenvalues, which is the cumulative variance contribution rate.

[0048] 4. Select the number of principal components: Set the cumulative variance contribution rate threshold to 95% (an empirical threshold to ensure that more than 95% of the original information is retained). Calculate the cumulative variance contribution rate of the first k features: the cumulative contribution rate of the first 64 features is 96.3% (more than 95%), and the first 63 features are 94.8% (less than 95%). Therefore, select the first 64 feature vectors as principal components.

[0049] 5. Generate the optimized feature matrix: The selected 64 feature vectors are arranged in descending order of eigenvalues, forming a 128×64 projection matrix (each row corresponds to an original feature dimension, and each column corresponds to a principal component). The standardized 26×128 feature matrix is ​​multiplied by the projection matrix to obtain a 26×64 optimized feature matrix, which is the optimized set of behavioral feature vectors. The dimension of each optimized feature vector is reduced from 128 to 64, with an information retention rate of 96.3%, while eliminating redundancy between features (e.g., two features with an original correlation coefficient of 0.85 have their correlation reduced to 0.12 in the principal component space).

[0050] Taking the optimized first feature vector (corresponding to the behavior of 0-5 seconds) as an example, its value is [0.18, 0.12, -0.09, ..., 0.07] (64 dimensions). Compared with the original 128-dimensional vector, the computational efficiency is improved by 50%, and it does not affect the accuracy of subsequent anomaly detection.

[0051] A multi-scale decomposition algorithm is used to decompose each feature vector in the behavioral feature vector set into high-frequency short-term dynamic features and low-frequency long-term trend features. Wavelet transform is used to achieve time-frequency domain feature separation, and finally a multi-scale behavioral feature representation is generated.

[0052] The purpose of multi-scale decomposition is to distinguish between the "short-term dynamic behavior" (such as sudden stops, quick turns, etc.) and the "long-term trend behavior" (such as continuous entry and exit from the campus, slow wandering, etc.) of the monitored object. Wavelet transform is the core algorithm for achieving time-frequency domain separation. Its advantage is that it can focus on the local features of the signal at different scales.

[0053] 1. Wavelet basis selection and decomposition parameter setting: Based on the frequency characteristics of the monitored behavioral features (short-term dynamics correspond to high-frequency signals, and long-term trends correspond to low-frequency signals), the db4 wavelet basis (Daubechies 4 wavelet, which has good temporal locality and frequency domain resolution, and is suitable for the non-stationary signal characteristics of behavioral features) is selected. The decomposition level is set to 3 levels because: a 3-level decomposition can divide the frequency range of the 64-dimensional feature vector into 4 frequency bands (1 low-frequency band + 3 high-frequency bands), which can both finely distinguish between short-term and long-term features and avoid the computational burden caused by too many decomposition levels.

[0054] 2. Decomposition using Discrete Wavelet Transform (DWT): For each 64-dimensional feature vector in the optimized set of behavioral feature vectors, a 3-level discrete wavelet transform is performed, as follows: Level 1 decomposition: The 64-dimensional vector is decomposed into one 32-dimensional low-frequency coefficient (A1, corresponding to coarse information of long-term trends) and one 32-dimensional high-frequency coefficient (D1, corresponding to high-frequency information of short-term dynamics). The low-frequency coefficient A1 is generated through a low-pass filter to capture slowly changing components in the feature vector (such as the trend of a pedestrian's average moving speed over 5 minutes); the high-frequency coefficient D1 is generated through a high-pass filter to capture rapidly changing components (such as a sudden change in a pedestrian's speed within 1 second).

[0055] Level 2 decomposition: The 32-dimensional low-frequency coefficients A1 obtained from the Level 1 decomposition are further decomposed to obtain 16-dimensional low-frequency coefficients A2 (more refined long-term trends) and 16-dimensional high-frequency coefficients D2 (medium-frequency short-term dynamics, such as the changes in a pedestrian's movements within 3-5 seconds).

[0056] Level 3 decomposition: The 16-dimensional low-frequency coefficients A2 obtained from the Level 2 decomposition are decomposed for the third time to obtain 8-dimensional low-frequency coefficients A3 (the final long-term trend characteristics) and 8-dimensional high-frequency coefficients D3 (the highest frequency short-term dynamics, such as the instantaneous actions of pedestrians within 1-2 seconds, such as suddenly raising their hands or bending over).

[0057] 3. Multi-scale feature integration: After the three-level decomposition, each 64-dimensional optimized feature vector is decomposed into one set of low-frequency coefficients and three sets of high-frequency coefficients, with the specific dimensional allocation as follows: Long-term trend characteristics: 8-dimensional low-frequency coefficient A3 (accounting for 12.5% ​​of the original dimensions), corresponding to the overall behavioral trend of the monitored object (such as the trend of "continuous influx" of pedestrians at the school gate during the morning rush hour, or the trend of "continuous outflow" during the evening rush hour).

[0058] Short-term dynamic features: The sum of the three high-frequency coefficients is 32 (D1) + 16 (D2) + 8 (D3) = 56 dimensions (accounting for 87.5% of the original dimensions), corresponding to instantaneous behaviors at different time scales (such as D3 capturing "sudden stop", D2 capturing "turning around after stopping", and D1 capturing "walking after turning around").

[0059] The "8-dimensional long-term trend features + 56-dimensional short-term dynamic features" decomposed from each optimized feature vector are recombined to form a 64-dimensional multi-scale behavioral feature vector (the dimensions are consistent with the optimized vector for easier subsequent processing). Taking an optimized feature vector at the school gate (corresponding to the morning rush hour of 7:30-7:35) as an example, its multi-scale features are: long-term trend feature A3=[0.85, 0.82, ..., 0.79] (8 dimensions, high values ​​indicate a clear "continuous influx" trend), and short-term dynamic features D1-D3=[0.12, -0.08, ..., 0.05] (56 dimensions, some high values ​​correspond to the rapid entry actions of individual pedestrians).

[0060] Finally, all the decomposed multi-scale behavioral feature vectors are arranged in chronological order to generate a multi-scale behavioral feature representation (the dimension is still 26×64, consistent with the optimized feature matrix). This representation contains both short-term dynamics and long-term trend information, providing multi-scale behavioral basis for subsequent anomaly measurement.

[0061] S203, using an adversarially trained Siamese network to measure the anomalies of the short-term dynamic features and long-term trend features, and generating an anomaly probability distribution map of the monitoring scene. Specifically, a twin network architecture can be constructed, which includes two feature extraction branches with shared weights, to handle short-term dynamic features and long-term trend features respectively. The core of Siamese networks is to achieve a unified feature mapping for two types of features (short-term dynamics and long-term trends) through "shared weighted dual branches," ensuring the consistency of the feature space and providing a foundation for subsequent semantic distance calculation. Combined with previously developed multi-scale behavioral features (56 dimensions for short-term dynamics and 8 dimensions for long-term trends), the architecture design needs to adapt to the dimensional differences between the two types of features while ensuring the effectiveness of feature extraction. The specific implementation is as follows: 1. Definition of the overall architecture of twin networks: The Siamese network comprises three parts: a feature input layer, a shared feature extraction branch, and a feature output layer. The two branches (branch A handles short-term dynamic features, and branch B handles long-term trend features) share the weight parameters of all layers (i.e., the convolutional kernels and fully connected layer weights of branch A are exactly the same as those of branch B, and are updated synchronously during training to avoid feature space misalignment due to branch differences). The network adopts a lightweight structure of "fully connected + batch normalization + ReLU activation," adaptable to the computing resources of monitoring terminals (such as edge GPU devices), avoiding latency caused by complex convolutional operations.

[0062] 2. The specific structure of the shared feature extraction branch: To address the dimensionality differences between short-term dynamic features (56 dimensions) and long-term trend features (8 dimensions), the branch input layer unifies them to the same dimensions through a "dimensionality adaptation layer" before entering the shared feature extraction process. Dimensional Adaptation Layer: Branch A takes 56-dimensional short-term dynamic features as input, which are expanded to 64 dimensions using a single fully connected layer (input dimension 56, output dimension 64). Branch B takes 8-dimensional long-term trend features as input, which are also expanded to 64 dimensions using a single fully connected layer (input dimension 8, output dimension 64), ensuring consistent input dimensions for subsequent shared layers. This layer uses ReLU as the activation function, with batch normalization parameters set to momentum 0.9 and epsilon = 1e-5 to avoid feature distribution shift caused by dimensionality expansion.

[0063] Shared Feature Extraction Layer: This layer consists of two fully connected layers. The first layer takes a 64-dimensional input and outputs a 128-dimensional output, using ReLU activation and batch normalization. The second layer takes a 128-dimensional input and outputs a 256-dimensional output, also using ReLU activation and batch normalization. Experiments have verified that this structure retains over 90% of the feature information while keeping computational load within the limits of edge devices (single-branch forward inference time < 10ms). Increasing the number of layers increases the inference latency to over 20ms, impacting real-time monitoring.

[0064] Feature output layer: 1 fully connected layer, with 256-dimensional input and 128-dimensional output (the final feature dimension is 128 because this dimension can ensure the richness of semantic information and reduce the complexity of subsequent distance calculation - the cosine distance calculation time of the 128-dimensional vector is only 50% of that of the 256-dimensional vector). There is no activation function, and the feature vector used to calculate the semantic distance is directly output.

[0065] 3. Architecture adaptation example: Taking the multi-scale features of campus gate surveillance as an example: Branch A inputs 56-dimensional short-term dynamic features (such as the instantaneous behavioral features of pedestrians "suddenly stopping" and "quickly turning around," with some dimension values ​​[0.8, 0.75, ..., 0.1]), which are expanded to 64 dimensions through a dimension adaptation layer, and then extracted by a sharing layer to output a 128-dimensional feature vector A=[0.23, -0.15, 0.42, ..., 0.08]; Branch B inputs 8-dimensional long-term trend features (such as the overall trend of pedestrians "continuously pouring in," with dimension values ​​[0.9, 0.85, ..., 0.8]), which are processed by the same adaptation and sharing layer to output a 128-dimensional feature vector B=[0.21, -0.17, 0.45, ..., 0.1]. Due to shared weights, the feature spaces of the two vectors are completely consistent, and semantic association can be directly calculated.

[0066] Adversarial training is performed on the Siamese network. A generator produces positive and negative sample pairs, and a discriminator learns to distinguish between normal and abnormal behavior patterns, thus generating an adversarial-trained Siamese network model. The purpose of adversarial training is to enhance the Siamese network's sensitivity to "abnormal behavior"—normal behavior exhibits a strong correlation between short-term and long-term characteristics (e.g., the short-term dynamics of "normal walking" match the long-term trend of "continuous movement"), while abnormal behavior shows a weak correlation (e.g., the short-term dynamics of "sudden stagnation" contradict the long-term trend of "continuous movement"). The training process employs a "generator-discriminator" adversarial framework, specifically implemented as follows: 1. Sample Pair Construction and Generator Design: Definition of positive and negative sample pairs: Positive sample pairs are matching pairs of "normal short-term dynamic features - normal long-term trend features", such as the feature pair of "pedestrians passing quickly (short-term) - continuous influx (long-term)" at the school gate during the morning rush hour. These are extracted from historical normal monitoring data, and a total of 10,000 pairs have been collected. Negative sample pairs are mismatched pairs of "abnormal short-term dynamic features - normal long-term trend features" or "normal short-term - abnormal long-term", such as the feature pair of "pedestrians lingering (short-term abnormal) - continuous influx (long-term normal)". There are only 500 pairs of these in the historical abnormal data, which need to be expanded by the generator.

[0067] Generator Structure: A fully connected generator with Gaussian noise injection is employed. The input consists of normal features (e.g., normal short-term features), and the output simulates anomalous features. The generator comprises three fully connected layers: the first layer takes a 56-dimensional (short-term) or 8-dimensional (long-term) input and outputs a 64-dimensional output; the second layer goes from 64 to 128 dimensions; and the third layer goes from 128 to 56 or 8 dimensions (consistent with the input dimensions). Each layer is activated using LeakyReLU (slope 0.2 to avoid gradient vanishing). When generating anomalous features, Gaussian noise (mean 0, variance 0.15; noise intensity determined experimentally—too large a variance renders the features meaningless, too small a variance fails to simulate anomalies) is injected after the output of the second layer. For example, injecting normal short-term features [0.8, 0.75, ...] with noise generates anomalous short-term features [0.1, 0.9, ...] (simulating lingering behavior). The generator expands the negative sample pairs to 10,000 pairs to balance the number of positive sample pairs.

[0068] 2. Discriminator design and adversarial training process: Discriminator structure: The input is a concatenated vector (256 dimensions) of 128-dimensional short-term features and 128-dimensional long-term features from the Siamese network. The output is the probability (0-1) of "true (positive sample pair)" or "false (negative sample pair)". The discriminator consists of three fully connected layers: the first layer is 256→128 dimensions, LeakyReLU; the second layer is 128→64 dimensions, LeakyReLU; and the third layer is 64→1 dimension, Sigmoid activation. The loss function is binary cross-entropy loss (formula: L=-y log (p)-(1-y) log (1-p), where y=1 is a positive sample and y=0 is a negative sample).

[0069] Adversarial training process: An "alternating training" strategy is employed, with a total of 100 iterations and a batch size of 32. Training the discriminator: Fix the weights of the generator and the Siamese network, input positive and negative sample pairs into the Siamese network to obtain the feature concatenation vector, then input it into the discriminator, calculate the loss and update the discriminator weights. The goal is to enable the discriminator to accurately distinguish between positive and negative samples (accuracy ≥ 90%). Training the generator and the Siamese network: Fix the discriminator weights, input the fake negative sample pairs generated by the generator into the Siamese network, input the concatenated feature vector into the discriminator, calculate the adversarial loss of the generator (the goal is to make the discriminator classify fake samples as real), and update the shared weights of the generator and the Siamese network so that the Siamese network can better capture the differences in anomalous features; Validation and optimization: Every 20 rounds, the validation set (2000 pairs of samples) is used for evaluation. If the discriminator accuracy drops below 85%, the generator noise variance is reduced to 0.12 to ensure training stability.

[0070] After training, an adversarially trained Siamese network model is generated, which is more sensitive to the differences in feature mappings of anomalous feature pairs—the output feature vectors of normal sample pairs have high similarity, while those of anomalous sample pairs have low similarity.

[0071] The multi-scale behavioral feature representation is input into the trained Siamese network model, the semantic distance between short-term dynamic features and long-term trend features is calculated, the degree of abnormality is quantified by the distance metric, and an abnormality score is generated. Multi-scale behavioral features are represented as a 26×64 matrix (26 time windows, each containing 56-dimensional short-term dynamic features + 8-dimensional long-term trend features). These features need to be input into the Siamese network window by window to calculate semantic distance and convert it into anomaly scores. The specific implementation is as follows: 1. Feature input and Siamese network forward inference: The features of 26 time windows are processed window by window: For the i-th window (i=1 to 26), the 56-dimensional short-term dynamic features S_i and the 8-dimensional long-term trend features L_i are separated and input into branches A and B of the Siamese network, respectively. Branch A processes S_i: through a dimension adaptation layer (56→64), a shared feature extraction layer (64→128→256), and an output layer (256→128), the short-term feature vector F_Si (128-dimensional) is obtained. Branch B processes L_i: through a dimension adaptation layer (8→64), a shared layer, and an output layer, the long-term feature vector F_Li (128 dimensions) is obtained.

[0072] During forward inference, the weights of the Siamese network are fixed (using parameters trained adversarially), the inference time for a single window is about 8ms, and the total inference time for 26 windows is < 220ms, which meets the real-time requirements (the response latency of the monitoring system needs to be < 500ms).

[0073] 2. Semantic distance calculation: Cosine distance is chosen as the semantic distance metric because it effectively measures the semantic similarity of high-dimensional features (unaffected by the absolute value of the features, making it more suitable for feature vectors output by Siamese networks). The formula is: cos_dist = 1 - (F_Si· F_Li) / (||F_Si|| × ||F_Li||), where F_Si·F_Li is the dot product of the two vectors, and ||F_Si|| and ||F_Li|| are the L2 norms of the two vectors. The cosine distance ranges from 0 to 1: the closer the distance is to 0, the more similar the short-term and long-term features are semantically (normal behavior); the closer the distance is to 1, the greater the semantic difference (abnormal behavior).

[0074] 3. Anomaly score generation: The cosine distance is normalized to the 0-1 interval (since the cosine distance is already in the 0-1 range, it only needs to be kept consistent here), and directly used as the anomaly score. The higher the score, the more severe the anomaly. For example: Normal window (e.g., window 5 during the morning rush hour at the school gate, corresponding to 10-15 seconds): F_Si=[0.23, -0.15,0.42, ..., 0.08], F_Li=[0.21, -0.17, 0.45, ..., 0.1], dot product = 0.23×0.21 + (-0.15)×(-0.17) + ... + 0.08×0.1≈102.4, ||F_Si||≈11.2, ||F_Li||≈11.3, cosine similarity≈102.4 / (11.2×11.3)≈0.8, cosine distance = 0.2, anomaly score = 0.2 (normal); Anomaly window (e.g., window 12, corresponding to seconds 24-29, pedestrian lingering): F_Si=[0.85, 0.72, -0.31, ..., 0.56], F_Li=[0.22, -0.18, 0.43, ..., 0.09], dot product ≈ 28.6, ||F_Si|| ≈ 12.5, ||F_Li|| ≈ 11.1, cosine similarity ≈ 28.6 / (12.5×11.1) ≈ 0.21, cosine distance = 0.79, anomaly score = 0.79 (highly anomaly).

[0075] After processing 26 windows, 26 anomaly scores are generated (e.g., [0.2, 0.25, ..., 0.79, 0.82]), forming a discrete anomaly sequence.

[0076] A two-dimensional probability distribution map of the monitoring scene is constructed based on the anomaly score. The kernel density estimation algorithm is used to transform the discrete anomaly score into a continuous probability distribution, and finally an anomaly probability distribution map of the monitoring scene is generated.

[0077] Discrete anomaly scores cannot intuitively reflect the continuous relationship between "time" and "anomaly probability". Therefore, a two-dimensional continuous distribution needs to be constructed using kernel density estimation (KDE). The two-dimensional dimension is defined as "time axis (horizontal axis) - anomaly score axis (vertical axis)", which intuitively presents the spatiotemporal distribution of anomaly probability in the monitoring scenario. The specific implementation is as follows: 1. Definition of a two-dimensional coordinate system: The horizontal axis (time axis) corresponds to the time range of 26 time windows, with a total monitoring duration of 60 seconds. Each window corresponds to 2 seconds (sliding step size), so the horizontal axis range is 0-60 seconds. A discrete point is marked every 2 seconds (a total of 26 points, corresponding to the start time of 26 windows, such as the 1st window corresponding to 0 seconds, the 2nd window corresponding to 2 seconds, ..., the 26th window corresponding to 50 seconds). The vertical axis (anomaly score axis) ranges from 0 to 1, corresponding to the range of anomaly scores. It is divided into 10 scales at 0.1 intervals (0, 0.1, 0.2, ..., 1.0). The vertical axis coordinates of discrete points are the anomaly scores of the corresponding windows (e.g., the coordinates of the 5th window score of 0.2 are (10, 0.2), and the coordinates of the 12th window score of 0.79 are (24, 0.79)).

[0078] 2. Kernel density estimation algorithm parameter settings and calculation: The core of kernel density estimation is to generate a continuous probability density surface by weighting each discrete point using a "kernel function". The key parameters and calculation process are as follows: Kernel function selection: The Gaussian kernel function (normal distribution kernel) is adopted because of its good smoothness and ability to effectively fit the distribution characteristics of abnormal scores. The formula is: K(x) = (1 / √(2π)) × e^(-x) 2 / 2), where x is the Euclidean distance between the point to be calculated and the discrete point; Bandwidth (h) selection: Bandwidth determines the smoothness of the kernel function. It is automatically calculated using the Silverman rule, with the formula h = 1.06 × σ × n^(-1 / 5), where σ is the standard deviation of 26 anomaly scores (e.g., σ=0.25), and n=26 (number of samples). The calculated value is h≈1.06×0.25×26^(-1 / 5)≈0.18 (too small a bandwidth will lead to fragmentation of the distribution, while too large a bandwidth will result in over-smoothing and loss of details; 0.18 is the optimal value). Continuous probability density calculation: Divide the two-dimensional plane into a 600×100 grid (horizontal axis 0-60 seconds, one grid point every 0.1 seconds; vertical axis 0-1, one grid point every 0.01 seconds). For each grid point (x, y), calculate the weighted sum of the kernel function values ​​of all 26 discrete points (x_i, y_i), which is the probability density of that point: f(x, y) = (1 / (n×h)) ×Σ[K((x - x_i) / h) × K((y - y_i) / h)].

[0079] For example, for grid point (24, 0.8), the kernel function values ​​calculated for the discrete point (24, 0.79) are: (x-x_i) / h=0, K(0)=0.3989; (y-y_i) / h=(0.8-0.79) / 0.18≈0.056, K(0.056)≈0.397, and the probability density after weighted sum is ≈0.08 (high probability); after calculating the kernel function values ​​for grid point (10, 0.3) and discrete point (10, 0.2), the probability density is ≈0.02 (low probability).

[0080] 3. Generation of anomaly probability distribution maps: The calculated grid point probability density is transformed into a two-dimensional distribution map in the form of a heatmap: probability density 0-0.02 is blue (low probability of anomaly), 0.02-0.05 is yellow (medium probability), and 0.05-0.1 is red (high probability). Taking the campus gate surveillance as an example, in the distribution map, the vertical axis area of ​​0.7-0.9 in the 24-30 second period (windows 12-15) shows a red high probability area (corresponding to abnormal pedestrian lingering), while the areas in the 0-10 second and 50-60 second periods are blue low probability areas (corresponding to normal passage), intuitively presenting the spatiotemporal distribution of anomalies and providing a visual basis for subsequent graded early warning.

[0081] S204. Based on the abnormal probability distribution map, perform multi-threshold hierarchical early warning and handling strategy matching to determine the final handling instruction, which serves as the response output of the monitoring system.

[0082] Specifically, a multi-level early warning threshold system can be constructed, setting multiple levels of thresholds based on different probability intervals of the anomaly probability distribution map to generate a dynamic threshold set; The core parameter of the anomaly probability distribution map is the "probability density" (range 0-0.1, derived from previous kernel density estimation results; for example, in a school gate monitoring scenario, the probability density of normal areas is mostly < 0.02, while that of abnormal areas is mostly ≥ 0.02). The multi-level early warning threshold system needs to combine the risk tolerance of the monitoring scenario with historical anomaly data, dividing it into three levels: "low risk," "medium risk," and "high risk" (balancing early warning accuracy and false alarm rate; too many levels can lead to complex decision-making, while too few levels cannot distinguish the degree of risk). The threshold settings need to be "dynamically adjustable" to adapt to real-time scenario changes (e.g., different risk standards for peak and off-peak traffic). The specific implementation is as follows: 1. Basic threshold interval division: Based on data from 1000 historical abnormal events at the school gate over the past 6 months, the probability density distribution of different risk events was statistically analyzed: Low-risk events (such as brief pauses of 2 seconds or slight loitering): 92% of these events have a probability density of 0.015-0.02. These events pose no safety risk and only require logging. Medium-risk events (such as lingering for 5-10 seconds, suspected items left behind): 88% of these events have a probability density of 0.02-0.05, requiring monitoring personnel to pay attention. High-risk events (such as staying for more than 10 seconds, gathering of people, abnormal behavior): 95% of the events have a probability density ≥0.05, requiring immediate intervention.

[0083] Based on this, the following basic thresholds are set: Low-risk threshold T1 = 0.02 (probability density ≥ 0.015 and < 0.02 is considered low-risk), Medium-risk threshold T2 = 0.05 (≥ 0.02 and < 0.05 is considered medium-risk), and High-risk threshold T3 = 0.1 (≥ 0.05 and ≤ 0.1 is considered high-risk). Each threshold corresponds to a clear risk definition, avoiding ambiguous boundaries (e.g., 0.02 does not belong to either low-risk or medium-risk, but is the starting point of medium-risk).

[0084] 2. Threshold dynamic adjustment mechanism: The base threshold is not fixed and needs to be dynamically adjusted based on real-time scene parameters (such as pedestrian density, time of day, and weather). The adjustment logic is implemented through a mapping of "scene parameters - threshold adjustment coefficient". Pedestrian density correction: When the real-time pedestrian flow at the entrance is ≥50 people / minute (common during the morning peak of 7:30-8:00), the low-risk threshold T1 is lowered to 0.015 (because even minor anomalies can cause congestion when pedestrian flow is high, earlier warnings are needed); when the pedestrian flow is ≤10 people / minute (after 22:00 at night), T1 is raised to 0.025 (to reduce false alarms at night); Time period adjustment: During weekday school hours (8:00-12:00), the medium-risk threshold T2 remains at 0.05; during holidays (such as National Day), T2 is lowered to 0.04 (the population is more complex during holidays, so vigilance is required). Weather correction: In rainy weather (camera field of view may be affected), the high-risk threshold T3 is lowered to 0.045 (to reduce the risk of missed reports); in sunny weather, it remains at 0.1.

[0085] The final set of dynamic thresholds is as follows: T1=0.015, T2=0.04, T3=0.045 for the morning rush hour scenario (7:30 AM, 60 people / minute); and T1=0.025, T2=0.05, T3=0.1 for the nighttime scenario (11:00 PM, 5 people / minute), ensuring that the thresholds always adapt to the risk requirements of the current scenario.

[0086] The anomaly probability distribution map is compared with a dynamic threshold set, and anomaly regions exceeding the threshold are identified by a region growing algorithm to generate an anomaly region identification map. The anomaly probability distribution map is a 600×100 grid matrix (horizontal axis 0-60 seconds, 0.1 seconds per grid; vertical axis 0-1, 0.01 score per grid). A region growing algorithm is needed to merge discrete "high-probability grid points" into continuous anomaly regions to avoid misjudging isolated points. Simultaneously, visual indicators are used to distinguish risk levels. The specific implementation is as follows: 1. Core process of the region growing algorithm: The essence of the region growing algorithm is to "start from the seed point and gradually merge adjacent grid points of the same type," adapting to the continuous characteristics of abnormal regions (such as the abnormal probability of people staying in the area remaining high over a continuous period of time). The steps are as follows: Step 1: Seed Point Selection. Traverse the grid matrix and mark grid points whose probability density exceeds the corresponding dynamic threshold as seed points, categorizing them by risk level: those exceeding T3 (high risk) are red seed points, those exceeding T2 but not exceeding T3 (medium risk) are yellow seed points, and those exceeding T1 but not exceeding T2 (low risk) are orange seed points. Taking the campus morning rush hour scenario (T3=0.045) as an example, 12 points, including grid points (24, 0.8) (24 seconds, score 0.8, probability density 0.06) and (24.1, 0.81) (24.1 seconds, score 0.81, probability density 0.062), are marked as red seed points.

[0087] Step 2: Region Growing. For each seed point, check its 4 neighboring grid points (up, down, left, right, excluding diagonals to avoid over-expansion of the region). If the probability density difference between the neighboring points and the current seed point is ≤0.01, and the neighboring points also exceed the corresponding threshold, then merge them into the current region. For example, the right neighbor (24.1, 0.81) of the red seed point (24, 0.8) has a probability density difference of 0.002 (≤0.01) and exceeds T3, so it is merged into the red region; the lower neighbor (24, 0.79) has a probability density of 0.058 (exceeding T3), so it is also merged.

[0088] Step 3: Growth Termination. Growth terminates when all neighboring points of a region no longer meet the condition of "difference ≤ 0.01 and exceeding the threshold". For example, if a red region grows from 24 seconds to 30 seconds, the probability density of the neighboring point (30.1, 0.8) is 0.04 (not exceeding T3=0.045), so growth stops, eventually forming a continuous red region of 24-30 seconds with a score of 0.7-0.9.

[0089] 2. Generation of anomaly area marker map: The grown areas are colored according to their risk level to generate a visual labeling map: High-risk area (red): Fill color is #FF0000, boundary line is a thick solid line (2px), and the area information is marked ("High-risk area: 24.0-30.0 seconds, score 0.70-0.90, probability density 0.045-0.09"). Medium-risk area (yellow): Fill color is #FFFF00, boundary line is a solid line (1px), labeled "Medium-risk area: 15.0-18.0 seconds, score 0.50-0.65, probability density 0.020-0.044"; Low-risk area (orange): Fill color is #FFA500, boundary line is thin solid line (0.5px), labeled "Low-risk area: 8.0-10.0 seconds, score 0.30-0.40, probability density 0.015-0.019"; Normal area (blue): Fill color is #0000FF, no annotation.

[0090] The indicator map overlays both timeline and rating scales, making it easier for monitoring personnel to intuitively locate the time range and severity of anomalies. For example, in a campus setting, the red high-risk area clearly covers the 24-30 second time period, indicating that there is high-risk behavior during this period.

[0091] Based on the abnormal area identification map, the handling strategy is matched, and the corresponding handling plan is selected from the strategy library according to the abnormality level, area size and duration to generate a set of candidate handling strategies. The response strategy library needs to predefine three-dimensional matching rules of "anomaly level - area size - duration" to cover the response needs of different scenarios and avoid a one-size-fits-all response (e.g., the response measures for high-risk small areas for a short time should be different from those for high-risk large areas for a long time). The specific implementation is as follows: 1. Strategy library construction and dimension definition: The policy database is stored using a relational database (such as MySQL). Each policy contains six fields: "Policy ID, Anomaly Level, Region Size (Number of Grid Points), Duration (Seconds), Handling Measures, and Execution Priority." Region size division: Small region (≤15 grid points, corresponding to an actual time of 1.5 seconds and a score range of 0.15), Medium region (16-50 grid points, corresponding to a time of 1.6-5 seconds and a score range of 0.16-0.5), Large region (>50 grid points, corresponding to a time of >5 seconds and a score range of >0.5). Duration classification: short duration (≤5 seconds), medium duration (6-20 seconds), long duration (>20 seconds); Execution priority: 1-5 (level 1 is the highest and is executed first).

[0092] Example entries for the strategy library: Strategy 1: ID=P001, Level=High Risk, Size=Large, Time=Long, Measures=“1. Trigger the security room's audible and visual alarm; 2. Start target tracking and recording; 3. Push the alarm to the security terminal; 4. Record the event log”, Priority=1; Strategy 2: ID=P002, Level=High Risk, Size=Small, Time=Short, Measures="1. Start recording; 2. Log; 3. Do not push real-time alerts", Priority=3; Strategy 3: ID=P003, Level=Medium Risk, Size=Medium, Time=Medium, Measures=“1. Pop-up reminder on the monitoring center screen; 2. Log", Priority=4; Strategy 4: ID=P004, Level=Low Risk, Size=Any, Time=Any, Measure="Only record event logs, summarize and analyze weekly", Priority=5.

[0093] 2. Strategy matching process: Three key parameters are extracted from the anomaly region identifier map and matched with the strategy library: Anomaly level assessment: Red areas correspond to high risk; Calculate the area size: count the number of grid points in the red area. For example, in the campus scene, the red area from 24 to 30 seconds contains 48 grid points (the middle area). Calculate the duration: The horizontal span of the red area is 30.0 - 24.0 = 6 seconds (medium time).

[0094] The strategy database is searched for entries with "Level = High Risk, Size = Medium, Time = Medium". Strategy P001 is matched (because "Large" in P001 includes "Medium" and "Long" in "Time" is a compatible match). At the same time, a supplementary search is conducted to see if there are more precise entries (such as no specific strategy for "High Risk - Medium - Medium"). Finally, the candidate handling strategy set is determined to be {all measures of strategy P001}, namely: "Trigger the security room's audible and visual alarm, start target tracking and recording, push alarm to the security terminal, and record the event log".

[0095] A risk assessment is performed on the set of candidate handling strategies, the optimal handling strategy is selected and specific execution instructions are generated, and finally the response instructions of the monitoring system are output.

[0096] Candidate strategies may have "redundant measures" or "cost conflicts" (e.g., triggering alarms and push notifications simultaneously requires assessment of necessity). A risk assessment is needed to select the optimal strategy, ensuring the measures are effective and the costs are controllable (e.g., avoiding frequent alarms that disrupt normal order). The specific implementation is as follows: 1. Risk assessment indicators and methods: A two-dimensional risk-cost assessment matrix was used, with the horizontal axis representing "risk mitigation effectiveness" (1-5 points, where 5 points indicates complete risk mitigation) and the vertical axis representing "response cost" (1-5 points, where 5 points indicates extremely high cost) to evaluate each candidate measure. Risk mitigation effectiveness scoring criteria: the ability of measures to intervene in abnormal events, such as "audio-visual alarms" which can quickly attract the attention of security guards, scoring 5 points; "logging" only retains evidence and has no real-time mitigation effect, scoring 1 point; The cost of handling is scored based on: resource consumption (such as storage and bandwidth) and interference level (such as noise and personnel interference). For example, the volume of "audio-visual alarm" is 80 decibels (which meets campus standards and does not cause excessive interference), and the cost is 2 points; "target tracking video" occupies 100MB / minute of storage (the system reserves 100GB of emergency storage), and the cost is 1 point.

[0097] Evaluation results of candidate measures in the campus scenario: Audible and visual alarm: mitigation effect 5 points, cost 2 points; Target tracking and recording: mitigation effect 4 points, cost 1 point; Push notification: mitigation effect 5 points, cost 1 point; Log recording: mitigation effect 1 point, cost 1 point.

[0098] 2. Optimal Strategy Selection and Instruction Generation: Selection Principle: Prioritize "high mitigation effect + low cost," and eliminate redundant measures (such as measures that are not duplicated). All candidate measures must meet the criteria of "mitigation effect ≥ 4 points and cost ≤ 2 points," and be retained without redundancy. Generate specific execution instructions (the instructions must include "execution target, parameters, time, and feedback requirements" to ensure feasibility): Command 1 (Audio-Visual Alarm): Execution Target = Audible and visual alarm device at the security room of the west gate of the campus, Parameters = "Alarm frequency 1 time / second, volume 80 decibels, duration 30 seconds", Execution Time = "2025-10-09 14:30:24", Feedback Requirement = "Return 'normal operation' signal within 1 second after device startup"; Command 2 (Target Tracking): Execution Target = Ximen No. 2 HD Camera (ID=CAM002), Parameters = "Resolution 1920×1080, Frame Rate 30fps, Tracking Area = Area corresponding to the screen for 24-30 seconds, Recording Storage Path = / monitor / alert / 20251009 / 143024.mp4, Retention Duration 7 days", Execution Time = "Start synchronously with alarm", Feedback Requirement = "Return 'Storage Normal' signal within 2 seconds after recording starts"; Command 3 (Push Alarm): Execution Target = Campus Security APP Server, Parameters = "Recipient = Security Captain (Mobile Number 138XXXX1234), Alarm Content = 'High-risk anomaly detected by West Gate monitoring: Personnel loitering for 24.0-30.0 seconds, area size 48 grid points, duration 6 seconds, immediate on-site verification recommended', Push Method = APP pop-up + SMS", Execution Time = "Within 1 second after alarm activation", Feedback Requirements = "Recipient returns 'Viewed' confirmation signal after reading"; Instruction 4 (Log): Execution target = Security system database (ID=DB001), Parameter = "Log fields: Event ID=ALERT202510091430, Time=2025-10-09 14:30:24, Level=High risk, Area=24.0-30.0 seconds / 0.70-0.90 score, Handling measures=Alarm + Tracking + Push, Handling result=Pending feedback", Execution time = "Within 5 seconds after all instructions are executed", Feedback requirement = "Return 'Storage successful' signal after log is written".

[0099] 3. Command output and verification: The four instructions are encapsulated in JSON format and sent to the corresponding execution devices via the campus security system's instruction bus. Simultaneously, the instruction execution status is displayed on the monitoring center screen (e.g., "Audio-visual alarm: Running" "Recording: Storage"). Once all device feedback signals are normal, a "Monitoring system response complete" message is output. The entire process has a delay of ≤10 seconds, meeting the response requirements for real-time monitoring (≤500ms).

[0100] Another embodiment of the present invention provides an artificial intelligence-based monitoring system, see [link to relevant documentation]. Figure 3 The system may include: The receiving module 301 is used to receive real-time monitoring data streams from a multimodal sensor array and generate a fused data sequence containing visual features and behavioral features based on the real-time monitoring data streams. The extraction module 302 is used to extract spatiotemporal features from the fused data sequence, construct a set of behavioral feature vectors, and decompose the feature vectors in the set of behavioral feature vectors into short-term dynamic features and long-term trend features to capture the multi-scale behavioral patterns of the monitored object. The quantization module 303 is used to perform anomaly quantification on the short-term dynamic features and long-term trend features using an adversarially trained Siamese network, and generate an anomaly probability distribution map of the monitoring scene. The handling module 304 is used to perform multi-threshold hierarchical early warning and handling strategy matching based on the abnormal probability distribution map, determine the final handling instruction, and serve as the response output of the monitoring system.

[0101] The above description, based on the embodiments shown in the figures, details the structure, features, and effects of the present invention. The above description is only a preferred embodiment of the present invention, but the present invention is not limited to the scope of implementation shown in the figures. Any changes made in accordance with the concept of the present invention, or equivalent embodiments modified to have equivalent changes, that do not exceed the spirit covered by the specification and figures, should be within the protection scope of the present invention.

Claims

1. A monitoring method based on artificial intelligence, characterized in that, The method includes: Receive real-time monitoring data streams from a multimodal sensor array, and generate a fused data sequence containing visual and behavioral features based on the real-time monitoring data streams; Spatiotemporal features are extracted from the fused data sequence to construct a set of behavioral feature vectors. The feature vectors in the set of behavioral feature vectors are decomposed into short-term dynamic features and long-term trend features to capture the multi-scale behavioral patterns of the monitored object. An anomaly metric is performed on the short-term dynamic features and long-term trend features using an adversarial-trained Siamese network to generate an anomaly probability distribution map of the monitoring scene. Based on the aforementioned anomaly probability distribution map, multi-threshold hierarchical early warning and handling strategies are matched to determine the final handling instruction, which serves as the response output of the monitoring system.

2. The method according to claim 1, characterized in that, The process of receiving real-time monitoring data streams from a multimodal sensor array and generating a fused data sequence containing visual and behavioral features based on the real-time monitoring data streams includes: Video surveillance data, infrared thermal imaging data, and depth image data are acquired synchronously by a multimodal sensor array. A timestamp alignment algorithm is used to ensure the temporal consistency of multi-source data and generate a synchronous multimodal data stream. The synchronous multimodal data stream is preprocessed, video data is processed using wavelet denoising algorithm, infrared data is processed using non-uniformity correction algorithm, and depth data is processed using point cloud registration algorithm to generate preprocessed multimodal data. For the preprocessed multimodal data, key point features of the monitoring target are extracted from the video data, temperature distribution features are extracted from the infrared data, and spatial motion trajectory features are extracted from the depth data to generate a multimodal feature set. A feature-level fusion algorithm is used to deeply fuse visual and behavioral features in a multimodal feature set. The feature representations of different modalities are weighted and fused through an attention mechanism, and finally a fused data sequence containing visual and behavioral features is generated.

3. The method according to claim 2, characterized in that, The step of extracting spatiotemporal features from the fused data sequence to construct a set of behavioral feature vectors, and decomposing the feature vectors in the set of behavioral feature vectors into short-term dynamic features and long-term trend features to capture the multi-scale behavioral patterns of the monitored object, includes: The fused data sequence is segmented into time series segments by using a sliding window algorithm to divide the continuous data stream into time series segments of equal length, thereby generating a set of time series data segments. Spatiotemporal features are extracted from each time series data segment, spatial features are extracted using a three-dimensional convolutional neural network, and time-dependent features are extracted using a recurrent neural network to generate a spatiotemporal feature vector. A set of behavioral feature vectors is constructed, and the spatiotemporal feature vectors are arranged in chronological order to form a feature matrix. The dimensionality is reduced by principal component analysis to generate an optimized set of behavioral feature vectors. A multi-scale decomposition algorithm is used to decompose each feature vector in the behavioral feature vector set into high-frequency short-term dynamic features and low-frequency long-term trend features. Wavelet transform is used to achieve time-frequency domain feature separation, and finally a multi-scale behavioral feature representation is generated.

4. The method according to claim 3, characterized in that, The step of using an adversarially trained Siamese network to measure anomalies in the short-term dynamic features and long-term trend features, and generating an anomaly probability distribution map of the monitoring scene, includes: Construct a twin network architecture, which includes two feature extraction branches with shared weights, to handle short-term dynamic features and long-term trend features respectively; Adversarial training is performed on the Siamese network. A generator produces positive and negative sample pairs, and a discriminator learns to distinguish between normal and abnormal behavior patterns, thus generating an adversarial-trained Siamese network model. The multi-scale behavioral feature representation is input into the trained Siamese network model, the semantic distance between short-term dynamic features and long-term trend features is calculated, the degree of abnormality is quantified by the distance metric, and an abnormality score is generated. A two-dimensional probability distribution map of the monitoring scene is constructed based on the anomaly score. The kernel density estimation algorithm is used to transform the discrete anomaly score into a continuous probability distribution, and finally an anomaly probability distribution map of the monitoring scene is generated.

5. The method according to claim 4, characterized in that, The step of matching multi-threshold graded early warning and handling strategies based on the anomaly probability distribution map to determine the final handling instruction, which serves as the response output of the monitoring system, includes: Construct a multi-level early warning threshold system, set multiple levels of thresholds according to different probability intervals of the anomaly probability distribution map, and generate a dynamic threshold set; The anomaly probability distribution map is compared with a dynamic threshold set, and anomaly regions exceeding the threshold are identified by a region growing algorithm to generate an anomaly region identification map. Based on the abnormal area identification map, the handling strategy is matched, and the corresponding handling plan is selected from the strategy library according to the abnormality level, area size and duration to generate a set of candidate handling strategies. A risk assessment is performed on the set of candidate handling strategies, the optimal handling strategy is selected and specific execution instructions are generated, and finally the response instructions of the monitoring system are output.

6. A monitoring system based on artificial intelligence, characterized in that, The system includes: The receiving module is used to receive real-time monitoring data streams from a multimodal sensor array and generate a fused data sequence containing visual features and behavioral features based on the real-time monitoring data streams. The extraction module is used to extract spatiotemporal features from the fused data sequence, construct a set of behavioral feature vectors, and decompose the feature vectors in the set of behavioral feature vectors into short-term dynamic features and long-term trend features to capture the multi-scale behavioral patterns of the monitored object. The quantization module is used to measure the anomalies of the short-term dynamic features and long-term trend features using an adversarially trained Siamese network, and generate an anomaly probability distribution map of the monitoring scene. The handling module is used to perform multi-threshold hierarchical early warning and handling strategy matching based on the anomaly probability distribution map, determine the final handling instruction, and serve as the response output of the monitoring system.

7. The system according to claim 6, characterized in that, The receiving module is specifically used for: Video surveillance data, infrared thermal imaging data, and depth image data are acquired synchronously by a multimodal sensor array. A timestamp alignment algorithm is used to ensure the temporal consistency of multi-source data and generate a synchronous multimodal data stream. The synchronous multimodal data stream is preprocessed, video data is processed using wavelet denoising algorithm, infrared data is processed using non-uniformity correction algorithm, and depth data is processed using point cloud registration algorithm to generate preprocessed multimodal data. For the preprocessed multimodal data, key point features of the monitoring target are extracted from the video data, temperature distribution features are extracted from the infrared data, and spatial motion trajectory features are extracted from the depth data to generate a multimodal feature set. A feature-level fusion algorithm is used to deeply fuse visual and behavioral features in a multimodal feature set. The feature representations of different modalities are weighted and fused through an attention mechanism, and finally a fused data sequence containing visual and behavioral features is generated.

8. The system according to claim 7, characterized in that, The extraction module is specifically used for: The fused data sequence is segmented into time series segments by using a sliding window algorithm to divide the continuous data stream into time series segments of equal length, thereby generating a set of time series data segments. Spatiotemporal features are extracted from each time series data segment, spatial features are extracted using a three-dimensional convolutional neural network, and time-dependent features are extracted using a recurrent neural network to generate a spatiotemporal feature vector. A set of behavioral feature vectors is constructed, and the spatiotemporal feature vectors are arranged in chronological order to form a feature matrix. The dimensionality is reduced by principal component analysis to generate an optimized set of behavioral feature vectors. A multi-scale decomposition algorithm is used to decompose each feature vector in the behavioral feature vector set into high-frequency short-term dynamic features and low-frequency long-term trend features. Wavelet transform is used to achieve time-frequency domain feature separation, and finally a multi-scale behavioral feature representation is generated.

9. A storage medium, characterized in that, The storage medium stores a computer program, wherein the computer program is configured to execute the method of any one of claims 1-5 when it is run.

10. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform the method of any one of claims 1-5.

Images