Intelligent image recognition security early warning system and method based on AI

Abstract

The invention discloses an intelligent image recognition security early warning system and method based on AI, relates to the technical field of artificial intelligence and security, and solves the problem that effective collection of multi-modal data is difficult to realize in a dynamic scene. An image in a complex environment is difficult to process to ensure the quality; real-time detection of abnormal behaviors and prediction of potential criminal actions are difficult to consider; the identification sensitivity is difficult to dynamically optimize according to scene features; and grading response is difficult to carry out on the early warning information. According to the invention, by fusing multispectral imaging and edge computing nodes and by means of an atmospheric scattering correction algorithm, an image in a complex environment is effectively processed; a dual-channel deep learning architecture is constructed, and real-time detection of abnormal behaviors and prediction of potential criminal actions are realized; through a dynamic threshold adjustment mechanism, the recognition sensitivity is optimized based on scene features; and a multi-mode interaction protocol and a self-adaptive execution engine are utilized to complete response of three-level early warning information.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and security technology, specifically an AI-based intelligent image recognition security early warning system and method. Background Technology

[0002] With the acceleration of urbanization and the continuous improvement of public safety systems, security monitoring systems have transformed from traditional passive video recording and storage to proactive intelligent early warning, with AI-based intelligent image recognition technology becoming the core driving force. In complex scenarios such as transportation hubs, commercial complexes, and large event venues, the requirements for the real-time performance and accuracy of dynamic target monitoring, abnormal behavior identification, and potential risk prediction are becoming increasingly stringent.

[0003] Existing AI-based intelligent image recognition security early warning systems and methods suffer from the following problems: difficulty in effectively acquiring multimodal data in dynamic scenes; difficulty in processing images in complex environments to ensure quality; difficulty in balancing real-time detection of abnormal behavior and prediction of potential criminal actions; difficulty in dynamically optimizing recognition sensitivity based on scene characteristics; and difficulty in providing graded responses to early warning information. Summary of the Invention

[0004] This invention aims to at least solve one of the technical problems existing in the prior art; to this end, this invention proposes an AI-based intelligent image recognition security early warning system and method to solve the above-mentioned problem. Specifically, the first aspect of this invention provides an AI-based intelligent image recognition security early warning system, comprising the following modules:

[0005] Image acquisition module: It integrates multispectral imaging equipment and edge computing nodes, and acquires multimodal data in dynamic scenes through real-time registration technology of infrared thermal imaging and visible light images;

[0006] Image processing module: Utilizes atmospheric scattering correction algorithms to process images in complex environments;

[0007] Image analysis module: Constructs a dual-channel deep learning architecture. One channel uses the YOLOv8 model to detect abnormal behavior in real time. Based on the cross-scene transfer learning mechanism with scene attribute labels, it constructs a multi-dimensional sample quality assessment model to screen the training set and iteratively update the sample library. The other channel uses the Transformer architecture to perform spatiotemporal correlation analysis on image sequences to predict potential criminal actions. It has a built-in dynamic threshold adjustment mechanism to automatically optimize the recognition sensitivity based on the multi-dimensional scene feature matrix.

[0008] Early warning response module: Utilizes a multimodal interaction protocol and an adaptive execution engine to respond to three levels of early warning information.

[0009] Furthermore, the image acquisition module includes the following steps:

[0010] Infrared thermal imaging sensors and visible light high-definition cameras deployed on edge computing nodes utilize hardware-level timestamps and PPS pulse synchronization technology to acquire multispectral images with sub-millisecond time synchronization.

[0011] Real-time environmental perception data is collected and multi-dimensional scene feature vectors are constructed. The scene feature vectors are then input into a lightweight decision tree model to generate sensor scheduling strategies and parameter adjustment instructions.

[0012] The signal-to-noise ratio (SNR) is obtained by calculating the ratio of the mean gray level of pixels in the foreground region to the standard deviation of gray level of pixels in the background region. The contrast is calculated by using the statistical value of the gray level difference between adjacent pixels. The registered image is evaluated by weighted SNR and contrast through edge computing nodes. If the evaluation result is lower than the preset threshold, a secondary parameter adjustment mechanism is triggered to feed the quality evaluation data back to the lightweight decision tree model.

[0013] The decision tree parameters are iteratively optimized based on feedback data, and the equipment adjustment instructions are regenerated.

[0014] Furthermore, the image processing module includes the following steps:

[0015] Lightweight FPGA edge computing nodes are used to perform image preprocessing based on depthwise separable convolution in real time at the acquisition end;

[0016] Weighted dark channel feature maps are extracted based on dark channel prior theory. These weighted dark channel feature maps are generated by sliding window minimum filtering and Sobel gradient weighting.

[0017] Based on the atmospheric scattering model, the ambient light intensity is calculated by selecting the top 0.1% of pixels in brightness using a weighted dark channel feature map; a soft matting algorithm is used to obtain the transmittance distribution map; and a non-degradable image is reconstructed based on the optimized transmittance map, using the following formula:

[0018] Where A is the ambient light intensity, and t(x) is the transmittance of the pixel at coordinate x; L in L(x) represents the pixel value of the pixel at coordinate x in the original image, and L(x) represents the pixel value of the pixel at coordinate x in the clear image, i.e., the pixel value of the pixel at coordinate x after processing.

[0019] An image quality quantification evaluation step is set up, which comprehensively evaluates the image by calculating the information entropy, average gradient, and contrast gain value of the processed image. If the evaluation result is lower than the preset threshold, it is automatically fed back to the transmittance optimization step.

[0020] Furthermore, a dual-channel deep learning architecture is constructed, with one channel employing a YOLOv8 model to detect anomalous behavior in real time, including the following steps:

[0021] A multimodal feature input layer is constructed to receive infrared thermal imaging and visible light images. Through a dynamic weight allocation mechanism, the fusion weights of the two modes are adaptively adjusted based on environmental perception data and scene complexity.

[0022] The YOLOv8 model is improved with lightweighting by replacing traditional convolutional layers with depthwise separable convolutions and adding an attention mechanism at the end of the backbone network.

[0023] An abnormal behavior sample library was built based on security scenarios, and the samples were trained using hybrid data augmentation techniques.

[0024] An embedded spatiotemporal context-aware sub-network is used to perform sequence modeling of continuous frame images using a lightweight Transformer architecture. The self-attention mechanism is used to capture the temporal correlation features of the target motion trajectory and action, generating a behavior feature vector containing temporal dimension information.

[0025] A multi-feature fusion decision layer is constructed to deeply fuse target detection features with spatiotemporal features, and the confidence level of abnormal behavior is output through a multilayer perceptron.

[0026] An abnormal behavior detection result caching mechanism is established. Through a dynamic sliding cache queue and a Bayesian evidence accumulation algorithm, similar abnormal behaviors detected in more than 3 consecutive frames are cached, stored, aggregated, and verified to generate final abnormal event labels and location information, which are then input into the early warning response module.

[0027] Furthermore, based on a cross-scene transfer learning mechanism with scene attribute labels, a multi-dimensional sample quality assessment model is constructed to screen the training set and iteratively update the sample library, including the following steps:

[0028] A cross-scenario transfer learning mechanism is introduced, embedding scenario attribute labels into the sample library;

[0029] A sample quality assessment model is constructed, in which sample clarity is calculated by weighting edge gradient strength and information entropy; the integrity of the calculation is evaluated based on the coverage of human keypoint detection; and the annotation accuracy is calibrated by cross-validation and manual review rate.

[0030] Set a quality threshold to filter low-quality samples and generate a sample quality score matrix for training set selection.

[0031] Design a dynamic sample library iteration mechanism, including: real-time access to new abnormal behavior cases in actual applications, which are then included in the incremental sample set after manual annotation and quality assessment;

[0032] Employing a federated learning framework, it aggregates feature distributions from multiple scenarios while protecting data privacy; it automatically identifies missing scenario feature dimensions in the sample library and triggers a targeted sample collection and supplementation process.

[0033] Furthermore, another channel uses the Transformer architecture to perform spatiotemporal correlation analysis on image sequences to predict potential criminal actions, including the following steps:

[0034] A temporal image feature extraction unit is constructed. For a multimodal image sequence of 16 consecutive preprocessed frames, the spatial features and inter-frame motion features of each frame are extracted through a 3D convolutional network to generate a spatiotemporal feature cube.

[0035] A lightweight Transformer encoder is designed, employing a hierarchical attention mechanism: the bottom layer spatial attention focuses on the target region and interaction relationship, the middle layer temporal attention captures the temporal dependence of actions, and the top layer global attention associates with scene context information; through multi-head self-attention computation, the long-range dependencies of target behavior in the spatiotemporal dimensions are modeled.

[0036] An action prediction head is constructed. Based on the encoded spatiotemporal feature vector, a prediction vector is output through a temporal convolutional network. The prediction vector includes the prediction results of action category, occurrence probability, and duration. The prediction vector is then output to the early warning response module.

[0037] Furthermore, a built-in dynamic threshold adjustment mechanism automatically optimizes recognition sensitivity based on a multi-dimensional scene feature matrix, including the following steps:

[0038] Construct a multi-dimensional scene feature matrix that includes pedestrian density, time characteristics, environmental characteristics, and regional characteristics;

[0039] A dynamic weight allocation model combining the analytic hierarchy process (AHP) and gradient descent algorithm;

[0040] A nonlinear mapping relationship between the feature matrix and the recognition threshold is constructed by using a residual network, and dynamic threshold parameters for anomaly detection and action prediction are output.

[0041] Set a threshold adjustment constraint mechanism to adaptively select either exponential function convergence or linear interpolation transition mode based on the magnitude of changes in scene characteristics.

[0042] Establish a closed-loop feedback system with reward and punishment mechanisms, use the PPO algorithm to dynamically optimize the threshold model based on the early warning results, set a penalty coefficient for missed reports in high-risk areas, and set an attenuation coefficient for false reports in densely populated places.

[0043] Furthermore, the early warning response module includes the following steps:

[0044] It receives abnormal behavior tags, potential criminal action prediction vectors, and scene feature matrices in real time, and transforms unstructured early warning elements into a three-dimensional risk tensor containing time urgency, spatial correlation, and threat severity through a tensor decomposition algorithm;

[0045] A dynamic hierarchical response mechanism is designed, which automatically matches three response levels based on the Euclidean distance calculation results of the risk tensor: Level 1 response displays low risk; Level 2 response displays medium risk; and Level 3 response displays high risk.

[0046] A multi-entity collaborative execution network is established, and a distributed message queue architecture is adopted to realize the asynchronous transmission and priority scheduling of early warning commands.

[0047] Compared with the prior art, the beneficial effects of the present invention are:

[0048] This invention integrates multispectral imaging equipment with edge computing nodes, utilizes real-time registration technology of infrared thermal imaging and visible light images to acquire multimodal data, and combines hardware-level timestamps and PPS pulse synchronization technology to achieve sub-millisecond time synchronization. At the same time, it optimizes the dynamic feedback of edge computing nodes and the iterative optimization of decision tree parameters.

[0049] This invention constructs a dual-channel deep learning architecture. One channel uses a lightweight improved YOLOv8 model combined with a spatiotemporal context-aware sub-network to detect abnormal behavior in real time. The other channel uses the hierarchical attention mechanism of the Transformer architecture to perform spatiotemporal correlation analysis on image sequences to predict potential criminal actions. It also incorporates a dynamic threshold adjustment mechanism based on a multi-dimensional scene feature matrix and the PPO algorithm, thus achieving accurate detection of abnormal behavior and forward-looking warning of potential risks.

[0050] This invention improves the model's generalization ability in different security scenarios by introducing a cross-scene transfer learning mechanism with scene attribute labels, combining it with a federated learning framework to aggregate features from multiple scenes, and designing a dynamic sample library iteration mechanism. Furthermore, it optimizes image quality by combining atmospheric scattering correction algorithm with FPGA edge computing, and improves the system's practicality and early warning efficiency in complex environments through a three-level early warning response mechanism and a multi-subject collaborative execution network. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 This is a system module diagram of the present invention.

[0053] Figure 2 This is a flowchart of the method of the present invention. Detailed Implementation

[0054] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0055] Please see Figure 1 As shown, a first aspect embodiment of the present invention provides an AI-based intelligent image recognition security early warning system, comprising the following modules:

[0056] Image acquisition module: It integrates multispectral imaging equipment and edge computing nodes, and acquires multimodal data in dynamic scenes through real-time registration technology of infrared thermal imaging and visible light images;

[0057] Image processing module: Utilizes atmospheric scattering correction algorithms to process images in complex environments;

[0058] Image analysis module: Constructs a dual-channel deep learning architecture. One channel uses the YOLOv8 model to detect abnormal behavior in real time. Based on the cross-scene transfer learning mechanism with scene attribute labels, it constructs a multi-dimensional sample quality assessment model to screen the training set and iteratively update the sample library. The other channel uses the Transformer architecture to perform spatiotemporal correlation analysis on image sequences to predict potential criminal actions. It has a built-in dynamic threshold adjustment mechanism to automatically optimize the recognition sensitivity based on the multi-dimensional scene feature matrix.

[0059] Early warning response module: Utilizes a multimodal interaction protocol and an adaptive execution engine to respond to three levels of early warning information.

[0060] Specifically, multispectral imaging equipment, including infrared thermal imaging sensors and visible light high-definition cameras, is deployed and connected to edge computing nodes. Sub-millisecond time synchronization of infrared and visible light images is achieved through hardware-level timestamps and PPS pulse synchronization technology, combined with real-time registration technology to fuse multimodal data. Edge computing nodes evaluate the signal-to-noise ratio and contrast of the registered image in real time; if the results are substandard, feedback is sent to a lightweight decision tree model to dynamically adjust sensor parameters and ensure data quality. Based on an atmospheric scattering correction algorithm, a weighted dark channel feature map is first extracted using dark channel prior theory, high-brightness pixels are selected to calculate ambient light intensity, and a soft matting algorithm is used to generate a transmittance distribution map. Finally, a non-degradable image is reconstructed according to a formula. An image quality evaluation stage is set up, using indicators such as information entropy and average gradient to verify the processing effect; if the results are substandard, the transmittance map is backtracked and optimized. A dual-channel deep learning architecture is built. One channel employs an improved YOLOv8 model, fusing infrared and visible light image features and combining a spatiotemporal context-aware sub-network to capture target motion trajectories. Abnormal behavior is verified through dynamic sliding buffer queues and Bayesian algorithms. The other channel utilizes a Transformer architecture to perform spatiotemporal correlation analysis on continuous image sequences, modeling long-range dependencies through a hierarchical attention mechanism to predict potential criminal actions. Simultaneously, based on multi-dimensional feature matrices such as pedestrian density and environmental factors, a dynamic threshold adjustment mechanism automatically optimizes recognition sensitivity. Receiving abnormal behavior labels, criminal action prediction results, and scene features, a three-dimensional risk tensor containing information such as time urgency and threat severity is generated using a tensor decomposition algorithm. A three-level response is matched based on the risk level, and a multimodal interaction protocol and distributed message queue are used to achieve hierarchical transmission and collaborative execution of early warning commands.

[0061] In this embodiment, the image acquisition module includes the following steps:

[0062] Infrared thermal imaging sensors and visible light high-definition cameras deployed on edge computing nodes utilize hardware-level timestamps and PPS pulse synchronization technology to acquire multispectral images with sub-millisecond time synchronization.

[0063] Real-time environmental perception data is collected and multi-dimensional scene feature vectors are constructed. The scene feature vectors are then input into a lightweight decision tree model to generate sensor scheduling strategies and parameter adjustment instructions.

[0064] The signal-to-noise ratio (SNR) is obtained by calculating the ratio of the mean gray level of pixels in the foreground region to the standard deviation of gray level of pixels in the background region. The contrast is calculated by using the statistical value of the gray level difference between adjacent pixels. The registered image is evaluated by weighted SNR and contrast through edge computing nodes. If the evaluation result is lower than the preset threshold, a secondary parameter adjustment mechanism is triggered to feed the quality evaluation data back to the lightweight decision tree model.

[0065] The decision tree parameters are iteratively optimized based on feedback data, and the equipment adjustment instructions are regenerated.

[0066] Specifically, infrared thermal imaging sensors and visible light high-definition cameras are deployed on edge computing nodes. Hardware-level timestamps and PPS pulse synchronization technology enable sub-millisecond time synchronization for multispectral image acquisition. Hardware-level timestamps precisely record the time each sensor acquires an image, while PPS pulses generate a precise pulse signal per second, like a precise clock ticking. This allows the infrared thermal imaging sensor and the visible light high-definition camera to complete image acquisition within a very short time difference, ensuring a high degree of temporal matching of multimodal data during subsequent fusion analysis. For example, in security monitoring scenarios, it can accurately capture the infrared thermal radiation characteristics and visible light appearance characteristics of a human body at the same moment, providing an accurate data foundation for behavior analysis and anomaly detection. Real-time environmental perception data is collected, covering information such as light intensity, temperature, humidity, and personnel density. This data is integrated to construct a multi-dimensional scene feature vector. The constructed scene feature vector is input into a lightweight decision tree model. The model analyzes the feature vector based on its internal decision logic and trained parameters, generating sensor scheduling strategies and parameter adjustment instructions. In scenes with frequent changes in lighting, the model may instruct the camera to dynamically adjust exposure parameters to ensure clear images. In densely populated areas, increasing the sensor's acquisition frequency ensures timely capture of human activity information, improving the effectiveness and timeliness of data collection. Signal-to-noise ratio (SNR) calculation: After multispectral image acquisition and registration, edge detection algorithms such as the Canny edge detection algorithm are used to segment the foreground and background regions in the image. For the foreground region, the grayscale values ​​of all pixels are statistically analyzed, and their arithmetic mean is calculated to obtain the mean grayscale value of the foreground region pixels. For the background region, the grayscale values ​​of all pixels are similarly statistically analyzed, and their standard deviation is calculated. Comparing the mean grayscale value of the foreground region pixels with the standard deviation of the background region pixels yields the SNR value, which reflects the contrast between signal strength and noise fluctuation in the image. Contrast calculation: The registered image is scanned pixel by pixel, and the absolute value of the difference in grayscale value between each pixel and its neighboring pixels is calculated. The absolute values ​​of all these neighboring pixel grayscale differences are statistically averaged to obtain the statistical value of the neighboring pixel grayscale difference, which is used to quantify the image contrast. Edge computing nodes are dynamically generated by a lightweight decision tree model based on real-time scene feature vectors according to preset rules. Weights are assigned to signal-to-noise ratio (SNR) and contrast, initially set to 0.6 and 0.4 respectively. The SNR and contrast calculation results are fused through weighted summation to obtain a comprehensive image quality score. If the comprehensive image quality score obtained through weighted evaluation is lower than a preset threshold of 0.7, it indicates that the quality of the currently acquired image does not meet the ideal standard. At this point, the edge computing node triggers a secondary parameter adjustment mechanism, feeding back the specific values ​​of SNR and contrast, as well as the comprehensive score, to the lightweight decision tree model.Based on this feedback data, the decision tree model iteratively optimizes its parameters, reanalyzes the characteristics of the current scene, and generates device adjustment instructions that better suit the needs of the current scene. These instructions may include adjusting the camera's focal length and aperture, changing the sensor's sensitivity, etc. Through a continuous feedback optimization process, the quality and effect of image acquisition are constantly improved. The threshold setting usually needs to be determined comprehensively based on factors such as the needs of the actual application scenario, device performance, and historical data statistics; it is a parameter that can be flexibly adjusted according to the actual situation.

[0067] In this embodiment, the image processing module includes the following steps:

[0068] Lightweight FPGA edge computing nodes are used to perform image preprocessing based on depthwise separable convolution in real time at the acquisition end;

[0069] Weighted dark channel feature maps are extracted based on dark channel prior theory. These weighted dark channel feature maps are generated by sliding window minimum filtering and Sobel gradient weighting.

[0070] Based on the atmospheric scattering model, the ambient light intensity is calculated by selecting the top 0.1% of pixels in brightness using a weighted dark channel feature map; a soft matting algorithm is used to obtain the transmittance distribution map; and a non-degradable image is reconstructed based on the optimized transmittance map, using the following formula:

[0071] Where A is the ambient light intensity, and t(x) is the transmittance of the pixel at coordinate x; L in L(x) represents the pixel value of the pixel at coordinate x in the original image, and L(x) represents the pixel value of the pixel at coordinate x in the clear image, i.e., the pixel value of the pixel at coordinate x after processing.

[0072] An image quality quantification evaluation step is set up, which comprehensively evaluates the image by calculating the information entropy, average gradient, and contrast gain value of the processed image. If the evaluation result is lower than the preset threshold, it is automatically fed back to the transmittance optimization step.

[0073] Specifically, lightweight FPGA edge computing nodes are deployed at the acquisition end, utilizing their parallel computing capabilities to perform image preprocessing operations in real time. The core employs depthwise separable convolution instead of traditional convolutional layers: standard convolution is decomposed into depthwise convolution, independent convolution for each channel, extracting spatial features, and pointwise convolution. This significantly reduces computational load and parameter size by approximately 80% while preserving key image features such as edges and textures. This step is completed directly after data acquisition, avoiding delays during the transmission of the original image and meeting the real-time requirements of security scenarios. Based on dark channel prior theory, a fixed-size sliding window is used to traverse each pixel of the preprocessed image, calculating the minimum value of pixels within the window to initially generate a dark channel feature map, reflecting areas where light is scattered and obscured in the image. Sobel edge detection is then performed on the original image, calculating the gradient value of each pixel. This gradient value is used as a weight and multiplied by the minimum value filtering result of the sliding window to generate a weighted dark channel feature map. Gradient weighting enhances the weight of edge regions in the dark channel features, addressing the problem of insufficient edge detail preservation in traditional dark channel algorithms, making it particularly suitable for accurate extraction of target contours in security scenarios. Based on an atmospheric scattering model, the top 0.1% of pixels with the highest brightness are selected from the weighted dark channel feature map. These pixels typically correspond to highlight areas unaffected by scattering, and their average gray value is taken as the ambient light intensity A to ensure the accuracy of ambient light assessment. A soft matting algorithm is used to optimize transmittance calculation, as transmittance reflects the proportion of light penetrating the atmosphere. This algorithm solves the problem of abrupt transmittance changes caused by traditional hard thresholding by introducing local smoothing constraints, generating a continuous and smooth transmittance distribution map that better reflects the real atmospheric scattering patterns. Based on the optimized transmittance map, a clear image is reconstructed using a formula. This formula eliminates image degradation caused by atmospheric scattering such as haze and backlighting, restoring target details such as faces and object outlines. An image quality assessment step is set up, calculating three indicators of the processed image: information entropy to measure the amount of information contained in the image; average gradient to reflect the image edge sharpness; and contrast gain value to indicate the improvement in contrast between the images before and after processing. The three indicators are weighted and summed, with weights of 0.4, 0.3, and 0.3 respectively. These weights can be dynamically adjusted based on the scenario. If the overall score is lower than the preset threshold of 0.8, the evaluation result is automatically fed back to the transmittance optimization stage. The smoothing parameters of the soft matting algorithm are readjusted, and the transmittance distribution map is iteratively optimized until the image quality meets the standards. The threshold value is typically determined by considering factors such as the needs of the actual application scenario, device performance, and historical data statistics; it is a parameter that can be flexibly adjusted according to the actual situation.

[0074] In this embodiment, a dual-channel deep learning architecture is constructed, one channel of which uses the YOLOv8 model to detect abnormal behavior in real time, including the following steps:

[0075] A multimodal feature input layer is constructed to receive infrared thermal imaging and visible light images. Through a dynamic weight allocation mechanism, the fusion weights of the two modes are adaptively adjusted based on environmental perception data and scene complexity.

[0076] The YOLOv8 model is improved with lightweighting by replacing traditional convolutional layers with depthwise separable convolutions and adding an attention mechanism at the end of the backbone network.

[0077] An abnormal behavior sample library was built based on security scenarios, and the samples were trained using hybrid data augmentation techniques.

[0078] An embedded spatiotemporal context-aware sub-network is used to perform sequence modeling of continuous frame images using a lightweight Transformer architecture. The self-attention mechanism is used to capture the temporal correlation features of the target motion trajectory and action, generating a behavior feature vector containing temporal dimension information.

[0079] A multi-feature fusion decision layer is constructed to deeply fuse target detection features with spatiotemporal features, and the confidence level of abnormal behavior is output through a multilayer perceptron.

[0080] An abnormal behavior detection result caching mechanism is established. Through a dynamic sliding cache queue and a Bayesian evidence accumulation algorithm, similar abnormal behaviors detected in more than 3 consecutive frames are cached, stored, aggregated, and verified to generate final abnormal event labels and location information, which are then input into the early warning response module.

[0081] Specifically, the multimodal feature input layer receives infrared thermal imaging frame sequences IR_t and visible light frame sequences RGB_t in parallel, where t is a timestamp; an environmental perception submodule is set up to collect ambient illuminance L_t, atmospheric visibility V_t, and target occlusion rate O_t in real time; a dynamic weight allocation network is constructed, taking (L_t, V_t, O_t) as input, and outputting fusion weights α_t and β_t after two layers of convolution and sigmoid activation, satisfying a sum of 1; IR_t and RGB_t are weighted and fused according to the fusion weights to generate a fused feature map F_t. A lightweight YOLOv8 detection subnetwork replaces all 3×3 convolutions in the C3 module of the YOLOv8 backbone network with depthwise separable convolutions; a lightweight coordinate attention module L-CA is inserted at the end of the backbone network, which includes: parallel branches of average pooling and max pooling along the channel dimension; 7×7 depthwise convolutions to generate attention weights; and a sigmoid function to output a spatial attention map A_t. A Ghost-PAN structure is adopted in the Neck layer; the detection head outputs the abnormal behavior category c_i and its bounding box b_i. The abnormal behavior sample library construction and training strategy is based on real-world security scene video, using a semi-automatic annotation tool to generate labels containing abnormal behaviors. Hybrid data augmentation strategies include: spatiotemporal synchronous random cropping: randomly selecting three consecutive frames within one second and simultaneously cropping a 512×512 region; cross-modal CutMix: stitching together complementary regions of IR_t and RGB_t at the same time; StyleGAN2-IR style transfer: performing infrared stylization on visible light frames to expand low-light scene samples. The Mosaic-4D method is used to stitch four sets of spatiotemporally synchronous samples into a 2×2 grid to improve the robustness of small target detection; class-balanced Focal Loss (γ=2, α=0.75) is adopted to address the long-tail distribution problem of the samples. A spatiotemporal context-aware subnetwork is constructed using a lightweight Transformer structure, ST-Former, containing two encoder layers, each with four self-attention heads. The input is a sequence of fused feature maps {F_t-7,…,F_t} spanning eight consecutive frames, with a frame interval Δt = 0.2 s. 3D separable position encoding is employed, simultaneously encoding spatial position (x,y) and temporal position t. The output is a spatiotemporal behavior feature vector, containing the target's motion trajectory and the temporal correlation features of the action. A multi-feature fusion decision layer concatenates the YOLOv8 detection features with the ST-Former output V_t. An abnormal behavior confidence score is output through a two-layer multilayer perceptron (MLP-Fusion). An abnormal event candidate is triggered when the confidence score is ≥0.7.An abnormal behavior caching and aggregation verification mechanism is implemented. A sliding cache queue Q is established, with its length N dynamically adjusted by the intensity of scene motion. Abnormal candidates that meet the IoU>0.5 and have the same category in N consecutive frames are aggregated. A Bayesian evidence accumulation algorithm is used to calculate the cumulative confidence. When the cumulative confidence is ≥0.85, the final abnormal event label and location information are generated and input into the early warning response module.

[0082] In this embodiment, a multi-dimensional sample quality assessment model is constructed based on a cross-scene transfer learning mechanism with scene attribute labels to screen the training set and iteratively update the sample library, including the following steps:

[0083] A cross-scenario transfer learning mechanism is introduced, embedding scenario attribute labels into the sample library;

[0084] A sample quality assessment model is constructed, in which sample clarity is calculated by weighting edge gradient strength and information entropy; the integrity of the calculation is evaluated based on the coverage of human keypoint detection; and the annotation accuracy is calibrated by cross-validation and manual review rate.

[0085] Set a quality threshold to filter low-quality samples and generate a sample quality score matrix for training set selection.

[0086] Design a dynamic sample library iteration mechanism, including: real-time access to new abnormal behavior cases in actual applications, which are then included in the incremental sample set after manual annotation and quality assessment;

[0087] Employing a federated learning framework, it aggregates feature distributions from multiple scenarios while protecting data privacy; it automatically identifies missing scenario feature dimensions in the sample library and triggers a targeted sample collection and supplementation process.

[0088] Specifically, considering the characteristics of security scenarios, a labeling system is constructed that includes basic environmental dimensions, such as light intensity (strong light, weak light, nighttime), weather type (sunny, rainy, foggy), area type (indoor, outdoor, densely populated areas, open areas), and dynamic feature dimensions, such as target density (high, medium, low); and motion intensity (intense and gentle). Each sample is assigned 10 to 15 scene attribute labels. Differentiated labels are added to samples from different scenarios, such as airports, campuses, and industrial areas, in the sample library, enabling knowledge transfer between scenarios through label association. Sample sharpness is evaluated by calculating edge gradient strength: the Sobel operator is used to extract the edge gradient of the sample image, and the percentage of pixels with gradient values ​​greater than a preset threshold of 30% is counted, reflecting the sharpness of the image edges; information entropy is calculated using the formula... Where, p iImage information content is measured by the pixel proportion of grayscale value i. The edge gradient strength and its corresponding weight (0.6) are weighted and summed with the information entropy and its corresponding weight (0.4) to obtain a sharpness score, filtering out blurry samples with scores less than 0.5. Behavioral integrity assessment uses a human keypoint detection model to identify 17 key nodes of the target in the sample, such as the head and limb joints. Coverage is calculated as the ratio of the number of effectively detected keypoints to the total number of keypoints. For example, the "fighting" behavior needs to cover the head, arms, and torso keypoints, with a coverage rate ≥ 80%. Integrity score is directly used as the score, filtering out samples with scores less than 0.8. Labeling accuracy assessment involves inputting samples into a trained base model for prediction, calculating the matching degree between the model's output label and the manually labeled label. Manual verification rate calibration involves manually verifying samples with a matching degree less than 0.9 in cross-validation, calculating the actual labeling error rate, and subtracting the error rate from 1 to obtain the labeling accuracy. A calibration score is then applied. The final accuracy score must be greater than 0.9 to ensure unambiguous labeling. The individual scores for clarity, behavioral completeness, and annotation accuracy are weighted and summed at weights of 0.3, 0.4, and 0.3 respectively to generate a comprehensive sample quality score. A comprehensive score of 0.8 is defined as a high-quality sample, directly included in the training set; scores greater than 0.6 and less than 0.8 are considered medium-quality samples, which are included after manual secondary optimization, such as supplementary annotation and enhanced clarity; scores less than 0.6 are considered low-quality samples and are directly filtered. The thresholds are adaptively fine-tuned for different scene labels. Real-time monitoring of false alarms or missed alarms in actual operation of the system extracts novel abnormal behaviors not covered by the existing sample library, such as drone intrusions and abnormal loitering. After manual annotation of scene attribute labels and passing quality assessment, these are included in the incremental sample set, and the main sample library is updated quarterly. A federated learning network is constructed by deploying nodes across multiple scenarios, such as different campuses and factories. Each node trains its model locally, uploading only model parameters rather than raw data to protect privacy. The central server aggregates the parameters from each node, generating a global model containing multi-scene feature distributions, which is then fed back to the sample libraries of each node to improve cross-scene adaptability. By using feature distribution analysis tools, the coverage of scene attribute labels in the sample library is checked regularly; targeted collection commands are automatically triggered to control the sensors of the corresponding scene to increase the data collection frequency and supplement missing samples until the proportion of the scene label samples is greater than 0.1, so as to avoid the performance degradation of the model in niche scenes.

[0089] In this embodiment, another channel performs spatiotemporal correlation analysis on the image sequence using the Transformer architecture to predict potential criminal actions, including the following steps:

[0090] A temporal image feature extraction unit is constructed. For a multimodal image sequence of 16 consecutive preprocessed frames, the spatial features and inter-frame motion features of each frame are extracted through a 3D convolutional network to generate a spatiotemporal feature cube.

[0091] A lightweight Transformer encoder is designed, employing a hierarchical attention mechanism: the bottom layer spatial attention focuses on the target region and interaction relationship, the middle layer temporal attention captures the temporal dependence of actions, and the top layer global attention associates with scene context information; through multi-head self-attention computation, the long-range dependencies of target behavior in the spatiotemporal dimensions are modeled.

[0092] An action prediction head is constructed. Based on the encoded spatiotemporal feature vector, a prediction vector is output through a temporal convolutional network. The prediction vector includes the prediction results of action category, occurrence probability, and duration. The prediction vector is then output to the early warning response module.

[0093] Specifically, for a multimodal image sequence of 16 consecutive preprocessed frames, infrared thermal imaging and visible light images are fused. The frame interval can be dynamically set according to the scene, such as 0.1 seconds / frame, covering 1.6 seconds of behavioral time sequence. The initial extraction of spatiotemporal features is completed through a 3D convolutional network: a lightweight 3D convolutional layer, such as a 3×3×3 convolutional kernel, is used. The first two dimensions (3×3) are used to extract spatial features of a single frame image, such as target contours and pose details, while the third dimension (3×3) is used to capture motion features between consecutive frames, such as limb displacement and velocity changes. Through multi-layer 3D convolution and pooling operations, the feature dimensions are gradually compressed, finally generating a spatiotemporal feature cube with dimensions [D×H×W×T], where D is the number of channels, H and W are the feature map sizes, and T = 16 is the number of frames in the time dimension. This cube contains both spatial details and temporal dynamic information. The encoder takes a spatiotemporal feature cube as input and models the spatiotemporal dependencies of target behavior through a hierarchical attention mechanism. The specific structure is as follows: The bottom-level spatial attention focuses on the target region and its interaction relationships within a single frame: For each frame's feature map in the spatiotemporal feature cube, spatial attention weights are calculated, such as region weight allocation based on the target bounding box, to enhance the feature responses of key regions such as the human body and suspicious objects, while suppressing background noise. The middle-level temporal attention captures the temporal dependencies of actions between consecutive frames: For a 16-frame feature sequence, self-attention calculation in the temporal dimension quantifies the correlation strength of features at different times, such as the temporal weight of the "raising hand" action and the subsequent "hitting" action. A multi-head self-attention mechanism, such as 4-head attention, is adopted to learn dependencies at different time scales in parallel. The top-level global attention associates scene context information: Combining scene attribute labels and global features, the attention distribution is adjusted. For example, in a "late-night alley + single person tailing" scenario, the global correlation weight between the tailer and the tailed is increased, strengthening the sensitivity to potential threats. By reducing the number of encoder layers, lowering the feature dimension, and replacing some fully connected layers with depthwise separable convolutions, the computational load is reduced while maintaining the effectiveness of the attention mechanism, thus adapting to the computing power limitations of edge computing nodes. Based on the encoded spatiotemporal feature vectors output by the encoder, an action prediction head is constructed through a temporal convolutional network to achieve multi-dimensional prediction of potential criminal actions.

[0094] A three-layer 1D temporal convolution is employed with a kernel size of 3. Dilated convolution expands the receptive field to capture action trends in long-term temporal features. Batch normalization and ReLU activation functions are applied after each convolution layer to enhance the non-linear expressive power of features.

[0095] The system outputs the category of potential criminal actions using a softmax classifier, such as robbery, theft, assault, and other preset high-risk actions. A sigmoid function outputs the confidence level of the action's occurrence, with a threshold of 0.6; actions exceeding this threshold are considered high-risk. Based on the rate of change of temporal features, the system predicts the duration of the action from the current moment to its completion, in seconds. The predicted vector, containing the action category, probability of occurrence, and duration, is transmitted to the early warning response module in real time.

[0096] In this embodiment, a built-in dynamic threshold adjustment mechanism is used to automatically optimize the recognition sensitivity based on a multi-dimensional scene feature matrix, including the following steps:

[0097] Construct a multi-dimensional scene feature matrix that includes pedestrian density, time characteristics, environmental characteristics, and regional characteristics;

[0098] A dynamic weight allocation model combining the analytic hierarchy process (AHP) and gradient descent algorithm;

[0099] A nonlinear mapping relationship between the feature matrix and the recognition threshold is constructed by using a residual network, and dynamic threshold parameters for anomaly detection and action prediction are output.

[0100] Set a threshold adjustment constraint mechanism to adaptively select either exponential function convergence or linear interpolation transition mode based on the magnitude of changes in scene characteristics.

[0101] Establish a closed-loop feedback system with reward and punishment mechanisms, use the PPO algorithm to dynamically optimize the threshold model based on the early warning results, set a penalty coefficient for missed reports in high-risk areas, and set an attenuation coefficient for false reports in densely populated places.

[0102] Specifically, the pedestrian density feature uses a target detection algorithm to count the number of people per unit area and quantifies it into a normalized value; the time feature includes time period labels and specific time codes, such as dividing 24 hours into 12 time windows and representing them with one-hot vectors; the environmental feature covers light intensity and quantifies it into a normalized value, weather type, such as sunny, rainy, and foggy, with one-hot encoding, and visibility is normalized; the regional feature includes: based on preset security level labels, including high, medium, and low levels. The final feature matrix has a dimension of [1×N], where N is the total number of features.

[0103] Combining the advantages of the Analytic Hierarchy Process (AHP) and gradient descent, dynamic weights are assigned to each dimension of the feature matrix: AHP initial weight setting: Based on security expert experience, the importance of the four types of features is compared pairwise to generate a judgment matrix; initial weights for each feature are obtained through eigenvalue decomposition, such as 0.3 for the regional feature, 0.2 for pedestrian density, 0.2 for time features, and 0.3 for environmental features in high-risk areas. Using the improvement in early warning accuracy after threshold adjustment as the objective function, the initial weights are used as the starting point, and the weight values ​​are iteratively optimized through gradient descent.

[0104] The complex relationship between the feature matrix and dynamic threshold is modeled using a residual network, outputting threshold parameters adapted to the current scene. A three-layer residual block is used, with input being a weighted multi-dimensional feature matrix × dynamic weight vector, and outputting two threshold parameters: anomaly detection threshold and action prediction threshold. The network mitigates the vanishing gradient problem through residual connections and learns the non-linear relationship between scene features and thresholds. Based on the magnitude of scene feature changes, an adaptive transition mode is selected to avoid abrupt threshold changes: the Euclidean distance between the current feature matrix and the matrix at the previous time step is used to quantify the degree of scene change; Δd ​​< 0.1 indicates a small change, and Δd ≥ 0.1 indicates a large change. For small changes, linear interpolation is used to smoothly transition the threshold from the old value to the new value within 10 frames, avoiding frequent fluctuations; for large changes, an exponential function convergence θ(t) = θ_new - (θ_new - θ_old) × e -0.5*t Here, θ(t) represents the dynamic threshold parameter at time t, which can be the anomaly detection threshold or the action prediction threshold, i.e., the threshold value at a certain moment during the transition process, which changes dynamically with time t. θ_new represents the target threshold calculated by the residual network, i.e., the threshold parameter that the system finally needs to converge to after a significant change in scene characteristics, such as a lower threshold adjusted after a missed detection in a high-risk area. θ_old represents the initial threshold, i.e., the threshold parameter currently used by the system before the change in scene characteristics, such as the original threshold before adjustment.

[0105] 0.5 represents the convergence rate coefficient, used to control how quickly the threshold converges from θ_old to θ_new. A larger value indicates faster convergence. t represents the time step, measured in frames, with each frame corresponding to one time step—the cumulative time calculated from the triggering of the threshold adjustment mechanism—used to quantify the duration of the transition process. Based on the PPO reinforcement learning algorithm, the threshold model state is dynamically optimized using the warning results, with the action being an adjustment of the threshold parameters by ±0.05. Penalty: For missed warnings in high-risk areas, a negative reward is given, forcing the model to lower the threshold for the corresponding scene; Attenuation: For false alarms in densely populated areas, a low negative reward is given, moderately increasing the threshold to reduce interference; Reward: For accurate warnings, a positive reward is given, reinforcing the rationality of the current threshold. Every hour, based on the cumulative reward and penalty values, the parameters of the residual network are updated using the PPO algorithm, allowing the threshold model to gradually adapt to the warning needs of the actual scenario.

[0106] In this embodiment, the early warning response module includes the following steps:

[0107] It receives abnormal behavior tags, potential criminal action prediction vectors, and scene feature matrices in real time, and transforms unstructured early warning elements into a three-dimensional risk tensor containing time urgency, spatial correlation, and threat severity through a tensor decomposition algorithm;

[0108] A dynamic hierarchical response mechanism is designed, which automatically matches three response levels based on the Euclidean distance calculation results of the risk tensor: Level 1 response displays low risk; Level 2 response displays medium risk; and Level 3 response displays high risk.

[0109] A multi-entity collaborative execution network is established, and a distributed message queue architecture is adopted to realize the asynchronous transmission and priority scheduling of early warning commands.

[0110] Specifically, the system receives three core inputs in real time: abnormal behavior labels, potential criminal action prediction vectors, and a multi-dimensional scene feature matrix. These unstructured elements are then transformed into a three-dimensional risk tensor using a tensor decomposition algorithm: **Time Urgency Dimension:** Combining the predicted duration of potential criminal actions with the number of frames of abnormal behavior, the urgency of the risk is quantified through weighted grading, with weights of 0.4 and 0.5 respectively. **Spatial Relevance Dimension:** Based on the location of abnormal behavior or potential actions, the spatial distance between the target location and key areas such as security posts and evacuation routes is calculated using bounding box coordinates and regional attributes in the scene features. **Threat Severity Dimension:** Integrating the abnormal behavior category, the probability of potential criminal actions, and the population density in the scene, a weighted sum is generated with weights of 0.4, 0.3, and 0.3 respectively. The final three-dimensional risk tensor is defined as [Time Urgency, Spatial Relevance, Threat Severity]. The Euclidean distance between the three-dimensional risk tensor and preset three-level baseline tensors is calculated by setting the historical average tensors of first, second, and third-level responses as the corresponding baseline tensors. The distance between the real-time risk tensor and each reference tensor is calculated using the Euclidean distance formula. The level corresponding to the reference tensor with the smallest distance is the current response level. Level 1 response indicates low risk: the system automatically records the event and displays a yellow warning icon on the monitoring terminal, requiring no immediate manual intervention. Level 2 response indicates medium risk: it triggers an audible and visual alarm, pushes a warning message to the handheld terminals of regional security personnel, and includes the event location and real-time video. Level 3 response indicates high risk: it activates the highest-level linkage mechanism, simultaneously triggering on-site audible and visual alarms, locking the monitoring screen of the incident area, automatically dialing the emergency command center, and pushing detailed risk information including predicted actions, location, and threat level.

[0111] A distributed message queue is used to build a collaborative network for efficient transmission and scheduling of early warning commands. A three-tiered priority queue is set up, allocating resources according to response level. Level 3 response commands enter the high-priority queue, level 2 commands enter the medium-priority queue, and level 1 commands enter the low-priority queue, avoiding resource contention that could lead to delays in high-risk commands. Multi-entity collaborative logic: Terminal nodes, including the monitoring center server, security personnel's handheld terminals, on-site alarm devices (audio-visual alarms, LED screens), receive commands by subscribing to their respective queues. Commands are encapsulated in JSON format, containing event ID, level, location, and timestamp, and sent asynchronously via the message queue to ensure no critical information is lost during network fluctuations. The receiving party processes the command and returns a confirmation receipt. When the queue is congested, processing of low-priority commands is automatically suspended, prioritizing the transmission and execution of high-priority commands.

[0112] Please see Figure 2 As shown, this invention is an AI-based intelligent image recognition security early warning method, comprising the following steps:

[0113] S1: Integrates multispectral imaging equipment with edge computing nodes, and acquires multimodal data in dynamic scenes through real-time registration technology of infrared thermal imaging and visible light images;

[0114] S2: Introduces an atmospheric scattering correction algorithm to process images in complex environments;

[0115] S3: Construct a dual-channel deep learning architecture. One channel uses the YOLOv8 model to detect abnormal behavior in real time. Based on the cross-scene transfer learning mechanism with scene attribute labels, a multi-dimensional sample quality assessment model is constructed to screen the training set and iteratively update the sample library. The other channel uses the Transformer architecture to perform spatiotemporal correlation analysis on image sequences to predict potential criminal actions. A built-in dynamic threshold adjustment mechanism automatically optimizes the recognition sensitivity based on the multi-dimensional scene feature matrix.

[0116] S4: Utilizes a multimodal interaction protocol and an adaptive execution engine to respond to three levels of early warning information.

[0117] The above embodiments are only used to illustrate the technical methods of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical methods of the present invention without departing from the spirit and scope of the technical methods of the present invention.

Claims

1. An AI-based intelligent image recognition security early warning system, characterized in that, Includes the following modules: Image acquisition module: It integrates multispectral imaging equipment and edge computing nodes, and acquires multimodal data in dynamic scenes through real-time registration technology of infrared thermal imaging and visible light images; Image processing module: Utilizes atmospheric scattering correction algorithms to process images in complex environments; Image analysis module: Constructs a dual-channel deep learning architecture. One channel uses the YOLOv8 model to detect abnormal behavior in real time. Based on the cross-scene transfer learning mechanism with scene attribute labels, a multi-dimensional sample quality assessment model is constructed to screen the training set and iteratively update the sample library. The other channel uses the Transformer architecture to perform spatiotemporal correlation analysis on image sequences to predict potential criminal actions. It has a built-in dynamic threshold adjustment mechanism that automatically optimizes recognition sensitivity based on a multi-dimensional scene feature matrix. Early warning response module: Utilizes a multimodal interaction protocol and an adaptive execution engine to respond to three levels of early warning information.

2. The AI-based intelligent image recognition security early warning system according to claim 1, characterized in that, The image acquisition module includes the following steps: Infrared thermal imaging sensors and visible light high-definition cameras deployed on edge computing nodes utilize hardware-level timestamps and PPS pulse synchronization technology to acquire multispectral images with sub-millisecond time synchronization. Real-time environmental perception data is collected and multi-dimensional scene feature vectors are constructed. The scene feature vectors are then input into a lightweight decision tree model to generate sensor scheduling strategies and parameter adjustment instructions. The signal-to-noise ratio (SNR) is obtained by calculating the ratio of the mean gray level of pixels in the foreground region to the standard deviation of gray level of pixels in the background region. The contrast is calculated by using the statistical value of the gray level difference between adjacent pixels. The registered image is evaluated by weighted SNR and contrast through edge computing nodes. If the evaluation result is lower than the preset threshold, a secondary parameter adjustment mechanism is triggered to feed the quality evaluation data back to the lightweight decision tree model. The decision tree parameters are iteratively optimized based on feedback data, and the equipment adjustment instructions are regenerated.

3. The AI-based intelligent image recognition security early warning system according to claim 1, characterized in that, The image processing module includes the following steps: Lightweight FPGA edge computing nodes are used to perform image preprocessing based on depthwise separable convolution in real time at the acquisition end; Weighted dark channel feature maps are extracted based on dark channel prior theory. These weighted dark channel feature maps are generated by sliding window minimum filtering and Sobel gradient weighting. Based on the atmospheric scattering model, the ambient light intensity is calculated by selecting the top 0.1% of pixels in brightness using a weighted dark channel feature map; a soft matting algorithm is used to obtain the transmittance distribution map; and a non-degradable image is reconstructed based on the optimized transmittance map, using the following formula: Where A is the ambient light intensity, and t(x) is the transmittance of the pixel at coordinate x; L in L(x) represents the pixel value of the pixel at coordinate x in the original image, and L(x) represents the pixel value of the pixel at coordinate x in the clear image, i.e., the pixel value of the pixel at coordinate x after processing. An image quality quantification evaluation step is set up, which comprehensively evaluates the image by calculating the information entropy, average gradient, and contrast gain value of the processed image. If the evaluation result is lower than the preset threshold, it is automatically fed back to the transmittance optimization step.

4. The AI-based intelligent image recognition security early warning system according to claim 1, characterized in that, A dual-channel deep learning architecture is constructed, with one channel employing the YOLOv8 model to detect anomalous behavior in real time. This includes the following steps: A multimodal feature input layer is constructed to receive infrared thermal imaging and visible light images. Through a dynamic weight allocation mechanism, the fusion weights of the two modes are adaptively adjusted based on environmental perception data and scene complexity. The YOLOv8 model is improved with lightweighting by replacing traditional convolutional layers with depthwise separable convolutions and adding an attention mechanism at the end of the backbone network. An abnormal behavior sample library was built based on security scenarios, and the samples were trained using hybrid data augmentation techniques. An embedded spatiotemporal context-aware sub-network is used to perform sequence modeling of continuous frame images using a lightweight Transformer architecture. The self-attention mechanism is used to capture the temporal correlation features of the target motion trajectory and action, generating a behavior feature vector containing temporal dimension information. A multi-feature fusion decision layer is constructed to deeply fuse target detection features with spatiotemporal features, and the confidence level of abnormal behavior is output through a multilayer perceptron; An abnormal behavior detection result caching mechanism is established. Through a dynamic sliding cache queue and a Bayesian evidence accumulation algorithm, similar abnormal behaviors detected in more than 3 consecutive frames are cached, stored, aggregated, and verified to generate final abnormal event labels and location information, which are then input into the early warning response module.

5. The AI-based intelligent image recognition security early warning system according to claim 4, characterized in that, Based on a cross-scene transfer learning mechanism with scene attribute labels, a multi-dimensional sample quality assessment model is constructed to screen the training set and iteratively update the sample library, including the following steps: A cross-scenario transfer learning mechanism is introduced, embedding scenario attribute labels into the sample library; A sample quality assessment model is constructed, in which sample clarity is calculated by weighting edge gradient strength and information entropy; the integrity of the calculation is evaluated based on the coverage of human keypoint detection; and the annotation accuracy is calibrated by cross-validation and manual review rate. Set a quality threshold to filter low-quality samples and generate a sample quality score matrix for training set selection. Design a dynamic sample library iteration mechanism, including: real-time access to new abnormal behavior cases in actual applications, which are then included in the incremental sample set after manual annotation and quality assessment; Employing a federated learning framework, it aggregates feature distributions from multiple scenarios while protecting data privacy; it automatically identifies missing scenario feature dimensions in the sample library and triggers a targeted sample collection and supplementation process.

6. The AI-based intelligent image recognition security early warning system according to claim 1, characterized in that, Another channel uses the Transformer architecture to perform spatiotemporal correlation analysis on image sequences to predict potential criminal actions, including the following steps: A temporal image feature extraction unit is constructed. For a multimodal image sequence of 16 consecutive preprocessed frames, the spatial features and inter-frame motion features of each frame are extracted through a 3D convolutional network to generate a spatiotemporal feature cube. A lightweight Transformer encoder is designed, employing a hierarchical attention mechanism: the bottom layer spatial attention focuses on the target region and interaction relationship, the middle layer temporal attention captures the temporal dependence of actions, and the top layer global attention associates with scene context information; through multi-head self-attention computation, the long-range dependencies of target behavior in the spatiotemporal dimensions are modeled. An action prediction head is constructed. Based on the encoded spatiotemporal feature vector, a prediction vector is output through a temporal convolutional network. The prediction vector includes the prediction results of action category, occurrence probability, and duration. The prediction vector is then output to the early warning response module.

7. The AI-based intelligent image recognition security early warning system according to claim 1, characterized in that, The built-in dynamic threshold adjustment mechanism automatically optimizes recognition sensitivity based on a multi-dimensional scene feature matrix, including the following steps: Construct a multi-dimensional scene feature matrix that includes pedestrian density, time characteristics, environmental characteristics, and regional characteristics; A dynamic weight allocation model combining the analytic hierarchy process (AHP) and gradient descent algorithm; A nonlinear mapping relationship between the feature matrix and the recognition threshold is constructed by using a residual network, and dynamic threshold parameters for anomaly detection and action prediction are output. Set a threshold adjustment constraint mechanism to adaptively select either exponential function convergence or linear interpolation transition mode based on the magnitude of changes in scene characteristics. Establish a closed-loop feedback system with reward and punishment mechanisms, use the PPO algorithm to dynamically optimize the threshold model based on the early warning results, set a penalty coefficient for missed reports in high-risk areas, and set an attenuation coefficient for false reports in densely populated places.

8. The AI-based intelligent image recognition security early warning system according to claim 1, characterized in that, The early warning response module includes the following steps: It receives abnormal behavior tags, potential criminal action prediction vectors, and scene feature matrices in real time, and transforms unstructured early warning elements into a three-dimensional risk tensor containing time urgency, spatial correlation, and threat severity through a tensor decomposition algorithm; A dynamic hierarchical response mechanism is designed, which automatically matches three response levels based on the Euclidean distance calculation results of the risk tensor: Level 1 response displays low risk; Level 2 response displays medium risk; and Level 3 response displays high risk. A multi-entity collaborative execution network is established, and a distributed message queue architecture is adopted to realize the asynchronous transmission and priority scheduling of early warning commands.

9. An AI-based intelligent image recognition security early warning method, using the AI-based intelligent image recognition security early warning system as described in any one of claims 1-8, characterized in that, Includes the following steps: S1: Integrates multispectral imaging equipment with edge computing nodes, and acquires multimodal data in dynamic scenes through real-time registration technology of infrared thermal imaging and visible light images; S2: Introduces an atmospheric scattering correction algorithm to process images in complex environments; S3: Construct a dual-channel deep learning architecture. One channel uses the YOLOv8 model to detect abnormal behavior in real time. Based on the cross-scene transfer learning mechanism with scene attribute labels, a multi-dimensional sample quality assessment model is constructed to screen the training set and iteratively update the sample library. The other channel uses the Transformer architecture to perform spatiotemporal correlation analysis on image sequences to predict potential criminal actions. It has a built-in dynamic threshold adjustment mechanism that automatically optimizes recognition sensitivity based on a multi-dimensional scene feature matrix. S4: Utilizes a multimodal interaction protocol and an adaptive execution engine to respond to three levels of early warning information.

Images