Monitoring information analysis method and system based on artificial intelligence

Abstract

The invention discloses a monitoring information analysis method and system based on artificial intelligence, and relates to the technical field of intelligent monitoring and information analysis, and the method comprises the steps: collecting a monitoring video stream from a monitoring device, and intercepting the monitoring video stream to obtain a monitoring image sequence; performing target identification and tracking on the monitoring image sequence by using a deep learning model, and extracting target motion feature parameters and spatial position information; constructing a spatial-temporal characteristic matrix, generating a target behavior track by using the spatial-temporal characteristic matrix, and inputting the target behavior track into a pre-trained behavior analysis model; and performing time sequence analysis on the target behavior track by adopting an attention mechanism, outputting an abnormal behavior probability value, and when the abnormal behavior probability value exceeds a preset threshold value, generating an abnormal event mark. According to the invention, at the video acquisition end, the improved adaptive white balance and multi-scale noise reduction algorithm is adopted to optimize the image quality, so that the problems of color restoration and noise interference in different scenes are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent monitoring and information analysis, and in particular to a monitoring information analysis method and system based on artificial intelligence. Background Art

[0002] In recent years, surveillance technology has undergone a major transformation from traditional analog video surveillance to intelligent video surveillance systems. Traditional surveillance technology mainly relies on analog cameras for video capture, which limits video quality and makes it impossible to achieve real-time analysis and intelligent recognition. With the development of digitization and networking, digital video surveillance systems using IP cameras and cloud storage technology have gradually emerged. These systems can not only efficiently store and manage video data, but also introduce basic motion detection functions. However, although the popularity of digital surveillance systems has brought many conveniences to the security industry, existing technologies still have significant shortcomings.

[0003] Current intelligent monitoring systems generally rely on feature-based target recognition and tracking algorithms, such as Haar features and HOG (Histogram of Oriented Gradients) features. Although these methods perform well under certain conditions, their accuracy drops significantly when dealing with scenes such as complex backgrounds, lighting changes, and target occlusion. In addition, existing behavior analysis models mostly use rule-based methods and lack deep learning of target behavior, resulting in limited ability to detect abnormal behavior. For example, many systems rely only on static thresholds to determine abnormal behavior and cannot adapt to dynamically changing environments and complex crowd behaviors. Furthermore, in the process of data processing, existing technologies often do not effectively utilize time series information, making it difficult to capture the spatiotemporal characteristics of target motion, which affects the accuracy of abnormal behavior discrimination. Summary of the invention

[0004] In view of the problems existing in the prior art, the present invention is proposed.

[0005] Therefore, the problem to be solved by the present invention is that the accuracy of existing methods is significantly reduced when dealing with scenes such as complex backgrounds, illumination changes, and target occlusion.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, an embodiment of the present invention provides a monitoring information analysis method based on artificial intelligence, which includes:

[0008] Collecting a monitoring video stream from a monitoring device and intercepting the monitoring video stream at a preset time interval to obtain a monitoring image sequence;

[0009] Using a deep learning model to perform target recognition and tracking on the surveillance image sequence, and extract target motion feature parameters and spatial position information;

[0010] Building a spatiotemporal feature matrix based on the target motion feature parameters and spatial position information, generating a target behavior trajectory using the spatiotemporal feature matrix, and inputting the target behavior trajectory into a pre-trained behavior analysis model;

[0011] The behavior analysis model uses an attention mechanism to perform time series analysis on the target behavior trajectory and outputs an abnormal behavior probability value. When the abnormal behavior probability value exceeds a preset threshold, an abnormal event marker is generated.

[0012] As a preferred solution of the monitoring information analysis method based on artificial intelligence of the present invention, the monitoring image sequence is obtained by the following steps:

[0013] Collect monitoring video streams from monitoring devices;

[0014] Intercepting the surveillance video stream at a preset time interval to obtain image data;

[0015] Saving the image data as an image sequence, and performing noise reduction processing on the image sequence to obtain the monitoring image sequence;

[0016] The monitoring image sequence includes timestamp information, monitoring device number information, geographic location information and camera orientation angle information.

[0017] As a preferred solution of the monitoring information analysis method based on artificial intelligence of the present invention, wherein: in the process of collecting the monitoring video stream from the monitoring device, the image is corrected in real time by a built-in correction algorithm;

[0018] The correction coefficient of the correction algorithm is calculated by the following formula:

[0019]

[0020] Where W(i, j) represents the white balance weight of pixel (i, j), I(i, j) is the current pixel value, μ is the image mean, σ is the standard deviation, H(i, j) is the hue value, and H ref is the reference hue, α and β are weight coefficients;

[0021] The monitoring video stream is intercepted using an inter-frame interpolation algorithm; the inter-frame interpolation algorithm is expressed by the following formula:

[0022]

[0023] Among them, F interp (t) is the interpolated frame, F(tk ) is the key frame, ω k is the adaptive weight coefficient, and λ is the time decay factor.

[0024] As a preferred solution of the monitoring information analysis method based on artificial intelligence described in the present invention, wherein: the deep learning model includes a backbone network, a spatial pyramid pooling module and a channel attention module;

[0025] The backbone network combines the spatial pyramid pooling module and the channel attention module to extract the monitoring image sequence features, which can be expressed as follows:

[0026] F(x)=W s ·[SPP(x)+α·SE(x)]+x

[0027] Among them, SPP is the spatial pyramid pooling module, SE is the channel attention module, and W s is the adaptive weight matrix, α is the channel attention coefficient 0.7;

[0028] The output of the spatial pyramid pooling module is calculated as:

[0029] SPP(x)=concat{Pool k (x)}, k∈{1, 2, 4}

[0030] Among them, Pool k (x) represents the maximum pooling operation of size k×k;

[0031] The calculation process of the channel attention module SE(x) is:

[0032] SE(x)=σ1(W2·δ(W1·GAP(x)))·x

[0033] Among them, GAP(x) is the global average pooling, W1 and W2 are weight matrices, δ and σ1 are ReLU and Sigmoid activation functions respectively.

[0034] As a preferred solution of the monitoring information analysis method based on artificial intelligence of the present invention, wherein: the target motion characteristic parameters include a velocity vector, a motion characteristic state vector and a state prediction;

[0035] The spatial position includes a spatial state vector, and the spatial state vector includes spatial coordinate information of the target;

[0036] The target motion feature parameters and spatial position information are obtained by performing multi-target tracking using a target tracking algorithm according to the target area detected by the deep learning model;

[0037] The velocity vector is extracted by calculating the correlation of the target tracking algorithm, and the calculation of the correlation is shown in the following formula:

[0038] A(d,t)=β·M(d,t)+(1-β)·C(d,t)

[0039] Among them, M(d, t) is the apparent similarity metric, C(d, t) is the motion state consistency metric, and β is the adaptive weight coefficient:

[0040] β=exp(-λ·||v d -v t || 2 )

[0041] Among them, v d and v t are the velocity vectors of the detection box and the tracker respectively, and λ is the speed difference sensitivity parameter;

[0042] The apparent similarity is calculated by the improved cosine metric, as shown in the following formula:

[0043]

[0044] Among them, f d and f t are the depth features of the detection frame and tracker respectively, Δt is the time difference, ∈ is the time attenuation coefficient;

[0045] The motion state consistency is calculated by Mahalanobis distance, as shown in the following formula:

[0046] C(d, t) = (y d -y t ) T ·S -1 ·(y d -y t )

[0047] Among them, y d and t is the state vector, and S is the state prediction covariance matrix.

[0048] As a preferred solution of the monitoring information analysis method based on artificial intelligence of the present invention, the spatial state vector is extracted by state prediction of the target tracking algorithm, as shown in the following formula:

[0049] x t =F t ·x t-1 +w t

[0050] y t =H t ·xt +v t

[0051] Among them, F t is the state transfer matrix, H t is the observation matrix, w t and v t is the noise vector, satisfying the adaptive covariance update:

[0052]

[0053] Among them, η1 is the process noise smoothing coefficient, μ1 is the observation noise smoothing coefficient, Q t is the process noise covariance matrix, R t is the observation noise covariance matrix, and is the outer product of the noise vector.

[0054] As a preferred solution of the monitoring information analysis method based on artificial intelligence of the present invention, when the probability value of abnormal behavior exceeds a preset threshold, confirmation is performed:

[0055] Perform sliding average on the probability value sequence of the most recent preset number of frames;

[0056] Calculate the standard deviation of the probability value sequence. When the standard deviation is less than the preset value, the abnormal state is considered stable.

[0057] Verify the spatiotemporal consistency of abnormal regions, requiring the overlap rate of abnormal regions in multiple consecutive frames to exceed 60%;

[0058] When all conditions of the exception confirmation mechanism are met, an exception event marker is generated.

[0059] In a second aspect, an embodiment of the present invention provides a monitoring information analysis system based on artificial intelligence, which includes:

[0060] A data acquisition module is used to collect monitoring video streams from monitoring equipment and intercept the monitoring video streams at preset time intervals to obtain monitoring image sequences;

[0061] A feature extraction module is used to use a deep learning model to perform target recognition and tracking on the surveillance image sequence, extract target motion feature parameters and spatial position information; construct a spatiotemporal feature matrix based on the target motion feature parameters and spatial position information, generate a target behavior trajectory using the spatiotemporal feature matrix, and input the target behavior trajectory into a pre-trained behavior analysis model;

[0062] The abnormality analysis module is used to perform time series analysis on the target behavior trajectory using a behavior analysis model, output an abnormal behavior probability value, and generate an abnormal event mark when the abnormal behavior probability value exceeds a preset threshold.

[0063] In a third aspect, an embodiment of the present invention provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the processor executes the computer program, any step of the above-mentioned artificial intelligence-based monitoring information analysis method is implemented.

[0064] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, any step of the above-mentioned artificial intelligence-based monitoring information analysis method is implemented.

[0065] The beneficial effect of the present invention is to achieve intelligent processing of the entire process through multi-level technological innovation. First, at the video acquisition end, an improved adaptive white balance and multi-scale noise reduction algorithm are used to optimize image quality, solving the problems of color restoration and noise interference in different scenes. Secondly, at the video analysis level, the improved deep learning detection model is combined with the multi-target tracking algorithm, and the target detection and tracking performance in complex scenes is improved through feature fusion. In terms of behavioral analysis, the system innovatively introduces a spatiotemporal feature extraction mechanism and a multi-head attention network, and achieves accurate behavior recognition by deeply mining multi-dimensional information such as the target's motion trajectory and posture characteristics. Finally, in the anomaly detection link, a dynamic threshold adjustment strategy is adopted, combined with historical data analysis and real-time status evaluation, to construct an adaptive abnormal event judgment mechanism. Deep feature fusion and information transmission are realized between the various modules of the entire system to ensure the accuracy and reliability of the analysis results. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0067] Figure 1 The flowchart of the monitoring information analysis method based on artificial intelligence. DETAILED DESCRIPTION

[0068] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0069] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0070] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0071] The present invention is described in detail with reference to schematic diagrams. When describing the embodiments of the present invention, for the sake of convenience, the cross-sectional diagrams showing the device structure will not be partially enlarged according to the general scale, and the schematic diagrams are only examples, which should not limit the scope of protection of the present invention. In addition, in actual production, the three-dimensional dimensions of length, width and depth should be included.

[0072] At the same time, in the description of the present invention, it should be noted that the directions or positional relationships indicated by the terms "upper, lower, inner and outer" are based on the directions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore cannot be understood as limiting the present invention. In addition, the terms "first, second or third" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0073] In the present invention, unless otherwise clearly specified and limited, the terms "install, connect, connect" should be understood in a broad sense, for example: it can be a fixed connection, a detachable connection or an integral connection; it can also be a mechanical connection, an electrical connection or a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0074] Example 1

[0075] Reference Figure 1, which is the first embodiment of the present invention, and provides a monitoring information analysis method based on artificial intelligence, comprising:

[0076] S1. Collecting a surveillance video stream from a surveillance device and intercepting the surveillance video stream at a preset time interval to obtain a surveillance image sequence;

[0077] Specifically, a high-definition surveillance video stream is collected from a standardized surveillance device. The collection frame rate of the high-definition surveillance video stream is 30 frames per second, the resolution is 1920×1080 pixels, the video compression format adopts the H.264 encoding standard, and the video bit rate is set to 4Mbps.

[0078] During the process of collecting high-definition surveillance video streams, the image is corrected in real time through the built-in automatic white balance algorithm, the color temperature range is set between 2800K and 6500K, and the response time of the automatic white balance algorithm does not exceed 50 milliseconds; the high-definition surveillance video stream is regularly intercepted at a preset time interval of 10 seconds, and 300 frames of image data are retained during each interception process. The linear interpolation algorithm is used in the interception process to ensure continuity between frames, and the interpolation coefficient of the linear interpolation algorithm is set to 0.8.

[0079] The 300 frames of image data are saved as a surveillance image sequence in lossless compressed PNG format. The metadata of the surveillance image sequence contains timestamp information, surveillance equipment number information, geographic location information, and camera orientation angle information. Then the surveillance image sequence is subjected to adaptive denoising: the median filter algorithm is used to remove salt and pepper noise, and the filter window size is set to 5×5 pixels. Subsequently, the Gaussian filter algorithm is used to eliminate Gaussian noise, and the Gaussian kernel size is set to 3×3 pixels. The standard deviation is dynamically adjusted between 0.8 and 2.0. Finally, the processed image is contrast enhanced by using an adaptive histogram equalization algorithm, in which the block size is set to 8×8 pixels and the contrast limit threshold is set to 3.0.

[0080] For example, in the process of collecting high-definition surveillance video streams, the image is corrected in real time through a weight-adaptive white balance algorithm, and the calculation of the correction coefficient can be expressed by the following formula:

[0081]

[0082] Where W(i, j) represents the white balance weight of pixel (i, j), I(i, j) is the current pixel value, μ is the image mean, σ is the standard deviation, H(i, j) is the hue value, and H ref is the reference hue, α and β are weight coefficients and satisfy α+β=1.

[0083] Furthermore, the color temperature compensation value of the image is calculated based on the correction coefficient of the white balance algorithm:

[0084]

[0085] Among them, T base The basic color temperature is 2800K, ΔT is the adjustable range of 3700K, R(i, j) and B(i, j) are the red and blue channel values ​​respectively, and γ is the nonlinear adjustment coefficient with a value range of [0.8, 1.2].

[0086] The high-definition surveillance video stream is captured at a preset time interval of 10 seconds, and an improved adaptive inter-frame interpolation algorithm is used, which can be expressed as follows:

[0087]

[0088] Among them, F interp (t) is the interpolated frame, F(t k ) is the key frame, ω k is the adaptive weight coefficient, and λ is the time decay factor. 300 frames of image data are retained in each capture process; and the 300 frames of image data are saved as a monitoring image sequence in lossless compressed PNG format, and multi-scale denoising is performed on the monitoring image sequence. The denoising function is defined as:

[0089]

[0090] Among them, G(x, y) is the Gaussian kernel function, I(x, y) is the original image, represents the Laplace operator, and η is the adaptive adjustment coefficient:

[0091]

[0092] Among them, k is the proportional coefficient, θ is the gradient threshold, Represents the image gradient magnitude.

[0093] Finally, the processed image is contrast enhanced using an improved adaptive histogram equalization algorithm:

[0094]

[0095] Among them, CDF is the cumulative distribution function, C min and C max is the contrast range, CDF min and CDF max are the minimum and maximum values ​​of CDF respectively.

[0096] Through the above processing steps, a processed monitoring image sequence is obtained.

[0097] The better one is that by introducing a pixel-level weight adaptive mechanism in the white balance algorithm and combining it with a nonlinear dynamic color temperature compensation system, the color reproduction of the image is improved by 35%. The weight calculation takes into account the dual deviation of pixel brightness and reference hue, so that the adaptation range of white balance is expanded from the traditional 3000K-5500K to 2800K-6500K, and the scene adaptability is improved by 50%. In the inter-frame interpolation link, the combined action mechanism of the time domain attenuation factor and the adaptive weight coefficient is introduced. Through the exponential weight distribution strategy, the trajectory continuity of the moving target is significantly enhanced, and the motion artifacts are reduced by 60%. In the noise reduction processing link, the dynamic adjustment coefficient η is used to accurately suppress noise of different scales, so that the peak signal-to-noise ratio of the image is improved by 4.5dB. While maintaining edge details, the noise reduction effect is improved by 25% compared with the traditional Gaussian filter.

[0098] S2 uses a deep learning model to perform target recognition and tracking on the surveillance image sequence, and extracts target motion feature parameters and spatial position information, wherein the target motion feature parameters include motion speed, motion direction and acceleration.

[0099] Specifically, the improved YOLOv5 deep learning model is used to perform target recognition and tracking on the surveillance image sequence. The YOLOv5 model uses ResNet101 as the backbone network, introduces the spatial pyramid pooling module in the feature extraction layer, and the pooling sizes are 1×1, 2×2, and 4×4 pixels respectively. The channel attention coefficient is set to 0.7, and its feature extraction function can be expressed as follows:

[0100] F(x)=W s ·[SPP(x)+α·SE(x)]+x

[0101] Among them, SPP is the spatial pyramid pooling module, SE is the channel attention module, and W s is the adaptive weight matrix, α is the channel attention coefficient 0.7; the output of the spatial pyramid pooling module is calculated as:

[0102] SPP(x)=concat{Pool k (x)}, k∈{1, 2, 4}

[0103] Among them, Pool k (x) represents the maximum pooling operation of size k×k; the calculation process of the channel attention module SE(x) is:

[0104] SE(x)=σ1(W2·δ(W1·GAP(x)))·x

[0105] Among them, GAP(x) is the global average pooling, W1 and W2 are weight matrices, δ and σ1 are ReLU and Sigmoid activation functions respectively.

[0106] Furthermore, the target detection confidence threshold of the YOLOv5 model is set to 0.85, the non-maximum suppression threshold is set to 0.4, and the anchor box scale ratio is {8, 16, 32}; the improved DeepSORT algorithm is applied to the detected target area for multi-target tracking, and the association between the detection results and the tracker is achieved through the cascaded attention mechanism. The correlation calculation formula is:

[0107] A(d,t)=β·M(d,t)+(1-β)·C(d,t)

[0108] Among them, M(d, t) is the apparent similarity metric, C(d, t) is the motion state consistency metric, and β is the adaptive weight coefficient:

[0109] β=exp(-λ·||v d -v t || 2 )

[0110] Among them, v d and v t are the velocity vectors of the detection box and the tracker respectively, and λ is the speed difference sensitivity parameter; the apparent similarity is calculated by the improved cosine metric:

[0111]

[0112] Among them, f d and f t are the depth features of the detection frame and the tracker respectively, Δt is the time difference, ∈ is the time attenuation coefficient; the consistency of the motion state is calculated by the Mahalanobis distance:

[0113] C(d, t) = (y d -y t ) T ·S -1 ·(y d -y t )

[0114] Among them, y d and t is the state vector, S is the state prediction covariance matrix;

[0115] Furthermore, the DeepSORT algorithm uses an improved Kalman filter for state prediction, and the state transfer equation is:

[0116] x t =F t ·x t-1 +wt

[0117] y t =H t ·x t +v t

[0118] Among them, F t is the state transfer matrix, H t is the observation matrix, w t and v t is the noise vector, satisfying the adaptive covariance update:

[0119]

[0120] Among them, η1 is the process noise smoothing coefficient, μ1 is the observation noise smoothing coefficient, the value range is [0.8, 0.95], Q t is the process noise covariance matrix, R t is the observation noise covariance matrix, and is the outer product of the noise vector.

[0121] It should be noted that a deep feature cascade fusion mechanism is established between the YOLOv5 model and the DeepSORT algorithm. Specifically, in addition to outputting the target detection box, the YOLOv5 model also outputs the deep feature representation of the target through the feature extraction branch. This feature representation is directly used as the input for the apparent similarity calculation in the DeepSORT algorithm. At the same time, the detection confidence of YOLOv5 is used to dynamically adjust the association threshold of DeepSORT, realizing the adaptive coordination of detection and tracking. This cascade fusion mechanism significantly improves the accuracy and stability of multi-target tracking.

[0122] Furthermore, during the target tracking process, the target motion characteristic parameters are extracted, including instantaneous motion speed, cumulative displacement, motion direction angle and acceleration value, and the sampling frequency of the motion characteristic parameters is 10Hz; wherein the motion speed is calculated by the position difference of the target center point between adjacent frames, the motion direction is calculated by the heading angle cosine, and the acceleration is obtained by the second-order difference of the speed; at the same time, the spatial position information of the target is recorded, including the two-dimensional coordinates (x, y) of the target center point in the image coordinate system, the aspect ratio and area ratio of the target bounding box, and the update period of the spatial position information is 100ms;

[0123] The motion feature parameters were time-series smoothed using the exponentially weighted moving average method. The smoothing coefficient was set to 0.85 and the time window length was 2 seconds. The spatial position information was mapped to the real-world coordinate system through a projection transformation matrix, and the calibration error of the transformation matrix was controlled within ±5 cm.

[0124] S3 constructs a spatiotemporal feature matrix based on the target motion feature parameters and spatial position information, generates a target behavior trajectory using the spatiotemporal feature matrix, and inputs the target behavior trajectory into a pre-trained behavior analysis model;

[0125] A four-dimensional spatiotemporal feature matrix is ​​constructed based on the target motion feature parameters and spatial position information, and the dimension of the spatiotemporal feature matrix is ​​set to [T, N, D, C], where T represents the time dimension and is set to 128 time steps, N represents the number of targets and supports a maximum of 32 simultaneously tracked targets, D represents the feature dimension and is set to 256, and C represents the number of feature channels and is set to 4.

[0126] The spatiotemporal feature matrix is ​​constructed by hierarchical coding. An exponentially decaying sliding window is used for feature aggregation in the time dimension. The window size is set to 16, the decay coefficient is 0.9, and the overlap rate of adjacent windows is 0.5. A multi-scale feature fusion strategy is adopted in the feature dimension. The motion feature parameters and spatial position information are respectively mapped through three fully connected layers. The hidden layer dimensions are 512, 384 and 256 respectively. LeakyReLU is used as the activation function, and its slope parameter is set to 0.2.

[0127] The self-attention mechanism is introduced in the feature mapping process, the number of attention heads is set to 8, the attention weight is normalized by softmax, and the temperature parameter is set to 0.1. A residual connection structure is used in the channel dimension, and each feature channel corresponds to the velocity field, direction field, acceleration field and position field respectively. The feature interaction between channels is realized by 1×1 convolution, and the number of convolution kernels is 16;

[0128] Based on the spatiotemporal feature matrix, an improved temporal convolutional network is used to generate the target behavior trajectory, wherein the temporal convolutional network includes 5 dilated convolutional layers, the dilated rates are set to 1, 2, 4, 8, and 16 respectively, the convolution kernel size is 3, and the number of output channels of each layer is 64; a batch normalization layer is inserted between the convolutional layers, the momentum parameter is set to 0.9, and the epsilon value is set to 1e-5; the output of the temporal convolutional network is subjected to sequence modeling through a bidirectional long short-term memory network, the hidden layer dimension is 128, and the dropout ratio is set to 0.3; the target behavior trajectory finally obtained is represented as a temporal feature sequence with a sequence length of 64, and each temporal node contains the instantaneous state and cumulative state information of the target;

[0129] S4 The behavior analysis model uses an attention mechanism to perform time series analysis on the target behavior trajectory and outputs an abnormal behavior probability value. When the abnormal behavior probability value exceeds a preset threshold, an abnormal event marker is generated;

[0130] The behavior analysis model uses a multi-head cross-attention mechanism to perform temporal analysis on the target behavior trajectory. The attention mechanism includes a temporal attention module and a spatial attention module. Among them, the temporal attention module is configured with 16 attention heads, each with a dimension of 32 and a time step of 128. This module uses a position-aware self-attention calculation method, and the position encoding combines learnable absolute position encoding with sinusoidal position encoding, with an encoding dimension of 512. The spatial attention module uses a non-local attention mechanism to divide the attention calculation area into 3×3 grids, and local feature aggregation is performed in each grid. The features of different grids interact with global information through adaptive weights, and the weight coefficients are obtained through softmax normalization.

[0131] It should be noted that the calculation process of the multi-head cross-attention mechanism for time series analysis of the target behavior trajectory can be expressed as follows:

[0132] MultiHead(Q,K,V)=Concat(head1,...,head h )W O

[0133] The calculation of each attention head is:

[0134]

[0135] Preferably, a temporal gating function G(t) is introduced in the attention head calculation process to modulate the attention intensity at different time steps. The temporal gating function can be expressed as follows:

[0136] G(t)=σ1(W g ·[h t |m t ]+b g )

[0137] Among them, ht 为 The hidden state vector at time t, m t is the motion feature memory inherited from step S2, || represents feature concatenation, Q is the query matrix, K is the key matrix, V is the value matrix, and W O is the output projection matrix, h is the number of attention heads, Concat is the concatenation operation, is the query transformation matrix of the i-th attention head, is the key transformation matrix of the i-th attention head, is the value transformation matrix of the ith attention head, G(t) is the temporal gating function, and W g is the gating weight matrix, b g is the gate bias term.

[0138] The calculation of the attention score is based on the DeepSORT algorithm, which takes into account the position bias and motion state, and can be expressed as follows:

[0139]

[0140] Among them, R is the relative position encoding matrix, QK T is the attention score matrix, d k is the attention head dimension, and M is the motion correlation matrix inherited from S2:

[0141] M = ω·tanh(W m ·[v t -v t-1 ])

[0142] Among them, v t represents the velocity vector of the target at time t, v t-1 represents the velocity vector of the target at time t-1, ω is the adaptive weight coefficient, W m is the motion feature transformation matrix.

[0143] The output features of the attention mechanism are extracted through a three-layer temporal convolutional network, with convolution kernel sizes of [3, 5, 7] and output channels of [128, 256, 512]. After each convolution layer, a batch normalization layer and ReLU activation function are connected; residual connections are introduced between convolution layers, and the residual branch uses 1×1 convolution for channel matching. The extracted features are aggregated by global average pooling and global maximum pooling, and recalibrated by the channel attention module. The hidden layer dimension of the channel attention module is 256, and the output channel weight uses the Sigmoid activation function.

[0144] It should be noted that the feature aggregation process of the spatial attention module can be expressed as follows:

[0145]

[0146] where α i is the attention weight, φ(x i ) is the local feature transformation, ρ(p i ) is the position embedding, which is calculated as:

[0147]

[0148] Among them, W a is the attention weight matrix, x i , p i is the characteristics and position of the current grid, x j , p j for the features and positions of all meshes.

[0149] The aggregated features are input into a multi-layer perceptron for abnormal behavior scoring. The multi-layer perceptron contains three fully connected layers, with hidden layer dimensions of [512, 256, 128], and the middle layer uses a Leaky ReLU activation function with a dropout ratio of 0.4. Label smoothing regularization is introduced before the last layer, and the smoothing coefficient is set to 0.1. The output layer generates an abnormal behavior probability value through a Sigmoid function, with a value range of [0, 1].

[0150] It should be noted that the abnormal behavior score is ultimately the abnormal behavior probability value, and the overall calculation is as follows:

[0151] P anomaly =σ(W f ·F+b f )·exp(-λ·Δt)

[0152] Among them, F is the fusion feature, Δt is the time difference, and λ is the attenuation coefficient.

[0153] The dynamic threshold adjustment mechanism is introduced in the calculation process of the abnormal behavior probability value. The basic threshold is set to 0.75 and adaptively adjusted according to the following rules:

[0154] When more than 5 frames of high probability values ​​(greater than 0.9) are detected continuously, the threshold is increased by 0.05;

[0155] When no anomalies are detected within 20 consecutive frames, the threshold is reduced by 0.03;

[0156] The adjustment range of the threshold is limited to [0.65, 0.85];

[0157] The minimum time interval between adjacent threshold adjustments is 2 seconds;

[0158] When the abnormal behavior probability value exceeds the current threshold, the abnormal confirmation mechanism is started:

[0159] Perform sliding average on the probability value sequence of the most recent 30 frames, with an average window size of 10;

[0160] Calculate the standard deviation of the probability value sequence. When the standard deviation is less than 0.15, the abnormal state is considered stable.

[0161] Verify the spatiotemporal consistency of the abnormal area, requiring the overlap rate of the abnormal area to exceed 60% in 5 consecutive frames;

[0162] Perform rationality checks based on the target's motion state to exclude false anomalies caused by occlusion or tracking failure.

[0163] When all conditions of the abnormal confirmation mechanism are met, an abnormal event marker is generated, which contains the following information:

[0164] The timestamp and duration of the exception

[0165] Spatial range and center coordinates of the abnormal area

[0166] Target ID and motion status parameters

[0167] Cumulative anomaly probability value and confidence score

[0168] The contextual characteristics of the association.

[0169] The monitoring image sequence, target motion characteristic parameters, spatiotemporal characteristic matrix and abnormal behavior probability value corresponding to the abnormal event mark are integrated to form an abnormal event report.

[0170] Furthermore, this embodiment also provides a monitoring information analysis system based on artificial intelligence, including:

[0171] A data acquisition module is used to collect monitoring video streams from monitoring equipment and intercept the monitoring video streams at preset time intervals to obtain monitoring image sequences;

[0172] A feature extraction module is used to use a deep learning model to perform target recognition and tracking on the surveillance image sequence, extract target motion feature parameters and spatial position information; construct a spatiotemporal feature matrix based on the target motion feature parameters and spatial position information, generate a target behavior trajectory using the spatiotemporal feature matrix, and input the target behavior trajectory into a pre-trained behavior analysis model;

[0173] The abnormality analysis module is used to perform time series analysis on the target behavior trajectory using a behavior analysis model, output an abnormal behavior probability value, and generate an abnormal event mark when the abnormal behavior probability value exceeds a preset threshold.

[0174] This embodiment also provides a computer device, which is suitable for the monitoring information analysis method based on artificial intelligence, including a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute computer executable instructions to implement the monitoring information analysis method based on artificial intelligence proposed in the above embodiment.

[0175] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covering the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0176] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the monitoring information analysis method based on artificial intelligence as proposed in the above embodiment is implemented.

[0177] The storage medium proposed in this embodiment and the data storage method proposed in the above embodiment belong to the same inventive concept. The technical details not fully described in this embodiment can be found in the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0178] Example 2

[0179] This is the second embodiment of the present invention, which provides a monitoring information analysis method based on artificial intelligence. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments.

[0180] This experiment was tested in the monitoring system of a large commercial complex. The test scenes included key areas such as the main passage of the shopping mall, the elevator waiting area, and the parking lot entrance and exit. The experiment used Hikvision DS-2CD7A47FWD-LZS series network cameras, which are equipped with a 1 / 1.8-inch CMOS sensor and support 4K ultra-high-definition video acquisition at 30fps. The camera was installed at a height of 4.5 meters, with a pitch angle of 35° and a horizontal field of view of 95°, ensuring that the monitoring area covers an area of ​​about 200 square meters.

[0181] To verify the performance of the system, the experiment was conducted under different lighting conditions (10:00 am, 14:00 pm, and 20:00 pm) and different crowd densities (sparse, general, and dense). The test period lasted 30 days, and the cumulative amount of video data collected reached 2,160 hours. During the data collection phase, the system captured the screen at a preset 10-second time interval, and a total of 777,600 sets of monitoring image sequences were obtained.

[0182] In the image preprocessing stage, an improved adaptive white balance algorithm is used to perform color correction on the original image. By adjusting the weight coefficients α = 0.65 and β = 0.35, the system can maintain a stable color restoration effect under different lighting conditions. In the calculation of the color temperature compensation value, the nonlinear adjustment coefficient γ is set to 0.95, achieving a smooth transition from 2800K to 6500K. In the inter-frame interpolation algorithm, the time attenuation factor λ is set to 0.85, and the adaptive weight coefficient ω is dynamically adjusted in the range of [0.75, 0.95], which significantly improves the trajectory continuity of the moving target.

[0183] In the target detection and tracking stage, the improved YOLOv5 model uses the ResNet101 backbone network and introduces a spatial pyramid pooling module. By setting three scales of pooling operations (1×1, 2×2, 4×4) and combining the channel attention mechanism (the attention coefficient is set to 0.7), the model's ability to detect targets of different scales is significantly improved. In the DeepSORT algorithm, the speed difference sensitivity parameter κ of the correlation calculation is set to 0.8, and the time decay coefficient λ is set to 0.9, achieving a stable multi-target tracking effect.

[0184] In the behavior analysis stage, the spatiotemporal feature matrix uses the dimension configuration of [128, 32, 256, 4], and feature extraction is performed through hierarchical encoding. The multi-head cross attention mechanism is configured with 16 attention heads, each with a dimension of 32, to achieve accurate modeling of the target behavior trajectory. The abnormal behavior score adopts a dynamic threshold adjustment mechanism, with the basic threshold set to 0.75, and adaptively adjusted according to the real-time detection results. The experimental data is as follows:

[0185] Table 1 Experimental data table

[0186]

[0187] Through the analysis of experimental data, it can be found that the scheme of the present invention is significantly better than the traditional scheme and the basic model scheme in various performance indicators.

[0188] The average processing time of the solution in the standard environment is 31.2ms, which is 32% higher than the traditional solution A and 26.2% higher than the traditional solution B. Even in high-density crowd flow scenarios, the processing time is only 32.8ms, maintaining high real-time performance. This is mainly due to the improved feature extraction architecture and the optimized design of the multi-head attention mechanism.

[0189] The target detection accuracy of the present invention under standard environment reaches 94.5%, which is 9.2 percentage points higher than that of the traditional solution A; the target tracking accuracy reaches 92.7%, which is 10.1 percentage points higher than that of the traditional solution A. This performance improvement is mainly due to the spatial pyramid pooling module and channel attention mechanism introduced in the improved YOLOv5 model, as well as the optimized correlation calculation method in the DeepSORT algorithm.

[0190] The solution of the present invention improves the trajectory completeness to 90.4%, which is 12 percentage points higher than the traditional solution A. This is due to the improved inter-frame interpolation algorithm and the hierarchical coding mechanism of the spatiotemporal feature matrix, which enables the system to better handle complex scenes such as target occlusion and rapid motion.

[0191] Under standard conditions, the accuracy of abnormal behavior recognition of the solution of the present invention reaches 91.6%, the false alarm rate drops to 4.2%, and the missed alarm rate drops to 4.9%. Compared with the traditional solution A, the accuracy rate is increased by 8.4 percentage points, the false alarm rate is reduced by 4.3 percentage points, and the missed alarm rate is reduced by 4.3 percentage points. This significant improvement is mainly due to the application of the multi-head cross attention mechanism and the dynamic threshold adjustment strategy.

[0192] The solution of the present invention still maintains high performance in low-light environments and high-density scenes, with detection accuracy rates of 92.8% and 93.6% respectively, indicating that the system has good environmental adaptability. This is mainly due to the improved adaptive white balance algorithm and multi-scale feature fusion strategy.

[0193] In general, the experimental results prove that the present invention has significant technical advantages and practical value in practical application scenarios. The system not only achieves quantitative improvement in various performance indicators, but also shows good stability and reliability under different environmental conditions. These improvements provide more reliable technical support for real-time monitoring of abnormal behavior in practical scenarios.

[0194] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A monitoring information analysis method based on artificial intelligence, characterized in that: include, Collecting a monitoring video stream from a monitoring device and intercepting the monitoring video stream at a preset time interval to obtain a monitoring image sequence; Using a deep learning model to perform target recognition and tracking on the surveillance image sequence, and extract target motion feature parameters and spatial position information; Constructing a spatiotemporal feature matrix based on the target motion feature parameters and spatial position information, generating a target behavior trajectory using the spatiotemporal feature matrix, and inputting the target behavior trajectory into a pre-trained behavior analysis model; The behavior analysis model uses an attention mechanism to perform time series analysis on the target behavior trajectory and outputs an abnormal behavior probability value. When the abnormal behavior probability value exceeds a preset threshold, an abnormal event marker is generated.

2. The monitoring information analysis method based on artificial intelligence according to claim 1, characterized in that: The monitoring image sequence is obtained by the following steps: Collect monitoring video streams from monitoring devices; Intercepting the surveillance video stream at a preset time interval to obtain image data; Saving the image data as an image sequence, and performing noise reduction processing on the image sequence to obtain the monitoring image sequence; The monitoring image sequence includes timestamp information, monitoring device number information, geographic location information and camera orientation angle information.

3. The monitoring information analysis method based on artificial intelligence as claimed in claim 2, characterized in that: In the process of collecting the monitoring video stream from the monitoring device, the image is corrected in real time by using a built-in correction algorithm; The correction coefficient of the correction algorithm is calculated by the following formula: Where W(i, j) represents the white balance weight of pixel (i, j), I(i, j) is the current pixel value, μ is the image mean, σ is the standard deviation, H(i, j) is the hue value, and H ref is the reference hue, α and β are weight coefficients; The monitoring video stream is intercepted using an inter-frame interpolation algorithm; the inter-frame interpolation algorithm is expressed by the following formula: Among them, F interp (t) is the interpolated frame, F(t k ) is the key frame, ω k is the adaptive weight coefficient, and λ is the time decay factor.

4. The monitoring information analysis method based on artificial intelligence as claimed in claim 3, characterized in that: The deep learning model includes a backbone network, a spatial pyramid pooling module and a channel attention module; The backbone network combines the spatial pyramid pooling module and the channel attention module to extract the monitoring image sequence features, which can be expressed as follows: F(x)=W s ·[SPP(x)+α·SE(x)]+x Among them, SPP is the spatial pyramid pooling module, SE is the channel attention module, and W s is the adaptive weight matrix, α is the channel attention coefficient 0.7; The output of the spatial pyramid pooling module is calculated as: SPP(x)=concat{Pool k (x)},k∈{1,2,4} Among them, Pool k (x) represents the maximum pooling operation of size k×k; The calculation process of the channel attention module SE(x) is: SE(x)=σ1(W2·δ(W1·GAP(x)))·x Among them, GAP(x) is the global average pooling, W1 and W2 are weight matrices, δ and σ1 are ReLU and Sigmoid activation functions respectively.

5. The monitoring information analysis method based on artificial intelligence as claimed in claim 4, characterized in that: The target motion characteristic parameters include a velocity vector, a motion characteristic state vector and a state prediction; The spatial position includes a spatial state vector, and the spatial state vector includes spatial coordinate information of the target; The target motion feature parameters and spatial position information are obtained by performing multi-target tracking using a target tracking algorithm according to the target area detected by the deep learning model; The velocity vector is extracted by calculating the correlation of the target tracking algorithm, and the calculation of the correlation is shown in the following formula: A(d,t)=β·M(d,t)+(1-β)·C(d,t) Among them, M(d, t) is the apparent similarity metric, C(d, t) is the motion state consistency metric, and β is the adaptive weight coefficient: β=exp(-λ·||v d -in t || 2 ) Among them, v d and v t are the velocity vectors of the detection box and the tracker respectively, and λ is the speed difference sensitivity parameter; The apparent similarity is calculated by the improved cosine metric, as shown in the following formula: Among them, f d and f t are the depth features of the detection frame and tracker respectively, Δt is the time difference, ∈ is the time attenuation coefficient; The motion state consistency is calculated by Mahalanobis distance, as shown in the following formula: C(d,t)=(y d -y t ) T ·S -1 ·(y d -y t ) Among them, y d and t is the state vector, and S is the state prediction covariance matrix.

6. The monitoring information analysis method based on artificial intelligence as claimed in claim 5, characterized in that: The spatial state vector is extracted through the state prediction of the target tracking algorithm, as shown in the following formula: x t =F t ·x t-1 +w t y t =H t ·x t +v t Among them, F t is the state transfer matrix, H t is the observation matrix, w t and v t is the noise vector, satisfying the adaptive covariance update: Among them, η1 is the process noise smoothing coefficient, μ1 is the observation noise smoothing coefficient, Q t is the process noise covariance matrix, R t is the observation noise covariance matrix, and is the outer product of the noise vector.

7. The monitoring information analysis method based on artificial intelligence according to claim 6, characterized in that: When the abnormal behavior probability value exceeds the preset threshold, confirmation is performed: Perform sliding average on the probability value sequence of the most recent preset number of frames; Calculate the standard deviation of the probability value sequence. When the standard deviation is less than the preset value, the abnormal state is considered stable. Verify the spatiotemporal consistency of abnormal regions, requiring the overlap rate of abnormal regions in multiple consecutive frames to exceed 60%; When all conditions of the exception confirmation mechanism are met, an exception event marker is generated.

8. A monitoring information analysis system based on artificial intelligence, based on the monitoring information analysis method based on artificial intelligence according to any one of claims 1 to 7, characterized in that: include, A data acquisition module is used to collect monitoring video streams from monitoring equipment and intercept the monitoring video streams at preset time intervals to obtain monitoring image sequences; A feature extraction module is used to use a deep learning model to perform target recognition and tracking on the surveillance image sequence, extract target motion feature parameters and spatial position information; construct a spatiotemporal feature matrix based on the target motion feature parameters and spatial position information, generate a target behavior trajectory using the spatiotemporal feature matrix, and input the target behavior trajectory into a pre-trained behavior analysis model; The abnormality analysis module is used to perform time series analysis on the target behavior trajectory using a behavior analysis model, output an abnormal behavior probability value, and generate an abnormal event mark when the abnormal behavior probability value exceeds a preset threshold.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the monitoring information analysis method based on artificial intelligence described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the monitoring information analysis method based on artificial intelligence described in any one of claims 1 to 7 are implemented.

Images