Deep learning-based behavior analysis camera monitoring method and system
Through multimodal data fusion and dynamic weight adjustment, combined with spatiotemporal attention network and residual-convolution hybrid structure, the problems of high misjudgment rate of behavior analysis and loss of target identity in complex environments in the prior art are solved, and accurate perception of human behavior in complex environments and continuous and stable identification of target identity are achieved.
Patent Information
- Application Number
- CN202510547071.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-28
- Publication Date
- 2025-06-24
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing vision-based behavioral analysis system has a high misjudgment rate in complex environments, and cannot effectively capture the target motion characteristics in occlusion scenarios. It is easy to cause the loss of target identity in cross-camera synergy scenarios. It is difficult for mainstream deep learning models to dynamically integrate spatial and temporal features, resulting in significant deviations in the confidence of behavior classification and the real risk level.
An innovative architecture of multimodal data fusion and dynamic weight adjustment is adopted, combining spatiotemporal attention networks and residual-convolution hybrid structures, a closed-loop feedback mechanism for environmental parameters and anomaly index is designed, and a heterogeneous feature matching strategy is used in cross-camera collaborative tracking.
It realizes accurate perception of human behavior in complex environments, significantly improves the capture accuracy of human posture and motion trajectory in complex environments, ensures continuous and stable recognition of target identity in occlusion scenarios, and reduces the coupling between the abnormal detection boundary and environmental state.
Smart Images

Figure CN120198855A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of computer vision algorithms. More specifically, it particularly relates to a monitoring method for a behavior analysis camera based on deep learning. At the same time, the present invention also relates to a monitoring system for a behavior analysis camera based on deep learning. Background Art
[0002] Existing vision-based behavior analysis systems usually rely on single-modal data (such as RGB images) for anomaly detection and lack an adaptive suppression mechanism for environmental interferences (such as sudden changes in illumination and background noise), resulting in a high false positive rate; Traditional methods delimit the region of interest through fixed thresholds, unable to effectively capture the target motion characteristics in occluded scenarios, and in the cross-camera collaboration scenario, due to the lack of construction of a clothing-gait joint matching model, it is easy to cause the loss of target identity; in addition, mainstream deep learning models often adopt static network structures and are difficult to dynamically fuse spatio-temporal features. Especially when dealing with sudden noises or continuous occlusions, there is a significant deviation between the behavior classification confidence and the true risk level; Most systems adopt a unified storage strategy and do not implement differential processing for behavior segments with different risk levels, resulting in the loss of fine-grained information due to compression distortion of key data; in complex environments (such as rainy and foggy weather, low illumination at night), the coupling degree between the anomaly determination module of the existing solutions and environmental parameters is insufficient, and the detection boundary cannot be dynamically corrected through a feedback mechanism; In summary, we propose a monitoring method and system for a behavior analysis camera based on deep learning to effectively solve the above problems. Summary of the Invention
[0003] The purpose of the present invention is to solve the deficiencies existing in the prior art, and to propose a monitoring method and system for a behavior analysis camera based on deep learning. Through an innovative architecture that combines multi-modal data fusion and dynamic weight adjustment, it realizes the accurate perception of human behavior in complex environments; the joint design of a spatio-temporal attention network and a residual-convolution hybrid structure effectively captures the spatio-temporal dependence characteristics of the motion trajectory; the closed-loop feedback mechanism of environmental parameters and anomaly indices dynamically optimizes the determination boundary to suppress false alarms and missed detections; the heterogeneous feature matching strategy in cross-camera collaborative tracking ensures the continuous and stable identification of target identity in occluded scenarios.
[0004] To achieve the above purpose, the present invention provides the following technical solutions: A monitoring method for a behavior analysis camera based on deep learning, comprising the following steps: S1. Collect image sequences of target objects in the scene and environmental parameters through multi-modal sensors, and the environmental parameters include illumination intensity and background noise level; S2. Use the dynamic region of interest weight calculation module to prioritize the image sequence, generate a region of interest mask based on the movement trajectory and pose changes of the target object, and the weight calculation formula is: , where is the weight of the region of interest at time t, is the normalized value of the pixel change rate, is the human pose confidence score, is the background noise level, and α, β, γ are dynamic adjustment coefficients; S3. Use the spatio-temporal attention network to extract features from the weighted image sequence, and introduce a residual network module and a long short-term memory network to capture spatial and temporal dependencies respectively; S4. Generate an abnormal behavior index according to the output result of the classifier, and the calculation formula is: , where is the comprehensive anomaly index, is the risk weight of the k-th type of behavior, σ is a non-linear activation function, is the classification confidence, is the environmental correction coefficient; S5. When the abnormal behavior index exceeds the preset threshold, trigger the cross-camera collaborative tracking mechanism and generate an alarm signal, and at the same time mark and store the high-risk behavior segment in the priority queue.
[0005] Preferably, the multi-modal sensor described in step S1 includes a visible light camera, an infrared thermal imager, and an audio acquisition device, and the background noise level is calculated in real time through a frequency domain energy filtering algorithm; The storage strategy of the priority queue is: set different compression ratios according to the behavior risk level, retain the original resolution for high-risk segments, and downsample and store low-risk segments using an adaptive sampling rate; The generation process of the environmental correction coefficient includes: dividing the day / night mode by the light intensity, and constructing a piecewise function in combination with the influence of the environmental temperature on the infrared imaging resolution.
[0006] Preferably, the residual network module described in step S3 adopts a separable convolution structure, cascades dilated convolution and a standard convolution kernel to expand the receptive field, and the spatio-temporal attention network includes an axial attention mechanism, divides row / column attention blocks in the spatial dimension respectively, and constructs a skip connection structure in the time dimension; The parameter optimization formula of the separable convolution structure is: , where in the formula is the weight matrix of the dilated convolution kernel, is the weight matrix of the standard convolution kernel, is the learnable parameter for skip connection, and δ is the gating coefficient. represents the Hadamard product.
[0007] Preferably, the cross-camera collaborative tracking mechanism described in step S5 is specifically: a matching matrix is established based on the clothing color histogram and gait cycle characteristics of the target object, and trajectory relay is achieved in the topologically linked camera cluster.
[0008] Preferably, the output feature fusion calculation formula of the spatio-temporal attention network described in step S3 is: ψ, where is the final fused feature, is the spatial feature mapping function, is the temporal feature mapping function, T is the length of the time window, ⊙ represents element-wise multiplication, is a very small constant to prevent division by zero, is the spatial feature at the i-th time step, is the temporal feature at the i-th time step.
[0009] Preferably, in step S4, the calculation formula of the environmental correction coefficient is: , where is the current ambient light intensity, is the reference light intensity, is the environmental temperature, is the standard temperature of the human body surface, and η, κ are empirical coefficients.
[0010] Preferably, in step S2, the dynamic adjustment coefficient has the calculation formula: , where is the proportion of the occluded area of the target at time t, is the normalized value of the movement speed, μ, θ are parameters obtained through training, and the occluded area is jointly determined by the edge detection and region growing algorithms; The calculation formula of the dynamic adjustment coefficient β is: , where is the variance of the human joint angle change at time t, is the cosine value of the angle between the movement direction vector and the camera, is the class confusion degree output by the pose classifier, and ω, ζ are gain factors dynamically adjusted through reinforcement learning, and the confusion degree is measured by the entropy value of the classification probability distribution; The generation calculation formula of the dynamic adjustment coefficient γ is: , where is the moving window mean of the background noise. is the time differential of the noise variation, m is the length of the historical window, is the exponential decay parameter, is the noise sensitivity weight coefficient.
[0011] A monitoring system for a behavior analysis camera based on deep learning. The system is used to execute the above method. The system includes: a multi-modal data acquisition module, a dynamic region of interest generation module, a spatio-temporal feature extraction module, an abnormal behavior decision module, and a cross-device collaborative tracking module; The multi-modal data acquisition module integrates a visible light camera, an infrared thermal imager, and an audio device to obtain image sequences and environmental parameters in real time; The dynamic region of interest generation module generates a region of interest mask through a dynamic region of interest weight calculation module; The spatio-temporal feature extraction module stacks a residual network module and an axial attention network to output spatio-temporal fusion features; The abnormal behavior decision module performs environmental correction coefficient calculation and outputs a comprehensive abnormal index; The cross-device collaborative tracking module realizes trajectory relay according to the matching matrix and triggers a priority queue storage strategy.
[0012] Preferably, the specific construction method of the matching matrix is: splicing the CIE Lab space distribution of the clothing color histogram and the knee joint angle change rate in the gait cycle, and then mapping it to the camera topology map through bilinear interpolation; The storage control unit of the priority queue integrates a dynamic memory allocation algorithm, adjusts the erasure cycle of the solid-state drive in real time according to the preset list of high-risk behavior types, and the segments with the original resolution occupy independent storage partitions; The calculation result of the spatio-temporal fusion feature is synchronously input into the differential network module and the separable convolution structure through a serial-parallel conversion interface to realize multi-granularity feature cross-validation.
[0013] The technical effects and advantages of the present invention: A monitoring method for a behavior analysis camera based on deep learning provided by the present invention. Compared with the prior art, the present invention has the following effects: By integrating heterogeneous data streams of visible light, infrared, and audio sensors, and designing a joint weighting algorithm based on occlusion perception and movement speed, the system can still accurately lock the high-risk behavior area when the target is partially occluded or there is noise interference; compared with the traditional single-modal system, this mechanism significantly improves the capture accuracy of human postures and movement trajectories in complex environments by dynamically adjusting the region of interest weights. Specifically, it combines the environmental noise level to correct the feature extraction priority in real time, avoids feature drift caused by background interference, and at the same time ensures the semantic coherence of continuous action sequences through the long-term and short-term memory coupling of the spatio-temporal attention network; A dynamic suppression model is constructed by utilizing the time-domain differential characteristics and historical cumulative effect of background noise, enabling the feature extraction module to automatically adjust the convolution kernel parameters according to the noise intensity; this design breaks through the spectral truncation defect of traditional fixed filters and suppresses high-frequency interference while retaining effective motion information by online optimizing the noise sensitivity coefficient; in the scenario of sudden strong noise, through the synergistic effect of sliding window mean and exponential decay, the balance control of instantaneous interference suppression and persistent environmental noise is achieved, ensuring the stability of abnormal behavior index calculation. Brief Description of the Drawings
[0014] Figure 1 It is a flowchart of the monitoring method of the behavior analysis camera based on deep learning of the present invention; Figure 2 It is an architecture diagram of the monitoring system of the behavior analysis camera based on deep learning of the present invention. Detailed Embodiment
[0015] In order to make the purpose, technical solution and advantages of the present invention clearer, the following further details the present invention in conjunction with specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.
[0016] The present invention provides a monitoring method and system for a behavior analysis camera based on deep learning. Through an innovative architecture that combines multi-modal data fusion and dynamic weight adjustment, accurate perception of human behavior in complex environments is achieved; the joint design of a spatio-temporal attention network and a residual-convolution hybrid structure effectively captures the spatio-temporal dependence characteristics of motion trajectories; a closed-loop feedback mechanism of environmental parameters and abnormal indices dynamically optimizes the decision boundary to suppress false alarms and missed detections; a heterogeneous feature matching strategy in cross-camera collaborative tracking ensures continuous and stable identification of target identities in occlusion scenarios; at the same time, a storage resource allocation strategy based on risk levels balances system resource overhead and the integrity of key data; compared with the prior art, this solution has significant advantages in noise suppression, occlusion tolerance, cross-device collaboration, and environmental adaptability, providing highly robust behavior monitoring support for fields such as public security.
[0017] As Figure 1 shown, the monitoring method of the behavior analysis camera based on deep learning includes the following steps: S1. Collect the image sequence of the target object in the scene and environmental parameters through a multi-modal sensor. The environmental parameters include the light intensity and the background noise level. In step S1, the multi-modal sensor includes a visible light camera, an infrared thermal imager, and an audio acquisition device. The background noise level is calculated in real time through a frequency domain energy filtering algorithm. The storage strategy of the priority queue is as follows: Set different compression ratios according to the behavior risk level. High-risk segments retain the original resolution, and low-risk segments are downsampled and stored using an adaptive sampling rate. Environmental correction factor The generation process of includes: Divide the day / night mode by the light intensity, and construct a piecewise function in combination with the influence of the environmental temperature on the infrared imaging resolution.
[0018] S2. Use the dynamic region of interest weight calculation module to divide the priority of the image sequence, generate a region of interest mask based on the motion trajectory and posture change of the target object. The weight calculation formula is: , where is the weight of the region of interest at time t, is the normalized value of the pixel change rate, is the human posture confidence score, is the background noise level, and α, β, γ are dynamically adjustable coefficients; In step S2, the dynamically adjustable coefficient The calculation formula of is: , where is the proportion of the occluded area of the target at time t, is the normalized value of the motion speed, μ, θ are parameters obtained through training, and the occluded area is jointly determined by the edge detection and region growing algorithms; The calculation formula of the dynamically adjustable coefficient β is: , where is the variance of the human joint angle change at time t, is the cosine value of the included angle between the motion direction vector and the camera, is the class confusion degree output by the posture classifier, and ω, ζ are gain factors dynamically adjusted through reinforcement learning. The confusion degree is measured by the entropy value of the classification probability distribution; The generation calculation formula of the dynamically adjustable coefficient γ is: , where is the moving window mean of the background noise, is the time differential of the noise change, m is the length of the historical window, is the exponential decay parameter, is the noise sensitivity weight coefficient.
[0019] S3. Feature extraction is performed on the weighted image sequence through a spatio-temporal attention network, and a residual network module and a long short-term memory network are introduced to capture spatial and temporal dependencies respectively; in step S3, the residual network module adopts a separable convolution structure, cascading dilated convolution and a standard convolution kernel to expand the receptive field. The spatio-temporal attention network includes an axial attention mechanism, dividing row / column attention blocks in the spatial dimension respectively, and constructing a skip connection structure in the temporal dimension; The parameter optimization formula of the separable convolution structure is: , where, is the weight matrix of the dilated convolution kernel, is the weight matrix of the standard convolution kernel, is the learnable parameter of the skip connection, and δ is the gating coefficient represents the Hadamard product.
[0020] It should be noted that the output feature fusion calculation formula of the spatio-temporal attention network in step S3 is: ψ, where, is the final fused feature, is the spatial feature mapping function, is the temporal feature mapping function, T is the length of the time window, ⊙ represents element-wise multiplication, is a very small constant to prevent division by zero, is the spatial feature at the i-th time step, is the temporal feature at the i-th time step.
[0021] S4. Generate an abnormal behavior index according to the output result of the classifier, and the calculation formula is: , where, is the comprehensive anomaly index, is the risk weight of the k-th type of behavior, σ is the non-linear activation function, is the classification confidence, is the environmental correction coefficient; In step S4, the calculation formula of the environmental correction coefficient is: , where, is the current ambient light intensity, is the reference light intensity, is the environmental temperature, is the standard temperature of the human body surface, and η, κ are empirical coefficients.
[0022] S5. When When it exceeds the preset threshold, it triggers the cross-camera collaborative tracking mechanism and generates an alarm signal. At the same time, the high-risk behavior segment is marked and stored in the priority queue. The cross-camera collaborative tracking mechanism in step S5 is specifically as follows: a matching matrix is established based on the clothing color histogram and gait cycle characteristics of the target object, and trajectory relay is realized in the topologically linked camera cluster.
[0023] This embodiment also provides a monitoring system for a behavior analysis camera based on deep learning, such as Figure 2 , the system is used to execute the above method, and the system includes: a multi-modal data acquisition module, a dynamic region of interest generation module, a spatio-temporal feature extraction module, an abnormal behavior decision module, and a cross-device collaborative tracking module; The multi-modal data acquisition module integrates a visible light camera, an infrared thermal imager, and an audio device to obtain image sequences and environmental parameters in real time; The dynamic region of interest generation module generates a region of interest mask through a dynamic region of interest weight calculation module; The spatio-temporal feature extraction module stacks a residual network module and an axial attention network to output spatio-temporal fusion features; The abnormal behavior decision module performs environmental correction coefficient calculation and outputs a comprehensive abnormal index; The cross-device collaborative tracking module realizes trajectory relay according to the matching matrix and triggers the priority queue storage strategy. The specific construction method of the matching matrix is as follows: the CIE Lab space distribution of the clothing color histogram and the knee joint angle change rate in the gait cycle are heterogeneously feature spliced, and then mapped to the camera topology map through bilinear interpolation; The storage control unit of the priority queue integrates a dynamic memory allocation algorithm, adjusts the erasure cycle of the solid-state drive in real time according to the preset list of high-risk behavior types, and the segments with the original resolution occupy independent storage partitions; The calculation result of the spatio-temporal fusion feature is synchronously input into the difference network module and the separable convolution structure through a serial-parallel conversion interface to realize multi-granularity feature cross-validation.
[0024] In summary, compared with the prior art, the present invention has the following effects: 1. Dynamic adjustment mechanism for multi-modal data fusion: By integrating heterogeneous data streams of visible light, infrared, and audio sensors and designing a joint weighted algorithm based on occlusion awareness and movement speed, the system can accurately lock high-risk behavior areas even when the target is partially occluded or there is noise interference. Compared with traditional single-modal systems, this mechanism significantly improves the capture accuracy of human postures and movement trajectories in complex environments by dynamically adjusting the weights of regions of interest. Specifically, it combines the environmental noise level to real-time correct the feature extraction priority to avoid feature drift caused by background interference. At the same time, through the long-term and short-term memory coupling of the spatio-temporal attention network, the semantic coherence of continuous action sequences is guaranteed. 2. Noise-Adaptive Spatio-Temporal Feature Enhancement: Utilize the time-domain differential characteristics and historical cumulative effect of background noise to construct a dynamic suppression model, enabling the feature extraction module to automatically adjust the convolution kernel parameters according to the noise intensity. This design breaks through the spectral truncation defect of traditional fixed filters and suppresses high-frequency interference while retaining effective motion information by online optimizing the noise sensitivity coefficient. Particularly, in the scenario of sudden strong noise, through the synergistic effect of sliding window mean and exponential decay, the balance control of instantaneous interference suppression and persistent environmental noise is achieved to ensure the stability of abnormal behavior index calculation. 3. Identity Preservation Mechanism for Cross-Device Collaborative Tracking: Based on the heterogeneous feature matching matrix of clothing color histogram and gait cycle characteristics, seamless relay of target trajectories is achieved in the camera network topology. Compared with traditional face recognition-based solutions, this mechanism, in the case of target occlusion or low-resolution scenarios, establishes an identity consistency model resistant to perspective changes by fusing joint kinematic features and appearance attributes (CIE Lab color distribution), significantly reducing the identity jump rate in cross-camera tracking and providing continuous temporal context for subsequent behavior risk assessment. 4. Abnormal Decision Optimization with Environmental Parameter Feedback: Construct a piecewise correction function of light-temperature coupling and dynamically inject environmental parameters into the abnormal index calculation link, enabling the system to automatically adjust the decision threshold under day-night switching or extreme temperature conditions. In low-light environments, the reliability of pose confidence score is improved through the infrared imaging resolution compensation mechanism. At the same time, the introduction of noise historical trend analysis avoids mis-triggering of the decision-making module by instantaneous environmental interference. Compared with traditional fixed threshold solutions, this mechanism forms a closed-loop feedback between the abnormal detection boundary and the environmental state, significantly enhancing the robustness of the system in variable scenarios. 5. Risk-Driven Optimal Allocation of Storage Resources: Adopt a dynamic mapping strategy of behavioral risk level and storage compression ratio, implement memory partition management based on a priority queue to ensure that high-risk behavior segments are stored in a lossless format, while low-risk data reduces resource occupancy through adaptive downsampling; this design intelligently adjusts the solid-state drive write-erase cycle, extends the hardware life while ensuring fine-grained traceability of key events, and solves the problems of resource waste and key information loss in the traditional uniform storage mode.
[0025] Finally, it should be noted that the above are only preferred embodiments of the present invention and are not used to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, for those skilled in the art, they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. A monitoring method of a behavior analysis camera based on deep learning, characterized in that: The steps include: S1. Collecting an image sequence of a target object in a scene and environmental parameters through a multimodal sensor, wherein the environmental parameters include light intensity and background noise level; S2, using a dynamic region of interest weight calculation module to prioritize the image sequence and generate a region of interest mask based on the motion trajectory and posture change of the target object; S3, feature extraction of weighted image sequences is performed through spatiotemporal attention network, and residual network module and long short-term memory network are introduced to capture spatial and temporal dependencies respectively; S4, generating an abnormal behavior index according to the output result of the classifier; S5. When the abnormal behavior index exceeds the preset threshold, the cross-camera collaborative tracking mechanism is triggered and an alarm signal is generated. At the same time, the high-risk behavior fragments are marked and stored in the priority queue.
2. The monitoring method of a behavior analysis camera based on deep learning according to claim 1, characterized in that: The multimodal sensor in step S1 includes a visible light camera, an infrared thermal imager and an audio acquisition device, and the background noise level is calculated in real time by a frequency domain energy filtering algorithm; The storage strategy of the priority queue is: setting differential compression rates according to the behavior risk level, retaining the original resolution of high-risk clips, and using adaptive sampling rate to downsample and store low-risk clips; In step S2, the weight calculation formula is: ,in, is the weight of the region of interest at time t, is the normalized value of the pixel change rate, is the human posture confidence score, is the background noise level, α, β, γ are dynamic adjustment coefficients; In step S4, an abnormal behavior index is generated according to the output result of the classifier, and the calculation formula is: ,in, is the comprehensive abnormality index, is the risk weight of the k-th type of behavior, σ is a nonlinear activation function, is the classification confidence, is the environmental correction factor; The environmental correction factor The generation process includes: dividing the day / night mode by light intensity and building a piecewise function based on the effect of ambient temperature on infrared imaging resolution.
3. The monitoring method of a behavior analysis camera based on deep learning according to claim 1, characterized in that: The residual network module in step S3 adopts a separated convolution structure, cascading hole convolution and standard convolution kernel to expand the receptive field. The spatiotemporal attention network includes an axial attention mechanism, which divides row / column attention blocks in the spatial dimension and constructs a jump connection structure in the time dimension.
4. The monitoring method of a behavior analysis camera based on deep learning according to claim 1, characterized in that: The cross-camera collaborative tracking mechanism described in step S5 is specifically as follows: a matching matrix is established according to the clothing color histogram and gait cycle characteristics of the target object, and track relay is realized in the topologically linked camera cluster.
5. The monitoring method of a behavior analysis camera based on deep learning according to claim 1, characterized in that: The output feature fusion calculation formula of the spatiotemporal attention network in step S3 is: , where is the final feature after fusion, is the spatial feature mapping function, is the time feature mapping function, T is the time window length, ⊙ represents element-by-element multiplication, To prevent division by zero, is the spatial feature of the i-th time step, is the time feature of the i-th time step.
6. The monitoring method of a behavior analysis camera based on deep learning according to claim 2 is characterized in that: In step S4, the calculation formula of the environmental correction coefficient is: , where is the current ambient light intensity, is the reference light intensity, is the ambient temperature, is the standard temperature of human body surface, η,κ are empirical coefficients.
7. The monitoring method of a behavior analysis camera based on deep learning according to claim 2, characterized in that: In step S2, the dynamic adjustment coefficient The calculation formula is: ,in, is the proportion of the target blocked area at time t, is the normalized value of the motion speed, μ and θ are the parameters obtained through training, and the occlusion area is determined by the joint edge detection and region growing algorithm; The calculation formula of the dynamic adjustment coefficient β is: , where is the variance of the human joint angle at time t, is the cosine value of the angle between the motion direction vector and the camera, is the category perplexity output by the posture classifier, ω,ζ are gain factors dynamically adjusted by reinforcement learning, and the perplexity is measured by the entropy value of the classification probability distribution; The calculation formula for generating the dynamic adjustment coefficient γ is: , where is the sliding window mean of the background noise, is the time differential of the noise change, m is the length of the historical window, is the exponential decay parameter, is the noise sensitivity weight coefficient.
8. A monitoring system based on deep learning behavior analysis camera, characterized in that: The system is used to execute the method described in any one of claims 1 to 7, and the system includes: a multimodal data acquisition module, a dynamic interest region generation module, a spatiotemporal feature extraction module, an abnormal behavior decision module, and a cross-device collaborative tracking module; The multimodal data acquisition module integrates a visible light camera, an infrared thermal imager and an audio device to acquire image sequences and environmental parameters in real time; The dynamic region of interest generation module generates a region of interest mask through a dynamic region of interest weight calculation module; The spatiotemporal feature extraction module stacks the residual network module and the axial attention network to output spatiotemporal fusion features; The abnormal behavior decision module performs environmental correction coefficient calculation and outputs a comprehensive abnormality index; The cross-device collaborative tracking module implements track relay according to the matching matrix and triggers the priority queue storage strategy.
9. The monitoring system of a behavior analysis camera based on deep learning according to claim 8, characterized in that: The specific construction method of the matching matrix is: performing heterogeneous feature splicing on the CIE Lab spatial distribution of the clothing color histogram and the knee joint angle change rate in the gait cycle, and then mapping them to the camera topology map through bilinear interpolation; The storage control unit of the priority queue integrates a dynamic memory allocation algorithm, adjusts the erase and write cycle of the solid-state hard disk in real time according to a preset list of high-risk behavior types, and retains the original resolution fragments occupying independent storage partitions; The calculation results of the spatiotemporal fusion features are synchronously input into the difference network module and the separated convolution structure through the serial-to-parallel conversion interface to achieve multi-granularity feature cross-validation.
Citation Information
Cited By
Large-scale scene character behavior analysis method, system and device and storage medium
CN120525971A
Intelligent photoelectric system in complex maritime environment and data processing method thereof
CN120724051A
Multi-modal real-time action recognition method and system based on lightweight deep learning
CN120744781A
Data processing system and method based on distributed visible light sensor
CN120746823A
Target detection method and system combining router, gateway and camera
CN121030429A
