Multi-camera cooperative intelligent early warning method and system for abnormal events

Abstract

The invention provides a multi-camera cooperative intelligent early warning method and system for abnormal events, and belongs to the technical field of intelligent security and big data. The method comprises the following steps: collecting video data through a multi-view and multi-type camera, collecting environmental parameters through a sensor, extracting target appearance, motion features and environmental state features by using deep learning after preprocessing, and fusing to construct a multi-modal feature vector; and analyzing and identifying an abnormal event based on the trained abnormal event identification model, determining event information through multi-camera cooperative feature matching and geometric operation, and finally generating early warning information according to a preset strategy and linking the equipment. The abnormal event recognition accuracy and early warning timeliness are improved, and the method is suitable for the field of intelligent security and protection of complex scenes.

Description

Technical Field

[0001] This application relates to the fields of intelligent security and big data technology, specifically to an intelligent early warning method and system for abnormal events using multi-camera collaboration. Background Technology

[0002] With the development of intelligent security technology, single cameras, due to limitations in field of view and poor environmental adaptability, are unable to meet the demand for accurate early warning of abnormal events in complex scenarios. Existing technologies for multi-camera collaboration often suffer from low feature matching accuracy, insufficient fusion of environmental factors, and fixed early warning strategies. For example, relying solely on video data is susceptible to false alarms due to lighting and occlusion; multimodal feature fusion does not consider dynamic weight adjustment, making it difficult to adapt to scene changes; abnormal event localization relies on the coordinates of a single camera and lacks cross-camera geometric relationship verification; and early warning information updates and device linkage strength are not correlated with event development trends, resulting in insufficient flexibility. Therefore, there is an urgent need for a technical solution that combines multi-source data fusion, dynamic collaborative localization, and intelligent early warning strategies to improve the reliability and adaptability of abnormal event early warning. Summary of the Invention

[0003] This invention aims to address the limitations of single-camera perspective, poor adaptability of multimodal feature fusion, insufficient accuracy in anomaly location, and rigid early warning strategies in existing technologies. It provides a multi-camera collaborative intelligent early warning method and system for anomaly events. The method involves acquiring video and environmental data through multiple cameras and sensors. After preprocessing and feature extraction, a multimodal vector is constructed by fusing target appearance, motion features, and environmental state features. Anomalies are identified using a recognition model incorporating prior scene knowledge. Event information is determined by combining multi-camera collaborative feature matching and geometric operations. Finally, a dynamic early warning strategy triggers linked devices. The system includes modules for data acquisition, preprocessing, feature extraction, feature fusion, anomaly identification, collaborative location, and early warning linkage. These modules work together to implement the above processes. This solution improves the anti-interference capability and accuracy of anomaly event identification through multi-camera collaboration and multimodal feature fusion. It enhances the scene adaptability of feature extraction through spatiotemporal correlation modeling and dynamic weight adjustment. Through cross-camera geometric operations and dynamic early warning strategies, it achieves precise location and flexible response to anomalies. This approach is suitable for intelligent security needs in complex scenarios such as public places and industrial parks.

[0004] This application provides a method for intelligent early warning of abnormal events through multi-camera collaboration, including the following steps:

[0005] Video data of the target scene is collected using multiple cameras with different perspectives and types, and environmental parameter data of the target scene is collected through sensors;

[0006] The acquired video data is preprocessed, and the environmental parameter data is calibrated and normalized.

[0007] Deep learning algorithms are used to extract features from the preprocessed video data to obtain the appearance feature data and motion feature data of the target object, and environmental state feature data is extracted based on the environmental parameter data.

[0008] The appearance feature data and motion feature data are fused with the environmental state feature data to construct a multimodal feature vector;

[0009] The multimodal feature vectors are analyzed based on a pre-trained abnormal event recognition model to identify abnormal events;

[0010] Once an abnormal event is identified, the type, location, and severity of the abnormal event can be obtained by using the collaborative work of multiple cameras and the feature matching and geometric relationship of the target object in different camera images.

[0011] Based on the type, location, and severity of the abnormal event, corresponding early warning information is generated according to the preset early warning strategy, and relevant devices linked with the early warning system are triggered.

[0012] The multi-camera collaborative intelligent early warning method for abnormal events described in this application includes:

[0013] The different types of cameras include imaging devices adapted to low-light scenes and imaging devices adapted to complex lighting scenes.

[0014] The sensor includes a detection device for collecting physical and chemical parameters in the scene.

[0015] In the multi-camera collaborative intelligent early warning method for abnormal events described in this application, the preprocessing operation on the collected video data specifically includes:

[0016] Adaptive denoising based on dynamic scene changes analyzes the pixel variation amplitude between video frames and adjusts the convolution kernel size of the denoising filter according to a preset threshold range.

[0017] Contrast adjustment processing based on target contour enhancement adjusts the grayscale value range of the target contour region by piecewise linear transformation according to the grayscale distribution characteristics of the target contour region, thereby changing the contrast parameter.

[0018] In the multi-camera collaborative intelligent early warning method for abnormal events described in this application, the step of using a deep learning algorithm to extract features from the preprocessed video data includes:

[0019] The spatial attention mechanism generates a spatial weight matrix that matches the size of the video frame, assigning high weights to target object regions to enhance the extraction of key region appearance feature data.

[0020] The temporal attention mechanism calculates the temporal correlation weights of each frame in a video sequence and assigns high weights to frames with significant motion features to highlight the extraction of motion feature data during key time periods.

[0021] In the multi-camera collaborative intelligent early warning method for abnormal events described in this application, the process of acquiring motion feature data of the target object includes:

[0022] The motion state parameters and spatial coordinate data of the target object are collected at preset time intervals to construct a spatiotemporal correlation matrix;

[0023] Matrix operations are used to map and associate the motion state with changes in spatial location at different time points;

[0024] Motion feature data containing continuous time series is generated based on the correlation results.

[0025] In the multi-camera collaborative intelligent early warning method for abnormal events described in this application, the construction of the multimodal feature vector includes:

[0026] Real-time acquisition of the motion intensity of target objects and fluctuation values ​​of environmental parameters in the scene, and input into a preset weight calculation model;

[0027] The weight calculation model outputs the weight values ​​corresponding to each feature data and replaces the original weights to fuse multimodal feature data and construct a multimodal feature vector.

[0028] In the multi-camera collaborative intelligent early warning method for abnormal events described in this application, the step of analyzing the multimodal feature vectors based on a pre-trained abnormal event recognition model to identify abnormal events includes:

[0029] Introduce prior knowledge of the scene, including the inherent properties of objects in the scene and the conventional interaction relationships between objects;

[0030] Construct a scene semantic graph, with scene elements as nodes and relationships between elements as edges. Use the node association rules in the graph to constrain the model's logic for identifying abnormal events and determine whether the events conform to the norm.

[0031] In the multi-camera collaborative intelligent early warning method for abnormal events described in this application, the collaborative operation between the multiple cameras includes:

[0032] Extract the dynamic features of target objects within the overlapping area of ​​the camera's field of view, and achieve dynamic feature association through feature vector similarity calculation to complete feature matching;

[0033] The camera pose data is collected at a preset period to update the extrinsic parameter matrix. The pixel coordinates of the matching features are substituted to complete the geometric relationship calculation, and the type, location, and severity information of the abnormal event are obtained.

[0034] Secondly, this application provides an intelligent early warning system for abnormal events using multi-camera collaboration, characterized in that it includes:

[0035] The data acquisition module is used to acquire video data of the target scene using multiple cameras with different perspectives and different types, and to acquire environmental parameter data of the target scene through sensors;

[0036] The data preprocessing module is used to preprocess the acquired video data and calibrate and normalize the environmental parameter data.

[0037] The feature extraction module is used to extract features from the preprocessed video data using deep learning algorithms, to obtain the appearance feature data and motion feature data of the target object, and to extract environmental state feature data based on the environmental parameter data.

[0038] The feature fusion module is used to fuse the appearance feature data and motion feature data with the environmental state feature data to construct a multimodal feature vector;

[0039] An anomaly detection module is used to analyze the multimodal feature vector based on a pre-trained anomaly event detection model to identify anomaly events;

[0040] The collaborative localization module is used to obtain information about the type, location, and severity of an abnormal event by working together among multiple cameras and using feature matching and geometric relationships of the target object in images from different cameras after an abnormal event is identified.

[0041] The early warning linkage module is used to generate corresponding early warning information according to the type, location, and severity of the abnormal event, and trigger relevant devices linked with the early warning system.

[0042] The system also includes a memory and a processor. The memory contains a program for a multi-camera collaborative intelligent early warning method for abnormal events. When the processor executes the program for the multi-camera collaborative intelligent early warning method for abnormal events, it performs the following steps:

[0043] Video data of the target scene is collected using multiple cameras with different perspectives and types, and environmental parameter data of the target scene is collected through sensors;

[0044] The acquired video data is preprocessed, and the environmental parameter data is calibrated and normalized.

[0045] Deep learning algorithms are used to extract features from the preprocessed video data to obtain the appearance feature data and motion feature data of the target object, and environmental state feature data is extracted based on the environmental parameter data.

[0046] The appearance feature data and motion feature data are fused with the environmental state feature data to construct a multimodal feature vector;

[0047] The multimodal feature vectors are analyzed based on a pre-trained abnormal event recognition model to identify abnormal events;

[0048] Once an abnormal event is identified, the type, location, and severity of the abnormal event can be obtained by using the collaborative work of multiple cameras and the feature matching and geometric relationship of the target object in different camera images.

[0049] Based on the type, location, and severity of the abnormal event, corresponding early warning information is generated according to the preset early warning strategy, and relevant devices linked with the early warning system are triggered.

[0050] As can be seen from the above, this invention addresses the limitations of single-camera perspective, poor adaptability of multimodal fusion, insufficient positioning accuracy, and rigid early warning in existing technologies by providing a multi-camera collaborative intelligent early warning method and system for abnormal events. The method involves collecting video and environmental data through multiple cameras and sensors. After preprocessing, deep learning is used to extract target appearance, motion features, and environmental state features, which are then combined with dynamic weight fusion to construct a multimodal vector. Prior scene knowledge and semantic graphs are introduced to assist in identifying abnormal events. Multi-camera feature matching and geometric operations are used to determine event information. Finally, the early warning strategy is dynamically adjusted based on the event development model. The system includes modules for data acquisition, preprocessing, feature extraction, fusion, recognition, collaborative positioning, and early warning linkage, all of which operate collaboratively. This solution improves recognition accuracy through multi-source collaboration and dynamic fusion, and enhances response flexibility through cross-device positioning and adaptive early warning, making it suitable for intelligent security needs in complex scenarios.

[0051] Other features and advantages of this application will be set forth in the following description and will be apparent in part from the description or may be learned by practicing embodiments of this application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description and the accompanying drawings. Attached Figure Description

[0052] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This is a high-level flowchart of the intelligent early warning method for abnormal events using multi-camera collaboration provided in the embodiments of this application, which is used to identify abnormal events and generate early warning information.

[0054] Figure 2 A flowchart of an intelligent early warning method for abnormal events using multi-camera collaboration provided in an embodiment of this application;

[0055] Figure 3 A flowchart illustrating the preprocessing operation of the collected video data in the multi-camera collaborative intelligent early warning method for abnormal events provided in this application embodiment;

[0056] Figure 4 The flowchart illustrates the process of using a deep learning algorithm to extract features from preprocessed video data in the multi-camera collaborative intelligent early warning method for abnormal events provided in this application embodiment.

[0057] Figure 5 A block diagram of an intelligent early warning system for abnormal events with multi-camera collaboration provided in an embodiment of this application. Detailed Implementation

[0058] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0059] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first" and "second" are used only to distinguish descriptions and should not be construed as indicating or implying relative importance. It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0060] Please refer to Figure 1 , Figure 1 This is a high-level flowchart of an intelligent early warning method for abnormal events using multi-camera collaboration in some embodiments of this application. This high-level flowchart illustrates the complete execution logic of a multi-camera collaborative intelligent early warning method for abnormal events: First, by deploying multiple cameras and sensors with different perspectives and types, video data and environmental parameter data of the target scene are collected simultaneously to achieve comprehensive scene perception. Next, preprocessing operations are performed on the video data to optimize quality, while environmental parameter data is calibrated and normalized to provide standardized data for subsequent analysis. Subsequently, deep learning algorithms are used to extract the appearance and motion features of the target object from the preprocessed video data and environmental state features from the environmental parameter data. Then, the above three types of features are fused to construct a multimodal feature vector, which serves as the input for abnormal event recognition. Based on a pre-trained abnormal event recognition model, this vector is analyzed to complete the identification of the abnormal event. Once an abnormal event is identified, the type, location, and severity of the abnormal event are determined by feature matching and geometric relationship calculation of the target object in images from different cameras, leveraging the collaborative work between multiple cameras. Finally, based on this information, an early warning message is generated according to a preset early warning strategy, and related linkage devices are triggered, forming a closed-loop process from data collection, processing, analysis to early warning response.

[0061] Please refer to Figure 2 , Figure 2 This is a flowchart of an intelligent early warning method for abnormal events using multi-camera collaboration in some embodiments of this application.

[0062] The first aspect of this invention discloses a multi-camera collaborative intelligent early warning method for abnormal events, which is used in terminal devices such as computers and mobile phones. This multi-camera collaborative intelligent early warning method for abnormal events includes the following steps:

[0063] S201. Collect video data of the target scene using multiple cameras with different perspectives and types, and collect environmental parameter data of the target scene through sensors;

[0064] S202. Perform preprocessing on the acquired video data, and simultaneously calibrate and normalize the environmental parameter data.

[0065] S203. Use deep learning algorithms to extract features from the preprocessed video data, obtain the appearance feature data and motion feature data of the target object, and extract environmental state feature data based on the environmental parameter data.

[0066] S204. The appearance feature data and motion feature data are fused with the environmental state feature data to construct a multimodal feature vector;

[0067] S205. Analyze the multimodal feature vectors based on the pre-trained abnormal event recognition model to identify abnormal events;

[0068] S206. After an abnormal event is identified, through the collaborative work of multiple cameras, the feature matching and geometric relationship of the target object in different camera images are used to obtain information on the type, location and severity of the abnormal event.

[0069] S207. Based on the type, location, and severity information of the abnormal event, generate corresponding early warning information according to the preset early warning strategy and trigger relevant devices linked with the early warning system.

[0070] The intelligent early warning process for abnormal events in this technical solution begins with the data acquisition stage. Multiple cameras with different perspectives and adapted to diverse scenarios (including imaging devices adapted to low-light scenarios and those adapted to complex lighting scenarios) are deployed to collect video data of the target scene. Simultaneously, environmental parameter data is acquired using sensors capable of collecting physical and chemical parameters, achieving comprehensive and multi-dimensional perception of the scene. Subsequently, the data preprocessing stage is entered, where adaptive denoising based on dynamic scene changes (adjusting the convolution kernel size of the denoising filter according to a preset threshold range by analyzing the pixel variation amplitude between video frames) and target contour-based denoising are performed on the video data. Enhanced contrast adjustment processing (adjusting the grayscale range of the target contour region through piecewise linear transformation based on its grayscale distribution characteristics to optimize contrast) is performed. Simultaneously, environmental parameter data is calibrated and normalized to eliminate data noise and dimensional differences, providing a high-quality data foundation for subsequent processing. The feature extraction stage employs deep learning algorithms, using a spatial attention mechanism to generate a spatial weight matrix matching the video frame size to enhance the extraction of appearance features of the target object region. A temporal attention mechanism is used to calculate the temporal correlation weights of the video sequence to highlight motion feature extraction during key periods. Environmental state features are also extracted from the environmental parameter data, and these features are integrated into the motion feature acquisition process. By collecting motion state parameters and spatial coordinate data at preset time intervals, constructing a spatiotemporal correlation matrix, and mapping and associating motion and position changes at different time points through matrix operations, motion feature data containing continuous time series is generated, ensuring the accuracy and timeliness of feature extraction. In the feature fusion stage, the motion intensity of target objects and environmental parameter fluctuations in the scene are collected in real time, and the dynamic weight values ​​of each feature data are obtained by inputting them into a preset weight calculation model. The dynamic weight values ​​of each feature data are then used to replace the original weights, fusing appearance features, motion features, and environmental state features to construct a multimodal feature vector adapted to scene changes. During the abnormal event recognition process, the multimodal feature vector is divided based on the pre-trained model. The system analyzes and introduces prior knowledge of the scene (including the inherent properties of objects in the scene and the conventional interaction relationships between objects) and constructs a scene semantic graph (using scene elements as nodes and relationships between elements as edges). The model's recognition logic is constrained by the node association rules in the graph, improving the accuracy of abnormal event recognition. After an abnormal event is identified, the system uses multi-camera collaboration to extract the dynamic features of the target object within the overlapping field of view and completes feature matching through feature vector similarity calculation. At the same time, the system collects camera pose data at a preset period to update the extrinsic parameter matrix, substitutes the pixel coordinates of the matching features to complete geometric relationship calculations, and accurately obtains information on the type, location, and severity of the abnormal event.Ultimately, based on the above information, an event development model is constructed (based on continuously collected data on changes in environmental parameters surrounding the abnormal event and the trajectory data of the target object) to output parameters for the spread rate and impact range. The warning information generation interval and the activation level of the linked devices are adjusted according to a preset warning strategy to generate corresponding warning information and trigger the linked devices, forming a complete intelligent early warning closed loop from perception to response.

[0071] According to an embodiment of the present invention, it includes:

[0072] The different types of cameras include imaging devices adapted to low-light scenes and imaging devices adapted to complex lighting scenes.

[0073] The sensor includes a detection device for collecting physical and chemical parameters in the scene.

[0074] In the data acquisition phase, this solution employs various types of cameras, including imaging devices adapted to low-light scenarios (such as infrared cameras and starlight cameras, which can clearly capture target details in dimly lit environments such as nighttime and tunnels, avoiding image blurring or feature loss due to insufficient light) and imaging devices adapted to complex lighting scenarios (such as wide dynamic range cameras and anti-backlight cameras, which can effectively handle light interference in scenarios with strong direct sunlight and severe contrast between light and dark, ensuring that highlights are not overexposed and details in shadow areas are discernible). This combination of multiple device types enables comprehensive and clear acquisition of target scene video data under different lighting conditions. Simultaneously, the sensors used include those for scene acquisition. The system combines physical parameter detection equipment (such as temperature sensors, humidity sensors, and vibration sensors, which acquire physical characteristic data such as ambient temperature, humidity, and vibration amplitude) with chemical parameter detection equipment (such as gas sensors and smoke sensors, which monitor chemical characteristic data such as specific gas concentrations and smoke particle content in the environment). Through the coordinated acquisition of physical and chemical parameters, it provides multi-dimensional environmental data support for subsequent environmental state feature extraction and abnormal event identification. This complements various types of cameras, jointly constructing a comprehensive data acquisition system covering both visual and environmental information. This ensures the integrity and effectiveness of data acquisition in complex scenarios, laying the foundation for the accuracy of the entire early warning process.

[0075] Please refer to Figure 3 , Figure 3 This is a flowchart illustrating the preprocessing operation of the acquired video data in a multi-camera collaborative intelligent early warning method for abnormal events, as described in some embodiments of this application. According to embodiments of the present invention, the preprocessing operation of the acquired video data specifically includes:

[0076] S301. Adaptive denoising processing based on dynamic scene changes analyzes the pixel change amplitude between video frames and adjusts the convolution kernel size of the denoising filter according to a preset threshold range.

[0077] S302. Contrast adjustment processing based on target contour enhancement: According to the gray-scale distribution characteristics of the target contour region, the gray-scale value range of the region is adjusted by piecewise linear transformation to change the contrast parameter.

[0078] The preprocessing of the acquired video data is a crucial step in providing high-quality data for subsequent feature extraction. This includes two core processes: First, adaptive denoising based on dynamic scene changes. This involves analyzing the pixel variation amplitude between adjacent frames in the video sequence in real time to determine the motion intensity of dynamic elements in the scene. When the pixel variation amplitude falls within different preset threshold ranges, the convolution kernel size of the denoising filter is adjusted accordingly (for example, a smaller kernel is used when there are fewer dynamic elements and the pixel changes are gradual to retain more details, while a larger kernel is used when there are dense dynamic elements and drastic pixel changes to enhance the denoising effect). The first step is to effectively remove noise while avoiding feature loss due to over-smoothing. The second step is to perform contrast adjustment based on target contour enhancement. First, the target contour region in the video frame is identified through edge detection algorithm. Then, the gray-level distribution characteristics of the pixels in this region (such as gray-level mean and variance) are analyzed. Subsequently, piecewise linear transformation is used to adjust the gray-level range of this region (for example, for contour regions with gray-level values ​​concentrated in the low range, the gray-level distribution range is expanded to improve brightness contrast; for regions with dispersed gray-level values, non-critical gray-level ranges are compressed to strengthen contour edges). By changing the contrast parameters, the target contour becomes clearer and more distinguishable in complex backgrounds. The above preprocessing operations are connected with the subsequent feature extraction stage based on deep learning. The video data after denoising and contrast optimization can more accurately support the spatial attention mechanism for extracting the appearance features of key regions and the temporal attention mechanism for capturing the motion features of key time periods, ensuring the coherence and effectiveness of the data processing chain in the entire technical solution.

[0079] Please refer to Figure 4 , Figure 4 This is a flowchart illustrating the use of a deep learning algorithm to extract features from preprocessed video data in a multi-camera collaborative intelligent early warning method for abnormal events, as described in some embodiments of this application. According to embodiments of the present invention, the use of a deep learning algorithm to extract features from the preprocessed video data includes:

[0080] S401. Spatial attention mechanism generates a spatial weight matrix that matches the size of the video frame, and assigns high weights to the target object region to enhance the extraction of key region appearance feature data.

[0081] S402. The temporal attention mechanism calculates the temporal correlation weights of each frame in the video sequence and assigns high weights to frames with significant motion features to highlight the extraction of motion feature data in key time periods.

[0082] In this approach, when using deep learning algorithms to extract features from preprocessed video data, the scheme achieves accurate capture of key features of target objects through the synergistic effect of spatial and temporal attention mechanisms. The spatial attention mechanism targets the spatial dimension of video frames, generating a spatial weight matrix that perfectly matches the video frame size via a neural network. This matrix dynamically assigns weights based on the correlation between pixels and target objects—giving significantly higher weights to the areas containing target objects (such as human bodies, vehicles, and other core monitoring objects) than to background areas. This enhances the extraction accuracy of key area appearance feature data (including static attributes such as shape, color, and texture), providing a more recognizable appearance feature foundation for subsequent multimodal feature fusion. The temporal attention mechanism focuses on the temporal dimension of the video sequence. By calculating the temporal correlation weights between adjacent frames and multiple frames (such as temporal modeling methods based on LSTM or Transformer), it identifies frames with significant motion features (such as key frames of rapid object movement or sudden attitude changes) and assigns higher weights to these frames. This highlights the extraction of key time-period motion feature data (including dynamic attributes such as speed, acceleration, and trajectory changes), complementing the spatial attention mechanism. The aforementioned feature extraction method is compatible with the preprocessed video data (after adaptive denoising and contrast adjustment). The extracted appearance and motion features can directly support the construction of multimodal feature vectors and form a technical connection with the subsequent process of generating continuous time series motion features through the spatiotemporal correlation matrix. This ensures that the feature extraction stage focuses on key information while retaining complete temporal features, providing comprehensive and accurate input data for the abnormal event recognition model. This aligns with the core logic of "multi-dimensional feature fusion to improve recognition accuracy" in the overall technical solution.

[0083] According to an embodiment of the present invention, acquiring motion feature data of a target object includes:

[0084] The motion state parameters and spatial coordinate data of the target object are collected at preset time intervals to construct a spatiotemporal correlation matrix;

[0085] Matrix operations are used to map and associate the motion state with changes in spatial location at different time points;

[0086] Motion feature data containing continuous time series is generated based on the correlation results.

[0087] In acquiring motion feature data of the target object, this solution achieves accurate extraction of dynamic features through temporal data acquisition and spatial correlation analysis: First, motion state parameters (including velocity, acceleration, and direction of motion) and spatial coordinate data (based on camera imaging plane coordinates or calibrated 3D world coordinates) of the target object are collected at preset time intervals (which can be flexibly set according to the complexity of the scene, such as shorter intervals in high-density crowd scenes). These multi-source data across time dimensions are integrated to construct a spatiotemporal correlation matrix. The matrix's row dimensions correspond to time nodes, and the column dimensions cover motion state and spatial position parameters, achieving structured data storage. Subsequently, a mapping relationship between different time nodes is established through matrix operations (such as matrix multiplication and eigenvalue decomposition), quantifying the correlation between changes in motion state parameters and displacement in spatial position. For example, by calculating the correlation between the difference in state parameters and the difference in coordinates between adjacent time nodes, the continuity and abrupt changes of motion are captured. Finally, motion feature data containing continuous time series is generated based on the above correlation results. This data not only retains the instantaneous motion features of each time node but also reflects the motion trend (such as dynamic patterns such as uniform speed, acceleration, and turning) through temporal correlation. This process is integrated with the preprocessed video data. The generated motion feature data can be directly used as input for the temporal attention mechanism (highlighting key time period features through temporal weights). At the same time, it provides a dynamic feature foundation for multimodal feature fusion, ensuring the consistency of the technical logic between motion feature extraction and subsequent feature processing and anomaly recognition, and improving the accuracy of characterizing the motion patterns of target objects.

[0088] According to an embodiment of the present invention, constructing a multimodal feature vector includes:

[0089] Real-time acquisition of the motion intensity of target objects and fluctuation values ​​of environmental parameters in the scene, and input into a preset weight calculation model;

[0090] The weight calculation model outputs the weight values ​​corresponding to each feature data and replaces the original weights to fuse multimodal feature data and construct a multimodal feature vector.

[0091] In constructing the multimodal feature vector, this scheme achieves efficient feature fusion through dynamic weight adjustment: Real-time acquisition of the motion intensity of target objects in the scene (such as the rate of change of motion speed, displacement amplitude, and other parameters reflecting the intensity of motion) and environmental parameter fluctuations (such as changes in environmental indicators like temperature, humidity, and specific gas concentrations) is used. This real-time data is input into a pre-defined weight calculation model (which can be built based on neural networks or statistical learning algorithms and has dynamic adaptability through training with historical data). Based on the input motion intensity and environmental parameter fluctuations, the model calculates and outputs the corresponding weight values ​​for appearance feature data, motion feature data, and environmental state feature data, replacing the initially set fixed weights. Through weighted fusion, the three types of feature data are integrated to construct a multimodal feature vector that can adapt to dynamic changes in the scene. During this process, when the intensity of the target object's motion increases or environmental parameters fluctuate drastically, the model will correspondingly increase the weight ratio of motion features or environmental state features to ensure that the multimodal feature vector can accurately reflect the key information of the scene. This forms a technical connection with the aforementioned feature extraction stage (extraction of appearance, motion features, and environmental state features by spatial and temporal attention mechanisms), providing more targeted input data for subsequent abnormal event recognition models. At the same time, it conforms to the technical logic of "improving recognition accuracy through multimodal fusion" in the overall solution, avoiding the problem of insufficient adaptability of fixed weight fusion in complex scenarios.

[0092] According to an embodiment of the present invention, the step of analyzing the multimodal feature vector based on a pre-trained anomaly event recognition model to identify anomaly events includes:

[0093] Introduce prior knowledge of the scene, including the inherent properties of objects in the scene and the conventional interaction relationships between objects;

[0094] Construct a scene semantic graph, with scene elements as nodes and relationships between elements as edges. Use the node association rules in the graph to constrain the model's logic for identifying abnormal events and determine whether the events conform to the norm.

[0095] In the process of analyzing multimodal feature vectors based on a pre-trained abnormal event recognition model to identify abnormal events, this scheme strengthens the recognition logic by introducing prior scene knowledge. This prior scene knowledge includes the inherent attributes of various objects in the scene (such as the normal driving speed range of vehicles, the normal activity area of ​​pedestrians, etc.) and the normal interaction relationships between objects (such as the correspondence between pedestrians and zebra crossings, the association rules between fire-fighting facilities and fire sources, etc.). Simultaneously, a scene semantic graph is constructed based on this prior knowledge. Elements in the scene (such as people, vehicles, facilities, areas, etc.) are used as graph nodes, and logical relationships between elements (such as positional relationships, interaction rules, etc.) are used as edges between nodes. The recognition logic of the model is constrained by preset node association rules in the graph (such as "pedestrians should not remain in the motor vehicle lane" and "high-temperature areas are abnormally close to flammable materials"). When analyzing multimodal feature vectors, the model compares the extracted feature information with the association rules in the semantic graph to determine whether the current scene state conforms to normal logic, thereby accurately identifying abnormal events that deviate from the normal pattern. This process is technically connected with the aforementioned constructed multimodal feature vector (integrating appearance, motion, and environmental features). By introducing prior knowledge and semantic graphs, it makes up for the blind spots in the recognition of complex scenes by relying solely on data-driven models, improves the robustness of abnormal event recognition, and provides a clear recognition result basis for subsequent multi-camera collaborative positioning. This is in line with the core logic of "multi-dimensional information fusion to improve the accuracy of early warning" in the overall technical solution.

[0096] According to an embodiment of the present invention, the collaborative operation among the multiple cameras includes:

[0097] Extract the dynamic features of target objects within the overlapping area of ​​the camera's field of view, and achieve dynamic feature association through feature vector similarity calculation to complete feature matching;

[0098] The camera pose data is collected at a preset period to update the extrinsic parameter matrix. The pixel coordinates of the matching features are substituted to complete the geometric relationship calculation, and the type, location, and severity information of the abnormal event are obtained.

[0099] In the collaborative operation of multiple cameras, this solution achieves accurate location and information extraction of abnormal events through dynamic feature matching and geometric relationship calculation. Specifically, firstly, for the overlapping areas of the fields of view of each camera, the dynamic features of the target object within these areas are extracted (including key time-period motion features highlighted by the time attention mechanism and continuous time-series motion features). By calculating the similarity between feature vectors collected by different cameras (such as cosine similarity or Euclidean distance), dynamic feature association across devices is achieved, completing feature matching of the target object in multi-camera images and ensuring consistent recognition of the same target from different perspectives. Simultaneously, attitude data (including pitch angle, yaw angle, roll angle, etc.) of each camera is collected at preset intervals (such as 1-5 minutes based on scene stability). Based on this data, the extrinsic parameter matrix of the camera (used to describe the position and orientation of the camera in three-dimensional space) is updated in real time. The pixel coordinates of the target object obtained from the aforementioned feature matching are substituted into the updated extrinsic parameter matrix. Through geometric operations such as perspective transformation and triangulation, the two-dimensional image coordinates are converted into three-dimensional world coordinates. Finally, by combining the feature matching results and geometric operation data, the type of abnormal event (such as crowd gathering, vehicle collision, etc.), specific spatial location (accurate to the meter or sub-meter level), and severity information (such as quantitative indicators such as the number of people gathered and the force of the collision) are accurately determined. This collaborative process is linked to the identification results output by the abnormal event identification stage. It leverages the field-of-view coverage advantage of multiple cameras to compensate for the limitations of a single device's perspective. At the same time, it ensures positioning accuracy even under slight camera displacement (such as vibration or loose installation) through dynamic updates of the extrinsic parameter matrix. This provides accurate event information support for the execution of subsequent early warning strategies, which aligns with the core logic of the overall technical solution: "multi-device collaboration enhances adaptability to complex scenarios."

[0100] According to an embodiment of the present invention, the preset early warning strategy includes:

[0101] By continuously collecting data on changes in environmental parameters around abnormal events and the trajectory data of target objects, an event development model is constructed.

[0102] The event development model is used to analyze the spread trend of abnormal events and output parameters such as spread speed and impact range.

[0103] Based on the diffusion speed and impact range parameters, the generation interval of the early warning information is adjusted, and the activation level of the linkage equipment is adjusted accordingly.

[0104] This solution achieves graded response and precise handling of abnormal events by constructing a dynamic event development model: continuously collecting environmental parameter change data (such as temperature gradient, smoke concentration change rate, and physicochemical indicators such as the diffusion rate of harmful gases) and target object trajectory data (such as dynamic characteristics such as crowd flow direction, speed change points, and density changes) around the abnormal event, and inputting them into the event development model based on spatiotemporal sequence analysis; this model, by integrating the changing trends of multi-source data, uses Kalman filtering to predict the event diffusion path, and combines a fluid dynamics model to calculate the boundary of the impact range, outputting quantitative analysis results including diffusion speed (such as the speed of fire spread and the speed of the spread of panic among the crowd) and impact range parameters (such as the radius of the danger zone and the boundary of pollution diffusion); based on these parameters, the generation interval of early warning information is dynamically adjusted (such as shortening it to the second-level update when the diffusion speed increases), and the activation level of the linkage equipment is adjusted accordingly (such as upgrading from a primary audible and visual alarm to a full-area emergency broadcast, or expanding from a local sprinkler system to overall fire linkage). This process seamlessly integrates with the location and severity information of abnormal events output by the multi-camera collaborative positioning stage, forming a closed-loop mechanism of "monitoring-analysis-response". This ensures that the early warning strategy not only conforms to the objective laws of event development, but also dynamically optimizes resource allocation based on the real-time situation, effectively improving the timeliness and pertinence of emergency response in complex scenarios, which is in line with the "intelligent and adaptive" design concept of the overall technical solution.

[0105] According to an embodiment of the present invention, it further includes:

[0106] An edge-central federated learning architecture is constructed, where each edge node trains a sub-model based on local data, and the gradient is encrypted and uploaded to the central end after differential privacy processing.

[0107] The central node uses blockchain to verify the legality of gradients, weighted aggregates to generate a global model and feeds it back to the edge nodes. The global model dynamically adjusts the feature weights according to regional characteristics.

[0108] Construct a digital twin space that maps to the physical scene, and synchronize abnormal event information and emergency resource data;

[0109] The event propagation trend is simulated by a physics engine, and a decision scheme including equipment start-up priority and personnel scheduling path is generated by a genetic algorithm and output to the linked equipment.

[0110] This solution also constructs an edge-central federated learning architecture. Each edge node trains an anomaly recognition sub-model based on video data collected by local multi-cameras, sensor environmental parameters, and historical anomaly event data. During training, differential privacy technology is used to process the local model's gradient parameters to remove noise, and the processed gradient parameters are uploaded to the central node using encrypted transmission. After receiving the gradient parameters from each edge node, the central node uses blockchain technology to verify the source and integrity of the parameters. A weighted aggregation algorithm is then used to fuse the verified gradient parameters to generate a globally optimized model, which is then fed back to each edge node to update its local sub-model. Simultaneously, a feature weight dynamic adjustment module is included in the global model, which can adjust the feature weights according to the scene characteristics of different areas (such as population density, activity scale, etc.). The model adaptively adjusts the weights of appearance features, motion features, and environmental state features in its calculations, taking into account factors such as the law of nature and the degree of environmental interference. Simultaneously, a digital twin space, mapped 1:1 to the physical monitoring scene, is constructed within the abnormal event early warning process. This is achieved by real-time synchronization of abnormal event types and location information obtained from multi-camera collaborative positioning, along with environmental parameter changes and real-time status data of emergency resources (such as fire-fighting equipment and security personnel) collected by sensors. Within the digital twin space, a physics engine simulates the diffusion trend and potential chain reactions of abnormal events under different environmental variables. A genetic algorithm is then used to process the simulation results, generating a decision-making scheme that includes the priority of emergency equipment activation and the optimal path for security personnel dispatch. This decision-making scheme, along with the early warning information generated by the original early warning strategy, is then collaboratively output to the linked equipment. These steps complement the multi-camera data acquisition, feature extraction, abnormal event identification, and early warning linkage in the original solution. The federated learning mechanism enhances the model's adaptability to complex cross-regional scenarios, while the extrapolation capabilities of the digital twin strengthen the scientific rigor and accuracy of emergency response, all without altering the core technical logic of the original solution.

[0111] Please refer to Figure 5 , Figure 5 This is a block diagram of an intelligent early warning system for abnormal events with multi-camera collaboration provided in an embodiment of this application.

[0112] The second aspect of this invention also discloses an intelligent early warning system for abnormal events using multi-camera collaboration, comprising:

[0113] The data acquisition module 501 is used to acquire video data of the target scene using multiple cameras with different perspectives and different types, and to acquire environmental parameter data of the target scene through sensors;

[0114] The data preprocessing module 502 is used to preprocess the acquired video data and calibrate and normalize the environmental parameter data.

[0115] The feature extraction module 503 is used to extract features from the preprocessed video data using a deep learning algorithm, to obtain the appearance feature data and motion feature data of the target object, and to extract environmental state feature data based on the environmental parameter data.

[0116] The feature fusion module 504 is used to fuse the appearance feature data and motion feature data with the environmental state feature data to construct a multimodal feature vector;

[0117] Anomaly identification module 505 is used to analyze the multimodal feature vector based on a pre-trained anomaly event identification model to identify anomaly events;

[0118] The collaborative localization module 506 is used to obtain information on the type, location, and severity of the abnormal event by using the feature matching and geometric relationship of the target object in different camera images through the collaborative work of multiple cameras after an abnormal event is identified.

[0119] The early warning linkage module 507 is used to generate corresponding early warning information according to the type, location, and severity information of the abnormal event, and trigger relevant devices linked with the early warning system.

[0120] The system also includes a memory and a processor. The memory includes a program for a multi-camera collaborative intelligent early warning method for abnormal events. When the program for the multi-camera collaborative intelligent early warning method for abnormal events is executed by the processor, it implements the steps of the multi-camera collaborative intelligent early warning method for abnormal events as described in any one of the first aspects.

[0121] This invention discloses a multi-camera collaborative intelligent early warning method and system for abnormal events. By constructing a full-chain technology system of "multi-source perception - intelligent processing - accurate identification - collaborative positioning - dynamic response," it effectively solves the problems of limited single-device perspective, poor adaptability of feature fusion, insufficient event positioning accuracy, and rigid early warning strategies in traditional monitoring. Specifically, the solution first deploys multiple types of cameras adapted to low-light and complex lighting scenarios, combined with sensors that collect physical and chemical parameters, to achieve comprehensive capture of video and environmental data of the target scene. The video data undergoes adaptive denoising based on inter-frame pixel changes (dynamically adjusting the convolution kernel size) and target contour enhancement preprocessing based on grayscale transformation. Simultaneously, environmental parameters are calibrated and normalized to provide a high-quality data foundation for subsequent processing. The feature extraction stage incorporates deep learning's spatial and temporal attention mechanisms to enhance the extraction of appearance features of the target area and motion features during key time periods, respectively. A spatiotemporal correlation matrix is ​​constructed to generate continuous time-series motion features, ensuring the integrity of the feature dimensions. In the fusion stage, based on real-time collected target motion intensity and environmental parameter fluctuations, a dynamic weighting model is used to output dynamic weights, replacing fixed weights to fuse appearance, motion, and environmental features, constructing a multimodal feature vector adapted to scene changes. For anomaly recognition, prior scene knowledge (inherent object attributes and conventional relationships) is introduced, and a semantic graph is constructed. Graph node association rules constrain the model logic, improving recognition accuracy. After identifying an abnormal event, dynamic feature matching of overlapping areas of multiple camera fields of view (based on feature vector similarity calculation) is used, combined with geometric calculations of the extrinsic parameter matrix updated by camera pose data, to accurately locate the event type, location, and severity. Finally, based on the event development model (fusing environmental parameter changes and target trajectory data), diffusion trend parameters are output, dynamically adjusting the warning interval and the activation level of linked equipment, forming a closed loop from perception to response. The system achieves the implementation of technical solutions through the collaborative work of data acquisition, preprocessing, feature extraction, fusion, identification, collaborative positioning, and early warning linkage modules. It significantly improves the accuracy of abnormal event identification, the precision of positioning, and the timeliness of early warning in complex scenarios, providing an efficient solution for intelligent security in complex scenarios such as smart cities, industrial parks, and transportation hubs.

[0122] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0123] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0124] In addition, in the various embodiments of the present invention, each functional unit can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0125] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory, random access memory, magnetic disks, or optical disks.

[0126] Alternatively, if the integrated units of this invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

Claims

1. A method for intelligent early warning of abnormal events using multi-camera collaboration, characterized in that, Includes the following steps: Video data of the target scene is collected using multiple cameras with different perspectives and types, and environmental parameter data of the target scene is collected through sensors; The acquired video data is preprocessed, and the environmental parameter data is calibrated and normalized. Deep learning algorithms are used to extract features from the preprocessed video data to obtain the appearance feature data and motion feature data of the target object, and environmental state feature data is extracted based on the environmental parameter data. The appearance feature data and motion feature data are fused with the environmental state feature data to construct a multimodal feature vector; The multimodal feature vectors are analyzed based on a pre-trained abnormal event recognition model to identify abnormal events; Once an abnormal event is identified, the type, location, and severity of the abnormal event can be obtained by using the collaborative work of multiple cameras and the feature matching and geometric relationship of the target object in different camera images. Based on the type, location, and severity of the abnormal event, corresponding early warning information is generated according to the preset early warning strategy, and relevant devices linked with the early warning system are triggered.

2. The intelligent early warning method for abnormal events using multi-camera collaboration according to claim 1, characterized in that, include: The different types of cameras include imaging devices adapted to low-light scenes and imaging devices adapted to complex lighting scenes. The sensor includes a detection device for collecting physical and chemical parameters in the scene.

3. The intelligent early warning method for abnormal events using multi-camera collaboration according to claim 1, characterized in that, The preprocessing operation on the acquired video data specifically includes: Adaptive denoising based on dynamic scene changes analyzes the pixel variation amplitude between video frames and adjusts the convolution kernel size of the denoising filter according to a preset threshold range. Contrast adjustment processing based on target contour enhancement adjusts the grayscale value range of the target contour region by piecewise linear transformation according to the grayscale distribution characteristics of the target contour region, thereby changing the contrast parameter.

4. The intelligent early warning method for abnormal events using multi-camera collaboration according to claim 1, characterized in that, The step of using deep learning algorithms to extract features from preprocessed video data includes: The spatial attention mechanism generates a spatial weight matrix that matches the size of the video frame, assigning high weights to the target object region to enhance the extraction of key region appearance feature data. The temporal attention mechanism calculates the temporal correlation weights of each frame in a video sequence and assigns high weights to frames with significant motion features to highlight the extraction of motion feature data during key time periods.

5. The intelligent early warning method for abnormal events using multi-camera collaboration according to claim 1, characterized in that, When acquiring motion feature data of a target object, the following are included: The motion state parameters and spatial coordinate data of the target object are collected at preset time intervals to construct a spatiotemporal correlation matrix; Matrix operations are used to map and associate the motion state with changes in spatial location at different time points; Motion feature data containing continuous time series is generated based on the correlation results.

6. The intelligent early warning method for abnormal events using multi-camera collaboration according to claim 1, characterized in that, The construction of the multimodal feature vector includes: Real-time acquisition of the motion intensity of target objects and fluctuation values ​​of environmental parameters in the scene, and input into a preset weight calculation model; The weight calculation model outputs the weight values ​​corresponding to each feature data and replaces the original weights to fuse multimodal feature data and construct a multimodal feature vector.

7. The intelligent early warning method for abnormal events using multi-camera collaboration according to claim 1, characterized in that, The analysis of the multimodal feature vectors based on the pre-trained abnormal event recognition model to identify abnormal events includes: Introduce prior knowledge of the scene, including the inherent properties of objects in the scene and the conventional interaction relationships between objects; Construct a scene semantic graph, with scene elements as nodes and relationships between elements as edges. Use the node association rules in the graph to constrain the model's logic for identifying abnormal events and determine whether the events conform to the norm.

8. The intelligent early warning method for abnormal events using multi-camera collaboration according to claim 1, characterized in that, When the multiple cameras work together, it includes: Extract the dynamic features of target objects within the overlapping area of ​​the camera's field of view, and achieve dynamic feature association through feature vector similarity calculation to complete feature matching; The camera pose data is collected at a preset period to update the extrinsic parameter matrix. The pixel coordinates of the matching features are substituted to complete the geometric relationship calculation, and the type, location, and severity information of the abnormal event are obtained.

9. An intelligent early warning system for abnormal events using multi-camera collaboration, characterized in that: include: The data acquisition module is used to acquire video data of the target scene using multiple cameras with different perspectives and different types, and to acquire environmental parameter data of the target scene through sensors; The data preprocessing module is used to preprocess the acquired video data and calibrate and normalize the environmental parameter data. The feature extraction module is used to extract features from the preprocessed video data using deep learning algorithms, to obtain the appearance feature data and motion feature data of the target object, and to extract environmental state feature data based on the environmental parameter data. The feature fusion module is used to fuse the appearance feature data and motion feature data with the environmental state feature data to construct a multimodal feature vector; An anomaly detection module is used to analyze the multimodal feature vector based on a pre-trained anomaly event detection model to identify anomaly events; The collaborative localization module is used to obtain information about the type, location, and severity of an abnormal event by working together among multiple cameras and using feature matching and geometric relationships of the target object in images from different cameras after an abnormal event is identified. The early warning linkage module is used to generate corresponding early warning information according to the type, location, and severity of the abnormal event, and trigger relevant devices linked with the early warning system.

10. An intelligent early warning system for abnormal events using multi-camera collaboration, characterized in that: The system also includes a memory and a processor. The memory contains a program for a multi-camera collaborative intelligent early warning method for abnormal events. When the processor executes the program for the multi-camera collaborative intelligent early warning method for abnormal events, it performs the following steps: Video data of the target scene is collected using multiple cameras with different perspectives and types, and environmental parameter data of the target scene is collected through sensors; The acquired video data is preprocessed, and the environmental parameter data is calibrated and normalized. Deep learning algorithms are used to extract features from the preprocessed video data to obtain the appearance feature data and motion feature data of the target object, and environmental state feature data is extracted based on the environmental parameter data. The appearance feature data and motion feature data are fused with the environmental state feature data to construct a multimodal feature vector; The multimodal feature vectors are analyzed based on a pre-trained abnormal event recognition model to identify abnormal events; Once an abnormal event is identified, the type, location, and severity of the abnormal event can be obtained by using the collaborative work of multiple cameras and the feature matching and geometric relationship of the target object in different camera images. Based on the type, location, and severity of the abnormal event, corresponding early warning information is generated according to the preset early warning strategy, and relevant devices linked with the early warning system are triggered.

Images