A multi-camera surveillance data fusion and high-definition transmission system

Abstract

This invention discloses a multi-camera monitoring data fusion and high-definition transmission system, relating to the fields of video surveillance and data transmission technology. It includes a multi-camera data acquisition module, a data preprocessing module, a data fusion module, an encoding and transmission module, and a receiving end processing module, all connected in sequence. A control and scheduling module is also bidirectionally connected to these modules. This invention solves the problems of low data fusion accuracy and poor high-definition transmission stability in existing multi-camera monitoring systems, providing a system capable of accurate multi-modal data fusion and stable high-definition data transmission in weak network environments, greatly improving the anomaly detection accuracy and real-time performance of the monitoring system.

Description

Technical Field

[0001] This invention relates to the field of video surveillance and data transmission technology, and more specifically to a multi-camera surveillance data fusion and high-definition transmission system. Background Technology

[0002] With the increasing demand for security monitoring, multi-camera collaborative monitoring systems have been widely applied in smart cities, intelligent transportation, park security, and other fields. However, existing multi-camera monitoring systems still face many technical challenges. Some existing solutions focus on reducing data transmission volume by improving video encoding algorithms, but fail to fully consider the spatiotemporal correlation and collaborative processing requirements between data from multiple cameras, resulting in low effective utilization of monitoring information. Other solutions attempt to introduce data fusion technologies (such as attention mechanisms) to improve information integration capabilities, but their high model complexity makes it difficult to meet the stringent real-time requirements of monitoring scenarios. Furthermore, the fusion module and subsequent encoding and transmission stages are often designed independently, lacking system-level linkage optimization, resulting in poor transmission stability of high-definition video streams and limited overall system performance in weak network environments with fluctuating bandwidth.

[0003] Therefore, proposing a multi-camera surveillance data fusion and high-definition transmission system to solve the difficulties of existing technologies is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0004] In view of this, the present invention provides a multi-camera monitoring data fusion and high-definition transmission system, which solves the problems of low data fusion accuracy and poor high-definition transmission stability in existing multi-camera monitoring systems.

[0005] To achieve the above objectives, the present invention provides the following technical solution: A multi-camera surveillance data fusion and high-definition transmission system includes a multi-camera data acquisition module, a data preprocessing module, a data fusion module, an encoding and transmission module, and a receiving end processing module, which are sequentially and communicatively connected. A control and scheduling module is also bidirectionally and communicatively connected to the multi-camera data acquisition module, data preprocessing module, data fusion module, encoding and transmission module, and receiving end processing module. The multi-camera data acquisition module includes multiple cameras deployed in the area to be monitored, used to acquire raw video data streams and obtain acquired data; The data preprocessing module is used to preprocess the collected data to obtain preprocessed data; The data fusion module uses a fusion algorithm that combines cross-modal attention mechanism and residual fusion strategy to fuse the preprocessed data, output the fused data, and retain the original data backup for anomaly tracing. The encoding and transmission module enables data transmission through the synergistic effect of encoding and transmission. The receiver processing module is used for data decoding, error recovery, and image quality enhancement. The control and scheduling module is used to monitor the operating status of each module in real time, and to detect and alarm anomalies in the monitored scenario.

[0006] Optionally, the multi-camera data acquisition module has the ability to integrate heterogeneous imaging devices, and the types of cameras that can be connected include visible light cameras, infrared thermal imaging cameras, and depth sensing cameras.

[0007] Optionally, the data preprocessing module includes a timestamp alignment unit, an image registration unit, a filtering and denoising unit, and a region segmentation unit connected in sequence; Timestamp alignment unit, used to achieve time synchronization of multiple video streams; Image registration unit, used to achieve spatial alignment; A filtering and denoising unit is used to improve the signal-to-noise ratio; The region segmentation unit is used to perform preliminary analysis on the input video to delineate the region of interest, thereby enabling adaptive preprocessing.

[0008] Optionally, the data fusion module includes a cross-modal attention mechanism unit, a residual fusion unit, and an anomaly tracing unit connected in sequence; A cross-modal attention mechanism unit is used to calculate the feature importance among data from different cameras; The residual fusion unit is used to preserve the original detailed information and superimpose it with the attention-weighted result; The anomaly tracing unit backs up the original data for anomaly tracing.

[0009] Optionally, the encoding and transmission module includes a compression encoding unit and a transmission control unit connected in sequence; The compression coding unit is used to compress the fused video data; The transmission control unit is used to monitor network bandwidth, latency, and packet loss rate in real time and adjust encoding parameters.

[0010] Optionally, the receiving end processing module includes a data decoding unit, an error recovery unit, an image quality enhancement unit, a real-time behavior analysis unit, and an abnormal event detection unit connected in sequence; The data decoding unit is used to decode the received compressed bitstream and restore the video frames; An error recovery unit is used to detect and repair errors; The image enhancement unit is used to process the decoded and reconstructed video frames and output high-definition video; The real-time behavior analysis unit is used to analyze high-definition video and identify targets and their behavior patterns in the scene. The abnormal event detection unit is used to identify abnormal situations that violate rules or deviate from the norm based on the results of real-time behavior analysis and preset normal patterns, and immediately trigger an alarm.

[0011] Optionally, the control and scheduling module includes a status monitoring unit, a decision-making unit, and an instruction distribution unit connected in sequence; The status monitoring unit is used to collect status information from each module. Decision-making units are used to identify anomalies based on machine learning models. The instruction distribution unit is used to send parameter adjustment instructions to each module.

[0012] As can be seen from the above technical solution, compared with the prior art, the present invention provides a multi-camera monitoring data fusion and high-definition transmission system, which has the following beneficial effects: (1) This invention solves the problems of low data fusion accuracy and poor high-definition transmission stability in existing multi-camera monitoring systems, and provides a system that can achieve accurate fusion of multi-modal data and stable transmission of high-definition data in weak network environments, which greatly improves the anomaly detection accuracy and real-time performance of the monitoring system. (2) This invention supports the collaborative work of cameras with different resolutions and dynamically adjusts parameters according to the needs of the monitoring scenario. It is applicable to multiple fields and has wide applicability. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0014] Figure 1 This is a schematic diagram of a multi-camera surveillance data fusion and high-definition transmission system provided by the present invention. Detailed Implementation

[0015] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] See Figure 1As shown, this invention discloses a multi-camera surveillance data fusion and high-definition transmission system, comprising a multi-camera data acquisition module, a data preprocessing module, a data fusion module, an encoding and transmission module, and a receiving end processing module connected in sequence, and a control and scheduling module bidirectionally connected to the multi-camera data acquisition module, data preprocessing module, data fusion module, encoding and transmission module, and receiving end processing module; wherein, The multi-camera data acquisition module includes multiple cameras deployed in the area to be monitored, used to acquire raw video data streams and obtain acquired data; The data preprocessing module is used to preprocess the collected data to obtain preprocessed data; The data fusion module uses a fusion algorithm that combines cross-modal attention mechanism and residual fusion strategy to fuse the preprocessed data, output the fused data, and retain the original data backup for anomaly tracing. The encoding and transmission module enables data transmission through the synergistic effect of encoding and transmission. The receiver processing module is used for data decoding, error recovery, and image quality enhancement. The control and scheduling module is used to monitor the operating status of each module in real time, and to detect and alarm anomalies in the monitored scenario.

[0017] Furthermore, the multi-camera data acquisition module has the ability to integrate heterogeneous imaging devices, and the types of cameras that can be connected include visible light cameras, infrared thermal imaging cameras, and depth sensing cameras.

[0018] Furthermore, the data preprocessing module includes a timestamp alignment unit, an image registration unit, a filtering and denoising unit, and a region segmentation unit connected in sequence; Timestamp alignment unit, used to achieve time synchronization of multiple video streams; Image registration unit, used to achieve spatial alignment; A filtering and denoising unit is used to improve the signal-to-noise ratio; The region segmentation unit is used to perform preliminary analysis on the input video to delineate the region of interest, thereby enabling adaptive preprocessing.

[0019] Furthermore, the data fusion module includes a cross-modal attention mechanism unit, a residual fusion unit, and an anomaly tracing unit connected in sequence; A cross-modal attention mechanism unit is used to calculate the feature importance among data from different cameras; The residual fusion unit is used to preserve the original detailed information and superimpose it with the attention-weighted result; The anomaly tracing unit backs up the original data for anomaly tracing.

[0020] Furthermore, the encoding and transmission module includes a compression encoding unit and a transmission control unit connected in sequence; The compression coding unit is used to compress the fused video data; The transmission control unit is used to monitor network bandwidth, latency, and packet loss rate in real time and adjust encoding parameters.

[0021] Furthermore, the receiving end processing module includes a data decoding unit, an error recovery unit, an image quality enhancement unit, a real-time behavior analysis unit, and an abnormal event detection unit connected in sequence; The data decoding unit is used to decode the received compressed bitstream and restore the video frames; An error recovery unit is used to detect and repair errors; The image enhancement unit is used to process the decoded and reconstructed video frames and output high-definition video; The real-time behavior analysis unit is used to analyze high-definition video and identify targets and their behavior patterns in the scene. The abnormal event detection unit is used to identify abnormal situations that violate rules or deviate from the norm based on the results of real-time behavior analysis and preset normal patterns, and immediately trigger an alarm.

[0022] Furthermore, the control and scheduling module includes a status monitoring unit, a decision-making unit, and an instruction distribution unit connected in sequence; The status monitoring unit is used to collect status information from each module. Decision-making units are used to identify anomalies based on machine learning models. The instruction distribution unit is used to send parameter adjustment instructions to each module.

[0023] Specifically, it also includes a communication module, which is responsible for external network transmission, and a control and scheduling module, which coordinates the data and control flow interaction between internal components to jointly ensure the efficient and stable operation of the system.

[0024] The communication module includes a network transmission unit, a protocol encapsulation and parsing unit, a link control unit, and an internal bus interface unit; Network transmission unit: responsible for establishing physical and protocol-level connections with external networks to enable data transmission and reception; Protocol encapsulation and parsing unit: responsible for encapsulating the data to be transmitted into data packets that conform to standard network protocols (such as RTP / RTSP, TCP / IP), and parsing the received data packets; Link control unit: responsible for monitoring network status (bandwidth, latency, packet loss rate) and executing adaptive transmission strategies (such as bitrate adjustment, forward error correction, and retransmission requests). Internal bus interface unit: responsible for high-speed data exchange and control command transmission between various modules within the system.

[0025] In a specific embodiment, in a highway tunnel application scenario, a highway tunnel approximately 3 kilometers long has significant variations in internal lighting, which may lead to abnormal events such as traffic accidents, illegally parked vehicles, and pedestrians entering the tunnel, including the following: System deployment and module configuration: Multi-camera data acquisition module: A visible light high-definition PTZ camera is deployed every 50 meters inside the tunnel for panoramic monitoring and license plate recognition; infrared thermal imaging cameras are deployed at key locations on the tunnel top, including the entrance, exit, and curves, to detect the thermal outlines of vehicles and personnel even in smoky environments; depth sensing cameras are deployed at the tunnel entrance and exit to accurately detect the three-dimensional outlines and speed of vehicles, assisting in determining whether vehicles are exceeding height or speed limits. Data preprocessing module: Timestamp alignment unit: adds a unified and precise timestamp (synchronized to milliseconds) to all camera data, ensuring that the images from different cameras are from the same moment during subsequent fusion; Image registration unit: spatially aligns the images from cameras with different perspectives, i.e., registers the image from a side-view PTZ camera with the image from the thermal imaging camera directly below, so that the same physical location can correspond in different images; Filtering and denoising unit: suppresses image noise caused by insufficient light in the tunnel; Region segmentation unit: divides the video image into different regions of interest, such as "driving lane surface," "emergency stopping lane," and "pedestrian walkway," to facilitate targeted analysis later. Data Fusion Module: Cross-modal Attention Mechanism Unit: When a fire occurs in the tunnel and produces dense smoke, the visible light camera image becomes blurry. At this time, this unit will assign a very high weight (attention) to the data from the infrared thermal imaging camera, enabling it to penetrate the smoke and clearly display the heat distribution of the fire source and trapped vehicles; Residual Fusion Unit: The weighted thermal imaging features (highlighting high-temperature areas) are residually connected with the texture details of the visible light image (such as lane lines and signs) to generate a fused image that shows both the location of the fire and the tunnel environment; Anomaly Tracing Unit: While generating the fused image, the unit automatically retains a backup of the original data from all cameras. When the center receives a fire alarm, the management personnel can immediately retrieve the original infrared and visible light data for disaster tracing and analysis; Encoding and transmission module: Compression encoding unit: uses the H.265 encoding standard to compress the fused high-definition video stream, significantly reducing the data volume; Transmission control unit: monitors the network status from the tunnel to the monitoring center in real time. When it is found that the network bandwidth is reduced due to other services, it immediately notifies the compression encoding unit to dynamically reduce the output bit rate, prioritizing the smoothness and real-time performance of the video stream and avoiding stuttering. Receiver processing module (located in the highway monitoring center): Data decoding unit: uses GPU to quickly decode the received compressed bitstream; Error recovery unit: for the loss of a small number of data packets caused by unstable wireless transmission in the tunnel, it uses error hiding technology to fill in the gaps with the previous frame image to avoid pixelation; Image quality enhancement unit: performs super-resolution processing and color enhancement on the decoded video to make license plate numbers and vehicle details clearer; Real-time behavior analysis unit: tracks every vehicle in the tunnel in real time and analyzes its behavior, such as: vehicle speed, whether it changes lanes illegally, whether it stops in the emergency lane; Abnormal event detection unit: based on preset rules and AI models, it learns normal traffic flow patterns, detects abnormal situations where "an accident may occur ahead", and immediately pops up an alarm window on the monitoring screen and sounds an alarm. Control and Scheduling Module: Status Monitoring Unit: Real-time monitoring of "abnormally high temperature readings from infrared cameras in the middle of the tunnel" and "vehicles stopped as shown by visible light cameras in the area"; Decision Unit: Based on a machine learning model, it determines this to be an abnormal event of "high-risk fire accident"; Command Distribution Unit: Immediately sends a command to the data fusion module to force a switch to a fusion mode based on infrared data; sends a command to the encoding and transmission module to increase the transmission priority of this video path, ensuring that the center can see the clearest scene; sends a command to the multi-camera data acquisition module in the tunnel to control the cameras near the accident point to automatically turn to the accident area and zoom in to obtain more detailed scene information.

[0026] This invention utilizes a cross-modal attention mechanism to intelligently fuse visible light and infrared data in complex scenarios such as tunnel fires, providing more comprehensive and reliable on-site information than a single sensor. Through the coordinated adaptive encoding and transmission, it ensures uninterrupted and low-latency transmission of critical video streams under limited bandwidth. The system automatically detects accidents and issues alarms within seconds, winning valuable time for personnel evacuation and rescue, fully demonstrating the system's intelligence and practicality.

[0027] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0028] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A multi-camera surveillance data fusion and high-definition transmission system, characterized in that, It includes a multi-camera data acquisition module, a data preprocessing module, a data fusion module, an encoding and transmission module, and a receiving end processing module, which are sequentially and communicatively connected; and a control and scheduling module that is bidirectionally and communicatively connected to the multi-camera data acquisition module, data preprocessing module, data fusion module, encoding and transmission module, and receiving end processing module, respectively; wherein, The multi-camera data acquisition module includes multiple cameras deployed in the area to be monitored, used to acquire raw video data streams and obtain acquired data; The data preprocessing module is used to preprocess the collected data to obtain preprocessed data; The data fusion module uses a fusion algorithm that combines cross-modal attention mechanism and residual fusion strategy to fuse the preprocessed data, output the fused data, and retain a backup of the original data for anomaly tracing. The encoding and transmission module enables data transmission through the synergistic effect of encoding and transmission. The receiver processing module is used for data decoding, error recovery, and image quality enhancement. The control and scheduling module is used to monitor the operating status of each module in real time, and to detect and alarm anomalies in the monitored scenario.

2. The multi-camera monitoring data fusion and high-definition transmission system according to claim 1, characterized in that, The multi-camera data acquisition module has the ability to integrate heterogeneous imaging devices, and the types of cameras that can be connected include visible light cameras, infrared thermal imaging cameras, and depth sensing cameras.

3. The multi-camera monitoring data fusion and high-definition transmission system according to claim 1, characterized in that, The data preprocessing module includes a timestamp alignment unit, an image registration unit, a filtering and denoising unit, and a region segmentation unit connected in sequence. Timestamp alignment unit, used to achieve time synchronization of multiple video streams; Image registration unit, used to achieve spatial alignment; A filtering and denoising unit is used to improve the signal-to-noise ratio; The region segmentation unit is used to perform preliminary analysis on the input video to delineate the region of interest, thereby enabling adaptive preprocessing.

4. The multi-camera monitoring data fusion and high-definition transmission system according to claim 1, characterized in that, The data fusion module includes a cross-modal attention mechanism unit, a residual fusion unit, and an anomaly tracing unit connected in sequence; A cross-modal attention mechanism unit is used to calculate the feature importance among data from different cameras; The residual fusion unit is used to preserve the original detailed information and superimpose it with the attention-weighted result; The anomaly tracing unit backs up the original data for anomaly tracing.

5. The multi-camera monitoring data fusion and high-definition transmission system according to claim 1, characterized in that, The encoding and transmission module includes a compression encoding unit and a transmission control unit connected in sequence; The compression coding unit is used to compress the fused video data; The transmission control unit is used to monitor network bandwidth, latency, and packet loss rate in real time and adjust encoding parameters.

6. The multi-camera surveillance data fusion and high-definition transmission system according to claim 1, characterized in that, The receiving end processing module includes a data decoding unit, an error recovery unit, an image quality enhancement unit, a real-time behavior analysis unit, and an abnormal event detection unit connected in sequence. The data decoding unit is used to decode the received compressed bitstream and restore the video frames; An error recovery unit is used to detect and repair errors; The image enhancement unit is used to process the decoded and reconstructed video frames and output high-definition video; The real-time behavior analysis unit is used to analyze high-definition video and identify targets and their behavior patterns in the scene. The abnormal event detection unit is used to identify abnormal situations that violate rules or deviate from the norm based on the results of real-time behavior analysis and preset normal patterns, and immediately trigger an alarm.

7. The multi-camera monitoring data fusion and high-definition transmission system according to claim 1, characterized in that, The control and scheduling module includes a status monitoring unit, a decision-making unit, and an instruction distribution unit connected in sequence; The status monitoring unit is used to collect status information from each module. Decision-making units are used to identify anomalies based on machine learning models. The instruction distribution unit is used to send parameter adjustment instructions to each module.

Images