Monitoring data processing system and method

Abstract

The invention relates to the technical field of monitoring data processing, and discloses a monitoring data processing system, which comprises an optical imaging unit which comprises a plurality of optical lenses which are annularly distributed, synchronously captures a multi-view optical signal and converts the multi-view optical signal into a digital electric signal; the video processing unit is used for coding the digital electric signal and generating a video stream with time metadata; the space-time fusion unit is used for implementing four-dimensional binding of the space coordinates and the precise timestamps to the video streams; the transmission unit is used for transmitting the data processed by the time-space fusion unit to a cloud through a network; and the cloud processing unit is used for splicing the multi-view videos in a cloud server center. According to the invention, through cooperation of Beidou positioning and atomic clock microsecond-level time service, four-dimensional precise binding of a video stream and a physical space is realized, and the requirement of high-precision track restoration is met.

Description

Technical Field

[0001] The present invention relates to the technical field of monitoring data processing, and particularly to a monitoring data processing system and method. Background Art

[0002] Currently, intelligent monitoring systems generally adopt a multi-camera collaborative architecture, achieve panoramic monitoring through video stitching technology, and enhance data analysis capabilities by integrating positioning information.

[0003] However, traditional monitoring systems have significant technical bottlenecks in dealing with complex scenarios: in the multi-camera collaborative work, due to the lack of an accurate spatio-temporal synchronization mechanism, the video frames are misaligned with the geographical information, making it difficult to restore the spatio-temporal relationship of real events; existing video stitching technologies are sensitive to dynamic objects and light changes, prone to visible seams and artifacts, affecting the integrity of panoramic videos; encryption transmission schemes are difficult to balance security and real-time performance, and high-resolution video stream transmission often causes a sharp increase in latency. Summary of the Invention

[0004] The purpose of the present invention is to solve the problems existing in the prior art, and to propose a monitoring data processing system.

[0005] In order to achieve the above purpose, the present invention adopts the following technical solutions: A monitoring data processing system, comprising: An optical imaging unit, including a plurality of optical lenses distributed in a ring, synchronously capturing multi-view optical signals and converting them into digital electrical signals; A video processing unit, encoding the digital electrical signals and generating a video stream with time metadata; A spatio-temporal fusion unit, implementing the four-dimensional binding of spatial coordinates and accurate timestamps to the video stream; A transmission unit, transmitting the data processed by the spatio-temporal fusion unit to the cloud through a network; A cloud processing unit, stitching multi-view videos in the cloud server center.

[0006] Preferably, the optical imaging unit includes: An optical lens array sub-unit, including multiple groups of ultra-wide-angle optical lenses distributed in a ring, synchronously capturing ambient light signals from multiple views, covering 360° panoramic monitoring without dead angles; A CMOS / CCD sensor sub-unit, converting optical signals into 12-bit RAW format electrical signals; A dynamic backlight compensation sub-unit, automatically adjusting exposure parameters according to ambient light, suppressing overexposure and underexposure; An infrared supplementary light sub-unit, enabling active supplementary light in low light environments (≤0.1 lux); The adaptive white balance subunit corrects color deviation under different light sources in real time.

[0007] Preferably, the video processing unit includes: The signal conversion subunit uses the ISP image processing module of the Hisilicon 3519 chip to convert the RAW electrical signal into a YUV420 digital signal; The encoding processing subunit uses an H.265 / H.264 hardware encoder to compress the video stream and generate a standard H.265 / H.264 format; The metadata embedding subunit embeds the UTC timestamp in the video stream header; The video preprocessing subunit performs spatial domain noise reduction (PSNR improvement ≥ 5dB) and temporal jitter removal; The data buffer subunit temporarily stores the video frames to be processed and balances the data throughput.

[0008] Preferably, the spatio-temporal fusion unit specifically includes: The Beidou positioning subunit obtains the device's geographical coordinates in real time; The time synchronization subunit generates a synchronized UTC timestamp for each video frame; The spatio-temporal binding subunit establishes a one-to-one mapping relationship between the video frames and spatio-temporal tags; The data encapsulation subunit encapsulates the video stream, positioning data, and device status into a single transmission packet.

[0009] Preferably, the transmission unit includes: The multi-mode communication subunit uses a 4G Cat.12 module, a 5G NR module, and a WiFi6 chipset to support 4G / 5G / WiFi three-mode adaptive network access; The dynamic routing subunit automatically selects the optimal transmission path based on the QoS-aware scheduling algorithm; The encrypted transmission subunit performs end-to-end encrypted transmission of the video stream.

[0010] Preferably, the cloud processing unit includes: The video stitching subunit stitches multi-view videos into a seamless 360° panoramic video; The AI recognition subunit, based on the improved YOLOv5 model, analyzes sensitive behaviors in real time and generates alarm metadata; The data storage subunit, based on the distributed object storage system, archives the original videos and processing results for a long time; The alarm generation subunit comprehensively determines abnormal events and generates structured alarm logs; The data distribution subunit pushes the real-time video stream and alarm information to the end users.

[0011] Preferably, in the optical imaging unit, there are four optical lenses distributed in a ring.

[0012] A monitoring data processing method, using a monitoring data processing system, includes the following steps: S100: Synchronously collect ambient light signals through multiple optical lenses distributed in a ring, and convert them into raw electrical signals through a CMOS sensor array; S200: Use the Hisilicon 3519 chip to perform H.265 encoding processing on the raw electrical signals to generate an initial video stream with device identification; S300: Perform spatio-temporal binding operations on the main control circuit board, and embed the Beidou positioning coordinates and atomic clock timestamps into the video stream metadata; S400: Implement SM4 encryption on the bound data using a dynamic sharding encryption algorithm, and select a 4G / 5G transmission channel through a multi-mode network module; S500: The cloud server performs parallel processing on multiple video streams, including: S510: Implement sub-pixel stitching of multi-view videos based on the improved SIFT-RANSAC algorithm to generate a 360° panoramic video; S520: Identify 20 preset types of abnormal behaviors through a cascaded neural network model and generate an alarm data packet with spatial coordinates; S600: Use the WebRTC protocol to distribute the processing results to end users, including a panoramic video with AR annotations and a dynamic heat map.

[0013] Compared with the prior art, the beneficial effects of the present invention are: Full-time and full-space high-precision monitoring: Through the coordination of Beidou positioning and atomic clock microsecond-level timing, the present invention realizes the four-dimensional precise binding of the video stream and the physical space, meeting the high-precision trajectory restoration requirements.

[0014] 2. Extreme environment imaging optimization: The present invention combines dynamic backlight compensation and infrared supplementary light, enabling the system to have comparable imaging quality in extremely strong light to extremely dark environments.

[0015] 3. Efficient video processing: Based on the H.265 hardware encoding of the Hisilicon 3519 chip, the present invention can compress the 4K video bit rate to 8Mbps, with an encoding delay ≤ 30ms, supporting real-time video stream processing. Description of the Drawings

[0016] Figure 1 It is a schematic flow chart of a monitoring data processing system proposed by the present invention. Detailed Embodiments

[0017] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments.

[0018] Referring to Figure 1 , a monitoring data processing system includes: An optical imaging unit, including a plurality of optical lenses distributed in a ring, synchronously capturing multi-view optical signals and converting them into digital electrical signals; A video processing unit, encoding and processing the digital electrical signals and generating a video stream with time metadata; A spatio-temporal fusion unit, implementing the four-dimensional binding of spatial coordinates and precise timestamps to the video stream; A transmission unit, transmitting the data processed by the spatio-temporal fusion unit to the cloud through the network; A cloud processing unit, stitching multi-view videos at the cloud server center.

[0019] In this embodiment, the optical imaging unit includes: An optical lens array sub-unit, using 4 groups of 2 million pixel ultra-wide-angle lenses (model: Sony IMX335), synchronously capturing ambient light signals from multiple views, and achieving full coverage of 360° horizontally × 120° vertically after stitching 4 lenses; A CMOS / CCD sensor sub-unit, using a 1 / 2.8-inch back-illuminated CMOS (Sony IMX415), converting optical signals into 12-bit RAW format electrical signals; A dynamic backlight compensation sub-unit, based on the HDR algorithm synthesized by 3 frames (short / medium / long exposure times are 1 / 100s, 1 / 30s, 1 / 5s respectively), automatically adjusting exposure parameters according to ambient light, and suppressing overexposure and underexposure; An infrared supplementary light sub-unit, including 4 groups of 850nm wavelength infrared LED arrays, enabling active supplementary light in low illuminance environments (≤0.1 lux); An adaptive white balance sub-unit, using a color temperature sensor and combining a deep learning correction algorithm to correct color deviations under different light sources in real time.

[0020] In this embodiment, the video processing unit includes: A signal conversion sub-unit, using the ISP image processing module of the Hisilicon 3519 chip, for converting RAW electrical signals into YUV420 digital signals; An encoding processing sub-unit, using an H.265 / H.264 hardware encoder, for compressing the video stream and generating a standard H.265 / H.264 format; A metadata embedding sub-unit, embedding a UTC timestamp at the head of the video stream; Video preprocessing subunit, performing spatial domain noise reduction (PSNR improvement ≥ 5dB) and temporal jitter removal; Data buffer subunit, temporarily storing video frames to be processed and balancing data throughput.

[0021] In this embodiment, the spatio-temporal fusion unit specifically includes: Beidou positioning subunit, obtaining the device's geographical coordinates in real time; Time synchronization subunit, generating synchronous UTC timestamps for each video frame; Spatio-temporal binding subunit, establishing a one-to-one mapping relationship between video frames and spatio-temporal tags; Data encapsulation subunit, encapsulating video streams, positioning data, and device status into a single transmission packet.

[0022] In this embodiment, the transmission unit includes: Multi-mode communication subunit, using 4G Cat.12 module, 5G NR module, and WiFi6 chipset, supporting 4G / 5G / WiFi three-mode adaptive network access; Dynamic routing subunit, automatically selecting the optimal transmission path based on the QoS-aware scheduling algorithm; Encrypted transmission subunit, performing end-to-end encrypted transmission on the video stream.

[0023] In this embodiment, the cloud processing unit includes: Video stitching subunit, stitching multi-view videos into a seamless 360° panoramic video; AI recognition subunit, based on the improved YOLOv5 model, analyzing sensitive behaviors in real time and generating alarm metadata; Data storage subunit, long-term archiving of original videos and processing results based on a distributed object storage system; Alarm generation subunit, comprehensively judging abnormal events and generating structured alarm logs; Data distribution subunit, pushing real-time video streams and alarm information to end users.

[0024] Preferably, in the optical imaging unit, four optical lenses are annularly distributed.

[0025] A method for processing monitoring data, using a monitoring data processing system, includes the following steps: S100: Synchronously collecting ambient light signals through a plurality of annularly distributed optical lenses, and converting them into raw electrical signals through a CMOS sensor array; S200: Using the Hisilicon 3519 chip to perform H.265 encoding processing on the raw electrical signals, generating an initial video stream with device identification; S300: Perform spatio-temporal binding operation on the total control circuit board, and embed the Beidou positioning coordinates and atomic clock timestamps into the video stream metadata; S400: Implement SM4 encryption on the bound data using a dynamic sharding encryption algorithm, and select a 4G / 5G transmission channel through a multi-mode network module; S500: The cloud server performs parallel processing on multiple videos, including: S510: Implement sub-pixel stitching of multi-view videos based on the improved SIFT-RANSAC algorithm to generate a 360° panoramic video; S520: Identify 20 preset types of abnormal behaviors through a cascaded neural network model, and generate an alarm data packet with spatial coordinates; S600: Use the WebRTC protocol to distribute the processing results to end users, including the panoramic video with AR annotations and the dynamic heat map.

[0026] Among them, step S300 includes: S301: Obtain the three-dimensional spatial coordinates (accuracy ≤ 0.5m) in the ECEF coordinate system through the Beidou-3 RDSS module; S302: Synchronize the clock of the Hisilicon 3519 chip with the atomic clock using the PTP protocol, and the time synchronization error ≤ 1μs; S303: Embed four-dimensional spatio-temporal tags in the SEI field of the video stream, with the format <longitude, latitude, altitude, UTC time>.

[0027] Among them, the dynamic sharding encryption in step S400 includes: S401: Shard the video stream by 128KB, and generate a unique quantum random number key for each shard; S402: Parallelly execute the SM4-CTR mode encryption in the hardware encryption engine of the Hi3519 chip; S403: Bind the key index and the shard hash value through exclusive OR operation to generate a tamper-proof data packet.

[0028] Among them, the video stitching in step S510 includes: S511: Perform spherical projection transformation on the input video stream to compensate for lens parallax and distortion; S512: Use the improved RANSAC algorithm to eliminate the mismatched points caused by dynamic objects; S513: Use the Poisson fusion algorithm to eliminate the brightness jump (ΔY ≤ 3%) at the stitching seam.

[0029] Among them, the behavior recognition in step S520 includes: S521: Implement 30FPS real-time object detection through the MobileNetV3 network; S522: Execute high-precision classification (confidence level ≥ 90%) on the suspected target using the ResNet50 network; S523: Predict the target movement trajectory based on the GRU network and calculate the risk probability within the next 5 seconds.

[0030] Among them, step S600 includes: S601: Dynamically switch the video bitrate (1080P / 720P / 480P) according to the terminal network bandwidth; S602: Render the movement trajectory prediction line of the warning target in the AR overlay layer; S603: Trigger location-related warning push through the geofencing technology (fence accuracy ±2m).

[0031] As mentioned above, it is only the preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution and inventive concept of the present invention, makes equivalent substitutions or changes, and should be covered by the protection scope of the present invention.

Claims

1. A monitoring data processing system, characterized in that: Comprising: An optical imaging unit, including a plurality of optical lenses distributed in a ring, synchronously capturing multi-view optical signals and converting them into digital electrical signals; A video processing unit, encoding and processing the digital electrical signals and generating a video stream with time metadata; A spatio-temporal fusion unit, implementing the four-dimensional binding of spatial coordinates and precise timestamps to the video stream; A transmission unit, transmitting the data processed by the spatio-temporal fusion unit to the cloud through a network; A cloud processing unit, stitching multi-view videos at the cloud server center.

2. The monitoring data processing system according to claim 1, wherein: The optical imaging unit includes: An optical lens array sub-unit, including multiple groups of ultra-wide-angle optical lenses distributed in a ring, synchronously capturing ambient light signals from multiple views, covering 360° panoramic monitoring without dead angles; A CMOS / CCD sensor sub-unit, converting optical signals into 12-bit RAW format electrical signals; A dynamic backlight compensation sub-unit, automatically adjusting exposure parameters according to ambient light, suppressing overexposure and underexposure; An infrared supplementary light sub-unit, enabling active supplementary light in low illuminance environments (≤0.1 lux); An adaptive white balance sub-unit, real-time correcting color deviations under different light sources.

3. The monitoring data processing system according to claim 2, characterized in that: The video processing unit includes: A signal conversion sub-unit, using the ISP image processing module of the Hisilicon 3519 chip to convert RAW electrical signals into YUV420 digital signals; An encoding and processing sub-unit, using an H.265 / H.264 hardware encoder to compress the video stream and generate a standard H.265 / H.264 format; A metadata embedding sub-unit, embedding UTC timestamps at the head of the video stream; A video preprocessing sub-unit, performing spatial domain noise reduction (PSNR improvement ≥5 dB) and temporal jitter reduction; A data buffer sub-unit, temporarily storing video frames to be processed and balancing data throughput.

4. A monitoring data processing system according to claim 3, wherein: The spatio-temporal fusion unit specifically includes: A Beidou positioning sub-unit, real-time obtaining the geographical coordinates of the device; A time synchronization sub-unit, generating synchronous UTC timestamps for each video frame; A spatio-temporal binding sub-unit, establishing a one-to-one mapping relationship between video frames and spatio-temporal tags; A data encapsulation sub-unit, encapsulating the video stream, positioning data, and device status into a single transmission packet.

5. A monitoring data processing system according to claim 4, characterized in that: The transmission unit includes: A multi-mode communication sub-unit, using a 4G Cat.12 module, a 5G NR module, and a WiFi6 chipset, supporting 4G / 5G / WiFi three-mode adaptive network access; A dynamic routing sub-unit, automatically selecting the optimal transmission path based on the QoS-aware scheduling algorithm; An encrypted transmission sub-unit, performing end-to-end encrypted transmission of the video stream.

6. The monitoring data processing system according to claim 5, wherein: The cloud processing unit includes: A video stitching sub-unit, stitching multi-view videos into a seamless 360° panoramic video; An AI recognition sub-unit, based on an improved YOLOv5 model, real-time analyzing sensitive behaviors and generating alarm metadata; A data storage sub-unit, based on a distributed object storage system, long-term archiving the original videos and processing results; An alarm generation sub-unit, comprehensively determining abnormal events and generating structured alarm logs; A data distribution sub-unit, pushing real-time video streams and alarm information to end users.

7. A monitoring data processing system according to claim 6, characterized in that: In the optical imaging unit, there are four optical lenses distributed in a ring.

8. A monitoring data processing method, adopting a monitoring data processing system according to any one of claims 1-7, characterized in that: Including the following steps: S100: Synchronously collect ambient light signals through multiple annularly distributed optical lenses, and convert them into raw electrical signals via a CMOS sensor array; S200: Use a Hisilicon 3519 chip to perform H.265 encoding processing on the raw electrical signals to generate an initial video stream with device identification; S300: Perform spatio-temporal binding operations on the main control circuit board to embed Beidou positioning coordinates and atomic clock timestamps into the video stream metadata; S400: Implement SM4 encryption on the bound data using a dynamic sharding encryption algorithm, and select a 4G / 5G transmission channel through a multi-mode network module; S500: The cloud server performs parallel processing on multiple video streams, including: S510: Achieve sub-pixel stitching of multi-view videos based on the improved SIFT-RANSAC algorithm to generate a 360° panoramic video; S520: Identify 20 preset types of abnormal behaviors through a cascaded neural network model and generate an alarm data packet with spatial coordinates; S600: Use the WebRTC protocol to distribute the processing results to end users, including a panoramic video with AR annotations superimposed and a dynamic heat map.

Images