Wide-angle video monitoring system, method and equipment based on multiple cameras and medium

Abstract

The invention discloses a wide-angle video monitoring system, method and equipment based on multiple cameras and a medium, and relates to the technical field of video monitoring, the wide-angle video monitoring system comprises a multi-camera acquisition module, a variable-focus high-definition camera multi-angle acquisition module, a high-quality image obtained through distortion correction and an image splicing processing module, an improved algorithm is used for accurately matching feature points and fusing the images, and the high-quality image is obtained. The intelligent analysis module detects and tracks a target and analyzes behaviors in real time by means of an improved model, the storage and transmission module performs cluster storage, advanced coding cost reduction and high-speed link transmission, and the control module performs intelligent regulation and control according to multiple information and can also perform remote management. The system has the advantages of multi-angle acquisition without dead angles, good image splicing, accurate intelligent analysis, efficient storage and transmission, adaptability to severe environments, flexible power supply, guarantee of data safety, improvement of monitoring quality and efficiency, meeting of complex scene requirements, and enhancement of practicability, reliability and safety.

Description

Technical Field

[0001] The present invention relates to the field of video surveillance technology, and in particular to a wide-angle video surveillance system, method, equipment and medium based on multiple cameras. Background Art

[0002] In modern society, video surveillance is widely used in many fields such as public safety, production management, and traffic supervision. With the continuous expansion of application scenarios and the improvement of people's requirements for monitoring, traditional single-camera monitoring systems have gradually exposed many limitations.

[0003] Traditional single cameras have a fixed and narrow viewing angle, making it difficult to cover large areas. For example, in crowded and open places such as large shopping malls, airports, and stations, a single camera can only monitor local areas, leaving a large number of blind spots. To achieve comprehensive monitoring, multiple single cameras need to be deployed, which not only increases costs and installation and maintenance workload, but also makes it difficult to quickly build complete scene information because the images of each camera are independent of each other, which is not conducive to real-time monitoring and post-event retrospective analysis. At the same time, in complex scenarios, such as urban road intersections, a single camera cannot take into account traffic conditions and pedestrian activities in multiple directions, and is prone to missing key information.

[0004] Existing wide-angle video surveillance solutions have deficiencies in image acquisition and processing. Some wide-angle solutions that use fisheye lenses have severe image distortion. Although they can obtain wide-angle images, image deformation causes distortion of the shape and position of the target object, affecting subsequent analysis and recognition. Moreover, when processing multi-camera images, traditional image stitching technology has low feature point matching accuracy and slow stitching speed. In complex situations such as lighting changes and dynamic scenes, the stitching effect is poor, with obvious stitching seams or image dislocation, and cannot provide high-quality panoramic video images.

[0005] Traditional monitoring systems also face challenges in intelligent analysis and data processing. The accuracy and real-time performance of their target detection and behavior analysis algorithms are poor, making it difficult to accurately detect and track fast-moving targets or identify abnormal behaviors in complex backgrounds. At the same time, the storage and transmission efficiency of massive video data is low. Traditional storage methods take up a lot of storage space and are inconvenient to retrieve. Problems such as freezes and delays are prone to occur during transmission, which cannot meet the needs of real-time monitoring and emergency response. Therefore, it is urgent to develop a more advanced and efficient wide-angle video surveillance system. Summary of the invention

[0006] The present invention proposes a multi-camera based wide-angle video monitoring system, method, device and medium to solve the problems mentioned in the above-mentioned prior art.

[0007] In order to achieve the above object, the present invention adopts the following technical solution: a wide-angle video monitoring system based on multiple cameras, comprising the following modules: Multi-camera acquisition module: Composed of at least four cameras, the cameras have functions of autofocus, aperture adjustment and dynamic pixel adjustment. By combining an asymmetric fisheye lens and an improved distortion correction algorithm, panoramic video images are acquired. The formula of the improved distortion correction algorithm is , where (x, y) are the coordinates of the distorted image, and (f(x, y)) are the coordinates after correction, , and k1, k2, k3, k4 are distortion coefficients; Image stitching processing module: Using the Scale-Invariant Feature Transform combined with the Convolutional Neural Network algorithm to extract the feature points of each camera image, and then using the Random Sample Consensus combined with the Generative Adversarial Network algorithm to remove the mismatched points. The weighted fusion algorithm is used, and the formula is , where I(x, y) is the pixel value of the fused image, I i (x, y) is the pixel value of the i-th camera image, w i (x, y) is the weight coefficient, and α i (x, y) is the attention weight; Intelligent analysis module: Using an object detection and behavior analysis model based on the Transformer architecture and improved, introducing a position encoding improvement mechanism and a multi-head self-attention mechanism optimization strategy to analyze the stitched video images in real time. The model training dataset is expanded to include 200,000 images of different scenes and targets, detecting and tracking the targets in real time, and calculating the motion trajectory of the targets by combining the spatio-temporal attention mechanism. The formula is , where P(t) is the position of the target at time t, (x(t), y(t)) are the horizontal and vertical coordinates, is the target motion direction angle; Storage and transmission module: The video image storage adopts a distributed cluster storage architecture combined with erasure coding technology to ensure data reliability, and data is transmitted through the integration of 5G-Advanced and Wi-Fi6E dual communication links; Control module: Using an edge computing chip, according to the multi-source information fed back by the environmental perception module, calculating the control quantity u(t) through the improved fuzzy adaptive PID control algorithm formula , where , , are the proportional, integral, and differential coefficients that are dynamically adjusted with time and system state.

[0008] Furthermore, it also includes an environmental perception module, which is composed of temperature and humidity, light, air quality, wind speed and direction sensors, and real-time collects environmental parameters.

[0009] Furthermore, it also includes a power management module, which adopts a hybrid power supply mode of solar energy, wind energy and commercial power, and is equipped with an intelligent energy management system. Through the maximum power point tracking algorithm combined with an intelligent switching strategy, it automatically switches the power supply mode according to the energy collection situation, battery power and system power consumption, and preferentially utilizes renewable energy.

[0010] Furthermore, the cameras in the multi-camera acquisition module have multi-spectral night vision functions, integrating infrared, near-infrared and low-light sensors, and adopting 940nm infrared fill light, 850nm near-infrared fill light and low-light imaging technology to provide clear video images in the night environment, meeting the all-weather and all-environment monitoring requirements.

[0011] Furthermore, in the image stitching and processing module, before feature point matching, the image is first pre-processed based on the fusion of bilateral filtering and guided filtering, combining the edge-preserving denoising characteristics of bilateral filtering and the detail enhancement ability of guided filtering to improve the image quality, and then feature points are extracted through an improved SIFT-CNN algorithm.

[0012] Furthermore, in the intelligent analysis module, a model optimization technology combining knowledge distillation and adversarial training is adopted. The knowledge of the teacher model trained on the public dataset is distilled into the student model of this system, and at the same time, the robustness and generalization ability of the model are enhanced through adversarial training.

[0013] Furthermore, in the storage and transmission module, a data security management technology based on blockchain and federated learning is adopted. The distributed ledger characteristics of the blockchain are used to record the operation records of video data. At the same time, data collaborative training is carried out among multiple data owners through federated learning technology.

[0014] Furthermore, a method for a wide-angle video monitoring system based on multiple cameras includes the following steps: S1: Video image acquisition step: The control module adjusts the camera parameters in the multi-camera acquisition module according to the environmental parameters fed back by the environmental perception module, and at least four cameras at different angles acquire video images. The images are obtained through an asymmetric fisheye lens, and at the same time, an improved distortion correction algorithm is used to correct the distortion of the images; S2: Image stitching and processing step: For the acquired video images, first perform pre-processing based on the fusion of bilateral filtering and guided filtering, then use an improved SIFT-CNN algorithm to extract no less than 800 feature points for each image, and then remove the mismatched points through an improved RANSAC-GAN algorithm. According to the matched feature points, use a weighted fusion algorithm based on the attention mechanism to perform image fusion; S3: Intelligent analysis step: Input the spliced wide-angle video image into the object detection and behavior analysis model based on the improved Transformer architecture, optimize the strategy using the position encoding improvement mechanism and the multi-head self-attention mechanism, and calculate the object motion trajectory formula in combination with the spatio-temporal attention mechanism , perform real-time detection, tracking and behavior analysis on the objects in the video image, and issue a warning when abnormal behavior is detected; S4: Storage and transmission step: Encode the wide-angle video image with VVC, store it using a distributed cluster storage architecture combined with LRC erasure coding technology, and transmit the image to the monitoring center or the cloud through the dual communication links of 5G-Advanced and Wi-Fi6E.

[0015] Furthermore, a wide-angle video monitoring device based on multiple cameras includes the above-mentioned wide-angle video monitoring system based on multiple cameras, and a device shell. The shell is made of carbon fiber composite material, with functions of waterproof, dustproof, sunscreen and anti-collision, and the protection level reaches IP68. The device is built-in with an intelligent heat dissipation and temperature control system, which uses liquid cooling heat dissipation technology combined with an intelligent temperature control chip to automatically adjust the heat dissipation intensity according to the system temperature.

[0016] Furthermore, a computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned wide-angle video monitoring method based on multiple cameras are realized.

[0017] Compared with the existing technology, the beneficial effects of the present invention are: In terms of image acquisition, multiple cameras are flexibly configured at multiple angles, which can cover a larger area and effectively eliminate monitoring blind spots. The variable-focus high-definition camera combined with an advanced distortion correction algorithm ensures that the collected images are clear and distortion-free, and can provide high-quality original video images whether in large indoor spaces or outdoor complex scenes.

[0018] In image splicing processing, the feature point matching that combines deep learning and traditional algorithms and the weighted fusion algorithm based on the attention mechanism are used to achieve high-precision and fast image splicing. The generated seamless wide-angle video image has a perfect visual effect and provides a high-quality data source for subsequent intelligent analysis.

[0019] The intelligent analysis function is powerful. The model based on the improved Transformer architecture greatly improves the accuracy and real-time performance of object detection, tracking and behavior analysis. Whether it is a fast-moving vehicle or abnormal behavior in a crowd, it can be accurately identified and warned in a timely manner, providing strong support for security prevention and event handling.

[0020] In terms of storage and transmission, the distributed cluster storage combines advanced coding and erasure coding technologies, which can greatly reduce the storage cost while ensuring data reliability. The high-speed communication link enables low-latency real-time transmission of video images, ensuring that the monitoring center can obtain on-site information in a timely manner, facilitating quick decision-making and emergency response.

[0021] In addition, the system integrates a variety of advanced technologies, with good environmental adaptability and stability. The hybrid power supply mode ensures energy supply, and mechanisms such as intelligent heat dissipation and temperature control ensure the stable operation of the equipment in harsh environments. At the same time, data security management technologies ensure the security and credibility of video data, overall improving the practicability, reliability, and security of the monitoring system and meeting the monitoring needs in a variety of complex scenarios. Brief Description of the Drawings

[0022] Figure 1 It is a schematic block diagram of the wide-angle video monitoring system based on multiple cameras proposed by the present invention; Figure 2 It is a schematic block diagram of the wide-angle video monitoring method based on multiple cameras proposed by the present invention; Figure 3 It is a bar chart comparing the monitoring coverage ranges of a traditional single camera and this system; Figure 4 It is a line chart comparing the image sharpness of a traditional monitoring system and this system under different lighting conditions; Figure 5 It is a bar chart comparing the target detection accuracy of a traditional system and this system under different numbers of targets. Detailed Embodiment

[0023] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0024] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", etc. indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus cannot be understood as a limitation to the present invention.

[0025] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of the said features. In the description of the present invention, "a plurality of" means two or more unless otherwise specifically defined. In addition, the terms "mounted", "connected" and "coupled" should be understood in a broad sense. For example, it may be a fixed connection, a detachable connection, or an integral connection; it may be a mechanical connection or an electrical connection; it may be directly connected or indirectly connected through an intermediate medium, and it may be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances. The present invention will be further described in detail below with reference to the accompanying drawings.

[0026] Referring to Figures 1 to 5 : A wide-angle video surveillance system based on multiple cameras, comprising the following modules: Multiple camera acquisition module: It consists of at least four varifocal high-definition cameras installed at different angles, and these cameras have functions of automatic focusing, aperture adjustment and dynamic pixel adjustment. In actual installation, the angle between adjacent cameras is dynamically adjusted according to the monitoring scenario through an adaptive algorithm, and the range is 25° - 65°. For example, in the monitoring scenario of a large shopping mall, where the flow of people is large and the activity areas are scattered, the angle between adjacent cameras can be appropriately reduced to cover more areas; while in the monitoring of a long and narrow passage, increasing the angle can make more effective use of the camera resources. Panoramic video images are acquired by combining an asymmetric fisheye lens and an improved distortion correction algorithm. The formula of the improved distortion correction algorithm is , where (x, y) are the coordinates of the distorted image, (f(x, y)) are the coordinates after correction, , k1, k2, k3, k4 are distortion coefficients, and through this algorithm, the image distortion rate can be reduced, which is significantly improved compared with the traditional algorithm.

[0027] Image stitching and processing module: It adopts a feature point matching method that combines deep learning and traditional algorithms. First, the improved SIFT-CNN (Scale-Invariant Feature Transform combined with Convolutional Neural Network) algorithm is used to extract the feature points of each camera image. The number of feature points in each image is not less than 800 and the feature description is more accurate. Then, the improved RANSAC-GAN (Random Sample Consensus combined with Generative Adversarial Network) algorithm is used to remove the mismatched points, and the mismatching rate can be controlled below 5%. Then, image fusion is performed according to the matched feature points, and a weighted fusion algorithm based on the attention mechanism is used. The formula is , where I(x, y) is the pixel value of the fused image, I i (x, y) is the pixel value of the i-th camera image, w i(x, y) is the weight coefficient, α i (x, y) is the attention weight calculated according to the attention mechanism, which makes the splicing seam completely disappear and generates a seamless wide-angle video image with a more perfect visual effect.

[0028] Intelligent analysis module: Utilize the object detection and behavior analysis model based on and improved from the Transformer architecture. Introduce the position encoding improvement mechanism and the multi-head self-attention mechanism optimization strategy to perform real-time analysis on the spliced video image. The model training dataset is expanded to include 200,000 images of different scenes and targets, covering various target categories such as people, vehicles, animals, and objects. It can detect and track targets in real time, and the tracking accuracy is not less than 95%. By combining the spatio-temporal attention mechanism to calculate the movement trajectory of the target, the formula is , where P(t) is the position of the target at time t, (x(t), y(t)) are the horizontal and vertical coordinates, is the target movement direction angle, which can more accurately predict the target behavior and issue a warning when an abnormal behavior is detected. The detection accuracy of abnormal behaviors is increased to over 90%.

[0029] Storage and transmission module: In terms of video image storage, adopt a distributed cluster storage architecture. This architecture effectively improves the reliability and scalability of the storage system by dispersing data storage on multiple nodes. Combine erasure coding technology, especially the advanced LRC (Local Reconstruction Codes) erasure coding, which can encode data during data storage. When part of the data is damaged or lost, the original data can be recovered from the remaining data according to the coding rules, reducing the data redundancy rate and greatly saving storage space while ensuring data reliability. In terms of video coding, support the new generation of high-efficiency video coding standard VVC (Versatile Video Coding). VVC enables a compression ratio of up to 150:1 by adopting more advanced intra-frame prediction, inter-frame prediction, and transform coding technologies, further reducing the storage cost. Data transmission is carried out through the integration of 5G-Advanced and Wi-Fi6E dual communication links. The bandwidth of the 5G-Advanced link is not less than 500 Mbps, and the bandwidth of the Wi-Fi6E link is not less than 2.4 Gbps, enabling low-latency real-time transmission of ultra-high-definition video images to the monitoring center or the cloud.

[0030] Control module: Adopt a high-performance edge computing chip with real-time decision-making and intelligent regulation capabilities. According to multi-source information such as the environmental parameters feedback by the environmental perception module and the target analysis results of the intelligent analysis module, through the improved fuzzy adaptive PID control algorithm formula , where 、 、 They are proportional, integral, and differential coefficients that dynamically adjust with time and system status, automatically regulating camera parameters such as focal length, aperture, exposure time, pixel mode, etc., to obtain the best video image quality. At the same time, it can receive remote control instructions to achieve refined remote management of the entire monitoring system.

[0031] In the present invention, it also includes an environmental perception module, which consists of high-precision temperature and humidity sensors, light sensors, air quality sensors, and wind speed and direction sensors. The accuracy of the temperature and humidity sensors is ±0.3°C and ±1%RH, the measurement range of the light sensor is 0 - 20000Lux, and the accuracy is ±3%. The air quality sensor can detect the concentrations of various pollutants such as PM2.5, PM10, CO, SO2, etc., and the detection accuracy error does not exceed 5%. The measurement range of the wind speed and direction sensor for wind speed is 0 - 60m / s, the accuracy is ±0.5m / s, the measurement range for wind direction is 0 - 360°, and the accuracy is ±3°. This module collects environmental parameters in real time, providing comprehensive environmental information for the control module to adjust camera parameters and the intelligent analysis module to conduct target behavior analysis.

[0032] In the present invention, it also includes a power management module, which adopts a hybrid power supply mode of solar energy, wind energy, and commercial power. The equipped high-efficiency solar panel uses advanced photovoltaic materials and manufacturing processes, with a photoelectric conversion efficiency of over 22% and a power of not less than 200W, capable of efficiently converting solar energy into electrical energy under sufficient light conditions. The small wind turbine adopts permanent magnet synchronous motor technology, with a rated power of 100W, having the characteristic of starting at low wind speeds and being able to operate normally and generate electricity at a wind speed of 3m / s. The paired large-capacity lithium-graphene battery has advantages such as high energy density and long cycle life, with a capacity of 12V / 200Ah. The module is built with an intelligent energy management system that uses the MPPT (Maximum Power Point Tracking) algorithm to monitor the output power of the solar panel and the wind turbine in real time, dynamically adjusting the working point to ensure that it always operates near the maximum power point, improving the energy collection efficiency. Combining with an intelligent switching strategy, based on comprehensive judgment of multiple parameters such as energy collection, battery power, and system power consumption, renewable energy is preferentially used. When renewable energy is insufficient or the system power consumption surges, it automatically and smoothly switches to commercial power supply to ensure the stable operation of the system in different environments, effectively reducing the dependence on commercial power, and the energy consumption is reduced by more than 30% through actual tests.

[0033] In the present invention, the cameras in the multi-camera acquisition module have multi-spectral night vision capabilities and integrate high-performance infrared, near-infrared, and low-light sensors. The infrared sensor uses 940nm infrared fill light technology. The fill light in this band is not easily detectable by the human eye, has strong concealment, and the effective infrared distance is not less than 50 meters, enabling clear infrared imaging at relatively long distances. The near-infrared sensor uses 850nm near-infrared fill light technology, with high energy conversion efficiency and an effective distance of more than 80 meters. Even in a relatively far and dimly lit area, it can capture target information. The low-light sensor is equipped with ultra-low illumination low-light imaging technology, using advanced image enhancement algorithms and high-sensitivity photosensitive elements to fully collect weak light. In a low-light environment, it can convert weak light into high-quality electrical signals, and after processing, output images with a resolution of 1080P, effectively overcoming the problem of insufficient night light and providing clear video images for complex night environments to meet the all-weather and all-environment monitoring requirements.

[0034] In the present invention, in the image stitching and processing module, an advanced technology based on the fusion of bilateral filtering and guided filtering is adopted in the image preprocessing stage. Bilateral filtering uses its unique spatial proximity and pixel value similarity weight mechanism to effectively remove image noise while retaining the edge information of the image and preventing edge blurring. Guided filtering, on the other hand, uses the structural information of the guidance image to perform adaptive smoothing on the input image and enhance image details. The fusion of the two further improves the image quality. In the feature point extraction stage, the improved SIFT-CNN algorithm is used. This algorithm introduces the feature learning ability of the convolutional neural network (CNN) on the basis of the traditional SIFT algorithm. By automatically extracting multi-level and abstract features of the image through deep convolutional layers and combining the advantages of the SIFT algorithm's invariance to scale and rotation changes, the accuracy of feature point extraction is greatly improved. Compared with the traditional SIFT algorithm, the accuracy of feature point extraction is increased, laying a solid foundation for the precise stitching of subsequent images.

[0035] In the present invention, in the intelligent analysis module, a model optimization technology combining knowledge distillation and adversarial training is adopted. The knowledge of the teacher model trained on a large-scale public dataset is distilled into the student model of this system. At the same time, adversarial training is introduced. During the training process, adversarial samples are generated to let the model continuously learn to distinguish real samples and adversarial samples. In this way, the robustness of the model to various interferences and noises is enhanced, enabling it to adapt to more complex and changeable scenarios and improving the generalization ability. Through practical verification, this optimization technology reduces the model training time and improves the detection accuracy in complex scenarios.

[0036] In the present invention, in the storage and transmission module, a data security management technology based on blockchain and federated learning is applied to build a solid defense line for data security. By virtue of the distributed ledger feature of blockchain, operation information such as the uploading, downloading, and modification of video data is recorded on each node in the form of encrypted hash values. Each block contains the hash value of the previous block, forming a chain structure. Any tampering with the data operation records will result in hash value mismatches, thus achieving the immutability and traceability of data, and ensuring the transparency and compliance of data operations. The federated learning technology allows multiple data owners to collaboratively train a model without sharing the original data. Each participating party encrypts the data locally and trains the model, only uploading the model update parameters. Through a secure aggregation mechanism, the parameters of all parties are integrated. While protecting data privacy, the advantages of multiple parties' data are pooled to improve the model performance, greatly enhancing the data security and credibility.

[0037] In the present invention, a method for a wide-angle video surveillance system based on multiple cameras includes the following steps: Video image acquisition step: According to the environmental parameters fed back by the environmental perception module, the control module adjusts parameters such as the focal length, aperture, exposure time, and pixel mode of the cameras in the multi-camera acquisition module using an improved fuzzy adaptive PID control algorithm. At least four varifocal high-definition cameras at different angles acquire video images at a frame rate of 30fps, obtain large-view images through an asymmetric fisheye lens, and at the same time use an improved distortion correction algorithm to correct the distortion of the images and generate corrected video images.

[0038] Image stitching processing step: For the acquired video images, first perform preprocessing based on the fusion of bilateral filtering and guided filtering, then use an improved SIFT-CNN algorithm to extract no less than 800 feature points from each image, and then remove the mismatched points through an improved RANSAC-GAN algorithm, with the mismatching rate controlled below 5%. According to the matched feature points, use a weighted fusion algorithm based on the attention mechanism to perform image fusion and generate a seamless wide-angle video image.

[0039] Intelligent analysis step: Input the stitched wide-angle video image into an object detection and behavior analysis model based on an improved Transformer architecture. Use the position encoding improvement mechanism and the multi-head self-attention mechanism optimization strategy, combined with the spatio-temporal attention mechanism to calculate the target motion trajectory formula to perform real-time detection, tracking, and behavior analysis on the targets in the video image, and issue a warning when abnormal behavior is detected.

[0040] Storage and transmission steps: Perform VVC encoding on the wide-angle video image. VVC encoding utilizes advanced intra-frame prediction, inter-frame prediction and other technologies to greatly improve the compression efficiency and effectively reduce the data volume. When storing, adopt a distributed cluster storage architecture to disperse the data and store it on multiple nodes, enhancing reliability and scalability. Combine with the LRC erasure code technology to encode and process the data, which can be recovered when some data is damaged, significantly reducing the data redundancy rate. In terms of transmission, with the help of the 5G-Advanced and Wi-Fi6E dual communication links, 5G-Advanced provides a high-speed and stable connection, and Wi-Fi6E expands the frequency band to achieve low-latency real-time transmission of ultra-high-definition video images to the monitoring center or the cloud.

[0041] In the present invention, a wide-angle video monitoring device based on multiple cameras includes the above-mentioned wide-angle video monitoring system based on multiple cameras and a device housing. The housing is made of high-strength carbon fiber composite material, with functions of waterproof, dustproof, sun-proof and impact-proof, and the protection level reaches IP68, which can adapt to extremely harsh outdoor environments. The device is built-in with an intelligent heat dissipation and temperature control system, which combines liquid cooling heat dissipation technology with an intelligent temperature control chip, and can automatically adjust the heat dissipation intensity according to the system temperature to ensure the stable operation of the system within the ambient temperature range of -40°C to 80°C.

[0042] In the present invention, a computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned wide-angle video monitoring method based on multiple cameras are implemented.

[0043] The above is only the preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution and inventive concept of the present invention, makes equivalent substitutions or changes, and all should be covered within the protection scope of the present invention.

Claims

1. A wide-angle video surveillance system based on multiple cameras, characterized in that: Includes the following modules: Multi-camera acquisition module: It consists of at least four cameras. The cameras have the functions of autofocus, aperture adjustment and dynamic pixel adjustment. The panoramic video images are acquired by combining an asymmetric fisheye lens and an improved distortion correction algorithm. The formula of the improved distortion correction algorithm is: , where (x, y) is the distorted image coordinate, (f(x, y)) is the corrected coordinate, , k1, k2, k3, k4 are distortion coefficients; Image stitching processing module: Scale-invariant feature transformation combined with convolutional neural network algorithm is used to extract feature points of each camera image, and then random sampling is combined with generative adversarial network algorithm to remove mismatched points. Weighted fusion algorithm is used, and the formula is: , where I(x,y) is the pixel value of the fused image, I i (x,y) is the pixel value of the i-th camera image, w i (x,y) is the weight coefficient, α i (x,y) is the attention weight; Intelligent analysis module: Using the improved target detection and behavior analysis model based on the Transformer architecture, introducing the position encoding improvement mechanism and the multi-head self-attention mechanism optimization strategy, real-time analysis of spliced ​​video images, the model training data set is expanded to include 200,000 images of different scenes and targets, real-time detection and tracking of targets, and the motion trajectory of the target is calculated by combining the spatiotemporal attention mechanism. The formula is: , where P(t) is the position of the target at time t, (x(t), y(t)) is the horizontal and vertical coordinates, is the target motion direction angle; Storage and transmission module: Video image storage uses a distributed cluster storage architecture combined with erasure coding technology to ensure data reliability, and transmits data through integrated 5G-Advanced and Wi-Fi6E dual communication links; Control module: Using edge computing chip, based on the multi-source information fed back by the environment perception module, through the improved fuzzy adaptive PID control algorithm formula Calculate the control quantity u(t), where , , The proportional, integral, and differential coefficients are dynamically adjusted with time and system status.

2. The wide-angle video surveillance system based on multiple cameras according to claim 1, characterized in that: It also includes an environmental perception module, which is composed of temperature and humidity, light, air quality, wind speed and direction sensors to collect environmental parameters in real time.

3. The wide-angle video surveillance system based on multiple cameras according to claim 1, characterized in that: It also includes a power management module, which adopts a hybrid power supply mode of solar energy, wind energy and AC power. It has an intelligent energy management system. Through the maximum power point tracking algorithm combined with an intelligent switching strategy, it automatically switches the power supply mode according to energy collection conditions, battery power and system power consumption, giving priority to the use of renewable energy.

4. The wide-angle video surveillance system based on multiple cameras according to claim 1, characterized in that: The cameras in the multi-camera acquisition module have multi-spectral night vision function, integrated infrared, near-infrared and low-light sensors, and use 940nm infrared fill light, 850nm near-infrared fill light and low-light imaging technology to provide clear video images in night environments and meet all-weather and all-environment monitoring needs.

5. The multi-camera based wide-angle video surveillance system according to claim 1, characterized in that: In the image stitching processing module, before matching feature points, the image is first preprocessed based on the fusion of bilateral filtering and guided filtering, combining the edge-preserving denoising characteristics of bilateral filtering and the detail enhancement ability of guided filtering to improve image quality, and then feature points are extracted using the improved SIFT-CNN algorithm.

6. The multi-camera based wide-angle video surveillance system according to claim 1, characterized in that: In the intelligent analysis module, a model optimization technology combining knowledge distillation and adversarial training is adopted to distill the knowledge of the teacher model trained on the public data set into the student model of this system, and at the same time enhance the robustness and generalization ability of the model through adversarial training.

7. The multi-camera based wide-angle video surveillance system according to claim 1, characterized in that: In the storage and transmission module, data security management technology based on blockchain and federated learning is adopted, and the distributed ledger characteristics of blockchain are used to record the operation records of video data. At the same time, data collaborative training is carried out among multiple data owners through federated learning technology.

8. A method for applying the multi-camera wide-angle video surveillance system according to any one of claims 1 to 7, characterized in that: The following steps are involved: S1: Video image acquisition step: The control module uses the improved fuzzy adaptive PID control algorithm to adjust the camera parameters in the multi-camera acquisition module according to the environmental parameters fed back by the environmental perception module. At least four cameras at different angles acquire video images, and images are acquired through an asymmetric fisheye lens. At the same time, an improved distortion correction algorithm is used. Perform distortion correction on the image; S2: Image stitching processing steps: The collected video images are first preprocessed based on bilateral filtering and guided filter fusion, and then the improved SIFT-CNN algorithm is used to extract no less than 800 feature points for each image. Then the improved RANSAC-GAN algorithm is used to remove the mismatched points. According to the matched feature points, a weighted fusion algorithm based on the attention mechanism is used. Perform image fusion; S3: Intelligent analysis step: Input the spliced ​​wide-angle video image into the target detection and behavior analysis model based on the improved Transformer architecture, use the position encoding improvement mechanism and multi-head self-attention mechanism optimization strategy, and combine the spatiotemporal attention mechanism to calculate the target motion trajectory formula , perform real-time detection, tracking and behavior analysis of targets in video images, and issue warnings when abnormal behavior is detected; S4: Storage and transmission step: VVC encoding is performed on the wide-angle video image, and a distributed cluster storage architecture combined with LRC erasure coding technology is used for storage. The image is transmitted to the monitoring center or the cloud through the 5G-Advanced and Wi-Fi6E dual communication links.

9. A wide-angle video surveillance device based on multiple cameras, characterized in that: It includes a wide-angle video surveillance system based on multiple cameras as described in any one of claims 1 to 7, and a device shell, the shell is made of carbon fiber composite material, has waterproof, dustproof, sun-proof and impact-proof functions, and the protection level reaches IP68. The device has a built-in intelligent heat dissipation and temperature control system, which uses liquid cooling technology combined with an intelligent temperature control chip to automatically adjust the heat dissipation intensity according to the system temperature.

10. A computer-readable storage medium, characterized in that: A computer program is stored, and when the computer program is executed by a processor, the steps of the wide-angle video monitoring method based on multiple cameras as described in claim 8 are implemented.

Images