Computational imaging-based full-color day and night monitoring device

Abstract

The invention discloses a full-color day and night monitoring device based on computational imaging, which realizes day and night full-color imaging under the condition of no light supplement lamp by combining a low-illumination CMOS detector and an AI technology, remarkably improves the night monitoring effect, realizes the night full-color imaging effect by combining an innovative computational imaging algorithm with an AI processor platform, and improves the monitoring accuracy. Therefore, night imaging has rich color information as in the daytime, the accuracy of rear-end target monitoring and recognition is greatly improved, and meanwhile, the monitoring range and distance are greatly improved. Through innovative full-color bird target recognition, the method breaks through the mode that traditional bird recognition is only performed through shape and motion features, further improves the accuracy of bird recognition, not only plays a role in bird recognition and repelling, but also further enriches database classification information of bird interior monitoring, and improves the bird recognition efficiency. And the database content of the birds can be further enriched and improved through long-time use.

Description

Technical Field

[0001] The present application relates to the field of monitoring and bird-repelling technology, and in particular to a full-color day and night monitoring device based on computational imaging. Background Art

[0002] Day and night monitoring and bird repellent devices are usually used in special scenarios such as key power grid nodes, substations, airports, key farms, etc. to monitor people, vehicles, aircraft and birds, and to repel birds when bird targets are found. However, traditional day and night monitoring and bird repellent devices have the following major defects:

[0003] 1. Dependence on fill light: At night, traditional monitoring equipment needs to rely on fill light to achieve imaging. However, fill light not only affects the imaging effect, especially when monitoring at a long distance, but also has significant limitations on the monitoring range.

[0004] 2. Unsatisfactory imaging effect: Traditional night imaging is usually non-true color, which leads to low accuracy of target recognition, especially when color information is needed to identify birds, the recognition rate is greatly affected.

[0005] 3. Low bird recognition rate: Traditional bird recognition methods mainly rely on shape and motion features, which show low recognition rates in complex environments, especially at night or in low light conditions. Summary of the invention

[0006] The present application provides a full-color day and night monitoring device based on computational imaging, which aims to solve the problem that the prior art night imaging uses a fill light to assist, and the imaging is not true color, so it has a great impact on the recognition accuracy of the target at night, and cannot be combined with the color information of various birds for recognition, resulting in a low recognition rate.

[0007] A full-color day and night monitoring device based on computational imaging, the device comprising:

[0008] Full-color surveillance camera, which uses a low-light CMOS detector combined with computational imaging technology and a target computational imaging algorithm for full-color imaging during the day and night;

[0009] A monitoring processing module, which is based on an AI processor and is used to identify and distinguish people, vehicles, unmanned aerial vehicles and bird targets in real time, and classify and identify the targets based on their color characteristics;

[0010] Laser bird repellent, which uses a green laser combined with a target lens to focus the laser beam and form a specific beam pattern to repel most bird targets;

[0011] The communication module uses 5G communication technology to transmit monitoring information and bird-repelling information to the cloud platform in real time, and remotely update the target library through the 5G terminal;

[0012] The motion gimbal has a waterproof function of at least IP66 level, and is used to realize automatic scanning of the device and to link with the laser bird repellent and the camera.

[0013] In the above scheme, optionally, the target computation imaging algorithm of the full-color surveillance camera is based on an improved U-Net network structure, which is used to enhance the clarity and color reproduction of nighttime imaging through a combination of a denoising module and an edge information extraction module.

[0014] In the above solution, optionally, the improvement of the U-Net network structure includes:

[0015] In the denoising module, the jump connection in U-Net++ is used to make the connection of the denoising network denser, reduce the semantic gap between the feature mapping of the encoder and the decoder, and make the restoration effect of the night vision image clearer;

[0016] In the edge information extraction module, based on the VGG-16 network structure, feature learning and upsampling are performed on the edge information of the image to improve the edge texture effect of the image after denoising and generate the target color night vision image.

[0017] In the above scheme, optionally, the AI ​​processor of the monitoring processing module adopts an improved YOLO algorithm, which combines the movement characteristics and color characteristics of birds to improve the recognition accuracy of bird targets.

[0018] In the above scheme, optionally, the improvement of the YOLO algorithm includes:

[0019] In the Backbone part, residual connections and bottleneck structures are used to reduce the network size and improve performance;

[0020] In the Neck part, multi-scale feature fusion is used to fuse feature maps from different stages to enhance the feature representation capability.

[0021] In the Head part, by introducing the motion characteristics and color characteristics of birds, the movement trajectory of birds is tracked in continuous frames, and combined with the feather color distribution characteristics, the accuracy of bird recognition in complex backgrounds is improved.

[0022] In the above scheme, optionally, the U-Net network structure also realizes color image reconstruction under low light conditions by performing dimensionality reduction encoding and deconvolution processing on images taken at night, so that the final generated image is clearer and has richer color information.

[0023] In the above scheme, optionally, the YOLO algorithm predicts the bounding box and category probability directly from the complete image through a one-time evaluation, combined with the enhancement of motion features and color features, so that the algorithm has higher real-time performance and accuracy when processing target detection and classification in dynamic scenes.

[0024] In the above solution, optionally, the communication module has the function of remotely configuring and updating the device to adapt to different application scenarios and environmental requirements.

[0025] In the above scheme, optionally, the full-color surveillance camera is used to automatically adjust imaging parameters according to the real-time video stream to adapt to different lighting conditions and achieve full-color imaging during the day and night.

[0026] In the above scheme, optionally, the device is suitable for places such as airports, power grid nodes, substations and key farms that require day and night monitoring and bird repelling, and can achieve large-scale, high-precision all-weather monitoring and bird repelling functions without relying on fill lights.

[0027] Compared with the prior art, this application has at least the following beneficial effects:

[0028] Based on further analysis and research of the existing technical problems, this application recognizes that the existing technology uses a fill light to assist night imaging, and the imaging is not true color. Therefore, it has a great impact on the recognition accuracy of the target at night, and cannot be combined with the color information of various birds for recognition, resulting in a low recognition rate. This application is based on a full-color day and night monitoring and bird-repelling device based on computational imaging. By combining a low-light CMOS detector and computational imaging technology, full-color imaging during the day and at night is achieved without a fill light, which significantly improves the night monitoring effect. At the same time, the linkage function of the laser bird repellent and the motion gimbal makes the monitoring and bird-repelling process more efficient. This application uses an innovative computational imaging algorithm combined with an AI processor platform to achieve a full-color imaging effect at night, so that night imaging has very rich color information like daytime, greatly improving the accuracy of back-end target monitoring and recognition, and at the same time, the range and distance of monitoring are greatly improved (not dependent on the distance of the fill light). Through innovative full-color bird target recognition, it breaks through the traditional bird recognition method of only relying on shape and movement characteristics, and further improves the accuracy of bird recognition. It not only plays a bird identification and repelling function, but also further enriches the database classification information of bird monitoring. Long-term use can further enrich and improve the database content of birds. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A block diagram of the module architecture of a full-color day and night monitoring device based on computational imaging provided by one embodiment of the present application;

[0030] Figure 2A deep learning U-Net denoising network structure diagram provided for one embodiment of the present application;

[0031] Figure 3 A comparison chart of night imaging effects using the solution of the present application and the prior art is provided for one embodiment of the present application. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0033] In the description of this application: unless otherwise specified, the meaning of "plurality" is two or more. The terms "first", "second", "third", etc. in this application are intended to distinguish the objects referred to, and do not have special meanings in terms of technical connotations (for example, they should not be understood as emphasizing the importance or order, etc.). Expressions such as "including", "comprising", "having", etc. also mean "not limited to" (certain units, components, materials, steps, etc.).

[0034] In one embodiment, Figure 1 As shown, a full-color day and night monitoring device based on computational imaging is provided, the device comprising:

[0035] Full-color surveillance camera, which uses a low-light CMOS detector combined with AI technology and a target computing imaging algorithm for full-color imaging during the day and night;

[0036] A monitoring processing module, which is based on an AI processor and is used to identify and distinguish people, vehicles, unmanned aerial vehicles and bird targets in real time, and classify and identify the targets based on their color characteristics;

[0037] Laser bird repellent, which uses a green laser combined with a target lens to focus the laser beam and form a specific beam pattern to repel most bird targets;

[0038] The communication module uses 5G communication technology to transmit monitoring information and bird-repelling information to the cloud platform in real time, and remotely update the target library through the 5G terminal;

[0039] The motion gimbal has a waterproof function of at least IP66 level, and is used to realize automatic scanning of the device and to link with the laser bird repellent and the camera.

[0040] The full-color day and night monitoring and bird-repelling device based on computational imaging provided in this embodiment realizes full-color imaging during the day and night without fill light conditions by combining low-light CMOS detectors and AI technology, which significantly improves the night monitoring effect. At the same time, the linkage function of the laser bird repellent and the motion gimbal makes the monitoring and bird-repelling process more efficient.

[0041] In this embodiment, the target computation imaging algorithm of the full-color surveillance camera is based on an improved U-Net network structure, and is used to enhance the clarity and color reproduction of nighttime imaging through a combination of a denoising module and an edge information extraction module.

[0042] In this embodiment, the improvement of the U-Net network structure includes:

[0043] In the denoising module, the jump connection in U-Net++ is used to make the connection of the denoising network denser, reduce the semantic gap between the feature mapping of the encoder and the decoder, and make the restoration effect of the night vision image clearer;

[0044] In the edge information extraction module, based on the VGG-16 network structure, feature learning and upsampling are performed on the edge information of the image to improve the edge texture effect of the image after denoising and generate the target color night vision image.

[0045] like Figure 2 Figure 3 As shown, the innovative computational imaging algorithm (combined with the AI ​​processor platform) trains and learns the imaging of cameras using low-light CMOS detectors, and then applies the training and learning results to the processing of real-time video streams to generate color night vision images;

[0046] This algorithm has made improvements on the U-Net network structure, and is mainly composed of two parts: a denoising module and an edge information extraction module. First, in the denoising module, the original U-Net network structure (as shown in the attached figure) is used as the basic architecture, and the encoder on the left is used for downsampling and the decoder on the right is used for upsampling. In the encoding process, the original night vision image captured is processed by dimensionality reduction encoding using convolution and pooling operations in sequence, and then the decoding stage is entered. Deconvolution and upsampling are used to double the size of the feature map received from the encoding process. At the same time, the feature map obtained from the left is merged, and finally the processed image is obtained in the last layer of the network structure. On this basis, this algorithm innovatively applies the jump connection in U-Net++ to the original network, which can make the denoising network connection more dense, thereby achieving the purpose of reducing the semantic gap between the encoder and decoder feature maps, making the restored image clearer. Compared with the original network, the improved U-Net network structure uses more jump connections, which makes it easier to handle feature extraction and decoding recovery tasks. Next is the edge information extraction module. Since the edge texture effect of the image obtained by the previous algorithm after denoising needs to be improved, we consider adding an edge information extraction module with VGG-16 as the network architecture, performing feature learning and upsampling on the feature map, and then concatenating it into the final feature vector to finally restore the image. This edge information extraction module can not only extract features from the image processed by the denoising network, but also back-propagate the loss parameters of the denoised image, thereby improving the performance of the denoising network and ultimately restoring a more realistic color night vision image.

[0047] This embodiment uses a computational imaging algorithm based on an improved U-Net network structure, and the present invention can generate clearer and more realistic color night vision images, further improving the monitoring and recognition accuracy of nighttime targets.

[0048] In this embodiment, the AI ​​processor of the monitoring processing module adopts an improved YOLO algorithm, which combines the movement characteristics and color characteristics of birds to improve the recognition accuracy of bird targets.

[0049] Based on the traditional bird identification method based on shape and motion characteristics, an innovative full-color bird target recognition method has been added (also through AI training, learning and reasoning).

[0050] The present invention adopts the AI ​​target recognition YOLO algorithm, combines the motion characteristics of moving birds, and color characteristics to accurately identify bird targets.

[0051] YOLO is a target detection algorithm based on convolutional neural networks. Its characteristic is that it can directly predict bounding boxes and category probabilities from the complete image in one evaluation, and directly optimize the detection performance end-to-end. The YOLO algorithm first divides the input image into multiple grids, each of which is responsible for predicting one or more bounding boxes and the classification probability of the targets to which these boxes belong. Specifically, the YOLO algorithm uses a single neural network model to predict the target bounding boxes and categories in the image. When the network processes the entire image, it outputs all the prediction results at once, which makes YOLO have extremely fast detection speed and also shows high accuracy when processing complex scenes.

[0052] The network structure of the algorithm mainly consists of three parts: Backbone, Neck and Head. The Backbone is responsible for feature extraction and uses a series of convolution and deconvolution layers. It also uses residual connections and bottleneck structures to reduce the size of the network and improve performance. The Neck is used for multi-scale feature fusion, which enhances feature representation capabilities by fusing feature maps from different stages of the Backbone. The Head is responsible for final target detection and classification, and includes a detection head and a classification head. The detection head contains a series of convolutional and deconvolutional layers to generate detection results. The classification head uses global average pooling to classify each feature map. By reducing the dimension of the feature map, it outputs the probability distribution of different categories.

[0053] The improved algorithm improves the recognition accuracy by introducing the motion and color features of birds on the basis of YOLO. For motion features, the algorithm tracks the motion trajectory of birds in continuous frames, which helps the model better understand the behavior patterns of birds and effectively distinguish other moving objects. The introduction of color features can further improve the recognition accuracy by analyzing the unique feather color distribution of birds. The fusion of these features makes the improved YOLO algorithm not only perform well in still images, but also can identify flying birds in video streams, especially for those with complex backgrounds or birds with similar colors to the background, which can more accurately detect and identify bird targets. The entire recognition process still relies on the end-to-end structure of the YOLO model, from the input image to the output of the bounding box and category label of the bird target, ensuring real-time and high efficiency. At the same time, combined with the enhancement of motion and color features, the model has higher robustness and reliability when processing dynamic scenes.

[0054] This embodiment uses an improved U-Net network structure through the combination of a denoising module and an edge information extraction module to significantly improve the clarity and detail expression of the generated night vision image, especially under low light conditions, and can provide richer visual information.

[0055] The improved YOLO algorithm achieves more accurate bird target recognition by combining the motion and color characteristics of birds, especially in complex backgrounds, effectively improving the recognition accuracy and efficiency in dynamic scenes.

[0056] In this embodiment, the improvements of the YOLO algorithm include:

[0057] In the Backbone part, residual connections and bottleneck structures are used to reduce the network size and improve performance;

[0058] In the Neck part, multi-scale feature fusion is used to fuse feature maps from different stages to enhance the feature representation capability.

[0059] In the Head part, by introducing the motion characteristics and color characteristics of birds, the movement trajectory of birds is tracked in continuous frames, and combined with the feather color distribution characteristics, the accuracy of bird recognition in complex backgrounds is improved.

[0060] In this embodiment, the U-Net network structure also realizes color image reconstruction under low light conditions by performing dimensionality reduction encoding and deconvolution processing on images taken at night, so that the final generated image is clearer and has richer color information.

[0061] The improved U-Net network structure performs dimensionality reduction encoding and deconvolution processing on images taken at night, so that high-quality color images can be generated even under low-light conditions, thereby enhancing the operability and effect of night monitoring.

[0062] In this embodiment, the YOLO algorithm directly predicts the bounding box and category probability from the complete image through a one-time evaluation, combined with the enhancement of motion features and color features, so that the algorithm has higher real-time performance and accuracy when processing target detection and classification in dynamic scenes.

[0063] In this embodiment, the communication module has the function of remotely configuring and updating the device to adapt to different application scenarios and environmental requirements.

[0064] In this embodiment, the full-color surveillance camera is used to automatically adjust imaging parameters according to the real-time video stream to adapt to different lighting conditions and achieve full-color imaging during the day and night.

[0065] This embodiment is suitable for places such as airports, power grid nodes, substations and key farms. It can achieve large-scale, high-precision all-weather monitoring and bird-repelling functions without relying on fill lights, significantly improving the monitoring and safety in these scenarios.

[0066] In this embodiment, the device is suitable for places such as airports, power grid nodes, substations and key farms that require day and night monitoring and bird repelling, and can achieve large-scale, high-precision all-weather monitoring and bird repelling functions without relying on fill lights.

[0067] This embodiment is a full-color day and night monitoring and bird-repelling device based on computational imaging. By combining low-light CMOS detectors and AI technology, full-color imaging is achieved during the day and night without fill light, which significantly improves the night monitoring effect. At the same time, the linkage function of the laser bird repellent and the motion pan-tilt makes the monitoring and bird-repelling process more efficient. This application achieves a full-color imaging effect at night through an innovative computational imaging algorithm combined with an AI processor platform, so that night imaging has very rich color information like daytime, greatly improving the accuracy of back-end target monitoring and recognition, while the monitoring range and distance are greatly improved (not dependent on the distance of the fill light). Through innovative full-color bird target recognition, it breaks through the traditional bird recognition method of only using shape and motion characteristics, further improving the accuracy of bird recognition, not only playing a bird recognition and repelling function, but also further enriching the database classification information of bird monitoring. Long-term use can further enrich and improve the database content of birds.

[0068] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A full-color day and night monitoring device based on computational imaging, characterized in that: The device includes: Full-color surveillance camera, which uses a low-light CMOS detector combined with computational imaging technology and a target computational imaging algorithm for full-color imaging during the day and night; A monitoring processing module, which is based on an AI processor and is used to identify and distinguish people, vehicles, unmanned aerial vehicles and bird targets in real time, and classify and identify the targets based on their color characteristics; Laser bird repellent, which uses a green laser combined with a target lens to focus the laser beam and form a specific beam pattern to repel most bird targets; The communication module uses 5G communication technology to transmit monitoring information and bird-repelling information to the cloud platform in real time, and remotely update the target library through the 5G terminal; The motion gimbal has a waterproof function of at least IP66 level, and is used to realize automatic scanning of the device and to link with the laser bird repellent and the camera.

2. The device according to claim 1, characterized in that The target computation imaging algorithm of the full-color surveillance camera is based on an improved U-Net network structure and is used to enhance the clarity and color reproduction of nighttime imaging through a combination of a denoising module and an edge information extraction module.

3. The device according to claim 2, characterized in that The improvements of the U-Net network structure include: In the denoising module, the jump connection in U-Net++ is used to make the connection of the denoising network denser, reduce the semantic gap between the feature mapping of the encoder and the decoder, and make the restoration effect of the night vision image clearer; In the edge information extraction module, based on the VGG-16 network structure, feature learning and upsampling are performed on the edge information of the image to improve the edge texture effect of the image after denoising and generate the target color night vision image.

4. The device according to claim 1, characterized in that The AI ​​processor of the monitoring processing module adopts an improved YOLO algorithm, which combines the movement characteristics and color characteristics of birds to improve the recognition accuracy of bird targets.

5. The device according to claim 4, characterized in that Improvements to the YOLO algorithm include: In the Backbone part, residual connections and bottleneck structures are used to reduce the network size and improve performance; In the Neck part, multi-scale feature fusion is used to fuse feature maps from different stages to enhance the feature representation capability. In the Head part, by introducing the motion characteristics and color characteristics of birds, the movement trajectory of birds is tracked in continuous frames, and combined with the feather color distribution characteristics, the accuracy of bird recognition in complex backgrounds is improved.

6. The device according to claim 3, characterized in that The U-Net network structure also achieves color image reconstruction under low-light conditions by performing dimensionality reduction encoding and deconvolution processing on images taken at night, making the final generated image clearer and having richer color information.

7. The device according to claim 5, characterized in that The YOLO algorithm predicts bounding boxes and category probabilities directly from the complete image through a one-time evaluation, combined with the enhancement of motion features and color features, so that the algorithm has higher real-time performance and accuracy when processing target detection and classification in dynamic scenes.

8. The device according to claim 1, characterized in that The communication module has the function of remotely configuring and updating the device to adapt to different application scenarios and environmental requirements.

9. The device according to claim 1, characterized in that The full-color monitoring camera is used to automatically adjust imaging parameters according to the real-time video stream to adapt to different lighting conditions and achieve full-color imaging during the day and night.

10. The device according to claim 1, characterized in that The device is suitable for places such as airports, power grid nodes, substations and key farms that require day and night monitoring and bird repelling. It can achieve large-scale, high-precision all-weather monitoring and bird repelling functions without relying on fill lights.

Images