A full-color day-and-night monitoring device based on computed imaging

Abstract

The application discloses a kind of full-color day and night monitoring devices based on computing imaging, through combining low-illumination CMOS detector and AI technology, realize in the condition without light compensation lamp day and night full-color imaging, significantly improve night monitoring effect, through the innovative computing imaging algorithm, in combination with AI processor platform, realize night full-color imaging effect, so that night imaging and day have very rich color information, greatly improve the accuracy of target monitoring and identification in rear end, while the range and distance of monitoring are greatly improved.Innovative full-color bird target identification, break through the traditional bird identification only through shape and motion characteristics, further improve the accuracy of bird identification, not only play the function of bird identification and drive, but also further enrich the database classification information of bird monitoring, long time use can further enrich and improve the database content of bird.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of monitoring and bird repelling, in particular to a full-color day and night monitoring device based on computational imaging. BACKGROUND

[0002] The day and night monitoring and bird repelling device is usually applied to key power grid nodes, substations, airports, key farms and other special scenes to monitor people, vehicles, aircraft and birds, and to repel birds when bird targets are found. However, the traditional day and night monitoring and bird repelling device has the following main defects:

[0003] 1. Dependence on fill light: At night, the traditional monitoring equipment needs to rely on fill light to achieve imaging. However, fill light not only affects the imaging effect, especially at a long distance, but also has a significant limitation on the monitoring range.

[0004] 2. Non-ideal imaging effect: Traditional night imaging is usually non-true color, which leads to low accuracy of target identification, especially when color information is needed for bird identification, the identification rate is greatly affected.

[0005] 3. Low bird identification rate: The traditional bird identification method mainly relies on shape and motion features, which shows low identification rate in complex environments, especially at night or in insufficient light. SUMMARY

[0006] The present application provides a full-color day and night monitoring device based on computational imaging, which aims to solve the problem that the existing technology uses fill light to assist night imaging, and the imaging is non-true color, which greatly affects the accuracy of target identification at night, and cannot identify the color information of various birds, resulting in low identification rate.

[0007] A full-color day and night monitoring device based on computational imaging, the device comprises:

[0008] A full-color monitoring camera, the camera uses a low-illumination CMOS detector combined with computational imaging technology, and uses a target computational imaging algorithm for day and night full-color imaging;

[0009] A monitoring processing module, the monitoring processing module is based on an AI processor, for real-time identification and differentiation of people, vehicles, unmanned aerial vehicles and bird targets, and classification identification combined with the color characteristics of the targets;

[0010] A laser bird repeller, the laser bird repeller uses green laser combined with a target lens to focus the laser beam and form a specific beam pattern for repelling most bird targets;

[0011] A communication module, which is selected from 5G communication technology, transmits monitoring information and bird repelling information to a cloud platform in real time, and remotely updates a target library through a 5G terminal;

[0012] A motion gimbal, which has at least IP66 level waterproof function, is used to realize automatic scanning of the device and linkage with the laser bird repeller and the camera.

[0013] In the above scheme, optionally, the target calculation imaging algorithm of the full-color monitoring camera is based on an improved U-Net network structure, and through the combination of a denoising module and an edge information extraction module, the clarity and color restoration of night imaging are enhanced.

[0014] In the above scheme, optionally, the improvement of the U-Net network structure includes:

[0015] In the denoising module, the skip connection in U-Net++ is adopted, so that the connection of the denoising network is more intensive, the semantic gap of feature mapping between the encoder and the decoder is reduced, and the restoration effect of the night vision image is better and clearer;

[0016] In the edge information extraction module, based on the VGG-16 network structure, the edge information of the image is feature learned and up-sampled, the edge texture effect of the image after denoising processing is improved, and a target color night vision image is generated.

[0017] In the above scheme, optionally, the AI processor of the monitoring processing module adopts an improved YOLO algorithm, which combines the motion features and color features of birds to improve the recognition accuracy of bird targets.

[0018] In the above scheme, optionally, the improvement of the YOLO algorithm includes:

[0019] In the Backbone part, residual connection and bottleneck structure are adopted to reduce network size and improve performance;

[0020] In the Neck part, multi-scale feature fusion is used to fuse feature maps from different stages to enhance feature representation capability;

[0021] In the Head part, the motion features and color features of birds are introduced to track the motion trajectory of birds in consecutive frames, and the feather color distribution features are combined to improve the bird recognition accuracy in complex backgrounds.

[0022] In the above scheme, optionally, the U-Net network structure also realizes color image reconstruction under low light conditions by dimensionality reduction coding and deconvolution processing of night shooting images, so that the finally generated image is clearer and has more rich color information.

[0023] In the above scheme, optionally, the YOLO algorithm directly predicts bounding boxes and class probabilities from complete images through one-time evaluation, combined with the enhancement of motion features and color features, so that the algorithm has higher real-time performance and accuracy in processing target detection and classification in dynamic scenes.

[0024] In the above scheme, optionally, the communication module has the function of remotely configuring and updating the device to adapt to different application scenarios and environmental requirements.

[0025] In the above scheme, optionally, the full-color monitoring camera is used to automatically adjust the imaging parameters according to the real-time video stream to adapt to different lighting conditions and realize day and night full-color imaging.

[0026] In the above scheme, optionally, the device is suitable for places such as airports, power grid nodes, substations and key farms that need day and night monitoring and bird repelling, and can realize large-scale, high-precision all-weather monitoring and bird repelling without relying on light supplementing lamps.

[0027] Compared with the prior art, the present application has at least the following beneficial effects:

[0028] Based on further analysis and research of the problems of the prior art, it is realized that the prior art night imaging adopts a light supplementing lamp assisted mode, and the imaging is not true color, so the night target recognition accuracy has a great influence, cannot combine the color information of various birds for recognition, and the recognition rate is low. Based on the full-color day and night monitoring and bird repelling device based on computational imaging, by combining a low-illumination CMOS detector and a computational imaging technology, day and night full-color imaging is realized without a light supplementing lamp, which significantly improves the night monitoring effect. At the same time, the linkage function of the laser bird repeller and the motion gimbal makes the monitoring and bird repelling process more efficient. Through the innovative computational imaging algorithm combined with the AI processor platform, the night full-color imaging effect is realized, so that the night imaging has very rich color information as in the daytime, greatly improving the accuracy of target monitoring and recognition in the back end, and the range and distance of monitoring are greatly improved (distance without relying on light supplementing lamps). Through the innovative full-color bird target recognition, the traditional bird recognition method of only relying on shape and motion features is broken through, further improving the accuracy of bird recognition, not only playing a role in bird recognition and repelling, but also further enriching the database classification information of bird monitoring, and long-term use can further enrich and improve the database content of birds. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 The module architecture block diagram of the full-color day and night monitoring device based on computational imaging provided by an embodiment of the present application is shown in the following figure:

[0030] Figure 2A deep learning U-Net denoising network structure diagram provided for an embodiment of the present application is shown in the following figure:

[0031] Figure 3 A comparison diagram of the effect of the night imaging picture of the present application scheme and the prior art is provided for an embodiment of the present application. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical scheme and advantages of the present application more clear, the present application is further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0033] In the description of the present application: unless otherwise specified, the meaning of "multiple" is two or more. The terms "first", "second", "third" and the like in the present application are intended to distinguish the objects referred to, and do not have special meaning in the technical connotation aspect (for example, it should not be understood as emphasizing the importance or order, etc.). The expressions such as "include", "contain", "have" and the like also mean "not limited to" (some units, components, materials, steps, etc.).

[0034] In one embodiment, as shown in Figure 1 A full-color day and night monitoring device based on computational imaging is provided, which comprises:

[0035] A full-color monitoring camera, which uses a low-illumination CMOS detector combined with AI technology, and uses a target computational imaging algorithm for day and night full-color imaging;

[0036] A monitoring processing module based on an AI processor, which is used for real-time identification and differentiation of human, vehicle, unmanned aerial vehicle and bird targets, and classification identification combined with the color characteristics of the target;

[0037] A laser bird repeller, which uses green laser combined with target lens to focus laser beam and form a specific beam pattern, for driving most bird targets;

[0038] A communication module, which selects 5G communication technology, transmits monitoring information and bird repelling information to the cloud platform in real time, and updates the target library remotely through the 5G terminal;

[0039] A motion cloud platform, which has at least IP66 level waterproof function, is used to realize the automatic scanning of the device and the linkage with the laser bird repeller and the camera.

[0040] The full-color day-and-night monitoring and bird repelling device based on computational imaging provided by the embodiment realizes day-and-night full-color imaging under the condition of no supplementary light through the combination of a low-illumination CMOS detector and AI technology, significantly improves the night monitoring effect, and simultaneously, the linkage function of the laser bird repelling device and the motion pan-tilt makes the monitoring and bird repelling process more efficient.

[0041] In the embodiment, the target computational imaging algorithm of the full-color monitoring camera is based on an improved U-Net network structure, and through the combination of a denoising module and an edge information extraction module, is used to enhance the definition and color restoration of night imaging.

[0042] In the embodiment, the improvement of the U-Net network structure includes:

[0043] In the denoising module, the skip connection in U-Net++ is adopted, so that the connection of the denoising network is more intensive, the semantic gap of feature mapping between the encoder and the decoder is reduced, and the restoration effect of the night vision image is better and clearer;

[0044] In the edge information extraction module, based on the VGG-16 network structure, the edge information of the image is subjected to feature learning and up-sampling, the edge texture effect of the image after denoising processing is improved, and the target color night vision image is generated.

[0045] As shown in Figure 2 Figure 3 The innovative computational imaging algorithm (combined with an AI processor platform) trains and learns the camera imaging using a low-illumination CMOS detector, and then applies the training and learning results to the processing of real-time video streams to generate color night vision images.

[0046] The algorithm is improved on the U-Net network structure, and mainly consists of a denoising module and an edge information extraction module. First, in the denoising module part, the original U-Net network structure (as shown in the drawing) is used as the basic framework, the left encoder is used for downsampling, and the right decoder is used for upsampling. In the encoding process, convolution and pooling operations are used in turn to reduce the dimension of the original night vision image, and then enter the decoding stage, use deconvolution and upsampling to double the size of the feature map received from the encoding process, at the same time, merge the feature map obtained from the left side, and finally get the processed image in the last layer of the network structure. On this basis, the algorithm innovatively applies the skip connection in U-Net++ to the original network, so that the denoising network connection is more dense, thereby reducing the semantic gap between the encoder and the decoder feature mapping, making the restored image clearer. Compared with the original network, the improved U-Net network structure uses more skip connections, and is easier to handle feature extraction and decoding recovery tasks. Secondly, the edge information extraction module part, since the image obtained after the previous algorithm is denoised needs to be improved in edge texture effect, therefore, a VGG-16 network architecture edge information extraction module is added to the image, which learns and upsamples the feature map, and then concatenates into the final feature vector, and finally restores the image. The edge information extraction module not only extracts features from the denoised image, but also backpropagates the loss parameters of the denoised image, thereby improving the performance of the denoising network, and finally restoring a more realistic color night vision image.

[0047] In this embodiment, based on the improved U-Net network structure, the application can generate clearer and more realistic color night vision images, further improving the monitoring and identification accuracy of night targets.

[0048] In this embodiment, the AI processor of the monitoring processing module adopts an improved YOLO algorithm, which combines the motion characteristics and color characteristics of birds, and improves the identification accuracy of bird targets.

[0049] On the basis of the traditional bird identification through shape and motion characteristics, an innovative full-color bird target identification method (also through AI training and reasoning) is added.

[0050] The application adopts the AI target recognition YOLO algorithm, combines the motion characteristics and color characteristics of the moving birds, and accurately identifies the bird targets.

[0051] YOLO is a target detection algorithm based on convolutional neural network, which can directly predict bounding boxes and class probabilities from complete images in one evaluation, and directly optimize the detection performance end-to-end. YOLO algorithm first divides the input image into multiple grids, and each grid is responsible for predicting one or more bounding boxes and the classification probability of the target belonging to these boxes. Specifically, YOLO algorithm uses a single neural network model to predict the target bounding box and class in the image. The network outputs all prediction results at once when processing the entire image, which makes YOLO have extremely fast detection speed, and also shows high accuracy when dealing with complex scenes.

[0052] The network structure of the algorithm mainly consists of three parts: Backbone part, Neck part and Head part. The Backbone part is responsible for feature extraction, which uses a series of convolution and deconvolution layers, and uses residual connection and bottleneck structure to reduce the size of the network and improve performance. The Neck part is used for multi-scale feature fusion, which enhances the feature representation ability by fusing feature maps from different stages of Backbone. The role of Head part is the final target detection and classification, including a detection head and a classification head. The detection head contains a series of convolution and deconvolution layers for generating detection results, and the classification head uses global average pooling to classify each feature map, which reduces the dimension of the feature map and outputs the probability distribution of different classes.

[0053] The improved algorithm introduces the motion features and color features of birds to improve the accuracy of recognition. For motion features, the algorithm tracks the motion trajectory of birds in consecutive frames, helping the model better understand the behavior patterns of birds and effectively distinguish other moving objects. The introduction of color features can further improve the recognition accuracy by analyzing the unique color distribution of bird feathers. The fusion of these features makes the improved YOLO algorithm not only perform well in still images, but also able to recognize flying birds in video streams, especially in complex backgrounds or when the color of the bird is similar to the background, it can more accurately detect and recognize bird targets. The entire recognition process still relies on the end-to-end structure of the YOLO model, from input image to output bird target bounding box and class label, ensuring real-time and high efficiency, while combining motion and color features to enhance the model's robustness and reliability in handling dynamic scenes.

[0054] The improved U-Net network structure in this embodiment combines the denoising module and the edge information extraction module, which significantly improves the clarity and detail performance of the generated night vision image, especially in low light conditions, providing more rich visual information.

[0055] The improved YOLO algorithm combines the motion features and color features of birds to achieve more accurate bird target recognition, especially in complex backgrounds, effectively improving the recognition accuracy and efficiency in dynamic scenes.

[0056] In this embodiment, the improvement of the YOLO algorithm includes:

[0057] In the Backbone part, residual connection and bottleneck structure are adopted to reduce network size and improve performance;

[0058] In the Neck part, multi-scale feature fusion is used to fuse feature maps from different stages, enhancing the feature representation ability;

[0059] In the Head part, by introducing the motion features and color features of birds, the motion trajectory of birds in consecutive frames is tracked, and the feather color distribution features are combined to improve the bird recognition accuracy in complex backgrounds.

[0060] In this embodiment, the U-Net network structure also realizes the reconstruction of color images in low light conditions by dimensionality reduction encoding and deconvolution processing of night shooting images, making the finally generated images clearer and with more rich color information.

[0061] The improved U-Net network structure realizes the reconstruction of color images in low light conditions by dimensionality reduction encoding and deconvolution processing of night shooting images, so as to enhance the operability and effect of night monitoring.

[0062] In this embodiment, the YOLO algorithm directly predicts bounding boxes and class probabilities from complete images through one-time evaluation, combined with the enhancement of motion features and color features, so that the algorithm has higher real-time and accuracy in processing dynamic scene target detection and classification.

[0063] In this embodiment, the communication module has the function of remotely configuring and updating the device to adapt to different application scenarios and environmental requirements.

[0064] In this embodiment, the full-color monitoring camera is used to automatically adjust the imaging parameters according to the real-time video stream to adapt to different lighting conditions and realize day and night full-color imaging.

[0065] This embodiment is suitable for airports, power grid nodes, substations and key farms, etc. It can realize large-scale, high-precision all-weather monitoring and bird repelling function without relying on light supplementing lamps, significantly improving the monitoring and safety in these scenes.

[0066] In the embodiment, the device is suitable for places such as airports, power grid nodes, substations and key farms that need day and night monitoring and bird repelling. Without relying on light supplement lamps, the device realizes large-range, high-precision all-weather monitoring and bird repelling functions.

[0067] The embodiment is a full-color day and night monitoring and bird repelling device based on computational imaging. By combining a low-illumination CMOS detector and AI technology, the device realizes full-color imaging at day and night without light supplement lamps, significantly improving night monitoring effect. Meanwhile, the linkage function of the laser bird repeller and the motion gimbal makes the monitoring and bird repelling process more efficient. The application realizes night full-color imaging effect through innovative computational imaging algorithm combined with an AI processor platform, so that night imaging has very rich color information like daytime, greatly improving the accuracy of target monitoring and identification in the back end, and greatly improving the range and distance of monitoring (distance without relying on light supplement lamps). Through innovative full-color bird target identification, the application breaks through the traditional bird identification method that only uses shape and motion features, further improves the accuracy of bird identification, not only plays a role in bird identification and repelling, but also further enriches the database classification information of bird monitoring, and long-term use can further enrich and improve the database content of birds.

[0068] The technical features of the above embodiments can be combined in any manner. To make the description concise, not all possible combinations of the technical features in the above embodiments are described, however, as long as the combinations of the technical features do not exist contradictions, they should be considered as the scope of the present disclosure.

Claims

1. A full color day and night monitoring device based on computed imaging, characterized in that, The device comprises: a full-color monitoring camera, which adopts a low-illumination CMOS detector combined with a computing imaging technology, and uses a target computing imaging algorithm for day-and-night full-color imaging; a monitoring processing module, which is based on an AI processor and is used for real-time identification and differentiation of human, vehicle, unmanned aerial vehicle and bird targets, and classification identification combined with color features of the targets; a laser bird repeller, which adopts a green laser combined with a target lens to focus a laser beam and form a specific beam pattern, and is used for driving away bird targets; a communication module, which selects 5G communication technology to transmit monitoring information and bird repelling information to a cloud platform in real time, and performs remote updating of a target library through a 5G terminal; a motion gimbal, which has an IP66 level waterproof function and is used for automatic scanning of the device and linkage with the laser bird repeller and the camera.

2. The apparatus of claim 1, wherein, The target computing imaging algorithm of the full-color monitoring camera is based on an improved U-Net network structure, and through the combination of a denoising module and an edge information extraction module, it is used for enhancing the clarity and color restoration of night imaging.

3. The apparatus of claim 2, wherein, The improvements of the U-Net network structure include: In the denoising module, the skip connection in U-Net++ is adopted, so that the connection of the denoising network is more intensive, the semantic gap between the encoder and the decoder is reduced, and the restoration effect of the night vision image is better and clearer; In the edge information extraction module, based on the VGG-16 network structure, the edge information of the image is feature learned and up-sampled to improve the edge texture effect of the image after denoising processing, and a target color night vision image is generated.

4. The apparatus of claim 1, wherein, The AI processor of the monitoring processing module adopts an improved YOLO algorithm, which combines the motion features and color features of birds to improve the identification accuracy of bird targets.

5. The apparatus of claim 4, wherein, The improvements of the YOLO algorithm include: In the Backbone part, residual connection and bottleneck structure are adopted to reduce network size and improve performance; In the Neck part, multi-scale feature fusion is used to fuse feature maps from different stages to enhance feature representation capability; In the Head part, the motion features and color features of birds are introduced to track the motion trajectory of birds in consecutive frames, and the feather color distribution features are combined to improve the bird identification accuracy in complex backgrounds.

6. The apparatus of claim 3, wherein, The U-Net network structure also realizes color image reconstruction under low light conditions through dimensionality reduction coding and deconvolution processing of night shooting images, so that the finally generated image is clearer and has more rich color information.

7. The apparatus of claim 5, wherein, The YOLO algorithm directly predicts bounding boxes and class probabilities from complete images through one-time evaluation, and the enhancement of motion features and color features makes the algorithm have higher real-time performance and accuracy in processing target detection and classification in dynamic scenes.

8. The apparatus of claim 1, wherein, The communication module has the function of remotely configuring and updating the device to adapt to different application scenarios and environmental requirements.

9. The apparatus of claim 1, wherein, The full-color monitoring camera is used to automatically adjust the imaging parameters according to the real-time video stream to adapt to different lighting conditions and realize day-and-night full-color imaging.

10. The apparatus of claim 1, wherein, The device is suitable for places such as airports, power grid nodes, substations and farms that need day and night monitoring and bird repelling, and can realize large-range, high-precision all-weather monitoring and bird repelling without relying on light supplementing lamps.

Images