A true color night vision sight with target detection and highlighting function
By using high-resolution CMOS sensors and deep learning algorithms for object detection and highlighting in night vision scopes, the problems of imaging of existing night vision sights in pseudo-color, low resolution and poor penetration ability are solved, and the problem of imaging of existing night vision sights is solved, thus achieving the goal of realizing the Efficient target detection and display effects.
Patent Information
- Application Number
- CN202211355411.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-11-01
- Publication Date
- 2024-11-29
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing night vision sights are imaging in pseudo-color, the thermal imaging sensor has low resolution and low image contrast, and the thermal imaging sight cannot penetrate glass, fog, or high temperatures.
A true color night vision scope was designed, using a 1080P resolution CMOS sensor, combined with a high-performance ISP image processor, to quickly acquire high-definition images, and through neural network NPU hardware, and using deep learning algorithms (MobileNet SSD) for object detection. Highlight the contour detection algorithm.
The high resolution and high detail resolution of true color glow night vision scopes can be achieved, allowing for easier detection and highlighting of targets while penetrating certain environmental obstacles.
Smart Images

Figure CN116086245B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the technical field of sights, in particular to a true color night vision sight with a target detection and highlighting function. Background Art
[0002] At present, night vision sights mainly include low-light and infrared thermal imaging sights. Thermal imaging sights use thermal radiation meters to detect heat. Thermal radiation meters can measure the temperature difference between a heating object and its surroundings, and then use the points with this temperature difference to form an image. Low-light sights focus the weak light reflected by the target through the objective lens and form an image on the cathode surface of the image intensifier, amplify the infrared light step by step, and convert it into visible light, forming an image with sufficient brightness and clarity on the last stage of the fluorescent screen.
[0003] The current patent application number is CN201510727481.0. A night vision sight that combines infrared and low light has the following disadvantages:
[0004] 1. The image is in pseudo-color, which is different from people's normal visual habits. The reason why thermal imaging can form an image is the temperature difference between the object (human body) and the environment. If the temperature difference does not exist, the imaging effect of the thermal imaging scope will be very poor.
[0005] 2. The resolution of thermal imaging sensors is relatively low, the image contrast is low, and the ability to distinguish details is poor. Since infrared thermal imaging relies on temperature difference imaging, and the temperature difference of general targets is not large, the contrast of thermal imaging images is low, which makes the ability to distinguish details poor.
[0006] 3. Thermal imaging scopes cannot penetrate glass, fog, or high temperatures. Since infrared thermal imaging relies on temperature differences, once the temperature difference is blocked, no image can be formed. Summary of the invention
[0007] The purpose of the present invention is to provide a true color night vision sight with target detection and highlighting function, so as to solve the problems in the above-mentioned background technology that the existing night vision sights have false color imaging, the resolution of thermal imaging sensors is relatively low, and the thermal imaging sights cannot penetrate glass, fog, and high temperature. Since infrared thermal imaging relies on temperature difference, once the temperature difference is blocked, imaging cannot be performed.
[0008] To achieve the above-mentioned purpose, the present invention provides the following technical solutions: A true color night vision sight with target detection and highlighting function comprises an objective lens, a body, an eyepiece, an OLED display screen, a main board and a CMOS chip, wherein the objective lens is installed at the left end of the body, the eyepiece is installed at the right end of the body, the main board is installed inside the body, the CMOS chip is installed inside the objective lens, and the main board is installed inside the eyepiece.
[0009] Preferably, the mainboard is connected to the OLED display screen and the CMOS chip control.
[0010] Preferably, the mainboard adopts ISP and NPU processors, wherein the ISP processor is used for digital image processing, the NPU processor is used for neural network processing, and the deep learning neural network is used for target detection.
[0011] Preferably, the objective lens is used to magnify an object.
[0012] Preferably, the CMOS chip is used to convert optical signals into electrical signals.
[0013] Preferably, the eyepiece further magnifies the real image magnified by the objective lens.
[0014] Preferably, the OLED display screen is a display screen used for displaying processed images.
[0015] The working steps of a true color night vision sight with target detection and highlighting function are as follows:
[0016] Step 1: The CMOS chip converts the original RAW data into an NV12 format image through the ISP processor;
[0017] Step 2: The NV12 image data is processed by the NPU processor for MobileNet and SSD target detection, and the target image is output;
[0018] Step 3: Binarization of the grayscale of the target image detected by the NPU processor;
[0019] Step 4: Use Sobel algorithm to extract the contour edge of the binary image;
[0020] Step 5: Map the contour edge pixels into highlight colors in the NV12 image;
[0021] Step 6. Display the NV12 image on the OLED display.
[0022] In the MobileNet and SSD target detection step in step 2, the MobileNet neural network performs feature extraction and the SSD classifier performs target classification detection.
[0023] Among them, Mobilenet is mainly a lightweight deep network model proposed for mobile terminals.
[0024] Mobilenet introduces two hyperparameters to reduce the number of parameters and calculations:
[0025] Width Multiplier: Reduce the number of input and output channels
[0026] Resolution Multiplier: Reduces the size of the input and output feature maps;
[0027] The computational ratio of the standard convolution kernel is:
[0028]
[0029] The Sobel algorithm in step 4 includes two sets of 3x3 matrices;
[0030] When inputting an image for edge detection, take 3 rows and 3 columns of image data, multiply the image data with the value of the operator at the corresponding position and then add them to obtain Gx in the x direction and Gy in the y direction. Square the obtained Gx and Gy, add them, and then take the arithmetic square root to obtain Gxy.
[0031] in
[0032] Gx=(a3-a1)+(b3-b1)*2+(c3-c1)
[0033] Gy=(a1-c1)+(a2-c2)*2+(a3-c3)
[0034]
[0035] The approximate value is the sum of the absolute values of Gx and Gy. The calculated Gxy is compared with the set threshold. If Gxy is greater than the threshold, it means that the point is a boundary point and this point is displayed as a black dot, otherwise it is displayed as a white dot.
[0036] Compared with the prior art, the present invention has the following beneficial effects:
[0037] The true color night vision sight of the present invention adopts a 1080P resolution CMOS sensor, combined with a high-performance ISP image processor, which can quickly capture 30 frames of 1080P high-definition images per second. At the same time, it adopts neural network NPU hardware and uses a deep learning algorithm (MobileNet SSD) for target detection. At the same time, the contour detection algorithm is used to detect the contour of the detected target, and the contour is highlighted.
[0038] Compared with various existing night vision scopes, the true color glow night vision scope has higher resolution and stronger detail resolution ability. At the same time, it introduces deep learning algorithms, making it easier to detect and highlight targets. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] Figure 1 It is a structural schematic diagram of a true color night vision sight with target detection and highlighting function of the present invention;
[0040] Figure 2 for Figure 1Schematic diagram of the internal cross-sectional structure;
[0041] Figure 3 Flowchart for target detection, contour detection and edge highlighting for the present invention;
[0042] Figure 4 A flowchart for realizing the system of the present invention;
[0043] Figure 5 This is a diagram of the Mobilenet neural network structure of an embodiment of the present invention;
[0044] Figure 6 It is the Sobel operator matrix diagram of the present invention;
[0045] Figure 7 This is the numerical calculation diagram of the Sobel operator matrix of the present invention.
[0046] In the figure: objective lens 1, body 2, eyepiece 3, OLED display 4, main board 5, CMOS chip 6. DETAILED DESCRIPTION
[0047] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0048] See also Figure 1-2 This embodiment provides a technical solution: a true color night vision sight with a target detection and highlighting function, in which an objective lens is installed at the left end of a fuselage, an eyepiece is installed at the right end of the fuselage, a main board is installed inside the fuselage, a CMOS chip is installed inside the objective lens, a main board is installed inside the eyepiece, and the main board is controlled and connected to an OLED display and a CMOS chip.
[0049] Reference Figure 3 and Figure 4 As shown, the working steps of the true color night vision sight with target detection and highlighting function in this embodiment are as follows:
[0050] Step 1: The CMOS chip 6 converts the original RAW data into an NV12 format image through the ISP processor;
[0051] Step 2: The NV12 image data is processed by the NPU processor for MobileNet and SSD target detection, and the target image is output;
[0052] Step 3: Binarization of the grayscale of the target image detected by the NPU processor;
[0053] Step 4: Use Sobel algorithm to extract the contour edge of the binary image;
[0054] Step 5: Map the contour edge pixels into highlight colors in the NV12 image;
[0055] Step 6: Display the NV12 image on the OLED display 4.
[0056] In this embodiment, step 2 uses the MobileNet and SSD target detection algorithms:
[0057] First, the MobileNet neural network is used for feature extraction, and then the SSD classifier is used for target classification and detection.
[0058] Among them, Mobilenet is mainly a lightweight deep network model proposed for mobile terminals.
[0059] The main method used is Depthwise Separable Convolution to decompose the standard convolution kernel and reduce the amount of calculation.
[0060] Mobilenet introduces two hyperparameters to reduce the number of parameters and calculations:
[0061] Width Multiplier: Reduce the number of input and output channels
[0062] Resolution Multiplier: Reduces the size of the input and output feature maps;
[0063] The Mobilenet neural network structure diagram is as follows Figure 5 As shown:
[0064] The computational ratio of the standard convolution kernel is:
[0065]
[0066] Therefore, MobileNet can achieve fast feature extraction on ARM processors, and SSD can perform target detection on feature maps of multiple scales respectively, so that targets of all scales can be taken into account, small-scale feature maps predict large targets, and large-scale feature maps predict relatively small targets.
[0067] In this embodiment, since an ARM processor is used, the speed of the algorithm needs to be considered, so the present invention uses a more efficient Sobel operator for edge detection.
[0068] The Sobel operator contains two sets of 3x3 matrices, as follows Figure 6 As shown:
[0069] When inputting an image for edge detection, take 3 rows and 3 columns of image data, multiply the image data with the value of the operator at the corresponding position and then add them to obtain Gx in the x direction and Gy in the y direction. Square the obtained Gx and Gy, add them, and then take the arithmetic square root to obtain Gxy, as follows Figure 7 As shown, the approximate value is the sum of the absolute values of Gx and Gy. The calculated Gxy is compared with the threshold we set. If Gxy is greater than the threshold, it means that the point is a boundary point and this point is displayed as a black dot, otherwise it is displayed as a white dot.
[0070] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application. It should be noted that similar numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.
[0071] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A true color night vision sight with target detection and highlighting function, characterized in that: The sight comprises an objective lens (1), a body (2), an eyepiece (3), an OLED display screen (4), a mainboard (5) and a CMOS chip (6); the objective lens (1) is mounted on the left end of the body (2), the eyepiece (3) is mounted on the right end of the body (2), the mainboard (5) is mounted inside the body (2), the CMOS chip (6) is mounted inside the objective lens (1), the OLED display screen (4) is mounted inside the eyepiece (3), and the mainboard (5), the OLED display screen (4) and the CMOS chip (6) are control-connected; The mainboard (5) uses an ISP and an NPU processor, wherein the ISP processor is used for digital image processing, the NPU processor is used for neural network processing, and the deep learning neural network is used for target detection; Described sight working process is: Step 1, the CMOS chip (6) converts the original RAW data into an NV12 image through the ISP processor; Step 2: The NV12 image data is processed by the NPU processor for MobileNet and SSD target detection, and the target image is output; Step 3: Binarize the grayscale of the target image detected by the NPU processor; Step 4: Use Sobel algorithm to extract the contour edge of the grayscale binary image; Step 5: Map the contour edge pixels into highlight colors in the NV12 image; Step 6: Display the NV12 image obtained in step 5 on the OLED display screen (4).
2. The true color night vision sight with target detection and highlighting function according to claim 1, characterized in that: The objective lens (1) is used to magnify an object.
3. The true color night vision sight with target detection and highlighting function according to claim 1, characterized in that: The CMOS chip (6) is used to convert optical signals into electrical signals.
4. The true color night vision sight with target detection and highlighting function according to claim 1, characterized in that: The eyepiece (3) further magnifies the real image magnified by the objective lens.
5. The true color night vision sight with target detection and highlighting function according to claim 1, characterized in that: The OLED display screen (4) is a display screen used for displaying processed images.
6. The true color night vision sight with target detection and highlighting function according to claim 1, characterized in that: In the step 2, feature extraction is performed through the MobileNet neural network, and target classification detection is performed through the SSD classifier.
Citation Information
Patent Citations
Infrared light and shimmer two-phase fused night-vision sighting device
CN105300175A
Method for correcting certificate image inclination
CN101625760A
Infrared target segmentation method based on joint obviousness
CN103413303A
