Night vision device, night vision method and computer readable storage medium

Abstract

The present disclosure provides a night vision device, a night vision method and a computer-readable storage medium, which belongs to the technical field of night vision devices. The night vision device includes: a low-light camera, a normal camera, a control unit and a display unit; the low-light camera is configured to: collect a first image and transmit it to the control unit; the normal camera is configured to: collect a second image and transmit it to the control unit, and the first image and the second image are images collected in the same area; the control unit is configured to: determine the clear part in the first image and the clear part in the second image according to the clarity of the first image and the second image; fuse the clear part of the first image and the clear part of the second image to obtain the image to be displayed; the display unit is configured to: display the image to be displayed. In a scenario where a night vision device needs to be worn and the user also needs to control other control objects, the user can clearly observe the dark environment at a long distance and the bright environment at a close distance without adjusting the focal length.

Description

Technical Field

[0001] The present disclosure relates to the technical field of night vision devices, and in particular to a night vision device, a night vision method and a computer-readable storage medium. Background Art

[0002] Night vision goggles are an important night observation device, widely used in military, security and other fields. They collect weak light and convert it into visible images, allowing people to observe objects in dark conditions.

[0003] The existing night vision device's focusing methods are mainly divided into automatic focusing and manual focusing. Automatic focusing will automatically zoom in or out to determine the best focal length and then observe the target at the best focal length. Manual focusing is to adjust the focus to the best focal length that the user perceives to be clear when the user perceives the image to be blurry.

[0004] Since the night vision device with automatic focus will frequently zoom in or out the observation screen, it may cause dizziness to the user when used close to the eye (such as head-mounted night vision device). Therefore, the night vision device used close to the eye is usually used to manually adjust a fixed focal length by the user, and then the focus is fixed according to the fixed focal length.

[0005] However, in some scenarios where night vision goggles (for near-eye use) are required and users also need to control other objects, such as driving a vehicle, helicopter, or fixed-wing aircraft, users need to clearly observe the dark environment at a distance and the bright environment at a close distance (such as the dashboard). In order to meet this requirement, users need to manually adjust the focus of the night vision goggles, which makes it impossible to concentrate on controlling other objects, increasing the risk of control errors. Summary of the invention

[0006] The present disclosure provides a night vision device, a night vision method and a computer-readable storage medium; in some scenarios where the night vision device needs to be worn and the user also needs to manipulate other controlled objects, there is no need to adjust the focal length of the night vision device, so that the user can clearly observe the dark environment at a long distance and the bright environment at a close distance while concentrating on manipulating the controlled object, thereby reducing the risk of control errors.

[0007] The technical solution of the present disclosure is achieved as follows:

[0008] In a first aspect, the present disclosure provides a night vision device, which includes: a low-light camera, a normal camera, a control unit and a display unit; the low-light camera is configured to: capture a first image and transmit it to the control unit; the normal camera is configured to: capture a second image and transmit it to the control unit, and the first image and the second image are captured synchronously; the control unit is configured to: determine a clear part in the first image and a clear part in the second image according to the clarity of the first image and the second image; fuse the clear part of the first image and the clear part of the second image to obtain an image to be displayed; the display unit is configured to: display the image to be displayed.

[0009] In a second aspect, the present disclosure provides a night vision method, which includes: determining a clear part in a first image and a clear part in a second image based on the clarity of a first image and a second image acquired in the same area; fusing the clear part of the first image and the clear part of the second image to obtain an image to be displayed, the first image being an image captured by a low-light camera, and the second image being an image captured by a normal camera; and displaying the image to be displayed.

[0010] In a third aspect, the present disclosure provides a computer-readable storage medium storing a program or instruction, which, when executed by a processor, implements the steps of the night vision method described in the second aspect.

[0011] In a fourth aspect, the present disclosure provides a computer program product, wherein the computer program product includes a computer program or instructions. When the computer program product runs on a processor, the processor executes the computer program or instructions to implement the steps of the night vision method as described in the second aspect.

[0012] In a fifth aspect, the present disclosure provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the night vision method as described in the second aspect.

[0013] The night vision device provided by the present disclosure can obtain the low-light environment information of the far focus through the low-light camera, and obtain the environment information of the near focus with a certain brightness (such as the information on the dashboard) through the ordinary camera. In this way, the night vision device can not only observe the dark environment at night, but also clearly view the information of the nearby light, avoiding the user dizziness caused by automatic focusing when using the night vision device close to the eye, and also avoiding the problem of manual focusing being inconvenient and prone to safety accidents in some driving processes. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is one of the structural schematic diagrams of a night vision device provided by the present disclosure;

[0015] Figure 2 The second structural schematic diagram of a night vision device provided by the present invention;

[0016] Figure 3 An exemplary actual environment image provided for the present disclosure;

[0017] Figure 4 An image of the environment outside the windshield collected by the low-light camera provided in the present disclosure;

[0018] Figure 5 Images of the environment outside the windshield and the dashboard captured by the low-light camera provided by the present disclosure;

[0019] Figure 6 A dashboard image captured by a low-light camera provided by the present disclosure;

[0020] Figure 7 The dashboard image captured by the common camera provided by the present disclosure;

[0021] Figure 8 The images of the environment outside the windshield and the dashboard collected by the common camera provided by the present invention;

[0022] Fig. 9 An image of the environment outside the windshield collected by a common camera provided in the present disclosure;

[0023] Fig.10 The dashboard image captured by the RGB camera provided by the present disclosure;

[0024] Fig.11 The image to be displayed obtained by fusion provided by the present disclosure;

[0025] Fig.12 One of the schematic diagrams of fusing a first image and a second image provided by the present disclosure;

[0026] Fig.13 The second schematic diagram of fusing the first image and the second image provided by the present disclosure;

[0027] Fig.14 A schematic diagram of a night vision method provided by the present invention. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the present disclosure to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of the present application.

[0029] The terms "first", "second", etc. in the specification of this application are used to distinguish similar objects, rather than to describe a specific order or precedence. It should be understood that the terms used in this way can be interchangeable where appropriate, so that the present disclosure can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more.

[0030] The night vision device provided by the present invention is described in detail below through specific embodiments and application scenarios in conjunction with the accompanying drawings.

[0031] like Figure 1 As shown, the present disclosure provides a night vision device, which includes: a low-light camera 10, a common camera 11, a control unit 12 and a display unit 13.

[0032] The low-light camera 10 is configured to: capture a first image and transmit it to the control unit 12; the ordinary camera 11 is configured to: capture a second image and transmit it to the control unit; the control unit 12 is configured to: determine a clear portion in the first image and a clear portion in the second image according to the clarity of the first image and the second image; and fuse the clear portion of the first image and the clear portion of the second image to obtain an image to be displayed; the display unit 13 is configured to: display the image to be displayed.

[0033] Low illumination usually refers to light intensity less than 1 lux. In a low illumination environment, without fill light, the ordinary camera 11 cannot capture an image that can clearly identify the observed object visually, such as: the captured image is completely black or the observed object is blurred and unrecognizable.

[0034] The low-light camera 10 refers to a camera that can still capture images that can clearly identify the observed object visually in a dim light environment. Low-light cameras 10 with different configuration parameters can capture images that can clearly identify the observed object visually at different minimum light illuminations, such as a minimum light illumination of 0.001 lux or even lower. For example, the low-light camera 10 can be: a low-light complementary metal oxide semiconductor (CMOS) camera, a low-light charge coupled device (CCD) camera, a low-light scientific complementary metal oxide semiconductor (sCMOS) camera, etc. The present disclosure does not make specific limitations, but because the image of the observed object captured by the CMOS camera is different from the observed object observed by the actual human eye except that the color information is lost, CMOS cameras are often used in practical applications.

[0035] like Figure 2 As shown, the low-light camera 10 includes: a low-light lens 101 and a first image sensor 102. The low-light lens 101 is used to gather the long-distance low-light ambient light signal; the first image sensor 102 is used to perform photoelectric conversion on the long-distance low-light ambient light signal and output a channel of video data (continuous multiple frames of images, each frame of image is the first image) to the control unit 12.

[0036] When the focal length of the low-light camera 10 remains unchanged, when the user's observation object is in a low-light environment at a long distance, the observation object is clearly visible in the first image captured by the low-light camera 10; when the user's observation object is converted from a low-light environment at a long distance to a normal-light environment at a close distance, the first image captured by the low-light camera 10 is partially clear and partially blurred; when the user's observation object is in a normal-light environment at a close distance, the first image captured by the low-light camera 10 is blurred.

[0037] For example, the objects observed by the low-light camera 10 are the environment outside the windshield and the instrument panel in the cockpit. Figure 3 As shown in FIG. 1 , it is an actual environment image including the environment outside the windshield and the instrument panel in the cockpit. When the driver's observation object is the environment outside the windshield and it is low light, the first image captured by the low light camera 10 is as follows: Figure 4 As shown, from Figure 4 The given image can clearly identify various objects in the environment outside the windshield under low illumination. As the driver's field of view moves, the driver changes the observation object from the low illumination environment outside the windshield to the instrument panel in the cockpit. When the field of view of the low illumination camera 10 remains unchanged, the collected image is as follows: Figure 5 As shown, from Figure 5 It can be seen from the figure that the various objects in the low-light environment outside the windshield in the distance are clearly distinguishable, but the information on the bright dashboard nearby is blurred. When the driver's field of vision moves to observe the dashboard in the cockpit, the first image captured by the low-light camera 10 is as follows Figure 6 As shown, from Figure 6 It can be seen that the blurred image makes the information on the dashboard visually unrecognizable.

[0038] The ordinary camera 11 refers to a camera that can only capture images that can clearly identify the observed object visually when the light intensity is greater than the normal illumination. Normal illumination is relative to low illumination, and usually refers to the light intensity greater than or equal to 1 lux. Ordinary cameras 11 with different configuration parameters can capture images that can clearly identify the observed object visually at different normal illuminations, such as the range of normal illumination values ​​between 1 and 10. The ordinary camera 11 can be implemented as a monochrome camera.

[0039] like Figure 2 As shown, the ordinary camera 11 includes: an ordinary lens 111 and a second image sensor 112. The ordinary lens 111 is used to gather ambient light signals of normal illumination at a close distance; the second image sensor 112 is used to perform photoelectric conversion on the ambient light signals of normal illumination at a close distance, and output a channel of video data (continuous multiple frames of images, each frame of image is the second image) to the control unit 12.

[0040] When the focal length of the ordinary camera 11 remains unchanged, when the user's observation object is in a normal illumination environment at a close distance, the second image captured by the ordinary camera 11 is visually clear in that the observation object is visible; when the user's observation object is converted from a normal illumination environment at a close distance to a low illumination environment at a long distance, the second image captured by the ordinary camera 11 is partially clear and partially blurred; when the user's observation object is in a low illumination environment at a long distance, the second image captured by the ordinary camera 11 is blurred.

[0041] For example, the objects observed by the ordinary camera 11 are the environment outside the windshield and the instrument panel in the cockpit. In the case where the driver's observation object is the instrument panel in the cockpit, the second image captured by the ordinary camera 11 is as follows: Figure 7 As shown, from Figure 7 The information on the dashboard can be clearly identified in the image given. As the driver's field of view moves, the driver changes the observation object from the dashboard in the cockpit to the low-light environment outside the windshield. When the field of view of the ordinary camera 11 remains unchanged, the collected image is as follows: Figure 8 As shown, from Figure 8 It can be seen from the figure that the information on the bright instrument panel nearby is clearly discernible, but the various objects in the low-light environment outside the windshield in the distance are blurred. When the driver's field of vision moves to observe the low-light environment outside the windshield, the second image captured by the ordinary camera 11 is as follows: Fig. 9 As shown, from Fig. 9 It can be seen that the low-light environment outside the windshield is visually unrecognizable.

[0042] In addition, the ordinary camera 11 can also be a color (Red Green Blue, RGB) camera. Because in some near-eye application scenarios, it is necessary not only to see the nearby environment clearly, but also to clarify the true color of the observed object in the environment. For example, when a user is driving a vehicle, a helicopter, a fixed-wing aircraft, etc., he not only needs to see the low-light environment at a long distance, but also needs to observe the dashboard. The dashboard not only has digital information, but also usually distinguishes different warning information according to different colors.

[0043] However, the low-light camera 10 achieves nighttime observation by enhancing weak natural light (such as starlight, moonlight, etc.). Under low-light conditions, the sensor has limited ability to capture light, and the first image sensor 102 can only capture the intensity of light but not the color. Therefore, the image output by the low-light camera 10 is usually monochrome, such as the most common color of green. If the ordinary camera 11 is implemented as a monochrome camera, the output image is also black and white, and does not include the color information of the observed object.

[0044] Therefore, in some embodiments, the ordinary camera 11 is an RGB camera. The RGB camera can not only capture the instrument panel at a close distance, but also the real color of the instrument panel will be reflected in the image. Fig.10 The red color indicated in the dashed box indicates that the scale here is displayed in red, which is used to warn of excessive speed or excessive rotation speed. In this way, the driver can obtain more color information from the instrument panel to improve driving safety.

[0045] In an image, the larger the ratio of the area occupied by the visually clearly identifiable objects to the area occupied by the total objects included in the image, the higher the clarity of the image; the smaller the ratio of the area occupied by the visually clearly identifiable objects to the area occupied by the total objects included in the image, the lower the clarity of the image.

[0046] Since the first image and the second image are images collected from the same area, that is, the objects included in the first image and the second image are the same, the clarity of the first image and the second image can indicate the area ratio of the objects that can be clearly identified in the image. The control unit 12 is configured to: determine the clear part in the first image and the clear part in the second image according to the clarity of the first image and the second image; and fuse the clear part of the first image and the clear part of the second image to obtain the image to be displayed. Therefore, the clear part of the first image and the clear part of the second image are comprehensively considered to ensure that each object in the fused image to be displayed is visually clear and distinguishable. Fig.11 As shown, it is shown that according to Figure 5 and Figure 8 The fused image to be displayed.

[0047] The display unit 13 is configured to display the image to be displayed. Figure 2 As shown, the display unit 13 includes: a digital display screen 131 and an eyepiece 132. The digital display screen 131 is used to receive and display the video data (the first image and the second image) from the control unit 12; the eyepiece 132 is used to convert the optical signal of the digital display screen 131 into a virtual display screen at a fixed distance according to the design, so as to facilitate close viewing by human eyes.

[0048] In some embodiments, the clarity of the first image and the second image is determined using an image quality assessment algorithm, such as determining the clarity of the first image and the second image based on at least one of contrast, edge strength, structural similarity, peak signal-to-noise ratio, contrast, and edge sharpness.

[0049] For example, the clarity of the first image and the second image is determined according to contrast, edge strength, structural similarity and peak signal-to-noise ratio. The contrast, edge strength, structural similarity and peak signal-to-noise ratio of the first image are calculated, and each calculation result is normalized. The normalization result is weighted and summed to obtain the clarity of the first image. The clarity of the second image is calculated in the same way as the clarity of the first image, which will not be described in detail here.

[0050] In some embodiments, in the process of determining the clarity of the first image and the second image based on at least one of contrast, edge strength, structural similarity, and peak signal-to-noise ratio, the weight of each parameter (contrast, edge strength, structural similarity, peak signal-to-noise ratio) is determined according to the application scenario.

[0051] For example, in a search scene, the background of the image is usually consistent, such as the sea, desert, etc., and the weight of the contrast parameter can be increased; in an obstacle avoidance scene, such as a jungle, obstacles are usually more concerned, and the weight of the edge strength parameter can be increased. Specifically, according to the actual application scenario, the weights of the parameters used in the process of calculating the clarity in different scenarios can be preset, and the present disclosure does not make specific limitations on this.

[0052] Before fusing the first image and the second image, in order to ensure that each pixel in the two images corresponds to each other and realize effective fusion, that is, to ensure that the first image and the second image are images of the same scene captured at the same position and at the same time. However, since the positions of the low-light camera 10 and the ordinary camera 11 in the night vision device cannot be exactly the same, it is necessary to perform geometric calibration on the low-light camera 10 and the ordinary camera 11 to determine the difference in the field of view size and angle between the two.

[0053] The difference in angle between the low-light camera 10 and the ordinary camera 11 can be obtained by a calibration algorithm (such as the Zhang Zhengyou calibration method) to obtain the relative position and posture of the two, so as to determine the angle difference of the images taken by the two according to the relative position and posture of the two. The difference in the field of view size of the low-light camera 10 and the ordinary camera 11 is determined according to the parameters of the camera actually selected.

[0054] According to the difference in the field of view size and angle between the two, the first image and the second image are registered to ensure the pixel correspondence in the same scene. For different field of view sizes, an interpolation method (such as bilinear interpolation) can be used to adjust the image size.

[0055] In some embodiments, the control unit 12 is specifically configured to: determine a first fusion weight of the first image and a second fusion weight of the second image based on the clarity of the first image and the second image; determine a clear part in the first image and a clear part in the second image based on the first fusion weight and the second fusion weight, and the relative position relationship of the observation objects of the low-light camera 10 and the ordinary camera 11.

[0056] Once the relative position relationship of the observed objects is determined, the distribution position of the clear part in the first image and the distribution position of the clear part in the second image can be determined, and the clarity of the image can indicate the proportion of the clear part in the entire image.

[0057] For example, the observation objects are the low-light environment outside the windshield and the instrument panel in the cockpit. Figure 3 As shown, the low-light environment outside the windshield is located above the instrument panel in the cockpit. If all objects in the image can be clearly identified visually, the corresponding clarity is 1. When the clarity of the first image is less than 1, since the low-light environment outside the windshield is located above the instrument panel in the cockpit, it can be determined that the clear part is located in the upper part of the first image; when the clarity of the second image is less than 1, since the low-light environment outside the windshield is located above the instrument panel in the cockpit, it can be determined that the clear part is located in the lower part of the second image; if the clarity of the first image is 0.7 and the clarity of the second image is 0.5, the fusion weight of the first image is determined to be: 0.7 / (0.7+0.5)=7 / 12, and the fusion weight of the second image is: 0.5 / (0.7+0.5)=5 / 12, then it is determined that 7 / 12 of the upper part of the first image is clear, and 5 / 12 of the lower part of the second image is clear, and the image to be displayed is obtained by splicing the two clear parts.

[0058] This fusion method can fuse a clear image to be displayed based only on the positional relationship of the observed objects and the clarity of the image. The process is simple and has low requirements on computing power.

[0059] When the user switches from observing a long-distance low-light environment to observing a close-distance normal-light environment, multiple frames of images will be collected synchronously, that is, there will be multiple groups of first images and second images in the process. The corresponding fusion weight will be determined for each group of first images and second images. According to the relative position relationship between the fusion weight and the observed object, the part of the first image and the part of the second image included in the fused object to be displayed will change frame by frame to ensure that the user has no perception.

[0060] Exemplarily, the pixels of the first image and the second image are divided into rows, such as Fig.12As shown, it is divided into 10 rows (in practice, the number of rows of pixels is usually more, such as 1920 rows, which is only for illustration). When the user switches from observing a long-distance low-illuminance environment to observing a close-distance normal-illuminance environment, the clarity of the first image gradually decreases, and the clarity of the second image gradually increases. At this time, the fusion process is: the number of rows of the first image included in the fused image to be displayed gradually decreases, and the number of rows of the second image gradually increases upward. Fig.12 In the image to be displayed shown in , the first fusion weight is 0.5, the second fusion weight is 0.5, the first 5 rows in the image to be displayed are the first 5 rows of the first image, and the last 5 rows in the image to be displayed are the last 5 rows of the second image; Fig.13 In the image to be displayed shown in , the first fusion weight is 0.4, the second fusion weight is 0.6, the first 4 lines in the image to be displayed are the first 4 lines of the first image, and the last 6 lines in the image to be displayed are the last 6 lines in the second image. That is, the difference between two consecutive frames of images to be displayed is very small, and the user has no perception of the fusion process.

[0061] In this way, as the user's observation target changes, the fusion weight changes gradually, so that the displayed image changes gradually, ensuring that the user has no perception.

[0062] In order to make the fused image clearer, in some embodiments, the control unit 12 is specifically configured to: determine the clear part in the first image and the clear part in the second image according to the clarity of each first sub-area in the first image and the clarity of the corresponding second sub-area in the second image.

[0063] A sub-region division strategy is pre-set, and each acquired image is divided according to the sub-region division strategy, and the clarity of the image in each sub-region is determined. Since the division strategy is the same, each first sub-region in the first image corresponds to a second sub-region in the second image.

[0064] The images in each corresponding sub-region are fused according to the clarity, which can be specifically weighted summed according to the clarity, and different weights are set according to the different clarity of each first sub-region and the corresponding second sub-region, and the result of the weighted sum is used as the sub-region of the image to be displayed. If the clarity of the fused image is required to be higher, the fewer pixels each sub-region includes, that is, the smaller the unit sub-region is, the higher the clarity of the image obtained after fusion.

[0065] This fusion method can ensure that in the obtained image to be displayed, each sub-region is a relatively clear image of the first image and the second image, so that the final image to be displayed has higher clarity.

[0066] In some embodiments, the control unit 12 is specifically configured to: when the clarity of each first sub-area is greater than the clarity of the corresponding second sub-area, determine that the first sub-area belongs to the clear part of the first image; when the clarity of each first sub-area is less than the clarity of the corresponding second sub-area, determine that the second sub-area belongs to the clear part of the second image; when the clarity of each first sub-area is equal to the clarity of the corresponding second sub-area, determine that the first sub-area belongs to the clear part of the first image, and / or determine that the second sub-area belongs to the clear part of the second image.

[0067] Since the weighted sum of each first sub-region and the corresponding second sub-region requires more resources, directly determining the sub-region with high clarity as a sub-region of the image to be displayed can not only reduce the calculation pressure, but also ensure that each sub-region of the fused image to be displayed is the clearer part of the first image and the second image, thereby improving the clarity of the image to be displayed.

[0068] If it is necessary to further reduce computing power and latency, in some embodiments, the control unit 12 is specifically configured as follows: when the clarity of the first image is greater than or equal to a first clarity threshold and the clarity of the second image is less than a second clarity threshold, determining the first image as the image to be displayed; when the clarity of the first image is less than the second clarity threshold and the clarity of the second image is greater than or equal to the first clarity threshold, determining the second image as the image to be displayed; the first clarity threshold is greater than the second clarity threshold.

[0069] When using night vision goggles, users usually spend most of their time observing a low-light environment at a long distance, and occasionally need to observe a normal-light environment at a close distance. When users observe a low-light environment at a long distance, the clarity of the first image is very high, but the clarity of the second image may not be 0. At this time, if each frame of the image is fused and displayed, it will consume a high computing power. Therefore, when it is determined that the clarity of the first image is greater than or equal to the first clarity threshold (such as the maximum clarity is 1, and the first clarity threshold is 0.8), and the clarity of the second image is less than the second clarity threshold (such as 0.2), it can be explained that the user is observing a low-light environment at a long distance, so the image to be displayed is determined to be the first image. Similarly, when it is determined that the clarity of the second image is greater than or equal to the first clarity threshold, and the clarity of the first image is less than the second clarity threshold, it can be explained that the user is observing a normal-light environment at a close distance, so the image to be displayed is determined to be the second image. In this way, fusion calculation is performed only when the user switches the observation object, and after the observation object is switched, computing power is no longer consumed in fusion calculation, thereby saving computing power.

[0070] Since the low-light camera 10 is fixed-focus, it can clearly capture images in a long-distance low-light environment under low-light light, but the images captured by the low-light camera 10 in a close-range normal-light environment will be blurred, while the ordinary camera 11 is just the opposite, and the two complement each other. Therefore, there is usually no problem of too low clarity of both the first image and the second image. If there is, one situation is that the camera is faulty, and the other situation is that the distance to the obstacle is very close in the low-light environment (such as a wall in the close range in front). Therefore, in some embodiments, the control unit 12 is also configured to: when the clarity of the first image is less than the second clarity threshold and the clarity of the second image is less than the second clarity threshold, output a prompt message, the prompt message is used to indicate that the distance to the obstacle is less than the distance threshold or to detect whether the night vision device is faulty.

[0071] In this way, in special situations, users can be promptly prompted to check whether the night vision device is malfunctioning, and to be aware that they may be too close to obstacles.

[0072] In some embodiments, the control unit 12 is further configured to: when the clarity of the first image is less than a second clarity threshold and the clarity of the second image is less than the second clarity threshold, control the enhanced display of the first image and the second image.

[0073] The strategies that can be adopted to control the enhanced display of the first image and the second image include: adjusting the automatic exposure parameters, determining the contours of the targets in the first image and the second image by edge algorithms and enhancing them, etc. The enhanced display strategy is only an example, and more strategies that can improve the image clarity can be adopted according to different needs in actual applications, and the present disclosure does not limit it.

[0074] After controlling the first image and the second image to be enhanced and displayed, if either the clarity of the first image and the clarity of the second image is greater than or equal to a second clarity threshold, the clear parts of the first image and the second image after the enhanced display are determined according to the clarity of the first image and the second image after the enhanced display, and the image to be displayed is fused according to the clear parts of the first image and the second image after the enhanced display.

[0075] After controlling the enhanced display of the first image and the second image, if the clarity of the first image is still less than the second clarity threshold and the clarity of the second image is still less than the second clarity threshold, the above prompt information can be output.

[0076] In some embodiments, the control unit 12 is specifically configured to: enhance the target in the first image to obtain an enhanced displayed first image; adjust the display parameters of the second image to obtain an adjusted second image; and determine a clear portion in the enhanced displayed first image and a clear portion in the adjusted second image based on the clarity of the enhanced displayed first image and the adjusted second image.

[0077] The first image is a low-light image of the outside world captured by the low-light camera 10. Usually, the background is not given much attention, but the target displayed in the image (such as obstacles, search objects, etc.) is given more attention. Therefore, the targets other than the background in the first image are enhanced and displayed.

[0078] Specifically, the enhanced display of the target can be achieved by using edge detection and enhancement algorithms, image sharpening algorithms, etc., to highlight targets with high attention and key information, so that users can clearly obtain the required information in a low-light environment.

[0079] The key information included in the second image is usually the instrument panel or other information with bright light nearby. Therefore, the target is usually clear and fixed, and no edge enhancement algorithm is required. It is only necessary to modify the display parameters of the second image, such as increasing the contrast, brightness, etc., to ensure that the information displayed in the second image can be clearly displayed.

[0080] According to the clarity of the enhanced first image and the adjusted second image, the clear part of the enhanced first image and the clear part of the adjusted second image are determined, and the clear part of the enhanced first image and the clear part of the adjusted second image are fused to obtain the image to be displayed. Because the information in the image before fusion is ensured to be clearer, it is ensured that the image to be displayed after fusion can be clearly displayed regardless of whether it is information in a long-distance low-light environment or information in a close-distance normal-light environment.

[0081] Based on the same inventive concept as the above technical solution, see Fig.14 , which shows a flow chart of a night vision method provided by the present disclosure. The execution subject of the night vision method can be the above-mentioned control unit 12, and the night vision method includes the following steps S301 to S303.

[0082] In step S301, a clear portion in the first image and a clear portion in the second image are determined according to the clarity of the first image and the second image acquired in the same area.

[0083] In step S302, a clear portion of the first image and a clear portion of the second image are fused to obtain an image to be displayed.

[0084] The first image is an image captured by a low-light camera, and the second image is an image captured by a common camera.

[0085] In step S303, the image to be displayed is displayed.

[0086] In some embodiments of the present disclosure, the above step S301 determines the clear part in the first image and the clear part in the second image according to the clarity of the first image and the second image acquired in the same area, which can be specifically implemented by the following steps.

[0087] According to the clarity of the first image and the second image, a first fusion weight of the first image and a second fusion weight of the second image are determined.

[0088] A clear portion in the first image and a clear portion in the second image are determined according to the first fusion weight and the second fusion weight, and a relative position relationship between the observation objects of the low-light camera and the ordinary camera.

[0089] In some embodiments of the present disclosure, the above step S301 determines the clear part in the first image and the clear part in the second image according to the clarity of the first image and the second image acquired in the same area, which can be specifically implemented by the following steps.

[0090] According to the clarity of each first sub-region in the first image and the clarity of the corresponding second sub-region in the second image, a clear portion in the first image and a clear portion in the second image are determined.

[0091] In some embodiments of the present disclosure, determining the clear part in the first image and the clear part in the second image based on the clarity of each first sub-region in the first image and the clarity of the corresponding second sub-region in the second image can be specifically achieved through the following steps.

[0092] When the clarity of each first sub-region is greater than the clarity of the corresponding second sub-region, it is determined that the first sub-region belongs to a clear part in the first image.

[0093] When the clarity of each first sub-region is less than the clarity of the corresponding second sub-region, it is determined that the second sub-region belongs to a clear part in the second image.

[0094] When the clarity of each first sub-region is equal to the clarity of the corresponding second sub-region, it is determined that the first sub-region belongs to the clear part in the first image, and / or it is determined that the second sub-region belongs to the clear part in the second image.

[0095] In some embodiments of the present disclosure, the above step S301 determines the clear part in the first image and the clear part in the second image according to the clarity of the first image and the second image acquired in the same area, which can be specifically implemented by the following steps.

[0096] When the definition of the first image is greater than or equal to a first definition threshold and the definition of the second image is less than a second definition threshold, the first image is determined to be the image to be displayed.

[0097] When the definition of the first image is less than the second definition threshold and the definition of the second image is greater than or equal to the first definition threshold, the second image is determined to be the image to be displayed.

[0098] The first clarity threshold is greater than the second clarity threshold.

[0099] In some embodiments of the present disclosure, the night vision method further includes the following steps:

[0100] When the clarity of the first image is less than the second clarity threshold and the clarity of the second image is less than the second clarity threshold, a prompt message is output, wherein the prompt message is used to indicate that the distance to the obstacle is less than the distance threshold or to detect whether the night vision device is faulty.

[0101] In some embodiments of the present disclosure, the night vision method further includes the following steps:

[0102] When the definition of the first image is less than the second definition threshold and the definition of the second image is less than the second definition threshold, the first image and the second image are controlled to be displayed in an enhanced manner.

[0103] In some embodiments of the present disclosure, the above step S301 is implemented by determining the clear part in the first image and the clear part in the second image according to the clarity of the first image and the second image acquired in the same area.

[0104] The target in the first image is enhanced to obtain an enhanced first image.

[0105] The display parameters of the second image are adjusted to obtain an adjusted second image.

[0106] According to the clarity of the enhanced displayed first image and the adjusted second image, a clear portion in the enhanced displayed first image and a clear portion in the adjusted second image are determined.

[0107] In some embodiments of the present disclosure, the second image is an image captured by a color camera.

[0108] The present disclosure also provides a computer-readable storage medium storing at least one instruction, wherein the at least one instruction is used to be executed by a processor to implement the night vision method described in the above embodiments.

[0109] The present disclosure also provides a computer program product, which includes computer instructions stored in a computer-readable storage medium; a processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device executes to implement the night vision methods described in the above-mentioned embodiments.

[0110] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned night vision method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0111] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0112] In the several embodiments provided in the present disclosure, it should be understood that the disclosed systems, devices, servers and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0113] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0114] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0115] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0116] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the present disclosure can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any media that facilitates the transmission of computer programs from one place to another. Storage media can be any available media that can be accessed by a general or special-purpose computer.

[0117] It should be noted that the technical solutions described in the present disclosure can be combined arbitrarily without conflict.

[0118] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A night vision device, characterized in that: include: Low-light camera, ordinary camera, control unit and display unit; The low-light camera is configured to: collect a first image and transmit it to a control unit; The ordinary camera is configured to: collect a second image and transmit it to the control unit, wherein the first image and the second image are images of the same area collected at the same time; The control unit is configured to: determine a clear portion in the first image and a clear portion in the second image according to the clarity of the first image and the second image, wherein a portion of the first image corresponding to the environment outside the cockpit is the clear portion, and a portion of the second image corresponding to the environment inside the cockpit is the clear portion; and, fusing a clear portion of the first image with a clear portion of the second image to obtain an image to be displayed; The display unit is configured to: display the image to be displayed; The determining, according to the clarity of the first image and the second image, a clear portion in the first image and a clear portion in the second image comprises: Determining, according to the clarity of the first image and the second image, a first fusion weight of the first image and a second fusion weight of the second image, wherein the first fusion weight is used to indicate the area ratio of the first image in the fused image to be displayed, and the second fusion weight is used to indicate the area ratio of the second image in the fused image to be displayed; A clear portion in the first image and a clear portion in the second image are determined according to the first fusion weight and the second fusion weight, and a relative position relationship between the observation objects of the low-light camera and the ordinary camera.

2. The night vision device according to claim 1, characterized in that: The control unit is specifically configured as follows: When the clarity of the first image is greater than or equal to a first clarity threshold and the clarity of the second image is less than a second clarity threshold, determining that the first image is the image to be displayed; When the definition of the first image is less than a second definition threshold and the definition of the second image is greater than or equal to the first definition threshold, determining that the second image is the image to be displayed; The first clarity threshold is greater than the second clarity threshold.

3. The night vision device according to claim 2, characterized in that: The control unit is further configured to: When the clarity of the first image is less than a second clarity threshold and the clarity of the second image is less than the second clarity threshold, a prompt message is output, where the prompt message is used to indicate that the distance to the obstacle is less than the distance threshold or to detect whether the night vision device is faulty.

4. The night vision device according to claim 2, characterized in that: The control unit is further configured to: When the definition of the first image is less than a second definition threshold and the definition of the second image is less than a second definition threshold, the first image and the second image are controlled to be displayed in an enhanced manner.

5. The night vision device according to any one of claims 1 to 4, characterized in that: The control unit is specifically configured as follows: enhancing the target in the first image to obtain an enhanced displayed first image; adjusting display parameters of the second image to obtain an adjusted second image; According to the clarity of the enhanced displayed first image and the adjusted second image, a clear portion in the enhanced displayed first image and a clear portion in the adjusted second image are determined.

6. The night vision device according to any one of claims 1 to 4, characterized in that: The ordinary camera is a color camera.

7. A night vision method, characterized in that: The method comprises: Determining a clear portion in the first image and a clear portion in the second image according to the clarity of a first image and a second image of the same area acquired at the same time, wherein the first image is an image acquired by a low-light camera, and the second image is an image acquired by a common camera; fusing a clear portion of the first image and a clear portion of the second image to obtain an image to be displayed, wherein a portion of the first image corresponding to the environment outside the cockpit is a clear portion, and a portion of the second image corresponding to the environment inside the cockpit is a clear portion; displaying the image to be displayed; The determining, according to the clarity of the first image and the second image, a clear portion in the first image and a clear portion in the second image comprises: Determining, according to the clarity of the first image and the second image, a first fusion weight of the first image and a second fusion weight of the second image, wherein the first fusion weight is used to indicate the area ratio of the first image in the fused image to be displayed, and the second fusion weight is used to indicate the area ratio of the second image in the fused image to be displayed; A clear portion in the first image and a clear portion in the second image are determined according to the first fusion weight and the second fusion weight, and a relative position relationship between the observation objects of the low-light camera and the ordinary camera.

8. A computer-readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by the processor, the steps of the night vision method as claimed in claim 7 are implemented.

Images