Image processing method and device, night vision device and computer readable storage medium

Abstract

The invention provides an image processing method and device, a night vision device and a computer readable storage medium, and belongs to the technical field of image processing.The image processing method comprises the steps that an external environment image of the current position of a moving body is collected through light capture sensitivity matched with external environment illuminance; acquiring image optimization data of the current position; and according to the image optimization data, carrying out image enhancement on the external environment image, and / or adding a display object in the external environment image. According to the scheme, the image observed by the night vision device is clearer or comprises more information.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to an image processing method, device, night vision device, and computer-readable storage medium. Background Art

[0002] Night vision goggles are nighttime observation devices that can enhance or provide visual capabilities under low-light conditions and are widely used in military, security and other fields. Night vision goggles collect weak light and convert it into visible images, allowing users to see targets clearly under low-light conditions.

[0003] Night vision goggles are an important device in aircraft. When pilots are flying aircraft to perform night missions, the ambient light intensity is low and they need to use night vision goggles to achieve a longer viewing range and clearer vision to identify targets such as terrain, obstacles and runways.

[0004] However, existing night vision devices usually only convert the collected weak light signals from the outside world into images and display them, which cannot meet the driver's expectation of obtaining more information. Summary of the invention

[0005] The present disclosure provides an image processing method, device, night vision device and computer-readable storage medium, which can make the image observed by the night vision device clearer or include more information.

[0006] The technical solution of the present disclosure is achieved as follows: In a first aspect, the present disclosure provides an image processing method for use in night vision devices, the image processing method comprising: utilizing a light capture sensitivity that matches the external environment illumination to collect an image of the external environment at the current position of a moving object; obtaining image optimization data for the current position; performing image enhancement on the external environment image based on the image optimization data, and / or adding a display object to the external environment image.

[0007] In a second aspect, the present disclosure provides an image processing device, which includes: a collection part, an acquisition part and an image processing part; the collection part is configured to use a light capture sensitivity that matches the external environment illumination to collect an external environment image of the current position of the moving body; the acquisition part is configured to obtain image optimization data of the current position; the image processing part is configured to enhance the external environment image according to the image optimization data, and / or add a display object to the external environment image.

[0008] In a third aspect, the present disclosure provides an electronic device comprising a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the image processing method described in the first aspect.

[0009] In a fourth aspect, the present disclosure provides a computer-readable storage medium storing a program or instruction, which, when executed by a processor, implements the steps of the image processing method described in the first aspect.

[0010] In a fifth aspect, the present disclosure provides a computer program product, wherein the computer program product includes a computer program or instructions, and when the computer program product runs on a processor, the processor executes the computer program or instructions to implement the steps of the image processing method as described in the first aspect.

[0011] In a sixth aspect, the present disclosure provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the image processing method as described in the first aspect.

[0012] The present disclosure provides an image processing method, which optimizes image data to enhance the display of various objects in the collected external environment image, so that the external environment observed through a night vision device is clearer, or adds supplementary information to the external environment image to enrich the external information that can be obtained through the night vision device. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 A schematic diagram of the structure of a night vision device in the related art provided by the present disclosure; Figure 2 A schematic diagram of a flow chart of an image processing method provided by the present disclosure; Figure 3 A schematic diagram of another image processing method provided by the present disclosure; Figure 4 A schematic diagram of the actual external environment within a certain area outside the aircraft provided by the present disclosure; Figure 5 A schematic diagram of an image of the external environment observed through a night vision device when visibility is low provided by the present disclosure; Figure 6 A schematic diagram of historical image data provided by the present disclosure; Figure 7 A schematic diagram of an optimized image observed by a night vision device provided by the present disclosure; Figure 8 A schematic diagram of another optimized image observed by a night vision device provided by the present disclosure; Fig. 9 A flowchart of another image processing method provided by the present disclosure; Fig.10 A schematic diagram of another historical image data provided by the present disclosure; Fig.11 A schematic diagram of an image of the external environment with an expanded field of view observed by a night vision device provided by the present disclosure; Fig.12 A flowchart of another image processing method provided by the present disclosure; Fig.13 A schematic diagram of an optimized image provided by the present disclosure with aircraft altitude information and the altitude information of the highest building in the field of view added; Fig.14 A schematic diagram of an optimized image with an added overheight obstacle warning provided for the present disclosure; Fig.15 A structural block diagram of a night vision device provided by the present invention; Fig.16 A schematic diagram of the hardware architecture of a night vision device provided by the present disclosure; Fig.17 This is a structural block diagram of an image processing device provided by the present disclosure. DETAILED DESCRIPTION

[0014] The following will be combined with the drawings in the present disclosure to clearly describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present disclosure are within the scope of protection of the present disclosure.

[0015] The terms "first", "second", etc. in the specification of the present disclosure are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the present disclosure can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of the same type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification means at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0016] In the related art, the structure of night vision device is as follows: Figure 1As shown, it includes a low-light objective lens 10, a low-light image sensor 11, a display screen 12 and an eyepiece 13. The low-light objective lens 10 is used to converge light signals and transmit the converged light signals to the low-light image sensor 11, the low-light image sensor 11 is used to convert the light signals into images and push them to the display screen 12, the display screen 12 is used to display images, and the eyepiece 13 is used to convert the displayed image on the display screen 12 into a virtual image at a fixed distance, so that the human eye can watch it at a close distance.

[0017] This type of night vision device can only see targets in low-light environments at night, and cannot display more information. For example, during aviation flights, foggy weather is often encountered. Since the particles in the fog scatter light, the targets observed through the night vision device become blurred, and even targets at a long distance cannot be seen at all.

[0018] Therefore, the present disclosure aims to provide an image processing method capable of optimizing the display image of a night vision device.

[0019] In the following, in conjunction with the accompanying drawings, the image processing method provided by the present disclosure is applied to a night vision device as an example, and the image processing method provided by the present disclosure is described in detail through specific embodiments and application scenarios.

[0020] like Figure 2 As shown, the image processing method provided by the present disclosure may include the following steps S201 to S203.

[0021] In step S201, an image of the external environment at the current position of the moving object is collected using a light capture sensitivity that matches the external environment illumination.

[0022] The mobile object refers to a means of transportation, such as a vehicle, a ship, an aircraft, etc. In the subsequent description of this disclosure, the mobile object is taken as an aircraft as an example. Light capture sensitivity refers to the ability of night vision goggles to respond to changes in light. The higher the light capture sensitivity, the higher the amplification rate of the night vision goggles for light signals. Therefore, the lower the external environment illumination, the higher the light capture sensitivity required to capture a clear and recognizable image of the external environment. The above-mentioned light capture sensitivity that matches the external environment illumination refers to adjusting the light capture sensitivity of the night vision goggles so that the captured external environment image is clear and recognizable for the current external environment illumination.

[0023] The light capture sensitivity of the night vision goggles can be adjusted, such as by adjusting the brightness of the infrared light source. The higher the brightness of the infrared light source, the higher the light capture sensitivity, and the clearer the collected image of the external environment. Adjusting the focal length can change the amount of light entering the objective lens. Adjusting the focal length to narrow the field of view angle can increase the amount of light entering the lens, thereby improving the light capture sensitivity of the night vision goggles. The specific method of adjusting the light capture sensitivity is not limited in this disclosure.

[0024] The night vision device provided by the present disclosure includes a positioning device, which can locate the position of the aircraft based on the global positioning system (GPS), China's Beidou system, Russia's Global Navigation Satellite System (GLONASS), Europe's Galileo system, Japan's Quasi Zenith Satellite System (QZSS), India's Indian Regional Navigation Satellite System (IRNSS) and other positioning systems to obtain the current position of the aircraft. The current position is a spatial position, including latitude and longitude information and altitude information.

[0025] The night vision device is used to collect images of the external environment within a preset distance range outside the aircraft. The preset distance range is determined by the maximum viewing distance parameter of the night vision device.

[0026] In step S202, image optimization data of the current location is obtained.

[0027] Image optimization data is used to supplement the external environment image collected by the night vision device. Image optimization data can be pre-stored historical image data of the current location, or a contour map corresponding to the radar point cloud data of the current location, or interactive information received from other devices.

[0028] Specifically, the image optimization data may be pre-stored historical image data of the current location. Since the aircraft usually flies at a certain altitude, the external environment on the flight route usually changes little, and multiple frames of image data outside the aircraft can be collected at a certain update frequency at the same frequency as the night vision device when the field of vision is good during the day, and the location information is marked in each frame of image data according to the positioning of the positioning device, so that at least one frame of historical image data corresponding to the current location can be directly found based on the location information.

[0029] The image optimization data may be a contour map corresponding to the radar point cloud data. The night vision device includes a laser radar, which acquires radar point cloud data by emitting a laser beam and measuring its reflection time and intensity. Based on the collected radar point cloud data of various entities outside the aircraft at the current location, the contour of each entity in the radar point cloud data is extracted to obtain a contour map.

[0030] The image optimization data can be interactive information sent by other devices. The night vision device also includes a communication module, which is used to receive interactive information sent by other devices, such as the location information of other aircraft sent by other aircraft in the same crew as the aircraft, the location information of fixed targets sent by base stations, and the weather information sent by meteorological observation sensors.

[0031] In step S203, the external environment image is enhanced according to the image optimization data, and / or a display object is added to the external environment image.

[0032] For the collected external environment images, the image optimization data is used to enhance the display of each initial object in the external environment image, so that the external environment observed through the night vision device is clearer, or supplementary information is added to the external environment image to enrich the external information that can be obtained through the night vision device.

[0033] In this embodiment, the initial object is an image of an entity in an image of the external environment outside the aircraft captured by a night vision device, such as a building, other aircraft, flying birds, etc.; the background is opposite to the initial object, which does not contain a specific analysis target, but exists as a background to set off the object. For example, in an image including a building and the sky, the building is the initial object and the background is the sky. The object in the contour map corresponding to the radar point cloud data is the outline of each entity. The object in the historical image data is the image of the entity captured when the field of view is good.

[0034] In some embodiments, in combination Figure 2 ,like Figure 3 As shown, the image optimization data includes: object optimization data having the same acquisition field of view as the external environment image; in the above step S203, the external environment image is enhanced according to the image optimization data, which can be specifically implemented through the following step S203a; in the above step S203, a display object is added to the external environment image according to the image optimization data, which can be specifically implemented through the following step S203b.

[0035] In step S203a, when the visibility of the external environment at the current location is less than or equal to the visibility threshold, the initial object in the external environment image that matches the first object is enhanced and displayed according to the first object in the object optimization data that matches the initial object in the external environment image.

[0036] The visibility of the external environment at the current location can be determined by analyzing the external environment image collected outside the aircraft; it can also be the visibility sent by the meteorological observation sensor received by the communication module of the night vision device.

[0037] The same acquisition field of view of the object optimization data and the external environment image indicates that both collect information in the same area outside the aircraft. The object optimization data may be a contour map corresponding to the radar point cloud data having the same acquisition field of view as the external environment image, or may be historical image data having the same acquisition field of view as the external environment image, or may be a contour map and historical image data corresponding to the radar point cloud data having the same acquisition field of view as the external environment image.

[0038] In the case where the object optimization data is a contour map corresponding to the radar point cloud data, the initial object in the external environment image that matches the first object is enhanced and displayed according to the first object in the object optimization data that matches the initial object in the external environment image. Specifically, the contour point cloud data corresponding to the radar point cloud data is obtained by a contour extraction algorithm according to the radar point cloud data, the contour point cloud data in the radar coordinate system is transformed to the night vision device coordinate system by using the external parameters (rotation matrix and translation vector) obtained by calibration, and the point cloud data in the night vision device coordinate system is obtained. Then, the point cloud in the night vision device coordinate system is mapped to the image pixel coordinate system by using the internal parameters of the night vision device, and the contour map of the contours of each entity included in the radar point cloud data is obtained. For any initial object in the external environment image, if there is a first object (corresponding to the contour of the entity in the external environment image) whose similarity with the initial object exceeds the similarity threshold in the contour map, it is determined that the initial object matches the first object, and the contour of the initial object that matches it is enhanced and displayed by the first object, such as directly superimposing the first object that will match it on the initial object, so that the contour of each initial object in the external environment image is clearer.

[0039] In the case where the object optimization data is historical image data, the initial object in the external environment image that matches the first object is enhanced and displayed according to the first object in the object optimization data that matches the initial object in the external environment image. Specifically, the historical image data is registered according to the external environment image so that the pixels of the two images correspond one to one. For the registered historical image data and the external environment image, one-to-one matching is performed according to the objects. If there is a first object in the registered historical image data whose similarity with the initial object exceeds the similarity threshold, it is determined that the initial object matches the first object. For the first object that matches the initial object in the external environment image, the first object and the initial object that matches it are weighted according to the preset weight. Since the clarity of each object in the historical image data is very high, and the initial objects in the external environment image are blurred, the clarity of each initial object in the external environment image is improved by weighted summation.

[0040] In addition, when the object optimization data is a contour map and historical image data, the contour of the initial object can be enhanced by the contour map, and other details outside the contour of the initial object can be enhanced after weighted summation. For details, please refer to the specific process of enhancing the contour of the initial object by the contour map and enhancing the clarity of the initial object by the historical image data, which will not be repeated here.

[0041] In the related art, the contours in the external environment image are directly analyzed by an algorithm, and the contours in the external environment image are enhanced. In the case of low visibility, the initial object in the external environment image is not clear, and the extracted contours will be inaccurate through algorithm analysis, and even only the outermost rough contours of multiple initial objects in the area can be obtained, and the contours of more detailed single entities cannot be obtained. In the present disclosure, since the point cloud data is less affected by fog, the contours of each entity in the external environment at the current position determined based on this are clear, and the contours are superimposed on the initial object matched with it, so that each object in the final optimized image is clearly identifiable; or, the historical image data is an image collected under high visibility, and each entity in the historical image data is clearly identifiable, therefore, the initial object and the first object matching the initial object are weighted summed, so that each object in the final optimized image is clearer.

[0042] For example, Figure 4 As shown in the figure, it is a schematic diagram of the actual external environment in a certain area outside the aircraft. Figure 5 As shown in FIG. 1 , the visibility is less than the visibility threshold due to the heavy fog, and the external environment image of the aircraft is observed through the eyepiece 13 of the night vision device. The area surrounded by the curve indicated by the number 40 in the figure is a schematic diagram of the heavy fog. Figure 5 It can be seen that due to the heavy fog, the external environment observed through the night vision device becomes blurred. Figure 5 The images of the buildings shown in are the initial objects. Figure 6 As shown, the historical image data stored has the same acquisition field of view as the external environment image. Figure 6 In the historical graph data Figure 5 The first object matched by the initial object in the image is the remaining building images except the dotted frame 401. Based on the historical image data, after the initial object matched by the first object in the external environment image is enhanced and displayed by the first object, the optimized image observed by the night vision device is as follows: Figure 7 As shown in the figure, it can be seen that the objects that are of more concern during driving (such as the tall buildings in the figure) have become clear, and the background sky has not changed.

[0043] Since the aircraft is flying at a certain altitude, it is often affected by heavy fog, which makes the external environment observed by the night vision device unclear. In the embodiment of the present disclosure, the external environment image collected by the night vision device is enhanced and displayed through the pre-stored historical image data or the contour map corresponding to the radar point cloud data, so that the external environment observed by the night vision device is clearer, thereby improving driving safety.

[0044] In step S203b, according to the second object other than the first object in the object optimization data, a second object is added to the external environment image.

[0045] A second object other than the first object in the object optimization data, that is, an object in the object optimization data that does not match any object in the external environment image, is added at a position in the external environment image that corresponds to the second object in the object optimization data.

[0046] In the case where the object optimization data is a contour map corresponding to the radar point cloud data of the same acquisition field of view as the external environment image, the second object is added to the external environment image according to the second object other than the first object in the object optimization data, specifically: if the similarity between the second object in the contour map and each initial object in the external environment image is less than a similarity threshold, it is determined that the second object does not match any initial object in the external environment image. The second object is added at a position in the external environment image corresponding to the position of the second object in the contour map.

[0047] In the case where the object optimization data is historical image data, a second object is added to the external environment image according to the second object other than the first object in the object optimization data, specifically: if the similarity between the second object in the historical image data after registration and each initial object in the external environment image is less than a similarity threshold, it is determined that the second object does not match any initial object in the external environment image. The weight of the second object is set to 1, and a weighted sum is performed with the external environment image.

[0048] In the case where the object optimization data is a contour map and historical image data corresponding to the radar point cloud data with the same acquisition field of view as the external environment image, the historical image data can be first used to perform weighted summation with the external environment image, and then the second object in the contour map can be superimposed on the contour map in the weighted summed external environment image, so that the object added in the weighted summed external environment image is clearer. It is also possible to first superimpose the second object in the contour map on the contour map in the external environment image, and then perform weighted summation on the superimposed second object through the historical image data to supplement the details. The specific superposition and weighted summation refer to the description of adding the second object to the external environment image using the contour map and historical image data corresponding to the radar point cloud data, which will not be repeated here.

[0049] For example, in combination Figure 4 and Figure 5 It can be seen that due to the influence of heavy fog, Figure 4 The buildings in the dotted box 401 in the distance are completely obscured by the fog. Figure 5 In the example, the building in the dashed box 401 is not visible. Figure 6 The historical image data shown in FIG. 4 adds a non-existent second object, that is, an image of a building in a dotted frame 401, to the external environment image. Finally, the optimized image observed by the night vision device is as follows: Figure 8 As shown, objects farther away that cannot be observed due to heavy fog are added.

[0050] In the disclosed embodiment, when the visibility of the environment outside the aircraft is low, unclear initial objects in the external environment image are enhanced through a contour map or historical image data corresponding to the radar point cloud data at the same current location to make each object clearly visible, and distant objects obscured by fog in the external environment image are supplemented with a display, so that the pilot can obtain more accurate and rich external environment information through the night vision goggles.

[0051] In some embodiments, in combination Figure 2 ,like Fig. 9As shown, the image optimization data includes: field of view expansion data that is larger than the acquisition field of view of the external environment image; in the above step S203, a display object is added to the external environment image according to the image optimization data, which can be specifically implemented through the following steps S203c and S203d.

[0052] In step S203c, the supplementary objects in the field of view expansion data that are beyond the acquisition field of the external environment image are determined.

[0053] In step S203d, a supplementary object is added to the external environment image.

[0054] The field of view expansion data may be a contour map corresponding to radar point cloud data having a larger field of view than the external environment image, or may be historical image data having a larger field of view than the external environment image.

[0055] Specifically, deep learning or traditional algorithms (such as scale-invariant feature transform, accelerated robust feature and other algorithms) are used to extract feature points and feature descriptors of the field of view expansion data and the external environment image, and the extracted feature descriptors are matched to find the matching point pairs between the two images. The homography matrix between the images is calculated based on the matching point pairs (the matrix describes the projection transformation from one image to another), and the external environment image is perspective transformed using the homography matrix and projected onto the plane of the field of view expansion data. The transformed external environment image and the field of view expansion data are fused in the overlapping area to form a large field of view image containing more objects.

[0056] The supplementary object refers to the data of the part of the field of view expansion data that exceeds the field of view of the external environment image. If the field of view is measured by the field of view angle, the larger the field of view angle, the larger the field of view, and the smaller the field of view angle, the smaller the field of view. If the external environment image is acquired at a field of view angle of 90 degrees, and the field of view expansion data is acquired at a field of view angle of 120 degrees, then the part of the field of view expansion data that exceeds the 90-degree field of view angle is the supplementary object.

[0057] That is to say, the supplementary object is the data within the collection field of view of the extended field of view data outside the collection field of view of the external environment image. When the extended field of view data is a contour map corresponding to the radar point cloud data, the supplementary object is the other contours in the contour map except the contour corresponding to each initial object; when the extended field of view data is historical image data, the supplementary object is the other objects in the contour map except the objects corresponding to each initial object.

[0058] For example, Fig.10 As shown, the historical image data stored has a larger field of view than the external environment image acquisition. Figure 8 , the additional objects are Figure 8 Based on the historical image data, other building images other than the building images in Figure 8 The observation image of the night vision device shown in FIG. 1 is further optimized for the field of view, and the following is obtained: Fig.11 The image with the expanded field of view is shown. Fig.10 The supplementary objects in are added to Figure 8 middle.

[0059] Since night vision goggles are affected by their structure, it is very difficult to expand the field of view by changing the hardware structure. In addition, as the field of view of night vision goggles increases, the weight and volume of night vision goggles will increase exponentially, resulting in reduced head wearing comfort or inability to wear them. In the disclosed embodiment, the collected external environment images are supplemented by field of view expansion data, so that the field of view observed by the night vision goggles becomes larger, while ensuring the existing wearability of the night vision goggles (small volume and light weight).

[0060] In some embodiments, in combination Figure 2 ,like Fig.12 As shown, the image optimization data includes: auxiliary driving information; in the above step S203, according to the image optimization data, a display object is added to the external environment image, which can be specifically implemented through the following step S203e.

[0061] In step S203e, auxiliary driving information is added at a preset position of the external environment image.

[0062] Specifically, the auxiliary driving information is determined according to the received interactive information sent by the external device. The external device is a device that can communicate with the night vision device through the communication module of the night vision device, such as a base station, an aircraft in the same crew, a meteorological observation sensor, etc.

[0063] When the pilot wears night vision goggles to fly the aircraft, the auxiliary driving information can be added to the preset position of the external environment image to facilitate the driver to obtain more driving information. The auxiliary driving information is determined based on the interactive information sent by the external device to the communication module. For example, the current weather received from the meteorological observation sensor is used as auxiliary driving information; the current position determined by the positioning device is used as auxiliary driving information; the position of the target received from the base station is determined based on the target position and the current position, and the distance is used as auxiliary driving information; the rescue information received from the aircraft of the same crew is used as auxiliary driving information.

[0064] For example, during the flight of an aircraft, the height information of obstacles in the external environment is of particular interest, such as Fig.13, which is a schematic diagram of adding auxiliary driving information to the external environment image for indicating the height of the highest obstacle in the environment and the current flight altitude. In the external environment image observed by the night vision device, the solid line frame 1101 indicates that the height of the highest building in the current external environment is 100m, and the oval 1102 indicates that the current flight altitude is 130m.

[0065] For example, Fig.14 As shown, based on the location information of the super-high obstacle sent by the base station, the distance is determined according to the location information of the super-high obstacle and the current location, and the distance is displayed in the external environment image in the form of an alarm. The alarm symbol (including an exclamation mark in the triangle) marked 1201 in the figure corresponds to the 3km shown, indicating that there is a super-high obstacle 3km away from the current location.

[0066] In the disclosed embodiment, based on the interactive information sent by the external device, auxiliary driving information is added to the external environment image observed by the night vision device, so that the driver can obtain more information faster and more conveniently, thereby improving driving safety.

[0067] Fig.15 The structure block diagram of a night vision device shown in the present disclosure is as follows: Fig.15 As shown, it includes a low-illuminance objective lens 10, a low-illuminance image sensor 11, an image processing module 14, and a display unit 15. The low-illuminance objective lens 10 is connected to the low-illuminance image sensor 11, the low-illuminance image sensor 11 is connected to the image processing module 14, and the display unit 15 is connected to the image processing module 14 and the low-illuminance image sensor 11 respectively.

[0068] The low-light objective lens 10 is configured to gather the external environment light signal of the current position of the moving body. The low-light image sensor 11 is configured to convert the external environment light signal into the external environment image. The image processing module 14 is configured to obtain the image optimization data of the current position; and, according to the image optimization data, perform image enhancement on the external environment image, and / or add a display object to the external environment image; the display unit 15 is configured to: display the optimized image. The display unit 15 may include only a display screen, or may include a display screen and an eyepiece.

[0069] refer to Fig.16 The figure shows a schematic diagram of the hardware architecture of the night vision device.

[0070] In some embodiments, the night vision device further includes a positioning device 16 connected to the image processing module 14, and the positioning device 16 is configured to determine the current position of the moving object.

[0071] In some embodiments, the night vision device further includes a laser radar sensor 17 connected to the image processing module 14, and the laser radar sensor 17 is configured to obtain radar point cloud data at the current position of the moving object.

[0072] In some embodiments, the night vision device also includes a storage module 18 connected to the image processing module 14, and the storage module 18 is configured to store an image database and cache the current external environment image captured by the low-light image sensor 11, cache the current position determined by the positioning device 16, cache the radar point cloud data captured by the lidar sensor 17, cache the interactive information received by the communication module 19, and transmit the stored information to the image processing module 14.

[0073] In some embodiments, the night vision device further includes a communication module connected to the image processing module 14 , and the communication module 19 is configured to communicate with an external device to receive interaction information from the external device.

[0074] The functions that can be realized by the image processing module 14 are specifically described in the above step S203 and steps S203a to S203e, which will not be repeated here.

[0075] Based on the same concept as the above-mentioned image processing method, the present disclosure also provides an image processing device, such as Fig.17 As shown, the image processing device 1700 includes: a collection part 1701, an acquisition part 1702 and an image processing part 1703; the collection part 1701 is configured to use a light capture sensitivity that matches the external environment illumination to collect an external environment image of the current position of the moving body; the acquisition part 1702 is configured to obtain image optimization data of the current position; the image processing part 1703 is configured to enhance the external environment image according to the image optimization data, and / or add a display object to the external environment image.

[0076] In some embodiments, the image optimization data includes: object optimization data with the same acquisition field of view as the external environment image; the image processing part 1703 is specifically configured to enhance the display of the initial object in the external environment image that matches the first object in the object optimization data when the visibility of the external environment at the current location is less than or equal to the visibility threshold.

[0077] In some embodiments, the image processing part 1703 is specifically configured to add a second object to the external environment image according to the second object other than the first object in the object optimization data.

[0078] In some embodiments, the image optimization data includes: the image processing part 1703 is specifically configured to determine the supplementary objects in the field of view expansion data that are beyond the acquisition field of view of the external environment image; and add the supplementary objects in the external environment image.

[0079] In some embodiments, the acquisition part 1702 is specifically configured to extract the contour of each entity in the radar point cloud data based on the collected radar point cloud data of the current position to obtain a contour map; obtain historical image data corresponding to the current position from the image database, and the image optimization data includes at least one of the contour map and the historical image data.

[0080] In some embodiments, the acquisition part 1702 is specifically configured to determine the assisted driving information based on the interaction information received from the external device.

[0081] In some embodiments, the image optimization data includes: assisted driving information; the image processing part 1503 is specifically configured to add the assisted driving information at a preset position of the external environment image.

[0082] In the embodiment of the present disclosure, each part of the image processing device 1700 can implement the image processing method provided by the above method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described here.

[0083] The present disclosure also provides a computer-readable storage medium, which stores at least one instruction, and the at least one instruction is used to be executed by a processor to implement the image processing method described in the above embodiments.

[0084] The present disclosure also provides a computer program product, which includes computer instructions, which are stored in a computer-readable storage medium; a processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device executes to implement the image processing methods described in the above embodiments.

[0085] The disclosed embodiment further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned image processing method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0086] It should be understood that the chip mentioned in the embodiments of the present disclosure may also be referred to as a system-level chip, a system chip, a chip system, or a system-on-chip chip, etc.

[0087] In the several embodiments provided in the present disclosure, it should be understood that the disclosed systems, devices, servers and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0088] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0089] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0090] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present disclosure is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0091] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the present disclosure can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any media that facilitates the transmission of computer programs from one place to another. Storage media can be any available media that can be accessed by a general or special-purpose computer.

[0092] It should be noted that the technical solutions described in the present disclosure can be combined arbitrarily without conflict.

[0093] The above description is only a specific implementation mode of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present disclosure, which should be included in the protection scope of the present disclosure.

Claims

1. An image processing method, characterized in that: Applied to night vision devices, the image processing method includes: Using the light capture sensitivity that matches the external environment illumination, an image of the external environment at the current position of the moving object is collected; Obtaining image optimization data of the current location; According to the image optimization data, the external environment image is enhanced, and / or a display object is added to the external environment image.

2. The image processing method according to claim 1, characterized in that: The image optimization data includes: object optimization data having the same acquisition field of view as the external environment image; The step of performing image enhancement on the external environment image comprises: When the visibility of the external environment at the current location is less than or equal to a visibility threshold, the initial object in the external environment image that matches the first object is enhanced and displayed based on the first object in the object optimization data that matches the initial object in the external environment image.

3. The image processing method according to claim 2, characterized in that: The adding a display object to the external environment image includes: According to a second object other than the first object in the object optimization data, the second object is added to the external environment image.

4. The image processing method according to claim 1, characterized in that: The image optimization data includes: field of view expansion data that is larger than the acquisition field of view of the external environment image; the adding of display objects in the external environment image includes: Determining the supplementary objects in the visual field expansion data that are beyond the acquisition visual field of the external environment image; In the external environment image, the supplementary object is added.

5. The image processing method according to claim 1, characterized in that: The obtaining of the image optimization data of the current location includes: Extracting the contour of each entity in the radar point cloud data according to the collected radar point cloud data of the current location to obtain a contour map; From an image database, historical image data corresponding to the current location is obtained, and the image optimization data includes at least one of a contour map and historical image data.

6. The image processing method according to claim 1, characterized in that: The obtaining of the image optimization data of the current location includes: The assisted driving information is determined based on the interaction information received from the external device.

7. The image processing method according to claim 6, characterized in that: The image optimization data includes: auxiliary driving information; the adding of display objects in the external environment image includes: The auxiliary driving information is added at a preset position of the external environment image.

8. An image processing device, characterized in that: The image processing device comprises: a collection part, an acquisition part and an image processing part; The acquisition part is configured to acquire an image of the external environment at the current position of the moving object by using a light capture sensitivity that matches the external environment illumination; The acquisition part is configured to acquire the image optimization data of the current location; The image processing part is configured to perform image enhancement on the external environment image according to the image optimization data, and / or to add a display object to the external environment image.

9. A night vision device, characterized in that: The night vision device comprises: a low illumination objective lens, a low illumination image sensor, an image processing module, and a display unit, wherein the low illumination objective lens is connected to the low illumination image sensor, the low illumination image sensor is connected to the image processing module, and the display unit is connected to the image processing module and the low illumination image sensor respectively; The low-illuminance objective lens is configured to gather external ambient light signals at the current position of the moving object; The low illumination image sensor is configured to convert the external environment light signal into an external environment image; The image processing module is configured to obtain image optimization data of the current location; and, according to the image optimization data, performing image enhancement on the external environment image, and / or adding a display object to the external environment image to obtain an optimized image; The display unit is configured to display the optimized image.

10. A computer-readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the steps of the image processing method according to any one of claims 1 to 7 are implemented.

Images