A target detection system for night vision and a night vision target detection method

Abstract

This invention relates to a target detection system for night vision, comprising: an active infrared night vision device, a thermal imaging device, and an image generation unit; the active infrared night vision device clearly images objects within a preset first distance and blurs objects outside the first distance; the thermal imaging device clearly images objects outside the first distance and blurs objects within the first distance; the image generation unit performs image recognition based on the sharpness information in the image from the thermal imaging device to identify a target area corresponding to the first distance; and, based on the assumption that the target area is located on a plane perpendicular to the optical axis at the first distance, calculates the position information of the target object, and outputs a marker corresponding to the position information to a blank display area above the image screen of the active infrared night vision device for display. This invention improves operational safety.

Description

Technical Field

[0001] This invention relates to the field of night vision technology for automobiles, and particularly to a target detection system and method for night vision that combines active infrared night vision and thermal imaging technology. Background Technology

[0002] Existing night vision technologies are mainly divided into two categories: active infrared night vision and passive thermal imaging. Active infrared night vision images by emitting infrared beams and receiving the echoes reflected from the target. It can obtain very high resolution and rich details at medium and close distances (such as within 200 meters), but its effective range is limited by the power of the infrared illuminator and is easily affected by environmental factors such as smoke, fog, and haze.

[0003] Thermal imaging technology forms images by passively receiving infrared radiation emitted by objects themselves. It is not limited by lighting conditions, has strong penetration ability in smoke, and has a long detection range, reaching 300 meters or more.

[0004] Simultaneously viewing both the black-and-white image from an active infrared night vision device and the color thermal image from thermal imaging technology requires a high level of concentration from the driver, and it's easy to miss crucial details in either image, potentially leading to an accident. Furthermore, existing image fusion technologies cannot effectively fuse these images and involve enormous computational demands. Summary of the Invention

[0005] To address the above problems, this invention proposes a target detection system for night vision, characterized by comprising:

[0006] Active infrared night vision equipment, thermal imaging equipment, and image generation unit;

[0007] The active infrared night vision device is located inside the car windshield, and the thermal imaging device is located outside the car windshield. Both the active infrared night vision device and the thermal imaging device are located on the lateral symmetrical center line of the car, and are placed close to each other in the direction perpendicular to the ground, with their optical axes parallel to each other.

[0008] The active infrared night vision device is configured to clearly image objects within a preset first distance and to blur objects outside the first distance.

[0009] The thermal imaging device is configured to produce a clear image of objects beyond the first distance and a blurred image of objects within the first distance.

[0010] An image generation unit, connected to the thermal imaging device and the active infrared night vision device, is configured to: perform image recognition based on the sharpness information in the image captured by the thermal imaging device to identify a target area corresponding to a distance beyond the first distance; and, based on the assumption that the target area is located on a plane perpendicular to the optical axis at the first distance, determine whether there are one or more target objects within the target area that are within a second distance range from the optical axis of the thermal imaging device. If so, calculate the position information of the target object based on the assumption, and output the marker corresponding to the position information to a blank display area above the image screen of the active infrared night vision device for display, thereby achieving image synthesis.

[0011] According to some embodiments of the present invention, the optical system of the thermal imaging device is a fixed focal length lens, and half of the hyperfocal distance of the lens is set to be equal to the first distance.

[0012] According to some embodiments of the present invention, the image generation unit identifies the target region through the following steps:

[0013] The imaging image from the thermal imaging device is divided into multiple analysis grids;

[0014] For each analysis grid, the variance is calculated using the Laplacian operator;

[0015] The analysis grid with a variance value greater than a threshold is identified as the target region.

[0016] According to some embodiments of the present invention, the image generation unit calculates the position information by: detecting the edge of the target object on the side closest to the center line of the thermal imaging device within the target area, obtaining its deviation pixel value in the image coordinate system, and determining the position information by means of trigonometric function relationships based on the deviation pixel value, the preset field of view parameter of the thermal imaging device, and the first distance.

[0017] According to some embodiments of the present invention, the active infrared night vision device includes an infrared laser illuminator and a near-infrared image sensor, wherein the illumination range of the illuminator is matched with the field of view of the sensor.

[0018] According to some embodiments of the present invention, the active infrared night vision device pre-identifies the edges on both sides of the road and accordingly draws the road extension line in advance in the blank area.

[0019] This application also proposes a night vision target detection method based on the aforementioned target detection system for night vision, which includes:

[0020] The active infrared night vision device can clearly image objects within a preset first distance and blur objects outside the first distance.

[0021] The thermal imaging device can clearly image objects beyond the first distance and blur objects within the first distance.

[0022] The image generation unit performs image recognition based on the sharpness information in the image captured by the thermal imaging device to identify the target area corresponding to the first distance. Based on the assumption that the target area is located on a plane perpendicular to the optical axis at the first distance, it determines whether there are one or more target objects within the target area that are within a second distance range from the optical axis of the thermal imaging device. If there are, the position information of the target object is calculated based on the assumption, and the marker corresponding to the position information is output to the blank display area above the image screen of the active infrared night vision device for display, so as to achieve image synthesis.

[0023] According to some embodiments of the present invention, the image generation unit identifies the target region through the following steps:

[0024] The imaging image from the thermal imaging device is divided into multiple analysis grids;

[0025] For each analysis grid, the variance is calculated using the Laplacian operator;

[0026] The analysis grid with a variance value greater than a threshold is identified as the target region.

[0027] According to some embodiments of the present invention, the image generation unit calculates the position information by detecting a target object within the target area and obtaining its deviation pixel value in the image coordinate system, and determining the position information by means of trigonometric function relationships based on the deviation pixel value, the preset field of view parameter of the thermal imaging device and the first distance.

[0028] According to some embodiments of the present invention, the active infrared night vision device pre-identifies the edges on both sides of the road and accordingly pre-draws road extension lines in the blank areas. This allows the user to simultaneously perceive the distance of the markers relative to the sides of the road when subsequent markers are displayed.

[0029] With this invention, the operator can simultaneously observe distant dangers and nearby targets without switching between viewing screens, effectively solving the problem that viewing multiple screens can easily lead to driver distraction and inadequate attention to the screen, thus improving driving safety.

[0030] Moreover, this invention avoids complex focusing mechanisms and ranging algorithms, simplifies the system structure, reduces power consumption and computing resources, and improves the reliability of long-distance target screening. Attached Figure Description

[0031] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings without creative effort.

[0032] Figure 1 A block diagram of a target detection system for night vision according to some embodiments of the present invention is shown.

[0033] Figure 2 The diagram shows the distribution of active infrared night vision devices and thermal imaging devices for a target detection system for night vision according to some embodiments of the present invention in front of and behind a windshield.

[0034] Figure 3 A view is shown on a thermal imaging device for a target detection system for night vision according to some embodiments of the present invention.

[0035] Figure 4 This diagram illustrates the image distribution on an active infrared night vision device for a target detection system for night vision, according to some embodiments of the present invention.

[0036] Figure 5 A flowchart of a night vision target detection method according to some embodiments of the present invention is shown. Detailed Implementation

[0037] Exemplary embodiments of the present invention will now be described in more detail with reference to the accompanying drawings. These embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art. It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with and referenced to each other. Furthermore, the described embodiments are only some, not all, of the embodiments of the present invention.

[0038] It should be understood that the terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “described” as used herein may also mean including the plural forms. Any component, data, or structure mentioned in the embodiments of this application is generally understood to mean one or more unless expressly limited or shown to the contrary in the context. “A plurality” can refer to two or more. The terms “comprising,” “including,” “containing,” and “having” are inclusive and therefore indicate the presence of the stated features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or combinations thereof.

[0039] Furthermore, the term "and / or" in this application is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this application generally indicates that the preceding and following related objects have an "or" relationship.

[0040] Figure 1 Block diagrams of target detection systems for night vision, according to some embodiments of the present invention, are shown. Figure 1 As shown, the target detection system 100 for night vision of the present invention includes an active infrared night vision device 101, a thermal imaging device 102, and an image generation unit 103. The active infrared night vision device 101 and the thermal imaging device 102 are connected to the image generation unit 103 via data cables.

[0041] Figure 2 This diagram illustrates the distribution of active infrared night vision equipment and thermal imaging equipment in front of and behind a windshield for a target detection system for night vision, according to some embodiments of the present invention. Figure 2 As shown, the active infrared night vision device 101 is installed inside the windshield 200 (201 is the car roof) of the vehicle, and the thermal imaging device 102 is installed outside the windshield 200. Both the active infrared night vision device and the thermal imaging device are located on the lateral symmetrical center line of the vehicle, placed adjacent to each other in a direction perpendicular to the ground, with their optical axes 204 and 203 parallel to each other. The mounting surfaces of both devices are placed in close contact with the glass.

[0042] Figure 3 This diagram illustrates a view on a thermal imaging device for a target detection system for night vision according to some embodiments of the present invention. The active infrared night vision device is configured to provide a clear image of objects within a preset first distance and a blurred image of objects beyond the first distance. Figure 3 As shown, red squares 300 and 301 correspond to objects within a preset first distance, which are clearly imaged relative to surrounding objects. However, the rightmost edge of red square 300 (i.e., the edge closest to the center line of the thermal imaging device) is within a preset second distance from the center line in the image. Therefore, the marker corresponding to square 300 will be displayed. Figure 4 In the middle, see Figure 4 A circle.

[0043] The active infrared night vision device 101 may include, for example, a laser illuminator and a near-infrared CMOS sensor, with the illumination range matched to the sensor's field of view. Its lens supports fast autofocus from 5 to 200 meters.

[0044] The thermal imaging device is configured to produce a clear image of objects beyond the first distance and a blurred image of objects within the first distance.

[0045] The thermal imaging device 102 may, for example, employ a vanadium oxide long-wave infrared detector with a resolution of 640x512, equipped with a fixed-focal-length chromium lens. Half of the hyperfocal distance of this lens is precisely set to 200 meters, ensuring that objects beyond 200 meters are clearly imaged, while objects within 200 meters are relatively blurred.

[0046] An image generation unit, connected to the thermal imaging device and the active infrared night vision device, is configured to: perform image recognition based on the sharpness information in the image captured by the thermal imaging device to identify a target area corresponding to a distance beyond the first distance; and, based on the assumption that the target area is located on a plane perpendicular to the optical axis at the first distance, determine whether there are one or more target objects within the target area that are within a second distance range from the optical axis of the thermal imaging device. If so, calculate the position information of the target object based on the assumption, and output the marker corresponding to the position information to a blank display area above the image screen of the active infrared night vision device for display, thereby achieving image synthesis.

[0047] According to some embodiments of the present invention, assuming a first distance of 200 meters and a second distance of 1.5 meters, the thermal imaging device can employ a vanadium oxide long-wave infrared detector with a resolution of 640x512.

[0048] The image generation unit processing flow is as follows:

[0049] (1) Divide the 640x512 thermal image into 16x12 (i.e., each grid is 40x42.6 pixels) analysis grids.

[0050] (2) For each grid, calculate the Laplacian variance of its image in real time. For example, at time t, the variance of grid (8, 5) is found to be greater than a threshold.

[0051] (3) Determine that grid (8, 5) is a “clearer target area” and assume that its distance is 200 meters.

[0052] (4) Within the grid, a pixel cluster (target) with the highest brightness was detected, with its horizontal coordinate near the optical axis being Px=330 pixels.

[0053] (5) Given that the horizontal field of view of the thermal imaging lens is θ = 20° and the horizontal pixel count is W = 640, the central axis is at 320 pixels.

[0054] (6) Calculate the target distance:

[0055] Number of pixels deviating from the central axis: ΔP = |330 - 320| = 10 pixels

[0056] The angle of deviation from the central axis: β = ΔP × (θ / W) = 10 × (20 / 640) = 0.31°

[0057] Second distance: A = 200 meters × tan(β) ≈ 1.08 meters

[0058] (7) Similarly, the image generation unit further converts the position A=1.08 meters into the corresponding position on the active infrared night vision device and displays it in the central blank position at the top of the display screen. It should be understood that the sensor resolution, number of grid divisions, and specific calculation parameters mentioned above are only a preferred example of the present invention. Those skilled in the art can use other sharpness evaluation algorithms (such as gradient functions, frequency domain analysis, etc.) and target detection methods according to actual needs, and these variations all fall within the protection scope of the present invention.

[0059] When using this device, users can... Figure 4 The system can detect pedestrians within 200 meters (via active infrared), and simultaneously, the display will alert the user that a target approximately 1.08 meters to the left of their line of sight is located 200 meters away. Users can use this information to anticipate and respond to potential threats from a distance.

[0060] In this invention, it is assumed that all target dangerous objects in the clearly defined area are located on the vertical plane of the optical axis at 200 meters. In reality, they may be distributed at 300 meters. However, this invention can use this method to show the danger at 300 meters to the user in advance. It amplifies and warns of the danger at 300 meters, because the distance from 200 meters to 300 meters may only take a few seconds. By warning the user of the danger that may appear in those few seconds in advance, driving safety can be greatly improved.

[0061] According to some embodiments of the present invention, the optical system of the thermal imaging device is a fixed focal length lens, and half of the hyperfocal distance of the lens is set to be equal to the first distance.

[0062] According to some embodiments of the present invention, the image generation unit identifies the target region through the following steps:

[0063] The imaging image from the thermal imaging device is divided into multiple analysis grids;

[0064] For each analysis grid, the variance is calculated using the Laplacian operator;

[0065] The analysis grid with a variance value greater than a threshold is identified as the target region.

[0066] According to some embodiments of the present invention, the image generation unit calculates the position information by detecting the rightmost edge of the target object within the target area and obtaining its deviation pixel value in the image coordinate system, and determining the position information by means of trigonometric function relationships based on the deviation pixel value, the preset field of view parameter of the thermal imaging device and the first distance.

[0067] According to some embodiments of the present invention, the active infrared night vision device includes an infrared laser illuminator and a near-infrared image sensor, wherein the illumination range of the illuminator is matched with the field of view of the sensor.

[0068] According to some embodiments of the present invention, the active infrared night vision device pre-identifies the edges on both sides of the road and accordingly pre-draws road extension lines in the blank areas. This allows the user to simultaneously perceive the distance of the markers relative to the sides of the road when subsequent markers are displayed.

[0069] This application also proposes a night vision target detection method based on the aforementioned target detection system for night vision, such as... Figure 5 As shown, it includes the following steps:

[0070] S1. Clearly image objects within a preset first distance using an active infrared night vision device, and blurry image objects outside the first distance;

[0071] S2. Use a thermal imaging device to clearly image objects outside the first distance and to blur objects within the first distance;

[0072] S3. The image generation unit performs image recognition based on the sharpness information in the image of the thermal imaging device to identify the target area corresponding to the first distance; and based on the assumption that the target area is located on a plane perpendicular to the optical axis at the first distance, it determines whether there are one or more target objects in the target area that are within the second distance range of the optical axis of the thermal imaging device. If there are, the position information of the target object is calculated based on the assumption, and the marker corresponding to the position information is output to the blank display area above the image screen of the active infrared night vision device for display, so as to achieve image synthesis.

[0073] According to some embodiments of the present invention, the image generation unit identifies the target region through the following steps:

[0074] The imaging image from the thermal imaging device is divided into multiple analysis grids;

[0075] For each analysis grid, the variance is calculated using the Laplacian operator;

[0076] The analysis grid with a variance value greater than a threshold is identified as the target region.

[0077] According to some embodiments of the present invention, the image generation unit calculates the position information by detecting a target object within the target area and obtaining its deviation pixel value in the image coordinate system, and determining the position information by means of trigonometric function relationships based on the deviation pixel value, the preset field of view parameter of the thermal imaging device and the first distance.

[0078] According to some embodiments of the present invention, the active infrared night vision device pre-identifies the edges on both sides of the road and accordingly pre-draws road extension lines in the blank areas. This allows the user to simultaneously perceive the distance of the markers relative to the sides of the road when subsequent markers are displayed.

[0079] With this invention, the operator can simultaneously observe distant dangers and nearby targets without switching between viewing screens, effectively solving the problem that viewing multiple screens can easily lead to driver distraction and inadequate attention to the screen, thus improving driving safety.

[0080] Moreover, this invention avoids complex focusing mechanisms and ranging algorithms, simplifies the system structure, reduces power consumption and computing resources, and improves the reliability of long-distance target screening.

[0081] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A target detection system for night vision, characterized in that, include: Active infrared night vision equipment, thermal imaging equipment, and image generation unit; The active infrared night vision device is located inside the car windshield, and the thermal imaging device is located outside the car windshield. Both the active infrared night vision device and the thermal imaging device are located on the lateral symmetrical center line of the car, and are placed close to each other in the direction perpendicular to the ground, with their optical axes parallel to each other. The active infrared night vision device is configured to clearly image objects within a preset first distance and to blur objects outside the first distance. The thermal imaging device is configured to produce a clear image of objects beyond the first distance and a blurred image of objects within the first distance. An image generation unit, connected to the thermal imaging device and the active infrared night vision device, is configured to: perform image recognition based on the sharpness information in the image captured by the thermal imaging device, so as to identify the target area corresponding to the first distance; Based on the assumption that the target area is located on a plane perpendicular to the optical axis at the first distance, it is determined whether there are one or more target objects within the target area whose optical axes are within the second distance range of the thermal imaging device. If there are, the position information of the target object is calculated based on the assumption, and the marker corresponding to the position information is output to the blank display area above the imaging screen of the active infrared night vision device for display, so as to achieve image synthesis.

2. The target detection system for night vision according to claim 1, characterized in that, The optical system of the thermal imaging device is a fixed focal length lens, and half of the hyperfocal distance of the lens is set to be equal to the first distance.

3. The target detection system for night vision according to claim 1, characterized in that, The image generation unit identifies the target region through the following steps: The imaging image from the thermal imaging device is divided into multiple analysis grids; For each analysis grid, the variance is calculated using the Laplacian operator; The analysis grid with a variance value greater than a threshold is identified as the target region.

4. The target detection system for night vision according to claim 1, characterized in that, The image generation unit calculates the position information by detecting the edge of the target object on the side closest to the center line of the thermal imaging device within the target area, obtaining its deviation pixel value in the image coordinate system, and determining the position information by using trigonometric function relationships based on the deviation pixel value, the preset field of view parameter of the thermal imaging device, and the first distance.

5. The target detection system for night vision according to claim 1, characterized in that, The active infrared night vision device includes an infrared laser illuminator and a near-infrared image sensor, wherein the illumination range of the illuminator is matched with the field of view of the sensor.

6. The target detection system for night vision according to claim 1, characterized in that, Active infrared night vision equipment pre-identifies the edges on both sides of the road and accordingly draws the road extension line in advance in the blank display area.

7. A night vision target detection method based on the target detection system for night vision according to any one of claims 1-6, characterized in that, include: The active infrared night vision device can clearly image objects within a preset first distance and blur objects outside the first distance. The thermal imaging device can clearly image objects beyond the first distance and blur objects within the first distance. The image generation unit performs image recognition based on the sharpness information in the image captured by the thermal imaging device to identify the target area corresponding to the first distance. Based on the assumption that the target area is located on a plane perpendicular to the optical axis at the first distance, it is determined whether there are one or more target objects within the target area whose optical axes are within the second distance range of the thermal imaging device. If there are, the position information of the target object is calculated based on the assumption, and the marker corresponding to the position information is output to the blank display area above the imaging screen of the active infrared night vision device for display, so as to achieve image synthesis.

8. The night vision target detection method according to claim 7, characterized in that, The image generation unit identifies the target region through the following steps: The imaging image from the thermal imaging device is divided into multiple analysis grids; For each analysis grid, the variance is calculated using the Laplacian operator; The analysis grid with a variance value greater than a threshold is identified as the target region.

9. The night vision target detection method according to claim 7, characterized in that, The image generation unit calculates the position information by detecting a target object within the target area and obtaining its offset pixel value in the image coordinate system, and determining the position information based on the offset pixel value, the preset field of view parameter of the thermal imaging device, and the first distance through trigonometric function relationships.

10. The night vision target detection method according to claim 7, characterized in that, Active infrared night vision equipment pre-identifies the edges on both sides of the road and accordingly draws the road extension line in advance in the blank display area.

Images