Vehicle low-light vision aided optimization system and method based on quantum sensing

Abstract

The invention discloses a vehicle low-light vision aided optimization system and method based on quantum sensing, and relates to the field of aided driving. A quantum infrared sensor and a starlight-level camera are fixed through an aluminum alloy or carbon fiber support in a mode that optical axes are parallel or an included angle is fixed; a thermal imaging image shot by a quantum infrared sensor and an image shot by a starlight-level camera in a low-light environment are utilized, the thermal imaging image and a visible light image in the low-light environment are converted into a temperature grey image and a visible light grey image according to a frame image, and temperature information of the temperature grey image and texture details of the visible light grey image are extracted; and the two grey images are fused and converted to generate a color image containing temperature information and more details, and early warning is performed on a driver according to the temperature information and the contour area change rate, so that the driving safety is effectively enhanced.

Description

Technical Field

[0001] The present invention relates to the field of assisted driving technology, and in particular to a vehicle low-light vision assistance optimization system and method based on quantum sensing. Background Art

[0002] In the field of intelligent driving, vehicle-mounted images in low-light environments have problems such as low target recognition, failure to highlight ambient colors at night, and lack of texture details. Existing sensor fusion solutions have insufficient accuracy, and the texture of vehicle-mounted images is not clear enough when used at night, resulting in the driver's inability to observe road conditions well. To this end, the present invention proposes a vehicle low-light visual assistance optimization system and method, which uses a convolutional neural network training model to train the mapping relationship between temperature and environment, eliminates exhaust interference based on an information entropy algorithm, processes visible light image data, extracts and fuses key information of the two images to generate a new color image, analyzes the key information contained in the new color image, and issues a warning to the driver to enhance driving safety. Summary of the Invention

[0003] The purpose of the present invention is to provide a vehicle low-light vision assistance optimization system and method based on quantum sensing to solve the problems raised in the prior art.

[0004] To achieve the above objectives, the present invention provides the following technical solutions: a vehicle low-light vision assistance optimization method based on quantum sensing, and an intelligent management method comprising the following steps: Step S1: Fix the quantum infrared sensor and the starlight-level camera with their optical axes parallel or at a fixed angle to capture thermal imaging and visible light images; Step S1-1, using an aluminum alloy or carbon fiber rigid bracket to fix the quantum infrared sensor and the starlight-level camera, with the bracket deformation coefficient ≤ 0.5 μm / °C; Step S1-2: Use a high-precision electronic level to ensure that the angle error between the two devices in the horizontal and vertical directions is ≤1mrad; Step S1-3: A fixed angle x° is preset for the installation of the quantum infrared sensor and the starlight-level camera. The fixed angle must satisfy: ; Where FOV g , FOV v is the field of view of the infrared and visible light lenses; △θ=5 is a safety margin, which lays the foundation for subsequent data fusion and analysis.

[0005] Step S2: Collect a large number of thermal images with known temperature information as training data, train the model to learn the mapping relationship between thermal image features and temperature, and then use the trained model to predict new thermal images to obtain temperature values ​​and convert them into temperature grayscale images; Step S2-1: Collect a large number of thermal imaging images as the original data set, including thermal images of industrial equipment, human bodies, and natural environments. Use biquadric interpolation to unify the resolution. Normalize the pixels of the images with the unified resolution. Split the preprocessed data set into a training set, a validation set, and a test set with a ratio of 70%, 15%, and 15% respectively. Step S2-2: Use the U-Net architecture of the convolutional neural network (CNN) to build a model, and then use the CNN convolutional neural network to train the model to train the mapping relationship between the thermal imaging image and the temperature gray map; Step S2-3: After the quantum infrared sensor generates a thermal imaging image, the resolution of the thermal imaging image is normalized, and then the processed thermal imaging image is placed in the trained model. The temperature prediction map is obtained through model calculation, and then the temperature prediction map is converted into a temperature gray map.

[0006] Step S3, set the exhaust gas temperature gray map to 1 ref , the temperature grayscale image without exhaust is I test , the temperature gray map with exhaust gas is divided into N×N pixel blocks, the diffuse exhaust gas and concentrated exhaust gas are located, and the exhaust gas interference area is filled to obtain the temperature gray map without exhaust gas interference; Step S3-1: Divide the temperature grayscale image into N×N pixel blocks and calculate the information entropy of each pixel block. The calculation formula is: ; Where H(I) is the information entropy of pixel block I, The sum of grayscale values ​​i from 0 to 255, p(x,y),i is actually p(i), which represents the probability of grayscale value i appearing in the image. It means taking the logarithm of the probability p(i) of gray value i with base 2; Step S3-2: quantify the uncertainty of the distribution of each pixel block by the information entropy value, calculate the entropy mean u and standard deviation σ of the entire image, set the thresholds for diffuse exhaust and concentrated exhaust, and make a judgment based on the fact that the entropy value of diffuse exhaust is high and the entropy value of concentrated exhaust is low. The diffusion judgment formula is: ; The centralized determination formula is: ; In the formula, k1 and k2 are empirical coefficients, k1 is set to 1.2, and k2 is set to 0.8; Step S3-3, assuming that the diffusion type exhaust gas is T high , concentrated exhaust gas is T low , based on the exhaust gas determination, the exhaust gas mask is generated, and the calculation formula is: ; Where H(x,y) is the local information entropy, M(x,y) is the exhaust mask, and the pixel block area containing exhaust is marked according to the exhaust mask; Step S3-4: Set a pixel threshold x and calculate the standard deviation of the background area pixels. When the standard deviation is greater than the threshold x, use the median filling method: ; Where, I out (M) indicates the exhaust interference area M in the output image, median (I bg ) represents the pixel median value of the non-exhaust background area extracted from the input image; When the standard deviation is less than the threshold x, the mean value is used for filling. The filling method is: ; Where, I out (M) represents the exhaust interference area M in the output image, mean(I bg ) represents the non-interfering background region I extracted from the input image bg By using different filling methods, the pixel average value is used to accurately locate and eliminate the exhaust interference area in the thermal imaging image, providing a high-quality data foundation for the follow-up.

[0007] Step S4: performing median filtering noise reduction processing on the image captured by the starlight camera, dividing the image into multiple small areas, performing histogram equalization on each small area, and finally integrating all the small areas to generate a visible light grayscale image; Step S4-1: convert the color image captured by the starlight camera into a grayscale image using a weighted average method, and then normalize the image pixel values; Step S4-2: Use a 3×3 square filter window to fill the image, then extract all pixel values ​​within the window centered on the current pixel, sort them, and take the median as the new value of the current pixel; Step S4-3 divides the image into several small non-overlapping x×x pixel blocks, where x is the number of pixel blocks. The grayscale values ​​of all pixels in the area are obtained using the weighted average method. The number of pixels at each grayscale value is counted to establish a grayscale histogram H(g), where g is the grayscale level (0-255). The CDF function is used to calculate the accumulation of all H(i)s with grayscale levels from 0 to g, and then the CDF(g) is normalized to the interval [0,255].

[0008] Step S5: Using a bilinear interpolation method, the resolution of the temperature gray map converted from the thermal image is maintained consistent with the resolution of the visible light gray map converted from the visible light image. A two-dimensional coordinate system is constructed with the upper left corner of the temperature gray map and the upper left corner of the visible light image as the coordinate origin, and the origin of the two-dimensional coordinate system constructed by the visible light image is aligned with the origin of the coordinate system of the temperature gray map; Step S5-1: The resolution of the temperature gray image converted from thermal imaging is preset to W1×H1, and the resolution of the processed visible light image is W2×H2. Based on the temperature gray image converted from thermal imaging, the calculation formula is: ; ; Where S x is the horizontal scaling factor, s y is the vertical scaling factor, through S x With s y Determine the corresponding position of the target image pixels in the original image to adjust the image size; Step S5-2: For each pixel (x, y) of the target image in the target coordinate system, calculate the floating-point coordinates (u, v) in the original image, and extract the coordinates and values ​​of the four pixels around (u, v) in the original image. After mapping the target image coordinates to the floating-point coordinates of the original image, the corresponding position of the target pixel in the original image is located to provide a coordinate basis for interpolation; Calculate the decimal part of the floating-point coordinates and determine the interpolation weights in the horizontal and vertical directions, where a and b represent the relative positions between adjacent pixels. Adjacent pixel coordinates are: (u0,v0), (u0+1,v0), (u0,v0+1), (u0+1,v0+1) The bilinear interpolation method is used to achieve consistency of image resolution and alignment of coordinate systems.

[0009] Step S6-1: Calculate the gradient of the image in the horizontal and vertical directions. The calculation formula is: ; ; G x The matrix is ​​the x-direction template of the Sobel operator, G y The matrix is ​​the template of the Sobel operator in the y direction, and the number is the weight ratio of the pixel area; Step S6: Calculate the gradients of the two gray images using the Sobel operator, extract the temperature information of the temperature gray image and the texture details of the visible light grayscale, generate a new gray image, and convert the generated new gray image into a new color image; Step S6-1: Calculate the gradient of the image in the horizontal and vertical directions. The calculation formula is: ; ; G x The matrix is ​​the x-direction template of the Sobel operator, G y The matrix is ​​the template of the Sobel operator in the y direction, and the number is the weight ratio of the pixel area; Step S6-2: record the temperature gray map as T(x,y), and compare T(x,y) with G X Perform convolution and calculate the horizontal gradient of each pixel (x, y). The calculation formula is: ; Where i and j are the offsets within the kernel, and the boundary pixels can be ignored or filled with zeros; For T(x,y) and G y Perform convolution and calculate the vertical gradient of each pixel (x, y). The calculation formula is: ; Step S6-3: Calculate the gradient amplitude of the temperature grayscale image by combining the horizontal gradient and the vertical gradient. The calculation formula is: ; Step S6-4: denote the visible light grayscale image as V(x,y), and perform convolution using the same Sobel kernel as the temperature grayscale image. The calculation formula is: ; ; Reflects texture details of visible light images; Step S6-5: Normalize the temperature grayscale gradient and the visible light texture gradient and map them to the grayscale range of [0, 255]. The temperature gradient normalization calculation formula is: ; Where, Represents the gradient magnitude of the image at coordinate (x, y), and Represents the gradient amplitude T in the image G The minimum and maximum values ​​of (x,y); The calculation formula for visible light gradient normalization is: ; Where, 、 and 、 Corresponding to the minimum and maximum values ​​of the gradient map respectively; Step S6-6: Use the linear weighted method to fuse the images, use the previously trained model to convert the new gray image into a color image and generate a color image, so that the new image contains temperature information and has a more accurate color image.

[0010] Step S7: Mark key legends according to the temperature information of the color image, and issue a warning to the driver according to the area change rate.

[0011] Step S7-1: Use the cv2.findContours function to mark the contours of the person and the car, set the human body temperature threshold T1, and the vehicle temperature threshold T2; Step S7-2: When the temperature is greater than T1, it is determined to be a person; when the temperature is greater than T2, it is determined to be a vehicle. The temperature information is used to distinguish between people, vehicles and background, and a reminder is given to the driver; Step S7-3: Calculate the relationship between the outlines of the person and the vehicle over time. The calculation formula is: ; set up The threshold S max ,When the rate of change exceeds the threshold, it is determined that the vehicle speed is too fast and a warning is issued to the driver.

[0012] The vehicle low-light visual assistance optimization system includes a hardware integration module, a data processing module, a visible light preprocessing module, an image alignment module, an image fusion conversion module, and an early warning module; the hardware integration module includes a quantum infrared sensor and a starlight-level camera, which are responsible for shooting thermal imaging images and visible light images; the data processing module includes a training model, which processes thermal imaging images to generate a temperature gray map, and processes the exhaust interference gray map; the visible light preprocessing module is responsible for data cleaning of visible light images, and performing resolution consistency operations on the temperature gray map and the visible light gray map; the image alignment module is responsible for constructing a coordinate system and mapping the temperature gray map and the visible light gray map; the image fusion module is responsible for generating a new gray map containing temperature information and texture details, and converting it into a color image; the early warning module is responsible for marking and distinguishing between people and vehicles, determining the vehicle speed and issuing early warnings to the driver.

[0013] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention extracts the temperature information of the temperature gray image and the texture details of the light gray image, and fuses them to generate a more accurate image, thereby improving the accuracy of target analysis.

[0014] 2. The present invention eliminates the influence of exhaust gas by filling the exhaust gas interference in the thermal imaging image, and provides a reliable data basis for subsequent processing of visible light images.

[0015] 3. The present invention uses temperature information to mark key legends and timely warns drivers by judging the area change rate. Combined with high-quality image preprocessing and fusion, the warning is more accurate and driving safety is effectively improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 Schematic diagram of the process of the vehicle low-light vision assistance optimization method based on quantum sensing of the present invention; Figure 2 This is a schematic diagram of the structure of the vehicle low-light vision assistance optimization system based on quantum sensing of the present invention. DETAILED DESCRIPTION

[0017] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0018] Step S1: Fix the quantum infrared sensor and the starlight-level camera with their optical axes parallel or at a fixed angle to capture thermal imaging and visible light images; Step S1-1, using an aluminum alloy or carbon fiber rigid bracket to fix the quantum infrared sensor and the starlight-level camera, with the bracket deformation coefficient ≤ 0.5 μm / °C; Step S1-2: Use a high-precision electronic level to ensure that the angle error between the two devices in the horizontal and vertical directions is ≤1mrad; Step S1-3: A fixed angle x° is preset for the installation of the quantum infrared sensor and the starlight-level camera. The fixed angle must satisfy: ; Where FOV g , FOV v is the field of view of the infrared and visible light lenses; △θ=5 is a safety margin, which lays the foundation for subsequent data fusion and analysis.

[0019] Step S2: Collect a large number of thermal images with known temperature information as training data, train the model to learn the mapping relationship between thermal image features and temperature, and then use the trained model to predict new thermal images to obtain temperature values ​​and convert them into temperature grayscale images; Step S2-1: Collect a large number of thermal imaging images as the original data set, including thermal images of industrial equipment, human bodies, and natural environments. Use biquadric interpolation to unify the resolution. Normalize the pixels of the images with the unified resolution. Split the preprocessed data set into a training set, a validation set, and a test set with a ratio of 70%, 15%, and 15% respectively. Step S2-2: Use the U-Net architecture of the convolutional neural network (CNN) to build a model, and then use the CNN convolutional neural network to train the model to train the mapping relationship between the thermal imaging image and the temperature gray map; Step S2-3: After the quantum infrared sensor generates a thermal imaging image, the resolution of the thermal imaging image is normalized, and then the processed thermal imaging image is placed in the trained model. The temperature prediction map is obtained through model calculation, and then the temperature prediction map is converted into a temperature gray map.

[0020] Step S3, set the exhaust gas temperature gray map to 1 ref , the temperature grayscale image without exhaust is I test , the temperature gray map with exhaust gas is divided into N×N pixel blocks, the diffuse exhaust gas and concentrated exhaust gas are located, and the exhaust gas interference area is filled to obtain the temperature gray map without exhaust gas interference; Step S3-1: Divide the temperature grayscale image into N×N pixel blocks and calculate the information entropy of each pixel block. The calculation formula is: ; Where H(I) is the information entropy of pixel block I, The sum of grayscale values ​​i from 0 to 255, p(x,y),i is actually p(i), which represents the probability of grayscale value i appearing in the image. It means taking the logarithm of the probability p(i) of gray value i with base 2; Step S3-2: quantify the uncertainty of the distribution of each pixel block by the information entropy value, calculate the entropy mean u and standard deviation σ of the entire image, set the thresholds for diffuse exhaust and concentrated exhaust, and make a judgment based on the fact that the entropy value of diffuse exhaust is high and the entropy value of concentrated exhaust is low. The diffusion judgment formula is: ; The centralized determination formula is: ; In the formula, k1 and k2 are empirical coefficients, k1 is set to 1.2, and k2 is set to 0.8; Step S3-3, assuming that the diffusion type exhaust gas is T high , concentrated exhaust gas is T low , based on the exhaust gas determination, the exhaust gas mask is generated, and the calculation formula is: ; Where H(x,y) is the local information entropy, M(x,y) is the exhaust mask, and the pixel block area containing exhaust is marked according to the exhaust mask; Step S3-4: Set a pixel threshold x and calculate the standard deviation of the background area pixels. When the standard deviation is greater than the threshold x, use the median filling method: ; Where, I out (M) indicates the exhaust interference area M in the output image, median (I bg ) represents the pixel median value of the non-exhaust background area extracted from the input image; When the standard deviation is less than the threshold x, the mean value is used for filling. The filling method is: ; Where, I out (M) represents the exhaust interference area M in the output image, mean(I bg ) represents the non-interfering background region I extracted from the input image bg By using different filling methods, the pixel average value is used to accurately locate and eliminate the exhaust interference area in the thermal imaging image, providing a high-quality data foundation for the follow-up.

[0021] Step S4: performing median filtering noise reduction processing on the image captured by the starlight camera, dividing the image into multiple small areas, performing histogram equalization on each small area, and finally integrating all the small areas to generate a visible light grayscale image; Step S4-1: convert the color image captured by the starlight camera into a grayscale image using a weighted average method, and then normalize the image pixel values; Step S4-2: Use a 3×3 square filter window to fill the image, then extract all pixel values ​​within the window centered on the current pixel, sort them, and take the median as the new value of the current pixel; Step S4-3 divides the image into several small non-overlapping x×x pixel blocks, where x is the number of pixel blocks. The grayscale values ​​of all pixels in the area are obtained using the weighted average method. The number of pixels at each grayscale value is counted to establish a grayscale histogram H(g), where g is the grayscale level (0-255). The CDF function is used to calculate the accumulation of all H(i)s with grayscale levels from 0 to g, and then the CDF(g) is normalized to the interval [0,255].

[0022] Step S5: Using a bilinear interpolation method, the resolution of the temperature gray map converted from the thermal image is maintained consistent with the resolution of the visible light gray map converted from the visible light image. A two-dimensional coordinate system is constructed with the upper left corner of the temperature gray map and the upper left corner of the visible light image as the coordinate origin, and the origin of the two-dimensional coordinate system constructed by the visible light image is aligned with the origin of the coordinate system of the temperature gray map; Step S5-1: The resolution of the temperature gray image converted from thermal imaging is preset to W1×H1, and the resolution of the processed visible light image is W2×H2. Based on the temperature gray image converted from thermal imaging, the calculation formula is: ; ; Where S x is the horizontal scaling factor, s y is the vertical scaling factor, through S x With s y Determine the corresponding position of the target image pixels in the original image to adjust the image size; Step S5-2: For each pixel (x, y) of the target image in the target coordinate system, calculate the floating-point coordinates (u, v) in the original image, and extract the coordinates and values ​​of the four pixels around (u, v) in the original image. After mapping the target image coordinates to the floating-point coordinates of the original image, the corresponding position of the target pixel in the original image is located to provide a coordinate basis for interpolation; Calculate the decimal part of the floating-point coordinates and determine the interpolation weights in the horizontal and vertical directions, where a and b represent the relative positions between adjacent pixels. Adjacent pixel coordinates are: (u0,v0), (u0+1,v0), (u0,v0+1), (u0+1,v0+1) The bilinear interpolation method is used to achieve consistency of image resolution and alignment of coordinate systems.

[0023] Step S6-1: Calculate the gradient of the image in the horizontal and vertical directions. The calculation formula is: ; ; G x The matrix is ​​the x-direction template of the Sobel operator, G y The matrix is ​​the template of the Sobel operator in the y direction, and the number is the weight ratio of the pixel area; Step S6: Calculate the gradients of the two gray images using the Sobel operator, extract the temperature information of the temperature gray image and the texture details of the visible light grayscale, generate a new gray image, and convert the generated new gray image into a new color image; Step S6-1: Calculate the gradient of the image in the horizontal and vertical directions. The calculation formula is: ; ; G x The matrix is ​​the x-direction template of the Sobel operator, G y The matrix is ​​the template of the Sobel operator in the y direction, and the number is the weight ratio of the pixel area; Step S6-2: record the temperature gray map as T(x,y), and compare T(x,y) with G X Perform convolution and calculate the horizontal gradient of each pixel (x, y). The calculation formula is: ; Where i and j are the offsets within the kernel, and the boundary pixels can be ignored or filled with zeros; For T(x,y) and G y Perform convolution and calculate the vertical gradient of each pixel (x, y). The calculation formula is: ; Step S6-3: Calculate the gradient amplitude of the temperature grayscale image by combining the horizontal gradient and the vertical gradient. The calculation formula is: ; Step S6-4: denote the visible light grayscale image as V(x,y), and perform convolution using the same Sobel kernel as the temperature grayscale image. The calculation formula is: ; ; Reflects texture details of visible light images; Step S6-5: Normalize the temperature grayscale gradient and the visible light texture gradient and map them to the grayscale range of [0, 255]. The temperature gradient normalization calculation formula is: ; Where, Represents the gradient magnitude of the image at coordinate (x, y), and Represents the gradient amplitude T in the image G The minimum and maximum values ​​of (x,y); The calculation formula for visible light gradient normalization is: ; Where, 、 and 、 Corresponding to the minimum and maximum values ​​of the gradient map respectively; Step S6-6: Use the linear weighted method to fuse the images, use the previously trained model to convert the new gray image into a color image and generate a color image, so that the new image contains temperature information and has a more accurate color image.

[0024] Step S7: Mark key legends according to the temperature information of the color image, and issue a warning to the driver according to the area change rate.

[0025] Step S7-1: Use the cv2.findContours function to mark the contours of the person and the car, set the human body temperature threshold T1, and the vehicle temperature threshold T2; Step S7-2: When the temperature is greater than T1, it is determined to be a person; when the temperature is greater than T2, it is determined to be a vehicle. The temperature information is used to distinguish between people, vehicles and background, and a reminder is given to the driver; Step S7-3: Calculate the relationship between the outlines of the person and the vehicle over time. The calculation formula is: ; set up The threshold S max ,When the rate of change exceeds the threshold, it is determined that the vehicle speed is too fast and a warning is issued to the driver.

[0026] The vehicle low-light visual assistance optimization system includes a hardware integration module, a data processing module, a visible light preprocessing module, an image alignment module, an image fusion conversion module, and an early warning module; the hardware integration module includes a quantum infrared sensor and a starlight-level camera, which are responsible for shooting thermal imaging images and visible light images; the data processing module includes a training model, which processes thermal imaging images to generate a temperature gray map, and processes the exhaust interference gray map; the visible light preprocessing module is responsible for data cleaning of visible light images, and performing resolution consistency operations on the temperature gray map and the visible light gray map; the image alignment module is responsible for constructing a coordinate system and mapping the temperature gray map and the visible light gray map; the image fusion module is responsible for generating a new gray map containing temperature information and texture details, and converting it into a color image; the early warning module is responsible for marking and distinguishing between people and vehicles, determining the vehicle speed and issuing early warnings to the driver.

[0027] Example: The field of view of the quantum sensor is 45 degrees, and the field of view of the starlight camera is 60 degrees. According to the fixed angle formula, the angle between the two devices is 58 degrees. The temperature gray image is divided into 32*32 pixel blocks, and the information entropy is calculated according to the information entropy calculation formula; The entropy mean of the whole image is 1.2, and the standard deviation is 0.3. The exhaust type is determined according to the judgment formula and the exhaust mask is generated. The pixel threshold x is set to 15, the standard deviation of the background area pixels is calculated to be 20, and the exhaust interference area is filled with the median value I. bgis the pixel value of the non-exhaust background area, the median value is 80, and the pixel value after filling is 80.

[0028] The weighted average method is used to convert the color image into a grayscale image, and a 3*3 filter window is used to process the image. According to the sorting [20, 30, 40, 50, 60, 70, 80, 90, 100], the median 60 is taken as the new value of the pixel, and the image is divided into small 16*16 areas. The cumulative value is calculated by the CDF function and planned to the [0, 255] interval to enhance the image contrast.

[0029] The resolution of the temperature gray image is 320*240, and the resolution of the visible light image is 640*480. Taking the temperature gray image as the benchmark, the horizontal scaling factor is 0.5. The vertical scaling factor is 0.5. The pixel values ​​are calculated using bilinear interpolation, and the two images are fused and converted into color images using weighted multiplication. Early warnings are issued based on the temperature and area change rates.

[0030] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the invention can be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims, not the foregoing description, and all variations within the meaning and range of equivalents of the claims are intended to be included therein. Any reference sign in a claim should not be construed as limiting the claim to which it relates.

Claims

1. A vehicle low-light vision assistance optimization method based on quantum sensing, characterized by: The method comprises the following steps: Step S1: Fix the quantum infrared sensor and the starlight-level camera with their optical axes parallel or at a fixed angle to capture thermal imaging and visible light images; Step S2: Collect a large number of thermal images with known temperature information as training data, train the model to learn the mapping relationship between thermal image features and temperature, and then use the trained model to predict new thermal images to obtain temperature values ​​and convert them into temperature grayscale images; Step S3, set the exhaust gas temperature gray map to 1 ref , the temperature grayscale image without exhaust is I test , the temperature gray map with exhaust gas is divided into N×N pixel blocks, the diffuse exhaust gas and concentrated exhaust gas are located, and the exhaust gas interference area is filled to obtain the temperature gray map without exhaust gas interference; Step S4: performing median filtering noise reduction processing on the image captured by the starlight camera, dividing the image into multiple small areas, performing histogram equalization on each small area, and finally integrating all the small areas to generate a visible light grayscale image; Step S5: Using a bilinear interpolation method, the resolution of the temperature gray map converted from the thermal image is maintained consistent with the resolution of the visible light gray map converted from the visible light image. A two-dimensional coordinate system is constructed with the upper left corner of the temperature gray map and the upper left corner of the visible light image as the coordinate origin, and the origin of the two-dimensional coordinate system constructed by the visible light image is aligned with the origin of the coordinate system of the temperature gray map; Step S6: Calculate the gradients of the two gray images using the Sobel operator, extract the temperature information of the temperature gray image and the texture details of the visible light grayscale, generate a new gray image, and convert the generated new gray image into a new color image; Step S7: Mark key legends according to the temperature information of the color image, and issue a warning to the driver according to the area change rate.

2. The method for optimizing low-light vision assistance for vehicles based on quantum sensing according to claim 1, characterized in that: The specific steps of step S1 are as follows: Step S1-1, using an aluminum alloy or carbon fiber rigid bracket to fix the quantum infrared sensor and the starlight-level camera, with the bracket deformation coefficient ≤ 0.5 μm / °C; Step S1-2: Use a high-precision electronic level to ensure that the angle error between the two devices in the horizontal and vertical directions is ≤1mrad; Step S1-3: A fixed angle x° is preset for the installation of the quantum infrared sensor and the starlight-level camera. The fixed angle must satisfy: ； Where FOV g , FOV v is the field of view of the infrared and visible light lenses; △θ=5 is the safety margin.

3. The method for optimizing low-light vision assistance for vehicles based on quantum sensing according to claim 2, characterized in that: The specific steps of step S2 are as follows: Step S2-1: Collect a large number of thermal imaging images as the original data set, including thermal images of industrial equipment, human bodies, and natural environments. Use biquadric interpolation to unify the resolution. Normalize the pixels of the images with the unified resolution. Split the preprocessed data set into a training set, a validation set, and a test set with a ratio of 70%, 15%, and 15% respectively. Step S2-2: Use the U-Net architecture of the convolutional neural network (CNN) to build a model, and then use the CNN convolutional neural network to train the model to train the mapping relationship between the thermal imaging image and the temperature gray map; Step S2-3: After the quantum infrared sensor generates a thermal imaging image, the resolution of the thermal imaging image is normalized, and then the processed thermal imaging image is placed in the trained model. The temperature prediction map is obtained through model calculation, and then the temperature prediction map is converted into a temperature gray map.

4. The method for optimizing low-light vision assistance for vehicles based on quantum sensing according to claim 3, characterized in that: The specific steps of step S3 are as follows: Step S3-1: Divide the temperature grayscale image into N×N pixel blocks and calculate the information entropy of each pixel block. The calculation formula is: ； Where H(I) is the information entropy of pixel block I, The sum of grayscale values ​​i from 0 to 255, p(x,y),i is actually p(i), which represents the probability of grayscale value i appearing in the image. It means taking the logarithm of the probability p(i) of gray value i with base 2; Step S3-2: quantify the uncertainty of the distribution of each pixel block by the information entropy value, calculate the entropy mean u and standard deviation σ of the entire image, set the thresholds for diffuse exhaust and concentrated exhaust, and make a judgment based on the fact that the entropy value of diffuse exhaust is high and the entropy value of concentrated exhaust is low. The diffusion judgment formula is: ； The centralized determination formula is: ； In the formula, k1 and k2 are empirical coefficients, k1 is set to 1.2, and k2 is set to 0.8; Step S3-3, assuming that the diffusion type exhaust gas is T high , concentrated exhaust gas is T low , based on the exhaust gas determination, the exhaust gas mask is generated, and the calculation formula is: ； Where H(x,y) is the local information entropy, M(x,y) is the exhaust mask, and the pixel block area containing exhaust is marked according to the exhaust mask; Step S3-4: Set a pixel threshold x and calculate the standard deviation of the background area pixels. When the standard deviation is greater than the threshold x, use the median filling method: ； Where, I out (M) indicates the exhaust interference area M in the output image, median (I bg ) represents the pixel median value of the non-exhaust background area extracted from the input image; When the standard deviation is less than the threshold x, the mean value is used for filling. The filling method is: ； Where, I out (M) represents the exhaust interference area M in the output image, mean(I bg ) represents the non-interfering background region I extracted from the input image bg The average pixel value.

5. The method for optimizing low-light vision assistance for vehicles based on quantum sensing according to claim 4, characterized in that: The specific steps of step S4 are as follows: Step S4-1: convert the color image captured by the starlight camera into a grayscale image using a weighted average method, and then normalize the image pixel values; Step S4-2: Use a 3×3 square filter window to fill the image, then extract all pixel values ​​within the window centered on the current pixel, sort them, and take the median as the new value of the current pixel; Step S4-3 divides the image into several small non-overlapping x×x pixel blocks, where x is the number of pixel blocks. The grayscale values ​​of all pixels in the area are obtained using the weighted average method. The number of pixels at each grayscale value is counted to establish a grayscale histogram H(g), where g is the grayscale level (0-255). The CDF function is used to calculate the accumulation of all H(i)s with grayscale levels from 0 to g, and then the CDF(g) is normalized to the interval [0,255].

6. The method for optimizing low-light vision assistance for vehicles based on quantum sensing according to claim 5, characterized in that: The specific steps of extracting the key information of the gray image in step S5 are as follows: Step S5-1: The resolution of the temperature gray image converted from thermal imaging is preset to W1×H1, and the resolution of the processed visible light image is W2×H2. Based on the temperature gray image converted from thermal imaging, the calculation formula is: ； ； Where S x is the horizontal scaling factor, s y is the vertical scaling factor, through S x With s y Determine the corresponding position of the target image pixels in the original image to adjust the image size; Step S5-2: For each pixel (x, y) of the target image in the target coordinate system, calculate the floating-point coordinates (u, v) in the original image, and extract the coordinates and values ​​of the four pixels around (u, v) in the original image. After mapping the target image coordinates to the floating-point coordinates of the original image, the corresponding position of the target pixel in the original image is located to provide a coordinate basis for interpolation; Calculate the decimal part of the floating-point coordinates and determine the interpolation weights in the horizontal and vertical directions, where a and b represent the relative positions between adjacent pixels. Adjacent pixel coordinates are: (u0,v0), (u0+1,v0), (u0,v0+1), (u0+1,v0+1) The bilinear interpolation method is used to achieve consistency of image resolution and alignment of coordinate systems.

7. The method for optimizing low-light vision assistance for vehicles based on quantum sensing according to claim 6, characterized in that: The specific steps of extracting key information from the gray image in step S6 are as follows: Step S6-1: Calculate the gradient of the image in the horizontal and vertical directions. The calculation formula is: ； ； G x The matrix is ​​the x-direction template of the Sobel operator, G y The matrix is ​​the template of the Sobel operator in the y direction, and the number is the weight ratio of the pixel area; Step S6-2: record the temperature gray map as T(x,y), and compare T(x,y) with G X Perform convolution and calculate the horizontal gradient of each pixel (x, y). The calculation formula is: ； Where i and j are the offsets within the kernel, and the boundary pixels can be ignored or filled with zeros; For T(x,y) and G y Perform convolution and calculate the vertical gradient of each pixel (x, y). The calculation formula is: ； Step S6-3: Calculate the gradient amplitude of the temperature grayscale image by combining the horizontal gradient and the vertical gradient. The calculation formula is: 。 8. The method for optimizing low-light vision assistance for vehicles based on quantum sensing according to claim 6, characterized in that: The specific steps of gray image fusion in step S6 are as follows: Step S6-4: denote the visible light grayscale image as V(x,y), and perform convolution using the same Sobel kernel as the temperature grayscale image. The calculation formula is: ； ； Reflects texture details of visible light images; Step S6-5: Normalize the temperature grayscale gradient and the visible light texture gradient and map them to the grayscale range of [0, 255]. The temperature gradient normalization calculation formula is: ； Where, Represents the gradient magnitude of the image at coordinate (x, y), and Represents the gradient amplitude T in the image G The minimum and maximum values ​​of (x,y); The calculation formula for visible light gradient normalization is: ； Where, 、 and 、 Corresponding to the minimum and maximum values ​​of the gradient map respectively; Step S6-6: Use the linear weighted method to fuse the images, use the previously trained model to convert the new gray image into a color image, and generate a color image.

9. The method for optimizing low-light vision assistance for vehicles based on quantum sensing according to claim 7, characterized in that: The specific steps of using temperature to distinguish between people and vehicles and the background in step S7 are as follows: Step S7-1: Use the cv2.findContours function to mark the contours of the person and the car, set the human body temperature threshold T1, and the vehicle temperature threshold T2; Step S7-2: When the temperature is greater than T1, it is determined to be a person; when the temperature is greater than T2, it is determined to be a vehicle. The temperature information is used to distinguish between people, vehicles and background, and a reminder is given to the driver; Step S7-3: Calculate the relationship between the outlines of the person and the vehicle over time. The calculation formula is: ； set up The threshold S max ,When the rate of change exceeds the threshold, it is determined that the vehicle speed is too fast and a warning is issued to the driver.

10. A quantum sensing-based vehicle low-light vision assistance optimization system, characterized by: The vehicle low-light vision assistance optimization system includes the following modules: The vehicle low-light visual assistance optimization system includes a hardware integration module, a data processing module, a visible light preprocessing module, an image alignment module, an image fusion conversion module, and an early warning module; the hardware integration module includes a quantum infrared sensor and a starlight-level camera, which are responsible for shooting thermal imaging images and visible light images; the data processing module includes a training model, which processes thermal imaging images to generate a temperature gray map, and processes the exhaust interference gray map; the visible light preprocessing module is responsible for data cleaning of visible light images, and performing resolution consistency operations on the temperature gray map and the visible light gray map; the image alignment module is responsible for constructing a coordinate system and mapping the temperature gray map and the visible light gray map; the image fusion module is responsible for generating a new gray map containing temperature information and texture details, and converting it into a color image; the early warning module is responsible for marking and distinguishing between people and vehicles, determining the vehicle speed and issuing early warnings to the driver.

Images