Road obstacle visualization system under low illumination

Abstract

The invention relates to the technical field of image processing, in particular to a low-illuminance road obstacle visualization system which comprises the following steps: collecting video data of a road under low illuminance; obtaining a light source area according to a clustering result of the overexposure points; determining the illumination influence degree of the light source on the pixel points based on the distance between the pixel points in the non-light-source area and the light source fitting circle; determining the possibility that the pixel point is located in the glare area formed by the light source by combining the displacement of the light source area between the adjacent frames and the parameter change characteristics when the pixel point is located in the glare area, and obtaining the influence degree of the pixel point by the light source by combining the illumination influence degree; and according to the influence degree, enhanced video data is obtained, obstacles on the detected road are obtained, and visualization is performed through a visualization port. According to the method and the device, the pixel points of the glare area in each frame of image are adaptively enhanced, so that the quality of the enhanced image is improved, and the accuracy of obstacle detection in a road is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a road obstacle visualization system under low illumination. Background Art

[0002] When driving on a road with low illumination, it is difficult to accurately detect and visualize obstacles on the road due to the significant change in light intensity. When driving a car, obstacles within a certain range are detected through observation by the driver and perception by the on-board camera. Obstacles detected on the road are visualized on the display end to assist the vehicle in avoiding obstacles in advance. Therefore, when driving a car on a road with low illumination, how to quickly and accurately visualize obstacles on the road is a key link to ensure safety.

[0003] Since light sources under low illumination will produce glare effects, it is difficult to identify obstacles on the road with the human eye alone, which further makes it difficult to visualize obstacles on the road. At this time, it is necessary to use the on-board camera to collect road video data within the perception range. However, when driving under low illumination, the visibility of road obstacles is reduced, and the collected video data needs to be enhanced to improve the quality of the video data, thereby improving the visualization effect of the obstacles. The traditional enhancement method is to enhance each frame of the video data through gamma transformation, but this method is difficult to appropriately suppress the glare area, affecting the accuracy of the subsequent obstacle visualization results. Summary of the invention

[0004] In order to solve the problem that the traditional gamma transform enhancement method is difficult to appropriately suppress the glare area of ​​the road image under low illumination, which affects the obstacle detection result, the present application provides a road obstacle visualization system under low illumination.

[0005] The present application provides a road obstacle visualization system under low illumination, which includes the following modules: A data acquisition module, used to collect video data of roads under low illumination; The video data analysis module is used to screen overexposed points according to the brightness values ​​of the pixels in each frame of the video data, and obtain the light source area in each frame of the image based on the clustering results of the overexposed points; determine the degree to which each pixel is affected by the lighting of each light source based on the distance between the pixel in the non-light source area in each frame of the image and the light source fitting circle; and determine the possibility that the pixel in the non-light source area is located in the glare area formed by the light source by combining the displacement size of the light source area between adjacent frames, the brightness value when the pixel is located in the glare area, and the change characteristics of the degree of lighting affected by each light source; The data enhancement module is used to extract the glare area in each frame of the image according to the possibility that the pixel point in the non-light source area is located in the glare area formed by the light source; obtain the degree of influence of the light source on the pixel point in the glare area according to the possibility that the pixel point in the glare area is located in the glare area formed by each light source and the degree of influence of the illumination of each light source; enhance the glare area in the image according to the influence degree to obtain enhanced video data; The obstacle perception module is used to detect obstacles on the road based on the enhanced video data of the road; The visualization module is used to visualize the obstacle detection results through the visualization port.

[0006] Preferably, the method of obtaining the light source area in each frame of the image based on the clustering result of the overexposure points includes: The pixel points whose brightness value in each frame of image is greater than the preset brightness threshold are regarded as over-exposed points; The overexposed points in each frame of the image are divided into several clusters using a clustering algorithm, and the connected domain formed by all the overexposed points in each cluster is regarded as a light source area.

[0007] Preferably, the determining the degree to which each pixel is affected by the illumination of each light source comprises the following specific methods: The light source fitting circle of each light source area is obtained by fitting, and the minimum Euclidean distance between the pixel points in the non-light source area of ​​each frame image and the edge points of each light source fitting circle is calculated; The ratio of the radius of each light source fitting circle to the minimum Euclidean distance is used as the degree of illumination influence of each light source on each pixel.

[0008] Preferably, the method of determining the possibility that a pixel point in the non-light source area is located in the glare area formed by the light source includes: In each frame of the image, a line segment is obtained by connecting each pixel point in the non-light source area with the center of each light source fitting circle, and the brightness values ​​of all the pixel points in the non-light source area on the line segment are arranged in position order to obtain a brightness value sequence of the line segment; Arrange the degree of illumination influence of the same light source on all pixels in the non-light source area on the connecting line segment in order of position to obtain a feature sequence of the connecting line segment; Determine the characteristic value of the glare area formed by each light source for the pixel point in the non-light source area based on the segmentation result of the brightness value sequence of the connecting line segments and the overall fitting deviation of the characteristic sequence; The pixel points in each frame image that have the same coordinates as the center of all light source fitting circles in the previous frame image are used as light source comparison points in each frame image; The normalized result of the ratio of the minimum Euclidean distance between the center of each light source fitting circle in each frame image and all light source comparison points in each frame image to the eigenvalue is used as the possibility that the pixel point in the non-light source area is located in the glare area formed by the light source.

[0009] Preferably, the specific method of determining the characteristic value of the glare area formed by each light source for the pixel point in the non-light source area includes: The mutation point in the brightness value sequence of the connecting line segment is obtained by using the mutation point detection method, and the mutation point is used as a segmentation point to complete the segmentation of the brightness value sequence, and the brightness values ​​between two adjacent mutation points in the brightness value sequence are combined into a brightness value array; Taking the characteristic sequence of the connecting line segment as the ordinate and the order value of the element value as the abscissa, a straight line fitting is performed on the elements in the characteristic sequence, the difference between the true value and the fitted value of each element in the characteristic sequence is calculated, and the accumulated result of the absolute values ​​of all the differences is taken as the overall fitting deviation; The sum of the coefficient of variation of the variance of all the brightness value arrays and the overall fitting deviation is used as the characteristic value of the glare area formed by each light source for the pixel point in the non-light source area.

[0010] Preferably, the method of extracting the glare area in each frame of the image according to the possibility that the pixel point in the non-light source area is located in the glare area formed by the light source includes the following specific methods: Compare the probability that each pixel point in the non-light source area in each frame of image is located in the glare area formed by each light source in each frame of image with a preset threshold; When the probability that a pixel point in a non-light source area is located in a glare area formed by any light source in each frame of the image is greater than a preset threshold, the pixel point is considered to be located in the glare area in the image; All pixel points located in the glare area of ​​the image are obtained and connected domains are extracted to obtain the glare area in each frame of the image.

[0011] Preferably, the obtaining of the degree to which the pixel points in the glare area are affected by the light source comprises the following specific methods: The product of the brightness value of each pixel in the glare area and the possibility that each pixel is located in the glare area formed by each light source is used as a numerator; The ratio of the molecule to the degree of illumination influence of the light source on each pixel in the glare area is used as a correction coefficient for the degree of illumination influence of the light source on the pixel in the glare area; The degree to which the pixel points in the glare area are affected by the light source is determined based on the correction coefficient.

[0012] Preferably, the determining the degree to which the pixel points in the glare area are affected by the light source based on the correction coefficient comprises the following specific methods: The sum of the correction coefficient of the degree of influence of the illumination of each light source on each pixel in the glare area and 1 is calculated, and the product of the sum and the degree of influence of the illumination of each light source on each pixel in the glare area is taken as the degree of influence of each light source on each pixel in the glare area.

[0013] Preferably, the step of enhancing the glare area in the image according to the influence degree to obtain enhanced video data includes the following specific methods: The ratio of the brightness value of each pixel in the glare area in each frame of the image to the maximum brightness value of the pixel in each frame of the image is used as a weight, and the weight is used to weight the cumulative sum of the degree of influence of all light sources on each pixel in the glare area; The product of the preset gamma value and the normalized result of the weighted result is used as the gamma value of each pixel in the glare area in each frame of the image; Performing gamma transformation on each frame of image based on the gamma values ​​of all pixels in the glare area of ​​each frame of image to obtain an enhancement result of each frame of image; The enhancement results of all frame images in the road video data are combined into enhanced video data.

[0014] Preferably, the method of detecting obstacles on the road according to the road-enhanced video data includes: The enhanced road video data is transmitted to the obstacle perception system, and the obstacle detection results in the video data are obtained using the detection model in the obstacle perception system.

[0015] The beneficial effects of this application are: The present application firstly screens overexposed points according to the brightness values ​​of pixels in each frame of the video data, and then obtains the degree of illumination influence of each light source on each pixel, and preliminarily quantifies the degree of influence of the light source on each pixel in each frame; secondly, considering that the light source in the actual scene usually scatters light in a circular shape to all directions, the light source fitting circle corresponding to each light source area is obtained, and then the analysis is performed based on the light source fitting circle to further improve the accuracy of the analysis result; then, the possibility of the pixel in the non-light source area being located in the glare area formed by the light source is determined by combining the displacement size of the light source area between adjacent frames and the brightness value when the pixel is located in the glare area and the change characteristics of the degree of illumination influence of each light source; through the possibility, it is further determined whether the pixel meets the image feature of being located in the glare area, and the glare area in the image is obtained based on this; then, the degree of influence of the pixel in the glare area on the light source is obtained by combining the possibility that the pixel in the glare area is in the glare area formed by each light source and the degree of illumination influence of each light source, and the quality of the enhanced image is improved by adaptively enhancing the different degrees of influence of the glare effect on the pixel in each glare area, so that the detection result of road obstacle detection in low illumination scenes is more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 This is a structural block diagram of a road obstacle visualization system under low illumination provided in this application. DETAILED DESCRIPTION

[0018] In order to further explain the technical means and effects adopted by the present application to achieve the predetermined invention purpose, the following is a detailed description of the specific implementation, structure, features and effects of a road obstacle visualization system under low illumination proposed by the present application in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0019] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

[0020] The following is a detailed description of a specific scheme of a road obstacle visualization system under low illumination provided by the present application in conjunction with the accompanying drawings.

[0021] See also Figure 1 , which shows a structural block diagram of a road obstacle visualization system under low illumination provided by an embodiment of the present application, the system includes the following modules: The data acquisition module 101 is used to acquire video data of the road.

[0022] The present application provides a road obstacle visualization system under low illumination, and its application scenario is to identify and visualize obstacles on the road in a timely manner when driving a vehicle on a road with low illumination. It is necessary to collect video data of the road through a data acquisition module; transmit the video data of the road to a video data analysis module to analyze the video data; transmit the analysis result of the video data to a data enhancement module for enhancement processing; finally, transmit the processing result of the video data to an obstacle perception module to visualize the obstacles on the road; therefore, it is necessary to collect video data of the road first.

[0023] Specifically, video data of the road is collected by a front camera of the car, and the video data includes a plurality of frame images.

[0024] At this point, the video data of the road is obtained.

[0025] The video data analysis module 102 is used to screen overexposed points according to the brightness values ​​of the pixels in each frame of the video data, and obtain the light source area in each frame of the image based on the clustering results of the overexposed points; determine the degree to which each pixel is affected by the lighting of each light source based on the distance between the pixel in the non-light source area in each frame of the image and the light source fitting circle; and determine the possibility that the pixel in the non-light source area is located in the glare area formed by the light source by combining the displacement size of the light source area between adjacent frames and the brightness value when the pixel is located in the glare area and the changing characteristics of the degree of lighting affected by each light source.

[0026] It should be noted that since the visibility of road obstacles is reduced when a vehicle is driving at night or in low light conditions, it is necessary to enhance each frame of the road video data separately to improve the accuracy of the obstacle visualization results in the video.

[0027] When each frame of the road video data is enhanced by gamma transformation, the on-board front camera is affected by multiple light sources such as street lights around the road and the lights of other cars on the road, which will cause glare areas such as halos and spots in each frame of the video data. At this time, when each frame of the road video data under low illumination is enhanced by gamma transformation, it is difficult to effectively suppress the glare area, which makes it difficult for the obstacle perception module to accurately detect obstacles on the road from the video data. Therefore, this application proposes a road obstacle visualization system under low illumination, which obtains the position of each light source in each frame of the video data, establishes a glare distribution model of each light source, and then effectively enhances each frame of the video data according to the glare distribution model, thereby improving the accuracy of road obstacle detection under low illumination.

[0028] Specifically, we first need to obtain the position of each light source in each frame of the video data. Each frame of the video data is recorded as an image; a brightness threshold is preset. , The specific value of can be set in different embodiments according to the actual situation, and this application does not impose any special restrictions. Preferably, in this embodiment, 3 The criterion is to obtain the value range of 3 times the standard deviation of the brightness value of all pixels in each frame of the image, and take the upper limit of the value range as the brightness threshold , the pixels in each frame whose brightness threshold is greater than the brightness threshold are regarded as overexposed points.

[0029] Furthermore, for any frame of image, take the i-th frame of image as an example. Clustering algorithm is used to cluster all overexposed points in the i-th frame of image to obtain several clusters, and the connected domain formed by all overexposed points in each cluster is regarded as a light source area. This is because in the video data collected under low illumination, the light source has the characteristic of relatively high brightness, so the connected domain where the overexposed points in each frame of image are gathered can be regarded as a light source area, and the degree of influence of the light source on the pixels in the subsequent non-light source area is determined based on the light source area in each frame of image.

[0030] It should be noted that the purpose of clustering overexposure points is that the light source corresponds to a connected domain in each frame of the image. Therefore, under the premise of being able to cluster the overexposure points, the clustering algorithms that can be used include but are not limited to the k-means clustering algorithm, the AP (Affinity Propagation clustering) clustering algorithm, and the DBSCAN (Density-Based Spatial Clustering of Applications with Noise). Preferably, this embodiment uses the AP clustering algorithm to divide all overexposure points in each frame of the image into several clusters. The AP clustering algorithm is a well-known technology in the field of image processing, and the specific process will not be repeated.

[0031] Furthermore, for all light source regions in the i-th frame image, considering that the light source in the actual scene usually scatters light in a circular shape, the least squares method is used to perform circular fitting on each light source region, and the light source fitting circle corresponding to each light source region is obtained respectively. Subsequently, analysis is performed based on the light source fitting circle to further improve the accuracy of the analysis result. Among them, the least squares method is a well-known technology in the field of image processing, and the specific process is not repeated here.

[0032] It should be understood that this embodiment only provides a circular fitting method, that is, using the least squares method to obtain the light source fitting circle of each light source area. Under the premise that circular fitting can be achieved, in other embodiments, the implementer may choose other fitting methods, and this application does not impose any special restrictions on this. Secondly, for any pixel point in a non-light source area, taking the kth pixel point as an example, the influence of the light source on the kth pixel point is evaluated based on the distance between the kth pixel point and the light source fitting circle and the influence range of the light source fitting circle, and the degree of influence of the kth pixel point on the illumination of each light source is determined. The calculation formula for the degree of influence of the illumination of the kth pixel point on the nth light source is as follows: In the formula, Indicates The pixel is affected by The lighting influence of each light source; Indicates The radius of the light source fitting circle; Indicates Pixels and The minimum Euclidean distance between the edge points of the light source fitting circle.

[0033] It should be noted that when The larger the radius of the light source fitting circle, the The higher the lighting effect of the first light source, the The stronger the influence of each light source on the surrounding pixels, It means the Pixels and The distance between the light sources is The closer the pixel is to the light source, then The light source is The more pixels there are, the stronger the impact.

[0034] It should be further explained that, for each pixel in the i-th frame image, the relative distribution positions between different light sources and the pixel are different, the size and strength of the light sources are different, and the influence of each light source on each pixel is not uniform, that is, the influence of each light source on each pixel cannot be accurately obtained based only on the distance between the light source and the pixel; in order to accurately obtain the influence of each light source on each pixel, it is also necessary to obtain the possibility that each pixel is in the glare area formed by each light source, and on this basis, correct the influence of each pixel on the illumination of each light source to obtain the actual influence of each pixel on each light source.

[0035] Specifically, the i-1th frame image is used as the comparison data of the i-th frame image. First, the centers of all light source fitting circles in the i-1th frame image are obtained respectively, where the coordinates of the ath center of the circle are marked as The pixel point in the i-th frame image with the same coordinates as the center of all light source fitting circles in the i-1-th frame image is taken as the light source comparison point, and the Euclidean distance between the center of each light source fitting circle in the i-th frame image and all light source comparison points is calculated respectively.

[0036] Secondly, for the pixel points in the i-th frame image that do not belong to the light source area, if the pixel points are located in the glare area such as light spot and halo caused by a certain light source, the pixel points between this pixel point and the center of the light source can be divided into two categories with obvious differences. One category is located in the glare area with basically equal brightness values, and the degree of influence of the illumination of the light source is basically similar; the other category is not located in the glare area, the brightness values ​​are randomly distributed, and the degree of influence of the illumination of the light source varies significantly with the distance.

[0037] Specifically, connect the kth pixel point in the i-th frame image with the center of the n-th light source fitting circle, and respectively The brightness values ​​of all pixels in the non-light source area and the degree of influence of the nth light source are arranged in order of position to obtain a line segment Brightness value sequence and feature sequence.

[0038] Secondly, the mutation point in the brightness value sequence is obtained by mutation point detection, and the mutation point is used as the segmentation point to complete the segmentation of the brightness value sequence, and the brightness values ​​between two adjacent mutation points in the brightness value sequence are combined into a brightness value array. Combined with the above analysis, if the kth pixel is located in the glare area formed by the nth light source, the variance of the partial brightness value array should be close to 0 after the mutation point segmentation, and the variance of the regional brightness value array should be randomly distributed.

[0039] It should be noted that the purpose of mutation point detection is to obtain mutation points in the brightness value sequence. The mutation point detection methods that can be used include but are not limited to Pettitt detection and Mann-Kendall detection. The present application does not impose any special restrictions on the specific method of mutation point detection. Preferably, in one embodiment of the present application, Pettitt detection is used to obtain the mutation points in the brightness value sequence. Mutation point detection is a well-known technology in the field of data analysis, and the specific process will not be repeated.

[0040] On the other hand, combined with the above analysis, if the kth pixel is located in the glare area formed by the nth light source, the line segment The characteristic sequence takes the element value as the ordinate and the order value of the element value as the abscissa. When a straight line is fitted to the elements in the characteristic sequence, a result with a small overall fitting deviation will be obtained. On the contrary, if the k-th pixel point is not located in the glare area formed by the n-th light source, there is no obvious correlation between the distribution and position of the elements in the characteristic sequence when performing straight line fitting, and the overall fitting deviation will be large. The overall fitting deviation is obtained by calculating the difference between the true value and the fitting value of each element in the characteristic sequence, and the cumulative result of the absolute values ​​of all the differences is used as the overall fitting deviation. Among them, straight line fitting is a well-known technology in the field of data analysis, and the specific process will not be repeated.

[0041] Here, the Euclidean distance and line segment between the center of the nth light source fitting circle in the i-th frame image and all light source comparison points are integrated. The segmentation result of the brightness value sequence and the fitting deviation of the feature sequence determine the possibility that the k-th pixel in the i-th frame image is located in the glare area formed by the n-th light source. The calculation formula is as follows: In the formula, Indicates that the kth pixel in the i-th frame image is in the The possibility of a light source forming a glare area, Is a line segment The coefficient of variation of the variance of all brightness value arrays obtained by segmenting the brightness value sequence, Is a line segment The overall fitting deviation when performing straight line fitting on the characteristic sequence of is the minimum value of the Euclidean distance between the center of the nth light source fitting circle and all light source comparison points in the i-th frame image. is a normalization function.

[0042] The light source area corresponding to the nth light source appears continuously in adjacent frames and the greater the relative position change, the more likely the nth light source is to belong to the glare area formed by the moving light source; and the eigenvalue Represents the line segment Whether the brightness value sequence and feature sequence of the k-th pixel in the non-light source area conform to the data characteristics when the k-th pixel point is in the glare area formed by the n-th light source.

[0043] In particular, for the first frame of the acquired video data, the probability that all pixels in the non-light source area in the first frame are in the glare area formed by the light source in the first frame is set to 0. Thus, the degree to which the pixels in each non-light source area in each frame are affected by the illumination of each light source, and the probability of being in the glare area formed by each light source are obtained.

[0044] The data enhancement module 103 is used to extract the glare area in each frame of the image based on the possibility that the pixel point in the non-light source area is located in the glare area formed by the light source; obtain the degree of influence of the light source on the pixel point in the glare area in combination with the possibility that the pixel point in the glare area is located in the glare area formed by each light source and the degree of influence of the lighting of each light source; enhance the glare area in the image according to the influence degree to obtain enhanced video data.

[0045] It should be noted that after obtaining the possibility that each pixel point in each non-light source area in each frame of image is in the glare area formed by each light source through the video data analysis module 102, the glare area range in each frame of image can be obtained according to the possibility that each pixel point in each frame of image is in the glare area formed by each light source.

[0046] Specifically, the probability that a pixel point in each non-light source area in the i-th frame image is located in the glare area formed by each light source in the i-th frame image is obtained respectively, and compared with a preset threshold value. In this embodiment, the preset threshold value is set to 0.7. When the probability that a pixel point in a non-light source area is located in the glare area formed by any light source in the i-th frame image is greater than the preset threshold value, it is considered that the pixel point is located in the glare area in the i-th frame image, and all pixel points located in the glare area in the i-th frame image are obtained and connected domain extraction is performed to obtain the glare area in the i-th frame image.

[0047] Furthermore, after obtaining the glare area in each frame of the image, although different light sources have different degrees of influence on the same pixel, each pixel in the glare area will be seriously affected by at least one light source. The higher the probability that a pixel in the glare area is in the glare area of ​​a certain light source, the greater the influence of the light source on the pixel should be. Combined with the degree of influence of the light source on the pixel, a correction coefficient of the degree of influence of the light source on the pixel in the glare area is obtained.

[0048] Specifically, for the pixel points in the glare area in the i-th frame image, taking the j-th pixel point as an example, the larger the brightness value of the j-th pixel point and the greater the possibility of being in the glare area of ​​the n-th light source, the greater the influence of the n-th light source on the j-th pixel point in the glare area. At this time, if the degree of influence of the n-th light source on the j-th pixel point in the glare area is smaller, it means that the degree of influence of the n-th light source on the j-th pixel point in the glare area is less accurate, that is, the degree of influence of the n-th light source on the j-th pixel point in the glare area needs to be corrected. The correction coefficient of the degree of influence of the n-th light source on the j-th pixel point is calculated as follows: In the formula, The correction coefficient representing the degree to which the j-th pixel in the glare area is affected by the illumination of the n-th light source; represents the brightness value of the jth pixel in the glare area; Indicates the possibility that the jth pixel in the glare area is in the glare area formed by the nth light source; Indicates the degree to which the j-th pixel in the glare area is affected by the illumination of the n-th light source.

[0049] It should be further explained that after obtaining the correction coefficient of the degree of influence of the illumination of each light source on each pixel in the glare area, the degree of influence of the illumination of each light source on each pixel in the glare area can be corrected to obtain the degree of influence of each light source on each pixel in the glare area.

[0050] Specifically, the degree of influence of the n-th light source on the j-th pixel in the glare area is obtained by multiplying the sum of the correction coefficient of the influence of the n-th light source on the j-th pixel in the glare area and 1 by the product of the influence of the n-th light source on the j-th pixel in the glare area.

[0051] Furthermore, after obtaining the degree to which each pixel in the glare area is affected by each light source, a gamma transform can be performed on each pixel in the glare area based on this. By giving a large gamma value to the pixel in the glare area that is greatly affected by each light source, and giving a small gamma value to the pixel in the glare area that is less affected by each light source, the glare area can be enhanced.

[0052] Specifically, an initial glare gamma value is preset , The specific value of can be set according to the actual situation, and this application does not impose any special restrictions on this. To describe, according to the degree to which each pixel in the glare area is affected by each light source, the gamma value of each pixel in the glare area is obtained. Among them, the calculation formula of the gamma value of the j-th pixel in the glare area in the i-th frame image is: In the formula, represents the gamma value of the jth pixel in the glare area of ​​the i-th frame image; Indicates the preset initial glare gamma value; represents the brightness value of the jth pixel in the glare area; represents the maximum brightness value in the glare area; N represents the number of light sources in the i-th frame image; Indicates the degree to which the j-th pixel in the glare area is affected by the n-th light source; Represents the maximum and minimum normalization function. The normalized object is all the pixels in the glare area in the i-th frame image. .

[0053] Secondly, according to the above process, the gamma value of the pixel point in the glare area of ​​each frame of the image is obtained respectively, and the gamma value of each pixel point in the glare area is used as the gamma value for gamma transformation of each pixel point in the glare area, and the brightness value of the pixel point in the glare area is gamma transformed to complete the enhancement of each frame of the image in the collected video data. Among them, gamma transformation is a well-known technology in the field of image enhancement, and the specific process will not be repeated.

[0054] At this point, the enhanced image is obtained.

[0055] The obstacle sensing module 104 is used to detect obstacles on the road according to the road enhanced video data.

[0056] Specifically, the enhancement results of all frame images in the video data of the road under low illumination are transmitted to the obstacle perception system built into the vehicle, and the obstacles in the video data are detected using the detection model in the obstacle perception system.

[0057] It should be noted that at present, the detection models in the obstacle perception system built into the vehicle are mainly based on deep learning detection models, including but not limited to convolutional neural network-based obstacle detection models, Transformer-based obstacle detection models, and YOLO target detection models. This application does not impose any special restrictions on the specific type of obstacle detection models. Preferably, in this embodiment, the YOLO target detection model is used to identify obstacles in the enhanced image. This is a well-known technology in the field of image processing, and the specific process will not be repeated.

[0058] It should be noted that after the enhanced image is obtained through the data enhancement module 103, the glare effect is weakened in the enhanced image, thereby improving the accuracy of identifying obstacles on the road.

[0059] The visualization module 105 is used to visualize the obstacle detection result through the visualization port.

[0060] Specifically, the obstacle sensing module 104 transmits the obstacle detection result obtained on the road under low illumination to the visualization module 105. After obtaining the obstacle detection result, the visualization module 105 visualizes the detection result on the display screen through the visualization port. The obstacle sensing module 104 can also transmit instructions to the early warning system in the vehicle to issue a safety prompt sound to avoid obstacles, thereby further effectively avoiding obstacles on the road under low illumination.

[0061] At this point, this embodiment is completed.

[0062] It should be noted that the above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

[0063] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

Claims

1. A road obstacle visualization system under low illumination, characterized in that: The system includes the following modules: A data acquisition module, used to collect video data of roads under low illumination; A video data analysis module is used to filter out overexposed points according to the brightness values ​​of pixels in each frame of the video data, and obtain the light source area in each frame of the video data based on the clustering results of the overexposed points; Determine the degree to which each pixel is affected by the illumination of each light source based on the distance between the pixel in the non-light source area in each frame of the image and the light source fitting circle; The possibility of a pixel point in a non-light source area being located in a glare area formed by a light source is determined by combining the displacement of the light source area between adjacent frames, the brightness value when the pixel point is located in the glare area, and the change characteristics of the degree of illumination influence by each light source; A data enhancement module is used to extract the glare area in each frame of the image based on the possibility that the pixel points in the non-light source area are located in the glare area formed by the light source; The degree of influence of the light source on the pixel in the glare area is obtained by combining the possibility that the pixel in the glare area is in the glare area formed by each light source and the degree of influence of the illumination by each light source; According to the degree of influence, the glare area in the image is enhanced to obtain enhanced video data; The obstacle perception module is used to detect obstacles on the road based on the enhanced video data of the road; The visualization module is used to visualize the obstacle detection results through the visualization port.

2. According to claim 1, a road obstacle visualization system under low illumination is characterized in that: The specific method of obtaining the light source area in each frame of the image based on the clustering result of the overexposure points includes: The pixel points whose brightness value in each frame of image is greater than the preset brightness threshold are regarded as over-exposed points; The overexposed points in each frame of the image are divided into several clusters using a clustering algorithm, and the connected domain formed by all the overexposed points in each cluster is regarded as a light source area.

3. The road obstacle visualization system under low illumination according to claim 1, characterized in that: The specific method of determining the degree to which each pixel is affected by the illumination of each light source is as follows: The light source fitting circle of each light source area is obtained by fitting, and the minimum Euclidean distance between the pixel points in the non-light source area of ​​each frame image and the edge points of each light source fitting circle is calculated; The ratio of the radius of each light source fitting circle to the minimum Euclidean distance is used as the degree of illumination influence of each light source on each pixel.

4. The road obstacle visualization system under low illumination according to claim 1, characterized in that: The specific method of determining the possibility that a pixel point in the non-light source area is located in the glare area formed by the light source includes: In each frame of the image, a line segment is obtained by connecting each pixel point in the non-light source area with the center of each light source fitting circle, and the brightness values ​​of all the pixel points in the non-light source area on the line segment are arranged in position order to obtain a brightness value sequence of the line segment; Arrange the degree of illumination influence of the same light source on all pixels in the non-light source area on the connecting line segment in order of position to obtain a feature sequence of the connecting line segment; Determine the characteristic value of the glare area formed by each light source for the pixel point in the non-light source area based on the segmentation result of the brightness value sequence of the connecting line segments and the overall fitting deviation of the characteristic sequence; The pixel points in each frame image that have the same coordinates as the center of all light source fitting circles in the previous frame image are used as light source comparison points in each frame image; The normalized result of the ratio of the minimum Euclidean distance between the center of each light source fitting circle in each frame image and all light source comparison points in each frame image to the eigenvalue is used as the possibility that the pixel point in the non-light source area is located in the glare area formed by the light source.

5. A road obstacle visualization system under low illumination according to claim 4, characterized in that: The specific method of determining the characteristic value of the glare area formed by each light source for the pixel point in the non-light source area includes: The mutation point in the brightness value sequence of the connecting line segment is obtained by using the mutation point detection method, and the mutation point is used as a segmentation point to complete the segmentation of the brightness value sequence, and the brightness values ​​between two adjacent mutation points in the brightness value sequence are combined into a brightness value array; Taking the characteristic sequence of the connecting line segment as the ordinate and the order value of the element value as the abscissa, a straight line fitting is performed on the elements in the characteristic sequence, the difference between the true value and the fitted value of each element in the characteristic sequence is calculated, and the accumulated result of the absolute values ​​of all the differences is taken as the overall fitting deviation; The sum of the coefficient of variation of the variance of all the brightness value arrays and the overall fitting deviation is used as the characteristic value of the glare area formed by each light source for the pixel point in the non-light source area.

6. The road obstacle visualization system under low illumination according to claim 1, characterized in that: The specific method of extracting the glare area in each frame of the image according to the possibility that the pixel point in the non-light source area is located in the glare area formed by the light source includes: Compare the probability that each pixel point in the non-light source area in each frame of image is located in the glare area formed by each light source in each frame of image with a preset threshold; When the probability that a pixel point in a non-light source area is located in a glare area formed by any light source in each frame of the image is greater than a preset threshold, the pixel point is considered to be located in the glare area in the image; All pixel points located in the glare area of ​​the image are obtained and connected domains are extracted to obtain the glare area in each frame of the image.

7. The road obstacle visualization system under low illumination according to claim 1, characterized in that: The specific method of obtaining the degree to which the pixel points in the glare area are affected by the light source includes: The product of the brightness value of each pixel in the glare area and the possibility that each pixel is located in the glare area formed by each light source is used as a numerator; The ratio of the molecule to the degree of illumination influence of the light source on each pixel in the glare area is used as a correction coefficient for the degree of illumination influence of the light source on the pixel in the glare area; The degree to which the pixel points in the glare area are affected by the light source is determined based on the correction coefficient.

8. A road obstacle visualization system under low illumination according to claim 7, characterized in that: The specific method of determining the degree to which the pixel points in the glare area are affected by the light source based on the correction coefficient is as follows: The sum of the correction coefficient of the degree of influence of the illumination of each light source on each pixel in the glare area and 1 is calculated, and the product of the sum and the degree of influence of the illumination of each light source on each pixel in the glare area is taken as the degree of influence of each light source on each pixel in the glare area.

9. The road obstacle visualization system under low illumination according to claim 1, characterized in that: The method of enhancing the glare area in the image according to the influence degree to obtain enhanced video data includes: The ratio of the brightness value of each pixel in the glare area in each frame of the image to the maximum brightness value of the pixel in each frame of the image is used as a weight, and the weight is used to weight the cumulative sum of the degree of influence of all light sources on each pixel in the glare area; The product of the preset gamma value and the normalized result of the weighted result is used as the gamma value of each pixel in the glare area in each frame of the image; Performing gamma transformation on each frame of image based on the gamma values ​​of all pixels in the glare area of ​​each frame of image to obtain an enhancement result of each frame of image; The enhancement results of all frame images in the road video data are combined into enhanced video data.

10. The road obstacle visualization system under low illumination according to claim 1, characterized in that: The method of detecting obstacles on the road according to the enhanced video data of the road includes: The enhanced road video data is transmitted to the obstacle perception system, and the obstacle detection results in the video data are obtained using the detection model in the obstacle perception system.

Images