A road obstacle visualization system under low illumination

Abstract

This application relates to the field of image processing technology, and particularly to a road obstacle visualization system under low illumination. The system includes: collecting video data of a road under low illumination; obtaining a light source area according to the clustering result of overexposed points; determining the degree of illumination influence of a pixel point by the light source based on the distance between the pixel point in the non-light source area and the fitting circle of the light source; combining the displacement size of the light source area between adjacent frames and the parameter change characteristics when the pixel point is in the glare area to determine the possibility that the pixel point is in the glare area formed by the light source, and combining the illumination influence degree to obtain the degree of influence of the pixel point by the light source; obtaining enhanced video data according to the degree of influence, detecting obstacles on the road, and visualizing them through a visualization port. This application adaptively enhances the pixel points in the glare area of each frame of image, improves the quality of the enhanced image, and thus improves the accuracy of obstacle detection on the road.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and particularly to a road obstacle visualization system under low illumination. Background Art

[0002] When driving on a road under low illumination, due to the significant change in light intensity, it is difficult to accurately detect obstacles on the road and visualize them. When driving a vehicle, the detection of obstacles within a certain range is carried out through the driver's observation and the perception of the in-vehicle camera, and the detected obstacles on the road are visualized on the display end to assist the vehicle in avoiding obstacles in advance. Therefore, when driving a vehicle on a road under low illumination, how to quickly and accurately visualize the obstacles on the road is a key link to ensure safety.

[0003] Since the light source under low illumination will produce a glare effect, it is difficult to identify obstacles on the road only by the human eye, which further leads to difficulties in visualizing obstacles on the road. At this time, it is necessary to use the in-vehicle camera to collect road video data within the sensing range. However, when driving under low illumination, the visibility of road obstacles decreases, and it is necessary to enhance the collected video data to improve the quality of the video data, thereby improving the visualization effect of obstacles. The traditional enhancement method is to enhance each frame of image in the video data through gamma transformation, but this method is difficult to appropriately suppress the glare area, affecting the accuracy of the subsequent obstacle visualization results. Summary of the Invention

[0004] To solve the problem that the traditional gamma transformation enhancement method is difficult to appropriately suppress the glare area in road images under low illumination, affecting the obstacle detection results, this application provides a road obstacle visualization system under low illumination.

[0005] This application provides a road obstacle visualization system under low illumination, and the system includes the following modules:

[0006] A data acquisition module, used to acquire video data of the road under low illumination;

[0007] A video data analysis module, used to screen overexposed points according to the brightness values of pixel points in each frame of the video data, and obtain the light source area in each frame of the image based on the clustering result of the overexposed points; determine the illumination influence degree of each pixel point by each light source based on the distance between the pixel points in the non-light source area of each frame of the image and the fitting circle of the light source; combine the displacement size of the light source area between adjacent frames, as well as the change characteristics of the brightness value and the illumination influence degree of each pixel point by each light source when the pixel point is located in the glare area, to determine the possibility that the pixel points in the non-light source area are located in the glare area formed by the light source;

[0008] A data enhancement module, which is used to extract the glare area in each frame of image according to the possibility that the pixel points in the non-light source area are located in the glare area formed by the light source; combine the possibility that the pixel points in the glare area are in the glare area formed by each light source and the degree of illumination influence of each light source to obtain the degree of influence of the pixel points in the glare area by the light source; enhance the glare area in the image according to the degree of influence to obtain the enhanced video data;

[0009] An obstacle perception module, which is used to detect obstacles on the road according to the enhanced video data of the road;

[0010] A visualization module, which is used to visualize the obstacle detection result through the visualization port.

[0011] Preferably, obtaining the light source area in each frame of image based on the clustering result of overexposed points includes the following specific method:

[0012] Taking the pixel points in each frame of image with brightness values greater than the preset brightness threshold as overexposed points;

[0013] Using a clustering algorithm to divide the overexposed points in each frame of image into several clustering clusters, and taking the connected domain formed by all overexposed points in each clustering cluster as a light source area.

[0014] Preferably, determining the degree of illumination influence of each pixel point by each light source includes the following specific method:

[0015] Using a fitting method to obtain the light source fitting circle of each light source area, and calculating the minimum Euclidean distance between the pixel points in the non-light source area of each frame of image and the edge points of each light source fitting circle;

[0016] Taking the ratio of the radius of each light source fitting circle to the minimum Euclidean distance as the degree of illumination influence of each pixel point by each light source.

[0017] Preferably, determining the possibility that the pixel points in the non-light source area are located in the glare area formed by the light source includes the following specific method:

[0018] In each frame of image, connecting the pixel points in each non-light source area with the center of each light source fitting circle to obtain a connection line segment, and arranging the brightness values of all pixel points in the non-light source area on the connection line segment in order of position to obtain the brightness value sequence of the connection line segment;

[0019] Arranging the degree of illumination influence of all pixel points in the non-light source area on the connection line segment by the same light source in order of position to obtain the feature sequence of the connection line segment;

[0020] Determine the eigenvalue of the pixel points in the non-light source area affected by the glare area formed by each light source based on the segmentation result of the brightness value sequence of the connection line segment and the overall fitting deviation of the feature sequence;

[0021] Take the pixel points in each frame of image with the same center coordinates of the fitting circles of all light sources in the adjacent previous frame of image as the light source comparison points in each frame of image;

[0022] Take the normalized result of the ratio of the minimum value of the Euclidean distance between the center of each light source fitting circle in each frame of image and all light source comparison points in each frame of image to the eigenvalue as the possibility that the pixel points in the non-light source area are located in the glare area formed by the light source.

[0023] Preferably, the method for determining the eigenvalue of the pixel points in the non-light source area affected by the glare area formed by each light source specifically includes:

[0024] Use the method of mutation point detection to obtain the mutation points in the brightness value sequence of the connection line segment, and take the mutation points as the segmentation points to complete the segmentation of the brightness value sequence. The brightness values between two adjacent mutation points in the brightness value sequence form a brightness value array;

[0025] Take the feature sequence of the connection line segment, use the element value as the ordinate and the order value of the element as the abscissa to perform linear fitting on the elements in the feature sequence, calculate the difference between the true value and the fitted value of each element in the feature sequence, and take the cumulative result of the absolute values of all the differences as the overall fitting deviation;

[0026] Take the sum of the coefficient of variation of the variances of all the brightness value arrays and the overall fitting deviation as the eigenvalue of the pixel points in the non-light source area affected by the glare area formed by each light source.

[0027] Preferably, the method for extracting the glare area in each frame of image according to the possibility that the pixel points in the non-light source area are located in the glare area formed by the light source specifically includes:

[0028] Compare the possibility that each pixel point in the non-light source area in each frame of image is located in the glare area formed by each light source in each frame of image with a preset threshold;

[0029] When the possibility that a pixel point in a non-light source area is located in the glare area formed by any light source in each frame of image is greater than the preset threshold, it is considered that the pixel point is located in the glare area in the image;

[0030] Obtain all the pixel points located in the glare area in the image and perform connected component extraction to obtain the glare area in each frame of image.

[0031] Preferably, the method for obtaining the degree of influence of the pixel points in the glare area by the light source specifically includes:

[0032] Use the product of the luminance value of each pixel in the glare area and the possibility that each pixel is within the glare area formed by each light source as the numerator;

[0033] Use the ratio of the numerator to the degree of illumination influence of each pixel in the glare area by the light source as the correction coefficient of the degree of illumination influence of the pixel in the glare area by the light source;

[0034] Determine the degree of influence of the pixels in the glare area by the light source based on the correction coefficient.

[0035] Preferably, the method of determining the degree of influence of the pixels in the glare area by the light source based on the correction coefficient includes the following specific method:

[0036] Calculate the sum of the correction coefficient of the degree of illumination influence of each pixel in the glare area by each light source and 1, and use the product of the sum value and the degree of illumination influence of each pixel in the glare area by each light source as the degree of influence of each pixel in the glare area by each light source.

[0037] Preferably, the method of enhancing the glare area in the image according to the degree of influence to obtain the enhanced video data includes the following specific method:

[0038] Use the ratio of the luminance value of each pixel in the glare area of each frame of image to the maximum luminance value of the pixels in each frame of image as the weight, and use the weight to weight the sum of the degrees of influence of each pixel in the glare area by all light sources;

[0039] Use the product of the preset gamma value and the normalized result of the weighted result as the gamma value of each pixel in the glare area of each frame of image;

[0040] Perform gamma transformation on each frame of image based on the gamma values of all pixels in the glare area of each frame of image to obtain the enhanced result of each frame of image;

[0041] Combine the enhanced results of all frames of images in the video data of the road to form the enhanced video data.

[0042] Preferably, the method of detecting obstacles on the road according to the enhanced video data of the road includes the following specific method:

[0043] Transmit the enhanced video data of the road to the obstacle perception system, and use the detection model in the obstacle perception system to obtain the obstacle detection result in the video data.

[0044] The beneficial effects of this application are:

[0045] This application first screens for overexposed points based on the brightness values of pixel points in each frame of video data, and then obtains the degree of illumination influence of each pixel point by each light source, preliminarily quantifying the degree of influence of each pixel point in each frame of image by the light source. Secondly, considering that light sources in the actual scene usually scatter light in a circular shape around, the light source fitting circle corresponding to each light source area is obtained, and subsequent analysis is carried out based on the light source fitting circle to further improve the accuracy of the analysis results. After that, combining the displacement size of the light source area between adjacent frames, as well as the change characteristics of the brightness value and the degree of illumination influence of each light source when the pixel point is in the glare area, the possibility that the pixel point in the non-light source area is in the glare area formed by the light source is determined. Based on this possibility, it is further determined whether the pixel point conforms to the image characteristics of being in the glare area, and the glare area in the image is obtained accordingly. Subsequently, combining the possibility that the pixel points in the glare area are in the glare area formed by each light source and the degree of illumination influence of each light source, the degree of influence of the pixel points in the glare area by the light source is obtained. By adaptively enhancing the different degrees of influence of the pixel points in each glare area by the glare effect, the quality of the enhanced image is improved, making the detection result more accurate when detecting road obstacles in a low-illumination scene. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0047] Figure 1 It is a structural block diagram of a road obstacle visualization system under low illumination provided by the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0048] In order to further elaborate on the technical means and effects adopted by the present application to achieve the intended invention purpose, the following, in combination with the drawings and preferred embodiments, details the specific implementation manners, structures, features and their effects of a road obstacle visualization system under low illumination proposed according to the present application. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures or characteristics in one or more embodiments can be combined in any suitable form.

[0049] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which this application belongs.

[0050] The following specifically describes the specific solution of a road obstacle visualization system provided by this application in conjunction with the accompanying drawings.

[0051] Please refer to Figure 1 , which shows a structural block diagram of a road obstacle visualization system provided by an embodiment of this application. The system includes the following modules:

[0052] The data acquisition module 101 is used to acquire video data of the road.

[0053] This application provides a road obstacle visualization system, and its application scenario is when driving on a road with low illumination, it is necessary to timely identify and visualize the obstacles on the road when driving a vehicle. It is necessary to acquire the video data of the road through the data acquisition module; transmit the video data of the road to the video data analysis module for analysis; transmit the analysis result of the video data to the data enhancement module for enhancement processing; finally, transmit the processing result of the video data to the obstacle perception module to visualize the obstacles on the road; therefore, it is first necessary to acquire the video data of the road.

[0054] Specifically, the video data of the road is acquired through the front camera of the vehicle, and the video data includes several frame images.

[0055] Thus, the video data of the road is obtained.

[0056] The video data analysis module 102 is used to screen out overexposed points according to the brightness values of pixel points in each frame image of the video data, obtain the light source area in each frame image based on the clustering result of the overexposed points; determine the illumination influence degree of each pixel point by each light source based on the distance between the pixel points in the non-light source area of each frame image and the fitting circle of the light source; combine the displacement size of the light source area between adjacent frames and the change characteristics of the brightness value and the illumination influence degree of each light source when the pixel point is in the glare area to determine the possibility that the pixel point in the non-light source area is in the glare area formed by the light source.

[0057] It should be noted that since the visibility of road obstacles decreases when driving a vehicle at night or under low light conditions, it is necessary to enhance each frame image in the video data of the road respectively to improve the accuracy of the visualization result of the obstacles in the video.

[0058] When traditional gamma transformation is used to enhance each frame of image in the video data of the road, due to light sources such as street lights around the road and car headlights on the road, the on-vehicle front camera is affected by multiple light sources, which will cause glare areas such as halos and light spots in each frame of image in the video data. At this time, when using gamma transformation to enhance each frame of image in the video data of the road under low illuminance, it is difficult to effectively suppress the glare area, resulting in the obstacle perception module being difficult to accurately detect obstacles on the road from the video data. Therefore, the present application proposes a road obstacle visualization system under low illuminance. By obtaining the position of each light source in each frame of image in the video data, establishing a glare distribution model for each light source, and then effectively enhancing each frame of image in the video data according to the glare distribution model, the accuracy of detecting road obstacles under low illuminance is improved.

[0059] Specifically, first, it is necessary to obtain the position of each light source in each frame of image in the video data. Each frame of image in the video data is respectively denoted as an image; a brightness threshold is preset , The specific value of can be set by itself in different embodiments in combination with the actual situation, and the present application does not make special restrictions. Preferably, in this embodiment, the 3 criterion is used to obtain the value range of three times the standard deviation of the brightness values of all pixel points in each frame of image, and the upper limit of the value range is used as the brightness threshold , and the pixel points in each frame of image with a brightness greater than the brightness threshold are used as overexposed points.

[0060] Furthermore, for any frame of image, taking the i-th frame of image as an example. The clustering algorithm is used to cluster and divide all overexposed points in the i-th frame of image to obtain several clustering clusters, and the connected domain formed by all overexposed points in each clustering cluster is used as a light source area. This is because in the video data collected under low illuminance, the light source has the characteristic of relatively high brightness. Therefore, the connected domain where the overexposed points are aggregated in each frame of image can be regarded as a light source area, and the influence degree of the light source on the pixel points in the subsequent non-light source area is based on the light source area in each frame of image.

[0061] It should be noted that the purpose of clustering overexposed points is that the light source corresponds to a connected region in each frame of the image. Therefore, on the premise that the overexposed points can be clustered and divided, the clustering algorithms that can be used include but are not limited to the k-means clustering algorithm, the AP (Affinity Propagation clustering) algorithm, and the DBSCAN (Density-Based Spatial Clustering of Applications with Noise). Preferably, in this embodiment, the AP clustering algorithm is used to divide all overexposed points in each frame of the image into several clustering clusters. The AP clustering algorithm is a well-known technology in the field of image processing, and the specific process will not be elaborated here.

[0062] Furthermore, for all light source regions in the i-th frame of the image, considering that in the actual scene, the light source usually scatters light in a circular shape around, the least squares method is used to perform circular fitting on each light source region, and the light source fitting circles corresponding to each light source region are obtained respectively. Subsequently, analysis is performed based on the light source fitting circles to further improve the accuracy of the analysis results. Among them, the least squares method is a well-known technology in the field of image processing, and the specific process will not be elaborated here.

[0063] It should be understood that this embodiment only provides a circular fitting method, that is, the least squares method is used to obtain the light source fitting circles of each light source region. On the premise that circular fitting can be achieved, in other embodiments, the implementer can choose other fitting methods, and this application does not make special restrictions on this. Secondly, for the pixel points in any non-light source region, taking the k-th pixel point as an example, the influence degree of the light source on the k-th pixel point is evaluated according to the distance between the k-th pixel point and the light source fitting circle and the influence range of the light source fitting circle, and the illumination influence degree of each light source on the k-th pixel point is determined. The calculation formula for the illumination influence degree of the k-th pixel point by the n-th light source is as follows:

[0064]

[0065] In the formula, represents the illumination influence degree of the -th pixel point by the -th light source; represents the radius of the -th light source fitting circle; represents the minimum Euclidean distance between the -th pixel point and the edge point of the -th light source fitting circle.

[0066] It should be noted that when the radius of the -th light source fitting circle is larger, it means that the The higher the illumination impact of a light source, that is, the stronger the impact of the th light source on the surrounding pixel points. represents the distance between the th pixel point and the th light source. When the th pixel point is closer to the th light source, the th light source has a stronger impact on the

[0067] It should be further noted that for each pixel point in the i-th frame of the image, due to the different relative distribution positions between different light sources and pixel points, the sizes and intensities of the light sources are different, and the influence degree of each light source on each pixel point is not uniform. That is, only based on the distance between the light source and the pixel point, the influence degree of each light source on each pixel point cannot be accurately obtained. In order to accurately obtain the influence degree of each light source on each pixel point, it is also necessary to obtain the possibility that each pixel point is in the glare area formed by each light source, and based on this, correct the illumination influence degree of each pixel point by each light source to obtain the actual influence degree of each pixel point by each light source.

[0068] Specifically, the (i - 1)-th frame of the image is used as the comparison data for the i-th frame of the image. First, the centers of all the light source fitting circles in the (i - 1)-th frame of the image are obtained respectively, and the coordinates of the a-th center are denoted as . The pixel points in the i-th frame of the image with the same center coordinates as all the light source fitting circles in the (i - 1)-th frame of the image are used as light source comparison points, and the Euclidean distances between the center of each light source fitting circle in the i-th frame of the image and all the light source comparison points are calculated respectively.

[0069] Secondly, for the pixel points in the i-th frame of the image that are not in the light source area, if the pixel point is in the glare area such as the light spot and halo caused by a certain light source, then the pixel points between this pixel point and the light source center can be divided into 2 categories with obvious differences. One category is the pixel points with basically equal brightness values in the glare area and basically similar illumination influence degrees by the light source; the other category is the pixel points not in the glare area, with randomly distributed brightness values and obvious differences in illumination influence degrees by the light source depending on the distance.

[0070] Specifically, connect the k-th pixel point in the i-th frame of the image with the center of the n-th light source fitting circle, and respectively arrange the brightness values and the illumination influence degrees by the n-th light source of all the pixel points in the non-light source area on the line segment in order of position to obtain the brightness value sequence and the feature sequence of the line segment .

[0071] Secondly, mutation points in the luminance value sequence are obtained by means of mutation point detection, and the mutation points are used as segmentation points to complete the segmentation of the luminance value sequence. The luminance values between two adjacent mutation points in the luminance value sequence form a luminance value array. Combining the above analysis, if the k-th pixel is located in the glare area formed by the n-th light source, after the mutation point segmentation, the variance of some of the luminance value arrays should be close to 0, and the variance of the regional luminance value arrays is randomly distributed.

[0072] It should be noted that the purpose of mutation point detection is to obtain mutation points in the luminance value sequence. The mutation point detection methods that can be used include but are not limited to Pettitt detection and Mann-Kendall detection. This application does not make special restrictions on the specific methods of mutation point detection. Preferably, in an embodiment of this application, Pettitt detection is used to obtain the mutation points in the luminance value sequence. Mutation point detection is a well-known technology in the field of data analysis, and the specific process will not be elaborated.

[0073] On the other hand, combining the above analysis, if the k-th pixel is located in the glare area formed by the n-th light source, then for the line segment when the characteristic sequence has the element value as the ordinate and the order value of the element value as the abscissa, a smaller overall fitting deviation will be obtained when performing linear fitting on the elements in the characteristic sequence. On the contrary, if the k-th pixel is not located in the glare area formed by the n-th light source, there is no obvious correlation between the element distribution and position in the characteristic sequence during linear fitting, and the overall fitting deviation will be larger. The way to obtain the overall fitting deviation is to calculate the difference between the true value and the fitting value of each element in the characteristic sequence, and take the cumulative result of the absolute values of all the differences as the overall fitting deviation. Among them, linear fitting is a well-known technology in the field of data analysis, and the specific process will not be elaborated.

[0074] Here, the Euclidean distance between the center of the fitting circle of the n-th light source in the i-th frame of image and all light source comparison points, the segmentation result of the luminance value sequence of the line segment and the fitting deviation of the characteristic sequence are combined to determine the possibility that the k-th pixel in the i-th frame of image is located in the glare area formed by the n-th light source. The calculation formula is as follows:

[0075]

[0076] In the formula, represents the possibility that the k-th pixel in the i-th frame of image is in the glare area formed by the -th light source, is the coefficient of variation of the variances of all the luminance value arrays obtained by segmenting the luminance value sequence of the line segment , is the line segment The overall fitting deviation when performing linear fitting on the feature sequence, is the minimum Euclidean distance between the center of the nth light source fitting circle in the ith frame image and all light source comparison points, is the normalization function.

[0077] Among them, the light source area corresponding to the nth light source appears continuously in adjacent frames and the greater the relative position change, it indicates that the nth light source is more likely to belong to the glare area formed by the moving light source; and the eigenvalue characterizes whether the brightness value sequence and feature sequence of the line segment conform to the data characteristics when the kth pixel point in the non-light source area is in the glare area formed by the nth light source.

[0078] Specifically, for the first frame image in the collected video data, the possibility that all pixel points in the non-light source area of the first frame image are in the glare area formed by the light source in the first frame image is set to 0. Thus, the degree of illumination influence of each pixel point in each non-light source area of each frame image by each light source, and the possibility of being in the glare area formed by each light source are obtained.

[0079] The data enhancement module 103, the data enhancement module, is used to extract the glare area in each frame image according to the possibility that the pixel points in the non-light source area are in the glare area formed by the light source; combine the possibility that the pixel points in the glare area are in the glare area formed by each light source and the degree of illumination influence by each light source to obtain the degree of influence of the pixel points in the glare area by the light source; enhance the glare area in the image according to the degree of influence to obtain the enhanced video data.

[0080] It should be noted that after obtaining the possibility that each pixel point in each non-light source area of each frame image is in the glare area formed by each light source through the video data analysis module 102, the glare area range in each frame image can be obtained according to the possibility that each pixel point in each frame image is in the glare area formed by each light source.

[0081] Specifically, the possibility that each pixel point in each non-light source area of the ith frame image is in the glare area formed by each light source in the ith frame image is obtained respectively and compared with a preset threshold. In this embodiment, the size of the preset threshold is set to 0.7. When the possibility that a pixel point in a non-light source area is in the glare area formed by any light source in the ith frame image is greater than the preset threshold, it is considered that this pixel point is in the glare area in the ith frame image. All pixel points located in the glare area in the ith frame image are obtained and connected component extraction is performed to obtain the glare area in the ith frame image.

[0082] Further, after obtaining the glare area in each frame of image, although different light sources have different degrees of influence on the same pixel point, each pixel point located in the glare area is at least severely affected by one light source. The higher the possibility that a pixel point in the glare area is in the glare area of a certain light source, the greater the influence of that light source on the pixel point. Combining the degree of illumination influence of the pixel point by that light source, a correction coefficient for the degree of illumination influence of the pixel point in the glare area by the light source is obtained.

[0083] Specifically, for the pixel point in the glare area of the i-th frame of image, taking the j-th pixel point as an example, the greater the brightness value of the j-th pixel point and the greater the possibility that it is in the glare area of the n-th light source, the greater the influence of the n-th light source on the j-th pixel point in the glare area. At this time, if the degree of illumination influence of the j-th pixel point in the glare area by the n-th light source is smaller, it means that the degree of illumination influence of the j-th pixel point in the glare area by the n-th light source is less accurate, that is, the degree of illumination influence of the j-th pixel point in the glare area by the n-th light source needs to be corrected more. The calculation method of the correction coefficient of the degree of illumination influence of the j-th pixel point by the n-th light source is as follows:

[0084]

[0085] In the formula, represents the correction coefficient of the degree of illumination influence of the j-th pixel point in the glare area by the n-th light source; represents the brightness value of the j-th pixel point in the glare area; represents the possibility that the j-th pixel point in the glare area is in the glare area formed by the n-th light source; represents the degree of illumination influence of the j-th pixel point in the glare area by the n-th light source.

[0086] It should be further noted that after obtaining the correction coefficient of the degree of illumination influence of each pixel point in the glare area by each light source, the degree of illumination influence of each pixel point in the glare area by each light source can be corrected accordingly to obtain the degree of influence of each pixel point in the glare area by each light source.

[0087] Specifically, the product of the sum of the correction coefficient of the degree of illumination influence of the j-th pixel point in the glare area by the n-th light source and 1, multiplied by the degree of illumination influence of the j-th pixel point in the glare area by the n-th light source, is used as the degree of influence of the j-th pixel point in the glare area by the n-th light source.

[0088] Further, after obtaining the influence degree of each pixel point in the glare area by each light source, each pixel point in the glare area can be gamma-transformed based on this. By giving a large gamma value to the pixel points in the glare area that are greatly affected by each light source and a small gamma value to the pixel points in the glare area that are slightly affected by each light source, the enhancement of the glare area is achieved.

[0089] Specifically, a preset initial glare gamma value , The specific value of which can be set according to the actual situation, and this application does not make special restrictions on this. Preferably, in this embodiment, is described. According to the influence degree of each pixel point in the glare area by each light source, the gamma value of each pixel point in the glare area is obtained. Among them, the calculation formula for the gamma value of the j-th pixel point in the glare area in the i-th frame image is:

[0090]

[0091] In the formula, represents the gamma value of the j-th pixel point in the glare area in the i-th frame image; represents the preset initial glare gamma value; represents the brightness value of the j-th pixel point in the glare area; represents the maximum brightness value in the glare area; N represents the number of light sources in the i-th frame image; represents the influence degree of the j-th pixel point in the glare area by the n-th light source; represents the maximum-minimum normalization function, and the object of normalization is of all pixel points in the glare area in the i-th frame image.

[0092] Secondly, according to the above process, the gamma values of the pixel points in the glare area of each frame of image are obtained respectively, and the gamma value of each pixel point in the glare area is used as the gamma value for gamma-transforming each pixel point in the glare area, and the brightness value of the pixel points in the glare area is gamma-transformed to complete the enhancement of each frame of image in the collected video data. Among them, gamma transformation is a well-known technology in the field of image enhancement, and the specific process will not be elaborated here.

[0093] So far, the enhanced image is obtained.

[0094] The obstacle perception module 104, the obstacle perception module, is used to detect obstacles on the road according to the enhanced video data of the road.

[0095] Specifically, the enhancement results of all frame images in the video data of the road under low illumination are transmitted to the obstacle perception system built in the vehicle, and the obstacles in the video data are detected by using the detection model in the obstacle perception system.

[0096] It should be noted that at the present stage, the detection models in the obstacle perception systems built in vehicles are mainly deep learning-based detection models, including but not limited to obstacle detection models based on convolutional neural networks, obstacle detection models based on Transformers, and YOLO object detection models. This application does not impose special restrictions on the specific types of obstacle detection models. Preferably, in this embodiment, the YOLO object detection model is used to identify obstacles in the enhanced image. This is a well-known technology in the field of image processing, and the specific process will not be elaborated here.

[0097] It should be noted that after obtaining the enhanced image through the data enhancement module 103, the glare effect is weakened in the enhanced image, thereby improving the accuracy of identifying obstacles on the road.

[0098] The visualization module 105 is used to visualize the obstacle detection results through the visualization port.

[0099] Specifically, the obstacle perception module 104 transmits the obtained obstacle detection results on the road under low illumination to the visualization module 105. After the visualization module 105 obtains the obstacle detection results, it visualizes the detection results on the display screen through the visualization port. The obstacle perception module 104 can also transmit instructions to the warning system in the vehicle to emit a safety prompt sound for avoiding obstacles, further effectively avoiding obstacles on the road under low illumination.

[0100] So far, this embodiment is completed.

[0101] It should be noted that: the above embodiments are only used to illustrate the technical solutions of this application, rather than to limit them; although this application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of this application, and should all be included in the protection scope of this application.

[0102] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between each embodiment can be referred to each other. The key point of each embodiment is to illustrate the differences from other embodiments.

Claims

1. A road obstacle visualization system under low illumination, characterized in that: The system includes the following modules: A data acquisition module, used to collect video data of roads under low illumination; A video data analysis module is used to filter out overexposed points according to the brightness values ​​of pixels in each frame of the video data, and obtain the light source area in each frame of the video data based on the clustering results of the overexposed points; Determine the degree to which each pixel is affected by the illumination of each light source based on the distance between the pixel in the non-light source area in each frame of the image and the light source fitting circle; The possibility of a pixel point in a non-light source area being located in a glare area formed by a light source is determined by combining the displacement of the light source area between adjacent frames, the brightness value when the pixel point is located in the glare area, and the change characteristics of the degree of illumination influence by each light source; A data enhancement module is used to extract the glare area in each frame of the image based on the possibility that the pixel points in the non-light source area are located in the glare area formed by the light source; The degree of influence of the light source on the pixel in the glare area is obtained by combining the possibility that the pixel in the glare area is in the glare area formed by each light source and the degree of influence of the illumination by each light source; According to the degree of influence, the glare area in the image is enhanced to obtain enhanced video data; The obstacle perception module is used to detect obstacles on the road based on the enhanced video data of the road; The visualization module is used to visualize the obstacle detection results through the visualization port.

2. According to claim 1, a road obstacle visualization system under low illumination is characterized in that: The specific method of obtaining the light source area in each frame of the image based on the clustering result of the overexposure points includes: The pixel points whose brightness value in each frame of image is greater than the preset brightness threshold are regarded as over-exposed points; The overexposed points in each frame of the image are divided into several clusters using a clustering algorithm, and the connected domain formed by all the overexposed points in each cluster is regarded as a light source area.

3. The road obstacle visualization system under low illumination according to claim 1, characterized in that: The specific method of determining the degree to which each pixel is affected by the illumination of each light source is as follows: The light source fitting circle of each light source area is obtained by fitting, and the minimum Euclidean distance between the pixel points in the non-light source area of ​​each frame image and the edge points of each light source fitting circle is calculated; The ratio of the radius of each light source fitting circle to the minimum Euclidean distance is used as the degree of illumination influence of each light source on each pixel.

4. The road obstacle visualization system under low illumination according to claim 1, characterized in that: The specific method of determining the possibility that a pixel point in the non-light source area is located in the glare area formed by the light source includes: In each frame of the image, a line segment is obtained by connecting each pixel point in the non-light source area with the center of each light source fitting circle, and the brightness values ​​of all the pixel points in the non-light source area on the line segment are arranged in position order to obtain a brightness value sequence of the line segment; Arrange the degree of illumination influence of the same light source on all pixels in the non-light source area on the connecting line segment in order of position to obtain a feature sequence of the connecting line segment; Determine the characteristic value of the glare area formed by each light source for the pixel point in the non-light source area based on the segmentation result of the brightness value sequence of the connecting line segments and the overall fitting deviation of the characteristic sequence; The pixel points in each frame image that have the same coordinates as the center of all light source fitting circles in the previous frame image are used as light source comparison points in each frame image; The normalized result of the ratio of the minimum Euclidean distance between the center of each light source fitting circle in each frame image and all light source comparison points in each frame image to the eigenvalue is used as the possibility that the pixel point in the non-light source area is located in the glare area formed by the light source.

5. A road obstacle visualization system under low illumination according to claim 4, characterized in that: The specific method of determining the characteristic value of the glare area formed by each light source for the pixel point in the non-light source area includes: The mutation point in the brightness value sequence of the connecting line segment is obtained by using the mutation point detection method, and the mutation point is used as a segmentation point to complete the segmentation of the brightness value sequence, and the brightness values ​​between two adjacent mutation points in the brightness value sequence are combined into a brightness value array; Taking the characteristic sequence of the connecting line segment as the ordinate and the order value of the element value as the abscissa, a straight line fitting is performed on the elements in the characteristic sequence, the difference between the true value and the fitted value of each element in the characteristic sequence is calculated, and the accumulated result of the absolute values ​​of all the differences is taken as the overall fitting deviation; The sum of the coefficient of variation of the variance of all the brightness value arrays and the overall fitting deviation is used as the characteristic value of the glare area formed by each light source for the pixel point in the non-light source area.

6. The road obstacle visualization system under low illumination according to claim 1, characterized in that: The specific method of extracting the glare area in each frame of the image according to the possibility that the pixel point in the non-light source area is located in the glare area formed by the light source includes: Compare the probability that each pixel point in the non-light source area in each frame of image is located in the glare area formed by each light source in each frame of image with a preset threshold; When the probability that a pixel point in a non-light source area is located in a glare area formed by any light source in each frame of the image is greater than a preset threshold, the pixel point is considered to be located in the glare area in the image; All pixel points located in the glare area of ​​the image are obtained and connected domains are extracted to obtain the glare area in each frame of the image.

7. The road obstacle visualization system under low illumination according to claim 1, characterized in that: The specific method of obtaining the degree to which the pixel points in the glare area are affected by the light source includes: The product of the brightness value of each pixel in the glare area and the possibility that each pixel is located in the glare area formed by each light source is used as a numerator; The ratio of the molecule to the degree of illumination influence of the light source on each pixel in the glare area is used as a correction coefficient for the degree of illumination influence of the light source on the pixel in the glare area; The degree to which the pixel points in the glare area are affected by the light source is determined based on the correction coefficient.

8. A road obstacle visualization system under low illumination according to claim 7, characterized in that: The specific method of determining the degree to which the pixel points in the glare area are affected by the light source based on the correction coefficient is as follows: The sum of the correction coefficient of the degree of influence of the illumination of each light source on each pixel in the glare area and 1 is calculated, and the product of the sum and the degree of influence of the illumination of each light source on each pixel in the glare area is taken as the degree of influence of each light source on each pixel in the glare area.

9. The road obstacle visualization system under low illumination according to claim 1, characterized in that: The method of enhancing the glare area in the image according to the influence degree to obtain enhanced video data includes: The ratio of the brightness value of each pixel in the glare area in each frame of the image to the maximum brightness value of the pixel in each frame of the image is used as a weight, and the weight is used to weight the cumulative sum of the degree of influence of all light sources on each pixel in the glare area; The product of the preset gamma value and the normalized result of the weighted result is used as the gamma value of each pixel in the glare area in each frame of the image; Performing gamma transformation on each frame of image based on the gamma values ​​of all pixels in the glare area of ​​each frame of image to obtain an enhancement result of each frame of image; The enhancement results of all frame images in the road video data are combined into enhanced video data.

10. The road obstacle visualization system under low illumination according to claim 1, characterized in that: The method of detecting obstacles on the road according to the enhanced video data of the road includes: The enhanced road video data is transmitted to the obstacle perception system, and the obstacle detection results in the video data are obtained using the detection model in the obstacle perception system.

Images