A Road Vehicle Detection Method in a Harsh Visual Environment

Abstract

The present invention discloses a road vehicle detection method under harsh visual environments. First, road video images are collected, and then a digital image enhancement algorithm based on dual-channel image information is constructed. An initial mapping is established through the bright-channel information of the image; then, the established mapping is corrected through the dark-channel information of the image to reduce information distortion caused by transmission during the image enhancement process; the parameter global brightness is obtained from the bright-channel result of the image; the global brightness and the corrected mapping relationship are substituted into the digital image empirical formula to obtain an enhanced image with dark removal enhancement for harsh visual environments; a deep learning vehicle detection model based on manual annotation and convolutional neural network is constructed, and the model is trained using the annotated data set to obtain a vehicle detection model, realizing vehicle detection under harsh visual environments. The present invention effectively improves the accuracy and precision of vehicle detection under harsh visual environments.

Description

Technical Field

[0001] The present invention belongs to the field of deep technology, and particularly relates to a method for detecting road vehicles. Background Art

[0002] With the rapid economic development, the number of urban motor vehicles has increased rapidly, traffic congestion has become increasingly serious, and traffic accidents have occurred more and more frequently. Urban traffic problems directly restrict the construction of cities and the development of the economy, and are closely related to people's daily lives. According to existing data, the probability of traffic accidents occurring in harsh visual environments is much higher than that under normal visual conditions. Therefore, realizing road vehicle detection in harsh visual environments and thus achieving early warnings for road targets such as motor vehicles and pedestrians is of great significance for protecting people's lives and health.

[0003] Generally speaking, harsh visual environments include low-light scenes caused by harsh weather such as heavy rain, fog, typhoon, sand and dust, or specific time periods such as night. For traffic flow monitoring, low-light environments are high-incidence periods of accidents. Therefore, realizing traffic flow monitoring in these harsh visual environments is actually an unavoidable problem for realizing intelligent transportation.

[0004] Since the 20th century, the progress of computer vision and network technology and the continuous improvement of related theories have provided more and more possibilities for the realization of intelligent transportation systems. However, most current object detection algorithms only focus on vehicle detection in normal visual environments. Compared with daytime, samples in night environments generally have problems such as blurred contours, uneven illumination, and difficult feature extraction. For traditional object detection algorithms such as frame difference methods, due to the too small pixel value difference between foreground targets and the background in night samples, the object detection effect is not good; for object detection algorithms based on machine learning or deep learning, there are also problems such as difficult feature extraction and poor learning effect. Summary of the Invention

[0005] In order to overcome the deficiencies of the prior art, the present invention provides a method for detecting road vehicles in harsh visual environments. First, road video images are collected, and then a digital image enhancement algorithm based on image dual-channel information is constructed. An initial mapping is established through the bright-channel information of the image; then the established mapping is corrected through the dark-channel information of the image to reduce information distortion caused by transmission during the image enhancement process; the parameter global brightness is obtained from the bright-channel result of the image; the global brightness and the corrected mapping relationship are substituted into the digital image empirical formula to obtain an enhanced image after removing darkness and enhancing for the harsh visual environment; a deep learning vehicle detection model based on manual annotation and convolutional neural network is constructed, and the model is trained using the annotated data set to obtain a vehicle detection model, realizing vehicle detection in harsh visual environments. The present invention effectively improves the accuracy and precision of vehicle detection in harsh visual environments.

[0006] The technical solution adopted by the present invention to solve its technical problems includes the following steps:

[0007] Step 1: Use a front-end information acquisition device to collect road videos in real time, and read road surveillance video images frame by frame at the back end;

[0008] Step 2: Construct a digital image enhancement algorithm based on image dual-channel information;

[0009] First, establish an initial mapping through the bright channel information of the image; correct the established mapping through the dark channel information of the image to reduce information distortion caused by transmission during the image enhancement process; obtain the global brightness parameter from the bright channel result of the image; substitute the global brightness and the corrected mapping relationship into the digital image empirical formula to obtain an enhanced image with dark removal enhancement for poor visual environments;

[0010] The digital image enhancement algorithm is based on the image exposure model:

[0011] A(x) = B(x)h(x) + C(1 - t(x))

[0012] In the formula, A(x) represents the input image, C is the environmental brightness, h(x) is the transmission parameter, and B(x) is the optimized image, that is, the image that has undergone light enhancement;

[0013] The formula is transformed to obtain the transformed image exposure model:

[0014]

[0015] Step 2-1: Establish an initial mapping through bright channel information: Assume that the transmission parameter in a local area of the image is a constant and perform a maximum filtering operation; combine with the prior theory of bright and dark channels to establish an initial mapping and calculate the transmission parameter initially obtained based on the bright channel;

[0016] Step 2-2: Perform correction processing through dark channel information: Correct the potential incorrect mapping obtained from the bright channel prior by calculating the difference between the bright and dark channel priors; derive the transmission parameter obtained based on the dark channel;

[0017] Calculate the difference between the transmission parameters obtained from the pixel points of the bright and dark channels, and compare the difference with a preset threshold. If the difference is less than the threshold, it is considered that the pixel point is located in a dark area and the mapping established through the bright channel information is unreliable. Increase the proportion of information obtained from the dark channel of the image for correction;

[0018] Step 2-3: Perform smoothing using a guided filter: After correcting the mapping through the dark channel information, the obtained mapping is smoothed by applying a guided filter to optimize the final transmission image; after the filtering process, the final transmission parameter h(x) is obtained;

[0019] Step 2-4: Calculate the environmental brightness based on the bright channel: First step, select the brightest 10% of the pixels in the bright channel; Second step, calculate the average pixel value of the brightest 10% of the pixels in the input image as the estimated value of the environmental brightness;

[0020] Step 2-5: Obtain the enhanced image for darkening enhancement in a harsh visual environment by combining the parameters obtained in Steps 2-1 to 2-4 with the digital image enhancement empirical formula;

[0021] Step 3: Construct a deep learning vehicle detection model based on manual annotation and convolutional neural network, and use the labeled dataset to train the model to obtain the vehicle detection model;

[0022] Input the enhanced image obtained in Step 2-5 into the model frame by frame to achieve vehicle detection in a harsh visual environment.

[0023] Preferably, the specific content of Step 2-1 is as follows:

[0024] Assume that the transmission parameter h(x) of the local area Ω(x) in the image is a constant, denoted as Perform a maximum filtering operation through the following formula:

[0025]

[0026] where A i , B i , C i respectively represent the input image, the image after exposure optimization, and the global environmental brightness, y represents the pixel point, and the superscript i in the formula takes one of the r, g, b channels;

[0027] Take the maximum value on both sides of formula (1) to obtain the formula:

[0028]

[0029] The expression of the bright channel of the image is:

[0030]

[0031] where A bright represents the bright channel information of the input image, and B bright represents the bright channel information of the image after exposure optimization;

[0032] Combining the image bright channel expression with the prior conclusion, the formula is obtained:

[0033]

[0034] Combining the deformed image exposure model with the above formula, we get:

[0035]

[0036] Get the formula:

[0037]

[0038] At this point, the initial mapping was established through the prior theory of bright channel and dark channel, and the transmission parameters obtained based on the bright channel were calculated.

[0039] Preferably, the step 2-2 is specifically: combining the image bright channel expression with the prior conclusion to derive the formula:

[0040] According to formula (4) and the prior conclusion of dark channel, we can get:

[0041]

[0042] Derivation of transmission parameters based on dark channel acquisition

[0043] The difference between the light and dark channels is calculated by:

[0044]

[0045] Among them, A dark (x) represents the dark channel of the image;

[0046] The difference is compared with the preset threshold α. If the difference is less than the threshold α, it is considered that the pixel is located in the dark area. The mapping established by the bright channel information is unreliable. The following correction formula is needed to increase the information obtained from the dark channel of the image:

[0047]

[0048] Where h(x) dark represents the medium transmission parameter obtained based on the dark channel, Represents the corrected medium transmission parameter.

[0049] Preferably, the threshold α is 0.4.

[0050] Preferably, in the deformed image exposure model, when the transmission parameter h(x) approaches zero, the first term on the right side of the formula will tend to infinity, making the restored image B become noise. To avoid this problem, a lower limit is imposed on the transmission parameter h(x), and a parameter h0 is introduced. The deformed image exposure model is written as the following formula:

[0051]

[0052] Preferably, h0 = 0.1.

[0053] The beneficial effects of the present invention are as follows:

[0054] Aiming at the high incidence of road traffic accidents in low-light and harsh visual environments such as at night and the poor effectiveness of existing detection means, the present invention proposes a method for detecting road vehicles in a harsh visual environment. By integrating digital image processing technology and deep learning models, automatic enhancement processing and target detection of input images are completed, effectively improving the accuracy and accuracy of vehicle detection in a harsh visual environment. Description of the Drawings

[0055] Figure 1 It is a schematic diagram of the system operation;

[0056] Figure 2 It is a flowchart of an image enhancement algorithm for a harsh visual environment based on dual-channel information;

[0057] Figure 3 It is a flowchart of a traditional image enhancement algorithm;

[0058] Figure 4 It is a flowchart of the system algorithm. Detailed Embodiments

[0059] The present invention will be further described below with reference to the drawings and embodiments.

[0060] Aiming at the high incidence of road traffic accidents in low-light and harsh visual environments such as at night and the poor effectiveness of existing detection means, a method for detecting road vehicles in a harsh visual environment is proposed. The system integrates digital image processing technology and deep learning models, aiming to improve the detection efficiency of road vehicles in a harsh visual environment, achieve early warning of road targets such as motor vehicles and pedestrians, and reduce the incidence of road traffic accidents.

[0061] Such as Figure 1 and Figure 4 , a method for detecting road vehicles in a harsh visual environment includes the following steps:

[0062] Step 1: Obtain the optical video information collected in real time by the front-end information collection devices deployed on the road, such as cameras, and read the road monitoring video frame by frame at the back end.

[0063] Step 2: Construct a digital image enhancement algorithm based on dual-channel image information: First, establish an initial mapping through the bright-channel information of the image; correct the established mapping through the dark-channel information of the image to reduce information distortion caused by transmission during the image enhancement process; obtain the global brightness of the parameters from the bright-channel result of the image; substitute the global brightness, the corrected mapping relationship, and other parameters into the digital image empirical formula to obtain the enhanced image after darkening enhancement for the harsh visual environment.

[0064] Step 3: Establish an initial mapping through the bright-channel information: Assume that the transmission parameter of a local area in the image is a constant and perform a maximum filtering operation. Combine with the prior theory of the bright / dark channels to establish an initial mapping and calculate the transmission parameter initially obtained based on the bright channel.

[0065] Step 4: Perform correction processing through the dark-channel information: By calculating the difference between the bright / dark channel priors, the potential incorrect mapping obtained from the bright-channel prior can be corrected. Similar to Step 3, the transmission parameter obtained based on the dark channel can be deduced. Calculate the difference between the transmission parameters obtained from the bright / dark channels, and compare the difference with a preset threshold. If the difference is less than the threshold, it is considered that the pixel point is in the dark area and the mapping established through the bright-channel information is unreliable, and the proportion of information obtained from the dark channel in the image is increased for correction.

[0066] Step 5: Perform smoothing processing using a guided filter: After correcting the mapping through the dark-channel information, since the previous formula derivation is based on some regional assumptions, the obtained mapping can be smoothed by applying a guided filter to optimize the final transmission image. After the filtering process, the final transmission parameter is obtained.

[0067] Step 6: Calculate the environmental brightness based on the bright channel: The calculation of the environmental brightness is mainly carried out in two steps: First, select the brightest 10% of the pixels in the bright channel; Second, calculate the average pixel value of these pixels in the input image as the estimated value of the environmental brightness.

[0068] Step 7: Obtain all the parameters required for image enhancement in the harsh visual environment based on Step 3, Step 4, Step 5, and Step 6, and combine with the digital image enhancement empirical formula to obtain the enhanced image after darkening enhancement for the harsh visual environment.

[0069] Step 8: A deep learning vehicle detection model based on manual annotation and convolutional neural network is constructed, and it is pre-trained using the annotated dataset to obtain a vehicle detection model under normal visual environment. The enhanced images based on Steps 3 to 7 are input into the model frame by frame to realize the integration of image enhancement and vehicle detection under harsh visual environment, and improve the road vehicle detection efficiency under harsh visual environment. Specific embodiment:

[0071] The digital image enhancement algorithm is based on the following common exposure model formula:

[0072] A(x) = B(x)h(x) + C(1 - t(x))

[0073] In the formula, A(x) represents the input image, C is the environmental brightness, h(x) is the light transmittance, and B(x) is the optimized image, that is, the image after light enhancement. The purpose of data enhancement is to perform light enhancement on the input image, that is, to obtain the optimized image B(x). To make the operation clearer, the formula is transformed to get:

[0074]

[0075] The core idea of the algorithm is to obtain the darkening-removed image by calculating several core parameters required in the above formula and combining the preset image enhancement model.

[0076] The flow chart of the harsh visual environment image enhancement algorithm based on dual-channel information is as Figure 2 shown; the flow chart of the traditional image darkening-removed enhancement algorithm is as Figure 3 shown.

[0077] From Figure 3 , the traditional image darkening-removed algorithm usually directly obtains the medium transmittance parameter h(x) through the dark channel information. By comparing Figure 2 with Figure 3 , it can be seen that compared with the traditional algorithm, the harsh visual environment image enhancement algorithm based on dual-channel information mainly establishes an initial mapping through the bright channel information, and then corrects it through the dark channel information, and obtains the transmittance parameter h(x) after guided filtering smoothing processing. This improvement enables the night vision image enhancement algorithm based on dual-channel information to achieve better darkening-removed effect.

[0078] The parts involved in the implementation process of the algorithm include Steps 3 to 7, and the specific implementation methods are as follows:

[0079] Step 3: Establish an initial mapping through the bright channel information

[0080] Since an image is usually composed of regions formed by different objects, it is first assumed that the transmittance parameter h(x) of a local region Ω(x) in the image is a constant, denoted as The maximum filtering operation is performed using the following formula:

[0081]

[0082] The value of the superscript i in the formula is one of the three channels r, g, and b. The formula for taking the maximum value on both sides of the formula is:

[0083]

[0084] The expression of the bright channel of the image is:

[0085]

[0086] According to the prior theory of bright / dark channels, the A value of the bright channel in the well-lit area should tend to 255, while the corresponding dark channel should tend to 0.

[0087] Combining the image bright channel expression with the prior conclusion, we can derive the formula:

[0088]

[0089] Combining the deformed image exposure model with the above formula, we get:

[0090]

[0091] The deformation formula is:

[0092]

[0093] At this point, through the prior theory of bright channel and bright channel, the initial mapping is established, and the transmission parameters obtained based on the bright channel are calculated.

[0094] Step 4: Correction processing using dark channel information

[0095] By calculating the difference between the bright / dark channel priors, the potential erroneous mapping obtained from the bright channel prior can be corrected. The prior conclusion of the dark channel has been given in step 3, and a similar formula can be given for the bright channel prior:

[0096]

[0097] Similarly, we can deduce the same as in step three Transmission parameters based on dark channel acquisition First, the difference between the light / dark channels is calculated using the following formula:

[0098] A difference (x) = A bright (x)-Adark (x)

[0099] After that, the difference is compared with a preset threshold α (the threshold α is obtained from pre-experiment tests, and in the case described in the present invention, α is taken as 0.4). If the difference is less than the threshold α, it is considered that the pixel point is located in the dark area, and the mapping established through the bright channel information is unreliable. It is necessary to increase the information obtained by the image from the dark channel through the following correction formula:

[0100]

[0101] Step Five: Use a guided filter for smoothing and obtain the transmission parameter

[0102] After correcting the mapping through the dark channel information, since the previous formula derivation is based on some regional assumptions, the obtained mapping can be smoothed by applying a guided filter to optimize the final transmitted image. After the filtering process, the final transmission parameter h(x) is obtained.

[0103] Step Six: Calculate the ambient brightness based on the bright channel

[0104] The calculation of the ambient brightness C is mainly carried out in two steps: First, select the brightest 10% of the pixels in the bright channel; Second, calculate the average pixel value of these pixels in the input image as the estimated value of the ambient brightness C.

[0105] The principle of calculating the ambient brightness C by this method is that the bright channel can be well approximated as the dark density of the global brightness in the input image. Therefore, the ambient brightness can be estimated through the bright channel. And the second step of taking the average value is to prevent the calculated ambient brightness from being too high.

[0106] Step Seven: Substitute into the algorithm model to generate a de-darkened image

[0107] Through the above Steps Three to Six, two parameters, namely the ambient brightness C and the transmission parameter h(x), can be calculated. Combining with the deformed image exposure model, the enhanced image after de-darkening can be calculated.

[0108] However, it should be noted that in the deformed image exposure model, when the transmission parameter h(x) approaches zero, the first term in the formula will tend to infinity, making the restored image B become noise. To avoid this problem, a lower limit should be imposed on the transmission parameter h(x). Therefore, a parameter h0 is introduced, and the deformed image exposure model is written as the following formula:

[0109]

[0110] In the case described in the present invention, h0 = 0.1 is selected.

[0111] Through the above steps, the system can automatically enhance the real-time video information collected by the front-end information acquisition device through digital image processing technology, and enhance the quality of the optical video collected in a harsh visual environment on the premise of not affecting the image effect in the normal visual environment.

[0112] After that, the present invention constructs a deep learning vehicle detection model based on manual annotation and convolutional neural network, and trains it in advance with the annotated data set to obtain a vehicle detection model in a normal visual environment. Then, the enhanced images based on steps three to seven are input into the model frame by frame to realize an integrated system for image enhancement and vehicle detection in a harsh visual environment, and improve the road vehicle detection efficiency in a harsh visual environment.

Claims

1. A road vehicle detection method under a harsh visual environment, characterized in that, It includes the following steps: Step 1: Use a front-end information collection device to collect road videos in real time, and read the road surveillance video images frame by frame at the back end; Step 2: Construct a digital image enhancement algorithm based on image dual-channel information; First, establish an initial mapping through the bright channel information of the image; Correct the established mapping through the dark channel information of the image to reduce information distortion caused by transmission during the image enhancement process; obtain the global brightness of the parameters from the bright channel result of the image; Substitute the global brightness and the corrected mapping relationship into the digital image empirical formula to obtain an enhanced image with dark removal enhancement for a harsh visual environment; The digital image enhancement algorithm is based on an image exposure model: A(x) = B(x)h(x) + C(1 - t(x)) In the formula, A(x) represents the input image, C is the environmental brightness, h(x) is the transmission parameter, and B(x) is the optimized image, that is, the image that has undergone light enhancement; Transform the formula to obtain the transformed image exposure model: Step 2-1: Establish an initial mapping through bright channel information: Assume that the transmission parameter in a local area of the image is a constant and perform a maximum filtering operation; Combine with the prior theory of bright and dark channels to establish an initial mapping and calculate the transmission parameter initially obtained based on the bright channel; Step 2-2: Perform correction processing through dark channel information: Correct the potential incorrect mapping obtained from the bright channel prior by calculating the difference between the bright and dark channel priors; Derive the transmission parameter obtained based on the dark channel; Calculate the difference between the transmission parameters obtained from the pixel points of the bright and dark channels, and compare the difference with a preset threshold. If the difference is less than the threshold, it is considered that the pixel point is in the dark area and the mapping established through the bright channel information is unreliable. Increase the proportion of information obtained from the dark channel of the image for correction; Step 2-3: Perform smoothing processing using a guided filter: After correcting the mapping through the dark channel information, perform smoothing processing on the obtained mapping by applying a guided filter to optimize the final transmission image; After the filtering process, obtain the final transmission parameter h(x); Step 2-4: Calculate the environmental brightness based on the bright channel: First step, select the brightest 10% of the pixels in the bright channel; Second step, calculate the average pixel value of the brightest 10% of the pixels in the input image as an estimated value of the environmental brightness; Step 2-5: According to the parameters obtained in Step 2-1 to Step 2-4, combine with the digital image enhancement empirical formula to obtain an enhanced image with dark removal enhancement for a harsh visual environment; Step 3: Construct a deep learning vehicle detection model based on manual annotation and convolutional neural network, and use the labeled dataset to train the model to obtain a vehicle detection model; Input the enhanced image obtained in Step 2-5 frame by frame into the model to achieve vehicle detection in a harsh visual environment.

2. The road vehicle detection method in a harsh visual environment according to claim 1, wherein, The specific content of Step 2-1 is as follows: Assume that the transmission parameter h(x) of the local region Ω(x) in the image is a constant, denoted as Perform a maximum filtering operation through the following formula: Among them, A i , B i , C i respectively represent the input image, the image after exposure optimization, and the global ambient brightness. y represents a pixel point. In the formula, the superscript i takes one of the three channels r, g, and b; Take the maximum value of both sides of formula (1) to obtain the formula: The expression of the bright channel of the image is: Among them, A bright represents the bright channel information of the input image, and B bright represents the bright channel information of the image after exposure optimization; Combine the expression of the bright channel of the image with the prior conclusion to obtain the formula: B bright B(x) = max c (max y∈Ω(x) (B i (y))) = 255 (4) Simultaneously transform the transformed image exposure model and the above formula to obtain: Obtain the formula: So far, through the prior theory of the bright channel and the dark channel, an initial mapping has been established, and the transmission parameters initially obtained based on the bright channel have been calculated.

3. The method for detecting road vehicles in a harsh visual environment according to claim 2, characterized in that, The specific content of step 2-2 is as follows: Combining the bright channel expression of the image with the prior conclusion to obtain the formula: Imitating formula (4) and the prior conclusion of the dark channel to obtain: B dark (x) = min c (min y∈Ω(x) B i (y)) = 0 (7) Derive the transmission parameters obtained based on the dark channel Calculating the difference between the bright and dark channels through the following formula: A difference (x) = A bright (x) - A dark (x) (8) Among them, A dark (x) represents the dark channel of the image; Comparing the difference with the preset threshold α. If the difference is less than the threshold α, it is considered that the pixel point is located in the dark area, and the mapping established through the bright channel information is unreliable. It is necessary to increase the information obtained by the image from the dark channel through the following correction formula: Among them, h(x) dark represents the medium transmission parameter obtained based on the dark channel, represents the corrected medium transmission parameter.

4. The method for detecting road vehicles in a harsh visual environment according to claim 3, wherein, The threshold α is taken as 0.

4.

5. A road vehicle detection method in a harsh visual environment according to claim 1, characterized in that, In the deformed image exposure model, when the transmission parameter h(x) approaches zero, the first term on the right side of the formula will tend to infinity, making the restored image B become noise. To avoid this problem, a lower limit is imposed on the transmission parameter h(x), and a parameter h0 is introduced. The deformed image exposure model is written as the following formula:

6. The method for detecting road vehicles in a harsh visual environment according to claim 5, characterized in that, The h0 = 0.1.

Images