Simultaneous estimation and internal calibration method of traffic road visibility based on video images

Abstract

The present invention discloses a method for simultaneously estimating and internally calibrating traffic road visibility based on video images. The method comprises the following steps: generating a permanent scatterer region image from a traffic video surveillance image; separating fog and background images in the permanent scatterer region image based on a matrix decomposition method; estimating traffic road visibility for the separated background image according to an atmospheric attenuation model; calculating the information entropy of the eigenvalue, and then internally calibrating traffic road visibility, and finally outputting a high-precision visibility value. The method of the invention can not only effectively suppress fog components in video images and enhance image contrast, but also accurately estimate traffic visibility and simultaneously realize internal calibration of visibility. The method has the advantages of low computational complexity, high precision, low cost, strong compatibility and scalability, and can be used in business applications such as traffic visibility monitoring and forecasting in traffic roads with video surveillance networks.

Description

Technical Field

[0001] The invention relates to a traffic road visibility simultaneous estimation and internal calibration method based on video images, belonging to the field of new generation information technology. Background Art

[0002] The presence of bad weather is a major source of traffic accidents. Atmospheric particles in these weather processes, such as rain, fog, snow, mist, haze and smoke, will greatly reduce the visual distance, reduce the visual contrast between the scene and the driver, and the clarity of the video image, posing challenges to camera-based advanced driver assistance systems, traffic flow monitoring, and intelligent highway management systems. Therefore, monitoring traffic road visibility is of great significance to improving traffic safety and traffic efficiency. At present, there are a small number of visibility sensors, but due to the high cost of equipment and station construction, they cannot cover the traffic road network. Compared with the sparsely distributed traffic visibility stations, the large number of video surveillance equipment installed on existing highways has formed a high-density monitoring network. If the visibility of the road can be estimated from the video, it will greatly improve the visibility monitoring and forecasting level of traffic roads. However, a large number of existing methods often focus on image defogging research. Most of them pursue visual effects and suppress fog as interference information. The fog information in the image is not utilized, and the visibility of the traffic road cannot be inverted. Summary of the invention

[0003] In view of the problems existing in the above-mentioned prior art, the present invention provides a method for simultaneously estimating and internally calibrating traffic road visibility based on video images, so that visibility can be estimated from video images and internal calibration can be achieved, thereby realizing high-precision road visibility monitoring.

[0004] In order to achieve the above object, the technical solution adopted by the present invention is: a method for simultaneously estimating and internally calibrating traffic road visibility based on video images, comprising the following steps:

[0005] Step S1: generating a permanent scatterer area image from a traffic video surveillance image;

[0006] Step S2: Separating the fog and background images in the permanent scatterer region image based on a matrix decomposition method;

[0007] Step S3: Then, for the separated background image, the visibility of the traffic road is estimated according to the atmospheric attenuation model;

[0008] Step S4: recalculate the information entropy of the eigenvalue, and then perform internal calibration on the traffic road visibility, and finally output a high-precision visibility value.

[0009] Furthermore, in step S1, two regions of the permanent scatterer region image within the video image monitoring range are selected and marked as PS1 and PS2 respectively;

[0010] The specific area selection requirements for attributes, color and distance are: the attributes are fixed and permanent on the ground, with the same material properties; the object color is white, and the backscattering produces white light to cover all wavelengths of visible light; it is distributed in the same plane and has a sufficient distance from the video surveillance equipment so that the incident angles of the video surveillance equipment to the two permanent scatterer areas are close.

[0011] Furthermore, the method for separating the fog and background image in the permanent scatterer area image based on the matrix decomposition method in step S2 is specifically as follows: assuming that the video image of each frame is F, whose size is m×n, the fog component image is A, and the background image is B, then F=A+B, and performing matrix decomposition on the image F, it can be obtained:

[0012] F = U∑V (1) where the matrices U and V are m×m and n×n unitary matrices respectively, and ∑ is an m×n diagonal matrix. Its specific form is

[0013]

[0014] The eigenvalues ​​satisfy λ 1 >λ 2 >…>λ m ;

[0015] Physically, these eigenvalues ​​correspond to the energy of different components of the image. The first and second eigenvalues ​​are the energy of the fog in the sky and non-sky areas. Therefore, these two eigenvalues ​​can be set to zero to obtain the defogging background image:

[0016]

[0017] Furthermore, in step S3, the method for estimating the visibility of the traffic road based on the atmospheric attenuation model for the separated background image is as follows: assuming that the background images without attenuation at the starting distances of the two permanent scatterer regions are and Where x and y represent the two-dimensional position of the image pixel;

[0018] According to the physical model of atmospheric attenuation, the backscattered light of the background objects in the permanent scatterer area is attenuated by atmospheric fog particles and the images are respectively and where r 1 and r 2 are the distances between the two scatterer areas and the video surveillance equipment, and β is the atmospheric extinction attenuation coefficient;

[0019] To improve reliability, the image intensities of the two scatterer regions in the background image B are averaged, and we can get

[0020]

[0021] and

[0022]

[0023] where |·| represents the total number of pixels in the image region, and are the average intensity values without attenuation at the starting distance of the background image;

[0024] Assume that the actual distance between the two scatterer regions on the same horizontal plane is the radial distance from the scatterer to the video device Generally, they are different; according to the elaborate design of the attributes, colors, and distances of the two scatterer regions in step S1, it can be made that and

[0025] Therefore, divide equation (4) by equation (5), and then according to the visibility v r calculation formula Derivation and simplification yield:

[0026]

[0027] Furthermore, the specific method for calculating the information entropy of the eigenvalue in step S4, and then performing internal calibration on the traffic road visibility and finally outputting a high-precision visibility value is as follows:

[0028] Define the probability P using the eigenvalue in equation (2) i , i = 1,..., m, specifically:

[0029]

[0030] Further calculate the information entropy H of the eigenvalue as:

[0031]

[0032] At the time without fog, the eigenvalue distribution of the image is relatively uniform; when fog appears, the first and second eigenvalues of the image are the main energy directions, and the eigenvalue distribution is uneven; according to the maximum entropy theory, the information entropy of the foggy image is small, while the information entropy value of the non-foggy image is large; analyzing the time-series image according to equation (8), it is easy to determine the critical entropy value H T when fog occurs; for the two special cases of very low visibility and very good visibility, two reliable visibility values are artificially determined according to the distances of the scatterer distributions in the image, and then the visibility value is calculated by equation (6), and the calibration factor f of high visibility is obtained by comparing these two artificially determined values with the formula calculated value hand the calibration factor f for low visibility l , and then the internal calibration operation for all visibility is formed as:

[0033]

[0034] In the formula is the final output visibility value.

[0035] The beneficial effects of the present invention are as follows: the method for simultaneous estimation and internal calibration of traffic road visibility based on video images of the present invention can effectively suppress the fog component in the video image and enhance the contrast of the image, and can also accurately estimate the traffic visibility and simultaneously realize the internal calibration of the visibility. It has the advantages of low computational complexity, high precision, low cost, strong compatibility and scalability, and can be used in business applications such as traffic visibility monitoring and forecasting in traffic roads with video surveillance networks. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 The figure is a schematic diagram of the workflow of a method for simultaneous estimation and internal calibration of traffic road visibility based on video images in an embodiment of the present invention.

[0037] Figure 2 The permanent scatterer region design result in the embodiment of the present invention.

[0038] Figure 3 This is the image defogging result in an embodiment of the present invention.

[0039] Figure 4 This is the estimation result under the condition of dense fog and low visibility in the embodiment of the present invention. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solution and advantages of the present invention more clear, the present invention is further described in detail below through the accompanying drawings and embodiments. However, it should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the scope of the present invention.

[0041] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by technicians in the technical field to which the present invention belongs. The terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0042] like Figure 1As shown in the figure, the workflow of a method for simultaneous estimation and internal calibration of traffic road visibility based on video images is as follows: a permanent scatterer area image is generated for the traffic video surveillance image; the fog and background image in the permanent scatterer area image are separated based on the matrix decomposition method; then, the traffic road visibility is estimated for the separated background image according to the atmospheric attenuation model; the information entropy of the eigenvalue is calculated, and then the traffic road visibility is internally calibrated, and finally a high-precision visibility value is output.

[0043] Specifically, a permanent scatterer area image is generated for the traffic video surveillance image. Two permanent scatterer areas within the video image surveillance range are generally selected, marked as PS1 and PS2 respectively. The specific design scheme is as follows:

[0044] (1) Properties: It is fixed and permanent on the ground and has the same material properties;

[0045] (2) Color: The object should be as white as possible, and backscattering produces white light to cover all wavelengths of visible light;

[0046] (3) Distance: Distributed in the same plane and at a sufficient distance from the video surveillance equipment, the incident angles of the video surveillance equipment to the two permanent scatterer areas are close.

[0047] Next, the fog and background images in the permanent scatterer area image are separated based on the matrix decomposition method. Assuming that the video image of each frame is F, its size is m×n, the fog component image is A, and the background image is B, then F=A+B. Matrix decomposition of image F can be obtained:

[0048] F=U∑V (1)

[0049] Wherein the matrices U and V are m×m and n×n unitary matrices respectively, and ∑ is an m×n diagonal matrix. The specific form is:

[0050]

[0051] The eigenvalues ​​satisfy λ 1 >λ 2 >…>λ m .

[0052] Physically, these eigenvalues ​​correspond to the energy of different components of the image. The first and second eigenvalues ​​are the energy of the fog in the sky and non-sky areas. Therefore, these two eigenvalues ​​can be set to zero to obtain the defogged background image:

[0053]

[0054] Then, the traffic road visibility is estimated based on the separated background image according to the atmospheric attenuation model. Assume that the background images with no attenuation at the starting distances of the two permanent scatterer regions are and Where x and y represent the two-dimensional position of the image pixel. According to the physical model of atmospheric attenuation, the backscattered light of the background object in the permanent scatterer area is attenuated by the atmospheric fog particles and the images are respectively and where r 1 and r 2 are the distances between the two scatterer regions and the video surveillance equipment, and β is the atmospheric extinction attenuation coefficient. To improve reliability, the image intensities of the two scatterer regions in the background image B are averaged to obtain

[0055]

[0056] as well as

[0057]

[0058] In the formula, |·| represents the total number of pixels in the image area. and is the average intensity value without attenuation at the starting distance of the background image. Assume that the distance between the two scatterer regions in the same horizontal plane is The radial distance from this scatterer to the video device Generally speaking, they are different. According to the solution of ingeniously designing the scatterer in claim 2, the careful design of the regional attributes, colors and distances of the two scatterers can make as well as Therefore, divide equation (4) by equation (5), and then calculate the visibility v according to the World Meteorological Organization standard: r The calculation formula The derivation and simplification can be obtained:

[0059]

[0060] Then calculate the information entropy of the eigenvalue, and then perform internal calibration on the traffic road visibility, and finally output a high-precision visibility value. Use the eigenvalue of formula (2) to define the probability P i , i = 1, ..., m, specifically:

[0061]

[0062] The information entropy H of the eigenvalue is further calculated as:

[0063]

[0064] When there is no fog, the eigenvalues ​​of the image are distributed evenly. When fog appears, the first and second eigenvalues ​​of the image are the main energy directions, and the eigenvalues ​​are unevenly distributed. According to the maximum entropy theory, the information entropy of foggy images is small, while the information entropy of fog-free images is large. By analyzing the time series images according to formula (8), it is easy to determine the critical entropy value H when fog occurs. T For the two special cases of very low visibility and very good visibility, two reliable visibility values ​​are artificially determined according to the distance of the image scatterer distribution, and then the visibility value is calculated by formula (6). The two artificially determined values ​​are compared with the calculated value to obtain the high visibility calibration factor f h and the calibration factor f for low visibility l , and then the internal calibration operation for all visibility is formed as

[0065]

[0066] In the formula is the final output visibility value.

[0067] In the embodiment, Figure 2 It can be seen that there are road boundary walls, street lamp poles and lane markings on the traffic road that can be selected as permanent scatterer areas. Relatively speaking, the lane markings are more in line with the three requirements of claim 2 in terms of attributes, color and distance, so this area is selected as the permanent scatterer area.

[0068] Depend on Figure 3 It can be seen that by comparing the grayscale images and intensity images of the two images without fog and with fog, the fog attenuates the intensity of the permanent scatterer and the fog component appears in the intensity image. The existing dark channel-based method cannot effectively eliminate the fog component. Figure 3 (c) There is still a lot of fog energy. By implementing the method of the present invention, the fog component is completely separated, and the obtained background image is different from the image without fog. Figure 3 (a) is close. Its intensity is smaller than that of the fog-free image because the scatterers are attenuated in the fog, which is a correct result.

[0069] Depend on Figure 4 It can be seen that under low visibility and dense fog weather conditions, the visibility result estimated by implementing the method of the present invention is close to the direct measurement result value, and the absolute average error between the two is 186 meters, which is better than the error required by the existing visibility measurement standard.

[0070] From the above description and the experimental results of image defogging and visibility estimation calibration, it can be seen that the method provided by the present invention can not only effectively suppress the fog component in the video image and enhance the contrast of the image, but also accurately estimate the traffic visibility and realize the internal calibration of the visibility at the same time. It has the advantages of low computational complexity, high precision, low cost, strong compatibility and scalability, and can be used in business applications such as traffic visibility monitoring and forecasting in traffic roads with video surveillance networks.

[0071] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modification, equivalent substitution or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for simultaneous estimation and internal calibration of traffic road visibility based on video images. It is characterized in that The following steps are involved: Step S1: generating a permanent scatterer area image from a traffic video surveillance image; In the step S1, two regions of the permanent scatterer region image within the video image monitoring range are selected and marked as PS1 and PS2 respectively; The specific area selection requirements for attributes, colors and distances are as follows: the attributes are fixed and permanent on the ground, with the same material attributes; the object color is white, and the backscattering produces white light to cover all wavelengths of visible light; it is distributed in the same plane and has a sufficient distance from the video surveillance equipment, so that the incident angles of the video surveillance equipment to the two permanent scatterer areas are close; Step S2: Separating the fog and background images in the permanent scatterer region image based on a matrix decomposition method; Step S3: Then, for the separated background image, the visibility of the traffic road is estimated according to the atmospheric attenuation model; In step S3, the method for estimating the visibility of the traffic road based on the atmospheric attenuation model for the separated background image is as follows: assuming that the background images without attenuation at the starting distances of the two permanent scatterer regions are and Where x and y represent the two-dimensional position of the image pixel; According to the physical model of atmospheric attenuation, the backscattered light of the background objects in the permanent scatterer area is attenuated by atmospheric fog particles and the images are respectively and where r 1 and r 2 are the distances between the two scatterer areas and the video surveillance equipment, and β is the atmospheric extinction attenuation coefficient; To improve reliability, the image intensities of the two scatterer regions in the background image B are averaged, and the following is obtained: as well as In the formula, |·| represents the total number of pixels in the image area. The average intensity value without attenuation from the starting distance of the background image; Assume that the distance between the two scatterer regions in the same horizontal plane is Radial distance from the scatterer to the video device Generally speaking, they are different; according to the careful design of the two scatterer area attributes, colors and distances in step S1, as well as Therefore, divide equation (4) by equation (5), and then calculate the visibility v according to the World Meteorological Organization standard: r The calculation formula The derivation and simplification can be obtained: Step S4: recalculate the information entropy of the eigenvalue, and then perform internal calibration on the traffic road visibility, and finally output a high-precision visibility value.

2. The method for simultaneously estimating and internally calibrating traffic road visibility based on video images according to claim 1, It is characterized in that The method for separating the fog and background image in the permanent scatterer region image based on the matrix decomposition method in step S2 is specifically as follows: assuming that the video image of each frame is F, whose size is m×n, the fog component image is A, and the background image is B, then F=A+B, and performing matrix decomposition on the image F, it can be obtained: F=U∑V (1) Wherein the matrices U and V are m×m and n×n unitary matrices respectively, and ∑ is an m×n diagonal matrix. The specific form is: The eigenvalues ​​satisfy λ 1 >λ 2 >…>λ m ; Physically, these eigenvalues ​​correspond to the energy of different components of the image. The first and second eigenvalues ​​are the energy of the fog in the sky and non-sky areas. Therefore, these two eigenvalues ​​can be set to zero to obtain the defogged background image:

3. The method for simultaneously estimating and internally calibrating traffic road visibility based on video images according to claim 1, It is characterized in that The specific method for calculating the information entropy of the characteristic value in step S4, and then internally calibrating the traffic road visibility, and finally outputting a high-precision visibility value is: The probability P is defined by the eigenvalue of (2) i , i = 1, ..., m, specifically: The information entropy H of the eigenvalue is further calculated as: When there is no fog, the eigenvalues ​​of the image are distributed evenly. When fog appears, the first and second eigenvalues ​​of the image are the main energy directions, and the eigenvalues ​​are unevenly distributed. According to the maximum entropy theory, the information entropy of foggy images is small, while the information entropy of fog-free images is large. By analyzing the time series images according to formula (8), it is easy to determine the critical entropy value H when fog occurs. T ; For the two special cases of very low visibility and very good visibility, two reliable visibility values ​​are artificially determined according to the distance of the image scatterer distribution, and then the visibility value is calculated by formula (6). The two artificially determined values ​​are compared with the calculated value to obtain the high visibility calibration factor f h and the calibration factor f for low visibility l , and then the internal calibration operation for all visibility is formed as: In the formula is the final output visibility value.

Images