A method for visibility inversion on highways

Abstract

This application relates to a visibility inversion method for highways. The method first processes the road image into grayscale and uses edge detection to obtain lane line contours. Then, it establishes a correspondence between real-world coordinates and imaging plane coordinates. Based on lane line standards such as the length of dashed lines and the distance between lines, parameters such as the height, viewing angle, and magnification of the highway camera are inverted. Finally, based on the lane line contours in the grayscale image, the actual length of the lane lines in the real world is calculated, and this actual length is used as the basis for visibility estimation. Experimental results demonstrate that by denoising the visibility curves of multiple images, the overall trend of visibility increasing over the given time period is obtained, which aligns with common sense. Therefore, this method proves to be relatively accurate in its visibility inversion method.

Description

Technical Field

[0001] This application relates to the field of visibility inversion technology, and more particularly to a method for visibility inversion on highways. Background Technology

[0002] Visibility inversion is a classic problem in atmospheric science research, with significant implications for military, transportation, environmental, and economic applications. Existing visibility detection methods include visual inspection, instrument-based methods, and image / video-based methods. Whether retrieving visibility information from AMOS data or video images, it falls under the category of inverse problems. Inverse problems are ill-posed, making them significantly more difficult to solve than direct problems. Therefore, choosing the right solution method for inverse problems is crucial.

[0003] Currently, visibility inversion data for highways generally relies on visibility observation data for estimation (such as constructing regression equations based on observation data), and there is a lack of technical solutions for accurately estimating highway visibility without relying on visibility observation data. Summary of the Invention

[0004] The following is an overview of the subject matter described in detail herein. This overview is not intended to limit the scope of the claims.

[0005] The main objective of this invention is to propose a visibility inversion method for highways, which can accurately estimate the visibility of highways without relying on visibility observation data.

[0006] To achieve the above objective, a first aspect of the present invention provides a visibility inversion method for highways, the visibility inversion method for highways comprising:

[0007] Acquire images of highways;

[0008] The road image is converted into a grayscale image, and the lane line contours are extracted from the grayscale image;

[0009] The lane line contour is converted into the actual length of the lane line in the real world based on a coordinate transformation model between real-world coordinates and imaging plane coordinates. The actual length of the lane line is then used as the basis for estimating highway visibility. The construction process of the coordinate transformation model includes:

[0010] Establish a conversion environment, which includes the road surface coordinate system (X) of the highway. ω Y ω Z ω ), the camera coordinate system (X) for capturing the road images c Y c Z c(and imaging plane, camera coordinate system Z) c The axis is in the same direction as the optical axis of the camera, and the origin of the camera coordinate system is Q. c Located at the optical center of the camera, plane X c Y c Parallel to the imaging plane, the road surface coordinate system Z ω The positive axis is perpendicular to the road surface and points downwards; the road surface coordinate system X... ω The axis is the Z-axis of the camera coordinate system. c The projection of the axis onto the road surface;

[0011] The transformation formula for constructing the coordinate transformation model based on the transformation environment is as follows:

[0012]

[0013] Where (X, Y) are the coordinates in the real world, ψ is the scaling factor between the imaging plane and the real world, H is the height of the camera, θ is the camera's tilt angle, (u, v) are the coordinates of the imaging plane, and (u0, v0) are the coordinates of the image center point in the imaging plane.

[0014] Based on the coordinates of the starting point and breakpoint of the lane line in the imaging plane and the lane line standard of the highway, estimate the values ​​of ψ, H and θ.

[0015] In some embodiments, the visibility inversion method for highways further includes:

[0016] A lane line sequence is formed based on the actual lengths of the lane lines in the real world, which are inverted from multiple road images;

[0017] A machine learning model is constructed, and the machine learning model is trained based on the lane line sequence to obtain a highway visibility inversion model after the machine learning model has been trained.

[0018] The target road image of the target highway is input into the highway visibility inversion model to obtain the actual length of the lane lines in the target highway as inverted by the highway visibility inversion model.

[0019] In some embodiments, converting the lane line profile into the actual length of the lane line in the real world according to a coordinate transformation model between real-world coordinates and imaging plane coordinates includes:

[0020] Extract the visible lane line start points and break points from the lane line contour;

[0021] Determine the coordinates of the lane line start point and break point in the imaging plane;

[0022] Based on the coordinate transformation model, the starting coordinates and breakpoint coordinates of the lane lines in the imaging plane are converted into the starting coordinates and breakpoint coordinates of the lane lines in the real world.

[0023] The actual length of the lane line is calculated based on the starting and ending coordinates of the lane line in the real world.

[0024] In some embodiments, extracting the lane line contours from the grayscale image includes:

[0025] Extract the image gradient of the grayscale image;

[0026] The grayscale image is binarized using a preset gradient threshold to obtain a binarized image of the grayscale image;

[0027] Extract the lane line contours from the binarized image.

[0028] In some embodiments, prior to extracting lane line contours from the binarized image, the highway visibility inversion method further includes:

[0029] The binarized image is segmented and denoised. The denoising includes: if less than 10 of the 25 adjacent pixels of the binarized image have a value of 0, then the area where the 25 adjacent pixels are located is determined to be without lane lines.

[0030] In some embodiments, the machine learning model is an LSTM network or an ARMA network.

[0031] In some embodiments, before training the machine learning model based on the lane line sequence, the highway visibility inversion method further includes:

[0032] The lane line sequence is subjected to window sliding filtering.

[0033] To achieve the above objective, a second aspect of the present invention provides a visibility inversion device for highways, the visibility inversion device for highways comprising:

[0034] Image acquisition unit, used to acquire road images of highways;

[0035] An edge extraction unit is used to convert the road image into a grayscale image and extract the lane line contours in the grayscale image;

[0036] The visibility inversion unit is used to convert the lane line profile into the actual length of the lane line in the real world based on a coordinate transformation model between real-world coordinates and imaging plane coordinates, and uses the actual length of the lane line as the basis for estimating highway visibility; wherein the construction process of the coordinate transformation model includes:

[0037] Establish a conversion environment, which includes the road surface coordinate system (X) of the highway. ω Y ω Z ω ), the camera coordinate system (X) for capturing the road images c Y c Z c (and imaging plane, camera coordinate system Z) c The axis is in the same direction as the optical axis of the camera, and the origin of the camera coordinate system is Q. c Located at the optical center of the camera, plane X c Y c Parallel to the imaging plane, the road surface coordinate system Z ω The positive axis is perpendicular to the road surface and points downwards; the road surface coordinate system X... ω The axis is the Z-axis of the camera coordinate system. c The projection of the axis onto the road surface;

[0038] The transformation formula for constructing the coordinate transformation model based on the transformation environment is as follows:

[0039]

[0040] Where (X, Y) are the coordinates in the real world, ψ is the scaling factor between the imaging plane and the real world, H is the height of the camera, θ is the camera's tilt angle, (u, v) are the coordinates of the imaging plane, and (u0, v0) are the coordinates of the image center point in the imaging plane.

[0041] Based on the coordinates of the starting point and breakpoint of the lane line in the imaging plane and the lane line standard of the highway, estimate the values ​​of ψ, H and θ.

[0042] To achieve the above objectives, a third aspect of the present invention provides an electronic device comprising: at least one control processor and a memory for communicatively connecting to the at least one control processor; the memory storing instructions executable by the at least one control processor, the instructions being executed by the at least one control processor to enable the at least one control processor to perform the highway visibility inversion method of the first aspect described above.

[0043] To achieve the above objectives, a fourth aspect of the present invention provides a computer-readable storage medium storing computer-executable instructions for causing a computer to perform the highway visibility inversion method of the first aspect described above.

[0044] One embodiment of this application provides a visibility inversion method for highways. This method first processes the road image into grayscale and uses edge detection to obtain lane line contours. Then, it constructs a correspondence between real-world coordinates and imaging plane coordinates. Based on lane line standards such as the length of dashed lines and the distance between lines, parameters such as the height, viewing angle, and magnification of the highway camera are inverted. Finally, based on the lane line contours in the grayscale image, the actual length of the lane lines in the real world is calculated, and this actual length is used as the basis for visibility estimation. Experimental results demonstrate that by denoising the visibility curves of multiple images, the overall trend of visibility increasing over the given time period is obtained, which aligns with common sense. Therefore, this method proves that the visibility inversion method provided is relatively accurate.

[0045] It is understood that the beneficial effects of the second to fourth aspects compared with the related technologies are the same as the beneficial effects of the first aspect compared with the related technologies. Please refer to the relevant description in the first aspect above, which will not be repeated here. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is a flowchart illustrating a method for retrieving visibility on a highway according to an embodiment of this application.

[0048] Figure 2 This is a schematic flowchart of extracting lane line contours from a grayscale image according to an embodiment of this application;

[0049] Figure 3 This application provides a grayscale image and a binarized image in one embodiment.

[0050] Figure 4 This is a filtered binarized image provided in one embodiment of this application;

[0051] Figure 5 This is a structural diagram of a coordinate transformation model provided in one embodiment of this application;

[0052] Figure 6 This is a grayscale image of calibrated points A-F6 provided in one embodiment of this application;

[0053] Figure 7This is a schematic diagram of a process for converting lane line contours into the actual length of lane lines in the real world based on a coordinate transformation model, according to one embodiment of this application.

[0054] Figure 8 This is a line graph of the visibility sequence of multiple road images provided in one embodiment of this application;

[0055] Figure 9 yes Figure 8 Filtered line chart;

[0056] Figure 10 This is a schematic diagram of a process for obtaining visibility prediction using a highway visibility inversion model, provided in one embodiment of this application.

[0057] Figure 11 This is a training diagram of an embodiment of the present application using LSTM as a machine learning model;

[0058] Figure 12 This is a schematic diagram illustrating the prediction results obtained from a highway visibility inversion model provided in one embodiment of this application;

[0059] Figure 13 This is a schematic diagram showing the comparison between the predicted sample and the actual value, as well as the root mean square error, provided in one embodiment of this application.

[0060] Figure 14 This is a schematic diagram of a filtered lane line sequence provided in one embodiment of this application;

[0061] Figure 15 This is a schematic diagram of the residual test results provided in one embodiment of this application;

[0062] Figure 16 This is a schematic diagram illustrating the prediction effect of a highway visibility inversion model provided in one embodiment of this application;

[0063] Figure 17 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application. Detailed Implementation

[0064] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0065] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0066] Reference Figure 1 One embodiment of this application provides a visibility inversion method for highways, which includes the following steps:

[0067] Step S110: Acquire images of the highway.

[0068] The purpose of this embodiment is to achieve accurate visibility estimation of highways without relying on visibility observation data. This is mainly achieved by using lane line outlines from highway surveillance videos combined with a coordinate transformation model between real-world coordinates and image coordinates to invert visibility on the highway. This is because visibility refers to the maximum distance at which a person with normal vision can identify an object from its background. "Visibility" in daylight means being able to see and recognize the outline and shape of an object. Based on this definition, road traffic marking outlines with relatively obvious contrast in the image, whose contrast changes are easily discernible to the human eye, are selected as the basis for visibility estimation. A transformation relationship between imaging plane coordinates and real-world coordinates is established to determine the actual distance of the lane lines. Therefore, in step S110, this embodiment first acquires road images of the highway, and then extracts the necessary information from the road images for subsequent visibility calculations. Road images can be captured by cameras installed on equipment (e.g., streetlights, height restriction poles, road signs, etc.) next to the highway. The camera's shooting angle is detailed in the subsequent embodiments.

[0069] Step S120: Convert the road image into a grayscale image and extract the lane line contours from the grayscale image.

[0070] In step S120, the road image is converted into a grayscale image, and lane line contours are extracted from the grayscale image. This embodiment differs from existing visibility prediction schemes in that it does not rely on visibility observation data to estimate visibility. Instead, it uses traditional imaging geometry to build a model and utilizes lane line contours with relatively obvious contrast in the image, whose contrast changes are easily discernible to the human eye, as the basis for visibility estimation. Therefore, this embodiment uses edge extraction technology to extract lane line contours from the grayscale image.

[0071] Reference Figure 2 In some embodiments, step S120 includes steps S310 to S330:

[0072] Step S310: Extract the image gradient of the grayscale image.

[0073] Step S320: Binarize the grayscale image using a preset gradient threshold to obtain a binarized grayscale image.

[0074] Step S330: Extract the lane line contours from the binarized image.

[0075] In actual operation, step S120 converts the RGB image to a grayscale image, calculates the image gradient, and then selects a threshold of 0.1 to perform binarization processing on the image. The result is as follows. Figure 3 As can be seen, the white portion in the binarized image is largely consistent with the portion that is perceptible to the human eye. Specifically, when extracting edges, the Sobel operator is used to extract the gradient of the grayscale image matrix A, and the edge image matrix G is extracted. The processing procedure is as follows:

[0076]

[0077] Noting that the binarized image contains a large amount of noise, which affects the extraction of lane line endpoint coordinates, this embodiment further includes step S410 before step S320:

[0078] Step S410: Segment and denoise the binarized image. Denoising includes: if less than 10 pixels out of 25 adjacent pixels in the binarized image have a value of 0, then the region containing these 25 adjacent pixels is determined to be without lane lines. The denoised image is as follows. Figure 4 As shown.

[0079] Step S130: Based on the coordinate transformation model between real-world coordinates and imaging plane coordinates, convert the lane line outline into the actual length of the lane line in the real world, and use the actual length of the lane line as the basis for estimating the visibility of the highway.

[0080] The geometric relationship between highway cameras and road imaging forms the basis of the visibility value calculation model; therefore, referring to... Figure 5 The process of constructing the coordinate transformation model includes the following steps S210-S230:

[0081] Step S210: Set up the conversion environment, which includes the highway road surface coordinate system (X). ω Y ω Z ω ), the camera coordinate system for capturing road images (X) c Y c Z c (and imaging plane, camera coordinate system Z) c The axis is in the same direction as the optical axis of the camera, and the origin of the camera coordinate system is Q. c Located at the optical center of the camera, plane X c Y c Parallel to the imaging plane, road surface coordinate system Z ω The positive axis is perpendicular to the road surface and points downwards; the road surface coordinate system X... ω The axis is the Z-axis of the camera coordinate system. c The projection of the axis onto the road surface.

[0082] Step S220: The transformation formula for constructing the coordinate transformation model based on the transformation environment is as follows:

[0083]

[0084] Where (X, Y) are the coordinates in the real world, ψ is the scaling factor between the imaging plane and the real world, H is the height of the camera, θ is the camera's tilt angle, (u, v) are the coordinates of the imaging plane, and (u0, v0) are the coordinates of the image center point in the imaging plane.

[0085] In step S220, ψ, H, and θ are relevant parameters of the camera. Therefore, in the subsequent step S330, calculations are performed, and the calculated values ​​are substituted into the above formula to perform coordinate transformation.

[0086] Step S230: Estimate the values ​​of ψ, H, and θ based on the starting point coordinates and breakpoint coordinates of the lane lines in the imaging plane and the lane line standards of the highway.

[0087] like Figure 6 The provided binarized RGB image captured by a camera near the Dasheng toll station is shown. To obtain the above parameters, it is necessary to use an object of known size in the image for calibration. According to relevant information, the length of the dashed line is 6m and the spacing between dashed lines is 9m. Six points (AF) can be marked on the dashed lines in the image. Using the three sets of relationships AB≈15m, DE≈15m, and DF≈9m, we can approximately solve for ψ=1.1984, H=4.0548m, and θ=20.8056. We also verify that BC≈5.7450m and EF≈6.0011m, so the conversion formula is approximately valid.

[0088] Reference Figure 7 In some embodiments of this application, step S130, which converts the lane line profile into the actual length of the lane line in the real world based on the coordinate transformation model between real-world coordinates and imaging plane coordinates, includes the following steps S510 to S540:

[0089] Step S510: Extract the visible lane line start points and break points from the lane line contour.

[0090] Step S520: Determine the coordinates of the lane line start point and break point in the imaging plane.

[0091] Step S530: Convert the starting point coordinates and breakpoint coordinates of the lane line in the imaging plane into the starting point coordinates and breakpoint coordinates of the lane line in the real world according to the coordinate transformation model.

[0092] Step S540: Calculate the actual length of the lane line based on the starting point coordinates and breakpoint coordinates of the lane line in the real world.

[0093] In step S530, the coordinate transformation model can convert the starting point coordinates and breakpoint coordinates (coordinates of the image imaging plane) of the lane line into the starting point coordinates and breakpoint coordinates of the lane line in the actual world coordinate system through the above formulas (2) and (3). After obtaining the coordinates of the two points in the actual world coordinate system, the actual length of the lane line can be obtained, and the actual length of the lane line can be used as the visibility value of the highway.

[0094] Assuming step S110 involves multiple road images, then steps S120 and S130 extract the visibility value of each road image, resulting in the following: Figure 8 The line graph shown.

[0095] The line chart shows an upward trend, but due to excessive noise, the specific trend is not readily apparent. Therefore, a sliding smoothing technique is used to reduce noise. The smoothing window is 10, and after three rounds of smoothing, the result is as follows: Figure 9 The curve shown in the figure clearly shows an upward trend, indicating that the fog gradually dissipates with sunrise. This aligns with actual natural conditions and, to some extent, proves that the visibility extraction model constructed in this embodiment is relatively accurate.

[0096] This method first processes the road image into grayscale and uses edge detection to obtain lane line contours. Then, it establishes the correspondence between real-world coordinates and imaging plane coordinates. Based on lane line standards such as the length of dashed lines and the distance between lines on highways, it inversely obtains parameters such as the height, viewing angle, and magnification of the highway camera. Finally, based on the lane line contours in the grayscale image, it calculates the actual length of the lane lines in the real world and uses this actual length as the basis for visibility estimation. Experimental results show that by denoising the visibility curves of multiple images, the overall trend of visibility increasing over the time period is obtained, which is consistent with common sense. Therefore, it can be proven that the visibility inversion method provided by this method is relatively accurate.

[0097] The above steps provide a method for accurately estimating highway visibility without relying on visibility observation data. In some embodiments, a method is provided that uses the estimated lane line sequence described above to train a machine learning model, and then uses the trained machine learning model to predict visibility, as described above. Figure 10 It also includes the following steps S610-S630:

[0098] Step S610: Form a lane line sequence based on the actual lengths of lane lines in the real world retrieved from multiple road images.

[0099] Step S620: Construct a machine learning model. Train the machine learning model based on the lane line sequence to obtain the highway visibility inversion model after the machine learning model has been trained.

[0100] Step S630: Input the target road image of the target highway into the highway visibility inversion model to obtain the actual length of the lane lines in the target highway inverted by the highway visibility inversion model.

[0101] Here, we take the Long Short-Term Memory (LSTM) model and the Autoregressive Moving Average (ARMA) model as examples to provide two implementation methods:

[0102] The first method (using an LSTM network as a machine learning model): To better fit the data and prevent training divergence, the lane line sequences are first standardized to zero mean and unit variance. The first 80 samples are used as the training set, and the last 20 samples are used as the test set. An LSTM network machine learning model is created, with each LSTM layer specified to have 200 hidden units. To prevent gradient explosion, the gradient threshold is set to 1, and an initial learning rate of 0.005 is specified. This learning rate is then multiplied by 0.2 after 125 epochs to reduce the learning rate, for a total of 250 epochs of training. The training process is as follows: Figure 11 .

[0103] The last 20 samples were tested in the model, and the results are as follows: Figure 12 The Forecast section represents the prediction results for the next 20 time steps from the highway visibility inversion model (obtained after training a machine learning model). Figure 12 (To the right of the dotted line). Figure 13 The comparison between the 20 predicted samples and the actual values, as well as the root mean square error, is presented. The comparison of the prediction results for these 20 samples shows that both the actual and predicted values ​​exhibit an upward trend, with a root mean square error of 6.7289, which is relatively small.

[0104] The second approach (using ARMA networks as a machine learning model): Treat the data sequence of predictive indicators over time as a random sequence. The dependencies between these random variables reflect the continuity of the original data over time. On one hand, there is the influence of influencing factors; on the other hand, there are inherent patterns of change. Let's assume the influencing factors are x1, x2, ..., x... k By regression analysis:

[0105] Y t =β1x1 + β2x2 + ... + β p x p +Z

[0106] Where Y is the observed value of the object to be predicted, and Z is the error. As the object to be predicted, Y... t Influenced by its own changes, its pattern can be represented by the following formula:

[0107] Y t =β1Yt-1 +β2Y t-2 +…+β p Y t-p +Z t

[0108] Therefore, the ARMA model expression is obtained as follows:

[0109] Y t =β0+β1Y t-1 +β2Y t-2 +…+β p +ε t +α1ε t-1 +α2ε t-2 +…+α q ε t-q

[0110] That is, the time series follows an autoregressive moving average mixture model of order (p, q).

[0111] The lane line sequence is filtered, and a moving average with a window size of 5 is applied. The filtering result is as follows. Figure 14 As shown, the stationarity of the filtered data was tested, and the result was non-stationary. Therefore, the data was first-differenced to obtain a stationary sequence. The order of the machine learning model was determined using the autocorrelation function (ACF) and partial autocorrelation function (PACF): p = 1, q = 5.

[0112] Next, to determine whether the choice of the ARMA machine learning model order was reasonable, a residual test was performed on the model. The residual is the signal remaining after subtracting the signal fitted by the machine learning model from the original signal. If the residual is randomly normally distributed and uncorrelated, it indicates that the residual is a white noise signal, meaning that the useful signal has already been extracted into the machine learning model. The residual test results are as follows: Figure 15 As shown. Figure 15 (a) is to check whether the residuals are close to a normal distribution. Ideally, the residuals should be close to a normal distribution. Figure 15 (b) and Figure 15 (c) Test the autocorrelation and partial autocorrelation of the residuals. Ideally, there should be no points in the graph that exceed the blue line. As can be seen from the graph, the residuals are close to a normal distribution and are independent of each other, so the machine learning model can be considered to meet the requirements.

[0113] The obtained highway visibility inversion model predicts the time series for the next 100 steps, and its prediction effect is as follows: Figure 16As shown in the forecast, visibility will increase as the fog gradually weakens, and the predicted visibility at time step 175 will reach 150 meters, meaning it will be around 8:30 AM. The model's predicted fog change trend aligns with natural laws, and the timing of fog dissipation is consistent with common sense, indicating that the highway visibility inversion model constructed using the ARMA network yields reasonable forecast results.

[0114] One embodiment of this application provides a visibility inversion device for highways. The visibility inversion device for highways includes an image acquisition unit, an edge extraction unit, and a visibility inversion unit.

[0115] The image acquisition unit is used to acquire images of highways.

[0116] The edge extraction unit is used to convert road images into grayscale images and extract lane line contours from the grayscale images.

[0117] The visibility inversion unit is used to convert lane line contours into the actual length of lane lines in the real world based on a coordinate transformation model between real-world coordinates and imaging plane coordinates. The actual length of the lane lines is then used as the basis for estimating highway visibility. The construction process of the coordinate transformation model includes:

[0118] Set up the conversion environment, which includes the highway road surface coordinate system (X). ω Y ω Z ω ), the camera coordinate system for capturing road images (X) c Y c Z c (and imaging plane, camera coordinate system Z) c The axis is in the same direction as the optical axis of the camera, and the origin of the camera coordinate system is Q. c Located at the optical center of the camera, plane X c Y c Parallel to the imaging plane, road surface coordinate system Z ω The positive axis is perpendicular to the road surface and points downwards; the road surface coordinate system X... ω The axis is the Z-axis of the camera coordinate system. c The projection of the axis onto the road surface.

[0119] The transformation formula for constructing the coordinate transformation model based on the transformation environment is as follows:

[0120]

[0121] Where (X, Y) are the coordinates in the real world, ψ is the scaling factor between the imaging plane and the real world, H is the height of the camera, θ is the camera's tilt angle, (u, v) are the coordinates of the imaging plane, and (u0, v0) are the coordinates of the image center point in the imaging plane.

[0122] Based on the coordinates of the starting point and breakpoint of the lane line in the imaging plane and the lane line standard of the highway, estimate the values ​​of ψ, H and θ.

[0123] It should be noted that the visibility inversion device for highways provided in this embodiment and the visibility inversion method embodiment for highways described above are based on the same inventive concept. Therefore, the relevant content of the visibility inversion method embodiment for highways described above is also applicable to the visibility inversion device embodiment for highways, and will not be described in detail here.

[0124] like Figure 17 This application also provides an electronic device, which includes:

[0125] At least one memory;

[0126] At least one processor;

[0127] At least one program;

[0128] The program is stored in memory, and the processor executes at least one program to implement the visibility inversion method for highways described above in this disclosure.

[0129] This electronic device can be any smart terminal, including mobile phones, tablets, personal digital assistants (PDAs), and in-vehicle computers.

[0130] The electronic devices according to embodiments of this application will now be described in detail.

[0131] The processor 1600 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention.

[0132] The memory 1700 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 1700 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1700 and is called by the processor 1600 to execute the highway visibility inversion method of the embodiments of this invention.

[0133] The input / output interface 1800 is used to implement information input and output.

[0134] The communication interface 1900 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0135] Bus 2000 transmits information between various components of the device (e.g., processor 1600, memory 1700, input / output interface 1800, and communication interface 1900);

[0136] The processor 1600, memory 1700, input / output interface 1800 and communication interface 1900 are connected to each other within the device via bus 2000.

[0137] This invention also provides a storage medium, which is a computer-readable storage medium storing computer-executable instructions for causing a computer to execute the aforementioned highway visibility inversion method.

[0138] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0139] The embodiments described in this invention are intended to more clearly illustrate the technical solutions of the embodiments of this invention, and do not constitute a limitation on the technical solutions provided by the embodiments of this invention. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this invention are also applicable to similar technical problems.

[0140] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present invention, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0141] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0142] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0143] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0144] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0145] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0146] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0147] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0148] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0149] The above is a detailed description of the preferred embodiments of this application. However, the embodiments of this application are not limited to the above-described implementation methods. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the embodiments of this application. All such equivalent modifications or substitutions are included within the scope defined by the claims of the embodiments of this application.

Claims

1. A method for visibility inversion on highways, characterized in that, The visibility inversion method for the highway includes: Acquire images of highways; The road image is converted into a grayscale image, and the lane line contours are extracted from the grayscale image; The lane line contour is converted into the actual length of the lane line in the real world based on a coordinate transformation model between real-world coordinates and imaging plane coordinates. The actual length of the lane line is then used as the basis for estimating highway visibility. The construction process of the coordinate transformation model includes: Establish a conversion environment, which includes the road surface coordinate system (X) of the highway. ω Y ω Z ω ), the camera coordinate system (X) for capturing the road images c Y c Z c (and imaging plane, camera coordinate system Z) c The axis is in the same direction as the optical axis of the camera, and the origin of the camera coordinate system is Q. c Located at the optical center of the camera, plane X c Y c Parallel to the imaging plane, the road surface coordinate system Z ω The positive axis is perpendicular to the road surface and points downwards; the road surface coordinate system X... ω The axis is the Z-axis of the camera coordinate system. c The projection of the axis onto the road surface; The transformation formula for constructing the coordinate transformation model based on the transformation environment is as follows: Where (X, Y) are the coordinates in the real world, ψ is the scaling factor between the imaging plane and the real world, H is the height of the camera, θ is the camera's tilt angle, (u, v) are the coordinates of the imaging plane, and (u0, v0) are the coordinates of the image center point in the imaging plane. Based on the coordinates of the starting point and breakpoint of the lane line in the imaging plane and the lane line standard of the highway, estimate the values ​​of ψ, H and θ.

2. The visibility inversion method for highways according to claim 1, characterized in that, The visibility inversion method for highways also includes: A lane line sequence is formed based on the actual lengths of the lane lines in the real world, which are inverted from multiple road images; A machine learning model is constructed, and the machine learning model is trained based on the lane line sequence to obtain a highway visibility inversion model after the machine learning model has been trained. The target road image of the target highway is input into the highway visibility inversion model to obtain the actual length of the lane lines in the target highway as inverted by the highway visibility inversion model.

3. The visibility inversion method for highways according to claim 2, characterized in that, The process of converting the lane line profile into the actual length of the lane line in the real world based on the coordinate transformation model between real-world coordinates and imaging plane coordinates includes: Extract the visible lane line start points and break points from the lane line contour; Determine the coordinates of the lane line start point and break point in the imaging plane; Based on the coordinate transformation model, the starting coordinates and breakpoint coordinates of the lane lines in the imaging plane are converted into the starting coordinates and breakpoint coordinates of the lane lines in the real world. The actual length of the lane line is calculated based on the starting and ending coordinates of the lane line in the real world.

4. The visibility inversion method for highways according to claim 2, characterized in that, The extraction of lane line contours from the grayscale image includes: Extract the image gradient of the grayscale image; The grayscale image is binarized using a preset gradient threshold to obtain a binarized image of the grayscale image; Extract the lane line contours from the binarized image.

5. The visibility inversion method for highways according to claim 4, characterized in that, Before extracting lane line contours from the binarized image, the highway visibility inversion method further includes: The binarized image is segmented and denoised. The denoising includes: if less than 10 of the 25 adjacent pixels of the binarized image have a value of 0, then the area where the 25 adjacent pixels are located is determined to be without lane lines.

6. The visibility inversion method for highways according to claim 2, characterized in that, The machine learning model is an LSTM network or an ARMA network.

7. The visibility inversion method for highways according to claim 2, characterized in that, Before training the machine learning model based on the lane line sequence, the visibility inversion method for highways further includes: The lane line sequence is subjected to window sliding filtering.

8. A visibility inversion device for highways, characterized in that, The visibility inversion device for the highway includes: Image acquisition unit, used to acquire road images of highways; An edge extraction unit is used to convert the road image into a grayscale image and extract the lane line contours in the grayscale image; The visibility inversion unit is used to convert the lane line profile into the actual length of the lane line in the real world based on a coordinate transformation model between real-world coordinates and imaging plane coordinates, and uses the actual length of the lane line as the basis for estimating highway visibility; wherein the construction process of the coordinate transformation model includes: Establish a conversion environment, which includes the road surface coordinate system (X) of the highway. ω Y ω Z ω ), the camera coordinate system (X) for capturing the road images c Y c Z c (and imaging plane, camera coordinate system Z) c The axis is in the same direction as the optical axis of the camera, and the origin of the camera coordinate system is Q. c Located at the optical center of the camera, plane X c Y c Parallel to the imaging plane, the road surface coordinate system Z ω The positive axis is perpendicular to the road surface and points downwards; the road surface coordinate system X... ω The axis is the Z-axis of the camera coordinate system. c The projection of the axis onto the road surface; The transformation formula for constructing the coordinate transformation model based on the transformation environment is as follows: Where (X, Y) are the coordinates in the real world, ψ is the scaling factor between the imaging plane and the real world, H is the height of the camera, θ is the camera's tilt angle, (u, v) are the coordinates of the imaging plane, and (u0, v0) are the coordinates of the image center point in the imaging plane. Based on the coordinates of the starting point and breakpoint of the lane line in the imaging plane and the lane line standard of the highway, estimate the values ​​of ψ, H and θ.

9. An electronic device, characterized in that, include: At least one control processor and a memory for communicatively connecting to the at least one control processor; The memory stores instructions that can be executed by the at least one control processor to enable the at least one control processor to perform the visibility inversion method for highways as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for causing a computer to perform the visibility inversion method for highways as described in any one of claims 1 to 7.

Images