A driving natural environment detection cloud platform

Abstract

The present application relates to natural environment detection, and specifically relates to a kind of driving natural environment detection cloud platform, including server, server receives the road image and wheel image that driving natural environment detection terminal sends by image receiving module, server identifies lane line in road image by lane line identification module, and based on lane line identification result, fuzzy area in road image is marked by fuzzy area marking module, server calculates the actual distance of fuzzy area based on lane line identification result by actual distance calculation module, and the depth profile feature analysis is carried out to fuzzy area by visibility coefficient acquisition module, to obtain the visibility coefficient of fuzzy area, server calculates the road visibility of clear area based on lane line identification result by road visibility calculation module;The technical scheme provided by the present application can effectively overcome the defects that the detection accuracy of road visibility, water accumulation, snow accumulation is lower in prior art.

Description

TECHNICAL FIELD

[0001] The present application relates to natural environment detection, in particular to a driving natural environment detection cloud platform. BACKGROUND

[0002] In the expressway scene, visibility is one of the most important objective natural environment factors affecting driving safety, and low visibility caused by fog may lead to serious traffic safety accidents. Therefore, how to accurately detect road visibility is a problem to be solved.

[0003] Under normal circumstances, a special visibility measuring instrument can be used to obtain the visibility distance, but this solution not only has a high budget cost, but also has a high implementation cost. In order to reduce the cost, the current main method is to obtain the visibility distance from the road image through a preset algorithm, wherein the preset algorithm can be a method of calculating the transmittance by the dark channel to convert the visibility distance. Some of these algorithms have high implementation complexity, and some are easily disturbed by noise, resulting in a large error between the calculated visibility distance and the actual situation, thereby leading to inaccurate visibility detection.

[0004] In addition, there are many objective natural environment factors affecting driving safety, such as uneven road surface, road water accumulation and road snow accumulation. Water accumulation on the road will cause the engine to be flooded and stall, and a small amount of water accumulation may also cause the vehicle to slide sideways and lose control. The snow on the road is difficult to melt in cold weather, and it is easy to be compacted by the continuous rolling of the wheels, which will also cause the vehicle to lose control. How to accurately detect the road water accumulation and snow accumulation is also a problem to be solved, which is of great significance to ensure driving safety. SUMMARY

[0005] (I) Technical problems to be solved

[0006] In view of the above shortcomings of the prior art, the present application provides a driving natural environment detection cloud platform, which can effectively overcome the defects of low detection accuracy of road visibility, water accumulation and snow accumulation in the prior art.

[0007] (II) Technical solutions

[0008] To achieve the above object, the present application is realized by the following technical solutions:

[0009] The driving natural environment detection cloud platform comprises a server, the server receives road images and wheel images sent by a driving natural environment detection terminal through an image receiving module, the server identifies lane lines in the road images through a lane line identification module, and labels a fuzzy area in the road images based on the lane line identification result through a fuzzy area labeling module, the server calculates an actual distance of the fuzzy area based on the lane line identification result through an actual distance calculation module, and performs deep contour feature analysis on the fuzzy area to obtain a visibility coefficient of the fuzzy area through a visibility coefficient acquisition module, the server calculates road visibility of a clear area based on the lane line identification result through a road visibility calculation module, and calculates road visibility of the fuzzy area based on the actual distance and the visibility coefficient of the fuzzy area;

[0010] The server extracts wheel edge information from the wheel images through a wheel edge information extraction module, and judges whether a straight line edge exists in the wheel edge information through a straight line edge judgment module, and calculates a road water depth based on the wheel edge information through a road water depth calculation module.

[0011] The server extracts a road background image from a road video through a road background image extraction module, and inputs the road background image into a road snow identification module, and the road snow identification module identifies road snow in the road background image through a trained road snow identification model.

[0012] Preferably, the fuzzy area labeling module labels the fuzzy area in the road images based on the lane line identification result, comprising:

[0013] Based on the lane lines and the number of lane lines in the road images, the farthest lane lines on both sides of the road images are labeled, and the next lane line adjacent to the farthest lane lines on both sides is predicted and labeled.

[0014] Preferably, the actual distance calculation module calculates the actual distance of the fuzzy area based on the lane line identification result, comprising:

[0015] Based on the lane lines in the road images, an affine matrix of the road images projected to an affine plane is calculated, and the actual distance of the fuzzy area is calculated according to the affine matrix.

[0016] Preferably, the wheel edge information extraction module extracts the wheel edge information from the wheel images, comprising:

[0017] Selecting a plurality of regions of interest in the wheel images, and extracting image features of all the regions of interest;

[0018] Based on the similarity between the image features of the regions of interest and the preset wheel image features, the regions of interest of the wheel are obtained.

[0019] Edge extraction is performed on the region of interest of the wheel to extract wheel edge information from the wheel image.

[0020] Preferably, the road water depth calculation module calculates the road water depth based on the wheel edge information, comprising:

[0021] If there is no straight line edge in the wheel edge, the road water depth is 0, otherwise, the road water depth is calculated according to the pixel coordinates of the wheel edge in the wheel image without road water, the preset wheel size and the straight line edge.

[0022] Preferably, the road water depth is calculated according to the pixel coordinates of the wheel edge in the wheel image without road water, the preset wheel size and the straight line edge, comprising:

[0023] According to the pixel coordinates of the wheel edge in the wheel image without road water, the first pixel coordinate corresponding to the maximum longitudinal coordinate and the second pixel coordinate with the minimum longitudinal coordinate located directly below the first pixel coordinate are obtained;

[0024] The number of first pixel blocks between the first pixel coordinate and the second pixel coordinate is obtained, the pixel block size is calculated according to the preset wheel size, and the road water depth is calculated according to the pixel block size and the straight line edge.

[0025] Preferably, the road water depth is calculated according to the pixel block size and the straight line edge, comprising:

[0026] The number of second pixel blocks between the first pixel coordinate and the straight line edge is obtained along the direction from the first pixel coordinate to the second pixel coordinate;

[0027] The road water depth is calculated according to the number of second pixel blocks, the pixel block size and the preset wheel size.

[0028] Preferably, the road background image extraction module extracts the road background image from the road video, comprising:

[0029] The road background image is extracted from the road image using a mixture Gaussian background modeling, and the road background modeling is performed according to the road background image;

[0030] The road background image is extracted by video analysis of the road video according to the road background modeling.

[0031] Preferably, the road background image is extracted from the road image using a mixture Gaussian background modeling, comprising:

[0032] The pixel value in the road image is matched with the Gaussian distribution background model, if the pixel value is successfully matched with the Gaussian distribution background model, the pixel value belongs to the background, otherwise the pixel value belongs to the foreground;

[0033] Extract all pixel values belonging to the background from the road image to obtain a road background image.

[0034] Preferably, the video analysis of the road video according to the road background modeling to extract the road background image comprises:

[0035] The road video is decoded to obtain a time stamp of the road video, and a time switch is set according to the time stamp to determine whether to extract the road background image from the road image by using the Gaussian mixture background modeling.

[0036] (Three) beneficial effects

[0037] Compared with the prior art, the driving natural environment detection cloud platform provided by the application has the following beneficial effects:

[0038] 1) The lane line recognition module recognizes the lane line in the road image, the fuzzy area labeling module labels the fuzzy area in the road image based on the lane line recognition result, the actual distance calculation module calculates the actual distance of the fuzzy area based on the lane line recognition result, and the visibility coefficient acquisition module analyzes the depth contour features of the fuzzy area to obtain the visibility coefficient of the fuzzy area. On the one hand, the road visibility of the clear area is calculated based on the lane line recognition result, and on the other hand, the road visibility of the fuzzy area is calculated based on the actual distance and the visibility coefficient of the fuzzy area, so that the road visibility can be accurately detected.

[0039] 2) The wheel edge information extraction module extracts wheel edge information from the wheel image, the straight line edge judgment module judges whether there is a straight line edge in the wheel edge, and the road water depth calculation module calculates the road water depth based on the wheel edge information. By judging whether there is a straight line edge in the wheel edge, the presence or absence of road water can be accurately detected, and based on the wheel edge information, the road water depth can be accurately detected.

[0040] 3) The road background image extraction module extracts the road background image from the road video, inputs the road background image into the road snow identification module, and the road snow identification module uses the trained road snow identification model to identify the road snow in the road background image. By extracting the road background image from the road video, the interference of foreground objects such as driving vehicles in the road image can be effectively reduced, and the accuracy of road snow detection can be effectively improved. BRIEF DESCRIPTION OF DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description only show some of the embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from these drawings without any creative effort.

[0042] Figure 1 The system schematic diagram of the present application;

[0043] Figure 2 The flowchart of calculating the road visibility in the present application. DETAILED DESCRIPTION

[0044] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without any creative effort belong to the protection scope of the present application.

[0045] A driving natural environment detection cloud platform, as shown in Figure 1 and Figure 2 , comprises a server, the server receives the road image and the wheel image sent by the driving natural environment detection terminal through an image receiving module, the server identifies the lane line in the road image through a lane line identification module, and labels the fuzzy area in the road image based on the lane line identification result through a fuzzy area labeling module, the server calculates the actual distance of the fuzzy area based on the lane line identification result through an actual distance calculation module, and analyzes the depth contour features of the fuzzy area to obtain the visibility coefficient of the fuzzy area through a visibility coefficient acquisition module, the server calculates the road visibility of the clear area based on the lane line identification result through a road visibility calculation module, and calculates the road visibility of the fuzzy area based on the actual distance and the visibility coefficient of the fuzzy area.

[0046] ①The fuzzy area labeling module labels the fuzzy area in the road image based on the lane line identification result, comprising:

[0047] Based on the lane line and the number of lane lines in the road image, the farthest lane lines on both sides of the road image are labeled, and the next lane line adjacent to the farthest lane lines on both sides is predicted and labeled.

[0048] ②The actual distance calculation module calculates the actual distance of the fuzzy area based on the lane line identification result, comprising:

[0049] Based on lane lines in the road image, an affine matrix of the road image projected to an affine plane is calculated, and an actual distance of the blur area is calculated according to the affine matrix.

[0050] The lane line recognition module recognizes lane lines in the road image, the blur area labeling module labels a blur area in the road image based on the lane line recognition result, the actual distance calculation module calculates an actual distance of the blur area based on the lane line recognition result, and the visibility coefficient acquisition module performs depth contour feature analysis on the blur area to acquire a visibility coefficient of the blur area. On one hand, the road visibility of the clear area is calculated based on the lane line recognition result, and on the other hand, the road visibility of the blur area is calculated based on the actual distance and the visibility coefficient of the blur area, so that accurate detection of the road visibility can be realized.

[0051] As shown in Figure 1 The server extracts wheel edge information from the wheel image through a wheel edge information extraction module, and judges whether a straight line edge exists in the wheel edge by using a straight line edge judgment module. The server calculates the road water depth based on the wheel edge information by using a road water depth calculation module.

[0052] ①The wheel edge information extraction module extracts wheel edge information from the wheel image, including:

[0053] A plurality of regions of interest are selected in the wheel image, and image features of all the regions of interest are extracted;

[0054] Based on the similarity between the image features of the regions of interest and preset wheel image features, the regions of interest of the wheel are acquired;

[0055] Edge extraction is performed on the regions of interest of the wheel to extract wheel edge information from the wheel image.

[0056] ②The road water depth calculation module calculates the road water depth based on the wheel edge information, including:

[0057] If no straight line edge exists in the wheel edge, the road water depth is 0, otherwise, the road water depth is calculated according to the pixel coordinates of the wheel edge in the wheel image when there is no road water, a preset wheel size and the straight line edge.

[0058] The road water depth is calculated according to the pixel coordinates of the wheel edge in the wheel image when there is no road water, the preset wheel size and the straight line edge, including:

[0059] According to the pixel coordinates of the wheel edge in the wheel image when there is no road water, a first pixel coordinate corresponding to a maximum longitudinal coordinate and a second pixel coordinate having a minimum longitudinal coordinate located directly below the first pixel coordinate are acquired;

[0060] acquire a first pixel block number between the first pixel coordinate and the second pixel coordinate, calculate a pixel block size according to a preset wheel size, and calculate the road water depth according to the pixel block size and the straight line edge.

[0061] The road water depth is calculated according to the pixel block size and the straight line edge, including:

[0062] acquire a second pixel block number between the first pixel coordinate and the straight line edge along a direction from the first pixel coordinate to the second pixel coordinate;

[0063] calculate the road water depth according to the second pixel block number, the pixel block size and the preset wheel size.

[0064] The above technical solution, the wheel edge information extraction module extracts the wheel edge information from the wheel image, the straight line edge judgment module judges whether there is a straight line edge in the wheel edge, and the road water depth calculation module calculates the road water depth based on the wheel edge information. By judging whether there is a straight line edge in the wheel edge, whether there is road water can be accurately detected, and based on the wheel edge information, the road water depth can be accurately detected.

[0065] As shown in Figure 1 The server extracts a road background image from a road video (collected and sent by a driving natural environment detection terminal installed on a driving vehicle) through a road background image extraction module, and inputs the road background image into a road snow identification module. The road snow identification module uses a trained road snow identification model to identify road snow from the road background image.

[0066] The road background image extraction module extracts the road background image from the road video, including:

[0067] The road background image is extracted from the road image using a mixture Gaussian background modeling, and the road background is modeled according to the road background image.

[0068] The road video is analyzed according to the road background modeling to extract the road background image.

[0069] 1) The road background image is extracted from the road image using a mixture Gaussian background modeling, including:

[0070] The pixel value in the road image is matched with the Gaussian distribution background model. If the pixel value is successfully matched with the Gaussian distribution background model, the pixel value belongs to the background, otherwise the pixel value belongs to the foreground.

[0071] All pixel values belonging to the background are extracted from the road image to obtain the road background image.

[0072] 2) video analysis on the road video according to road background modeling, extracting road background images, comprising:

[0073] Decoding the road video, obtaining the time stamp of the road video, setting the time switch according to the time stamp, and determining whether to extract the road background image from the road image by using the Gaussian mixture background modeling.

[0074] The technical scheme, the road background image extraction module extracts the road background image from the road video, inputs the road background image into the road snow identification module, and the road snow identification module identifies the road snow of the road background image by using the trained road snow identification model. By extracting the road background image from the road video, the interference of the foreground objects such as the driving vehicles in the road image can be effectively reduced, and the accuracy of the road snow detection can be effectively improved.

[0075] The above embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for some technical features; and these modifications or replacements will not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A cloud platform for detecting the natural environment while driving, characterized in that: The system includes a server that receives road images and wheel images from a vehicle natural environment detection terminal via an image receiving module. The server identifies lane lines in the road images via a lane line recognition module and annotates blurred areas in the road images based on the lane line recognition results using a blurred area annotation module. The server calculates the actual distance to the blurred areas based on the lane line recognition results via an actual distance calculation module and performs depth contour feature analysis on the blurred areas using a visibility coefficient acquisition module to obtain the visibility coefficient of the blurred areas. The server also calculates the road visibility of clear areas based on the lane line recognition results via a road visibility calculation module, and simultaneously calculates the road visibility of the blurred areas based on the actual distance and visibility coefficient of the blurred areas. The server extracts wheel edge information from the wheel image through the wheel edge information extraction module, and uses the straight edge judgment module to determine whether there are straight edges in the wheel edge. The server calculates the road water depth based on the wheel edge information through the road water depth calculation module. The server extracts road background images from road videos through a road background image extraction module and inputs the road background images into a road snow recognition module. The road snow recognition module uses a trained road snow recognition model to recognize the road snow in the road background images.

2. The vehicle natural environment detection cloud platform according to claim 1, characterized in that: The blurred region annotation module annotates blurred regions in the road image based on lane line recognition results, including: Based on the lane lines and the number of lane lines in the road image, the farthest lane lines on both sides of the road image are labeled, and the next lane line adjacent to the farthest lane lines on both sides is predicted and labeled.

3. The vehicle natural environment detection cloud platform according to claim 2, characterized in that: The actual distance calculation module calculates the actual distance to the blurred area based on the lane line recognition results, including: Based on the lane lines in the road image, the affine matrix of the road image projected onto the affine plane is calculated, and the actual distance of the blurred region is calculated based on the affine matrix.

4. The vehicle natural environment detection cloud platform according to claim 1, characterized in that: The wheel edge information extraction module extracts wheel edge information from the wheel image, including: Multiple regions of interest are selected in the wheel image, and image features of all regions of interest are extracted; The region of interest of the wheel is obtained based on the similarity between the image features of the region of interest and the preset wheel image features; Edge extraction is performed on the region of interest of the wheel to extract wheel edge information from the wheel image.

5. The vehicle natural environment detection cloud platform according to claim 4, characterized in that: The road water depth calculation module calculates the road water depth based on wheel edge information, including: If there is no straight edge in the wheel edge, the road water depth is 0; otherwise, the road water depth is calculated based on the pixel coordinates of the wheel edge in the wheel image when there is no road water, the preset wheel size, and the straight edge.

6. The vehicle natural environment detection cloud platform according to claim 5, characterized in that: The calculation of road water depth based on the pixel coordinates of the wheel edge in the wheel image when there is no road water accumulation, the preset wheel size, and the straight edge includes: Based on the pixel coordinates of the wheel edge in the wheel image when there is no road water, obtain the first pixel coordinate corresponding to the maximum ordinate, and the second pixel coordinate with the minimum ordinate located directly below the first pixel coordinate; Obtain the number of first pixel blocks between the first pixel coordinates and the second pixel coordinates, calculate the pixel block size based on the preset wheel size, and calculate the road water depth based on the pixel block size and the straight edge.

7. The vehicle natural environment detection cloud platform according to claim 6, characterized in that: The calculation of road water depth based on pixel block size and straight edge includes: Along the direction from the first pixel coordinate to the second pixel coordinate, obtain the number of second pixel blocks between the first pixel coordinate and the edge of the straight line; The depth of water accumulation on the road is calculated based on the number of second pixel blocks, the size of the pixel blocks, and the preset wheel size.

8. The vehicle natural environment detection cloud platform according to claim 1, characterized in that: The road background image extraction module extracts road background images from road videos, including: Road background images are extracted from road images using Gaussian mixture background modeling, and road background modeling is performed based on the road background images. Based on road background modeling, video analysis is performed on road videos to extract road background images.

9. The vehicle natural environment detection cloud platform according to claim 8, characterized in that: The method of extracting road background images from road images using Gaussian mixture background modeling includes: The pixel values ​​in the road image are matched with a Gaussian background distribution model. If the pixel value matches the Gaussian background distribution model, the pixel value belongs to the background; otherwise, the pixel value belongs to the foreground. Extract all pixel values ​​belonging to the background from the road image to obtain the road background image.

10. The vehicle natural environment detection cloud platform according to claim 9, characterized in that: The step of performing video analysis on road videos based on road background modeling and extracting road background images includes: The road video is decoded to obtain its timestamp. Based on the timestamp, a time switch is set to determine whether to use Gaussian mixture background modeling to extract the road background image from the road image.

Images