An apparatus and method for road detection and polarization imaging

Abstract

An apparatus and method for road detection and polarization imaging belong to the technical field of polarization imaging and detection. The present invention provides a vehicle-mounted high-definition polarization camera imaging system. By adding polarization elements and combining corresponding optical imaging elements, image acquisition is performed on the road surface in front of the vehicle during driving. Through the mutual cooperation among modules such as a monochromatic brightness information processing unit, a polarization ratio information processing unit, a white line detection unit, a road surface edge detection unit, a road surface shape estimation unit, a lane line search area determination unit, a lane line candidate point detection unit, and a lane line detection unit, etc., the polarization image of the road is detected, and brightness information is obtained to make the white line have a higher brightness level. Accurately identifying the white road lines on the road and guiding the vehicle's progress with the white road lines as a reference can make the recognition accuracy of the vehicle's driving road higher in rainy and snowy weather, and better play the role of vehicle-assisted safe driving.

Description

Technical Field

[0001] The present invention belongs to the technical field of polarization imaging and detection, and particularly relates to a device and method for road detection and polarization imaging. Background Art

[0002] With the acceleration of global road construction and the rapid improvement of the motorization level, the number of traffic accidents has been continuously rising. The frequent occurrence of traffic accidents not only causes economic losses in many aspects such as social security, treatment costs, and personnel disputes, but also threatens the safety of human life and property.

[0003] The main causes of road traffic accidents are: incorrect operation of drivers, failure of vehicle performance, and complex traffic environments. Among them, complex traffic environments are the primary cause of frequent traffic accidents. Especially in some adverse weather conditions (such as haze days, rain and snow days, cloudy days, etc.) and complex driving environments with variable road terrains, traffic accidents are more likely to occur. Therefore, how to effectively improve vehicle assisted safety driving technology and reduce the occurrence of traffic accidents is a hot topic that researchers are paying attention to.

[0004] Currently, there are various methods for vehicle assisted safety driving, but there is no report on using the accurate detection of white lines in the driving road image for vehicle assisted safety driving.

[0005] Therefore, there is an urgent need for a new technical solution in the existing technology to solve this problem. Summary of the Invention

[0006] The technical problem to be solved by the present invention is: to provide a device and method for road detection and polarization imaging to solve the technical problem that in the existing technology, vehicle assisted safety driving is not carried out by using the accurate detection of white lines in the driving road image.

[0007] A device for road detection and polarization imaging includes an in-vehicle high-definition polarization camera imaging system. The high-definition polarization camera imaging system includes a polarization imaging unit, a first polarization image memory, a second polarization image memory, and a display unit. The polarization imaging unit is used to obtain a first polarization image and a second polarization image of the vehicle driving road surface; the first polarization image memory is used to store the obtained first polarization image; the second polarization image memory is used to store the obtained second polarization image; the display unit displays each image;

[0008] A device for road detection and polarization imaging further includes a running road surface detection and recognition system. The running road surface detection and recognition system includes a white line detection unit, a road surface edge detection unit, a road surface shape estimation unit, a lane line search area determination unit, a lane line candidate point detection unit, a lane line detection unit, and a shape information storage unit;

[0009] The high-definition polarization camera imaging system further includes a monochromatic luminance information processing unit and a polarization ratio information processing unit;

[0010] The monochromatic luminance information processing unit is used to generate a monochromatic luminance image, and calculate and obtain the luminance information image data representing the luminance level of each pixel in the generated monochromatic luminance image based on the first polarization component and the second polarization component of the polarization image;

[0011] The polarization ratio information processing unit is used to obtain polarization ratio information, and the polarization ratio information processing unit calculates and obtains the corresponding polarization ratio information data of each pixel based on the first polarization image and the second polarization image;

[0012] The white line detection unit is used to detect and obtain the white line pixel information on the road surface from the luminance information image data obtained by the monochromatic luminance information processing unit;

[0013] The road surface edge detection unit scans the driving road surface with a light beam to obtain an image of the predicted road surface edge, segments the image on the same scan line, initially segments the image information of the road surface edge and the image information of the roadside buildings, and then further determines the image information of the road surface edge and the image information of the roadside buildings respectively through set threshold conditions and sends the corresponding image information data to the vehicle control unit for vehicle control;

[0014] The road surface shape estimation unit is used to estimate the shape of the road surface. The road surface shape estimation unit uses the polarization ratio information of each pixel to detect the planar alignment formed on the driving road surface, and separates the road surface from the roadside buildings on both sides of the road surface and at a certain angle to the road surface;

[0015] The lane line search area determination unit is used to determine the lane line search area under the estimated road surface shape;

[0016] The lane line candidate point detection unit is used to determine the lane line candidate points within the lane line search area;

[0017] The lane line detection unit is used to determine the lane lines within the lane line search area. When there are no lane lines in the lane line search area, the lane line detection unit is used to lower the polarization ratio threshold for detecting lane lines in the lane line search area;

[0018] The shape information storage unit is used to store the corresponding lane line information or road surface shape information in the images obtained by the white line detection unit, the road surface edge detection unit, and the road surface shape estimation unit, and synchronously display the corresponding information on the display unit in the form of an image for the driver to view, and also send it to the vehicle control unit for vehicle control.

[0019] A method for road detection and polarization imaging, using the device for road detection and polarization imaging, includes the following steps, and the following steps are carried out sequentially:

[0020] Step 1: The polarization imaging unit of the high-definition polarization camera imaging system receives the first polarized light and the second polarized light included in the reflected light from the objects in the imaging area, obtains the first polarization image and the second polarization image of the road surface in front of the vehicle, and stores them separately in the corresponding memories. The first polarized light and the second polarized light have different polarization directions;

[0021] Step 2: The monochromatic brightness information processing unit divides the first polarization image and the second polarization image into multiple processing areas in one-to-one correspondence, respectively obtains the brightness level characterization values of the first polarization image and the second polarization image in each processing area, and takes the sum of the brightness level characterization values of the first polarization image and the second polarization image in the same processing area as the combined brightness level value representing the processing area;

[0022] Step 3: The polarization ratio information processing unit obtains the polarization ratio information of the first polarization image and the second polarization image, and obtains the ratio of the difference between the brightness level characterization values of the first polarization image and the second polarization image in the same processing area to the combined brightness level value of the processing area;

[0023] Step 4: The white line detection unit extracts the values in the brightness level characterization values that satisfy the brightness level values representing the white line pixels, compares them with the set threshold, and determines the pixels corresponding to the values that meet the set threshold condition as the white line pixel information on the road surface and obtains the polarization ratio of the white line pixel information;

[0024] Step 5: The road surface edge detection unit uses a light beam to scan the driving road surface to obtain a predicted road surface edge image; according to the different polarization rates of the road surface and the roadside buildings, the image on the same scan line is segmented, and the image information of the road surface edge and the image information of the roadside buildings on the same scan line and their corresponding polarization ratio information are initially segmented and obtained;

[0025] Take the white line pixels obtained by the white line detection unit on the same scan line as reference pixels;

[0026] Compare the difference between the polarization ratio of the reference pixel information on the same scan line and the polarization ratio information of the road surface edge image with the set threshold. If the threshold condition is met, it is determined as the image data of the road surface edge and the obtained data is sent to the vehicle control unit for vehicle control;

[0027] Compare the difference between the polarization ratio of the reference pixel information on the same scan line and the polarization ratio information of the roadside building image with the set threshold. If the threshold condition is met, it is determined as the roadside building image data and the obtained data is sent to the display unit for display;

[0028] Step 6: Synchronously display the white line pixel information detected by the white line detection unit and the road edge image data detected by the road edge detection unit in the form of an image on the display unit for the driver to view;

[0029] Step 7: The road surface shape estimation unit uses the polarization ratio information obtained in Step 3 to detect the planar alignment formed on the driving road surface, and uses a set threshold to segment the road surface and the roadside buildings on both sides of the road surface and at a certain angle to the road surface, and obtains the estimated road surface shape, where the road surface shape includes the road surface inclination and width;

[0030] Step 8: The lane line search area determination unit obtains the lane line search area according to the road surface inclination and width in the estimated road surface shape;

[0031] Step 9: The lane line candidate point detection unit detects the lane lines on the road surface through the polarization ratio information within the obtained lane line search area, and the pixel points that meet the set threshold conditions are determined as lane line candidate points;

[0032] Step 10: The lane line detection unit determines whether the lane line candidate points are points on the lane line, and calculates the width of the line where each determined lane line point is located. If the calculated line width is within a predetermined threshold range, the line is determined as a lane line, and the polarization ratio information of each lane line candidate point located at the edge of the lane line is obtained;

[0033] If there are no lane line candidate points in the lane line search area or the width of the line where the lane line candidate points are located does not meet the set threshold, the lane line detection unit determines that no lane line is detected, and the lane line detection unit reduces the polarization ratio threshold used to detect the lane line in the lane line search area for the next detection;

[0034] The shape information storage unit stores the respective lane line information or road surface shape information in the images obtained by the white line detection unit, the road edge detection unit, and the road surface shape estimation unit, and synchronously displays the corresponding information in the form of an image on the display unit for the driver to view, and also sends it to the vehicle control unit for vehicle control.

[0035] The specific steps for segmenting the road surface and the roadside buildings on both sides of the road surface and at a certain angle to the road surface in Step 7 are: binarize the polarization ratio information obtained in Step 3, perform a labeling process on the binarized polarization ratio information according to the threshold of the predetermined parameters, and obtain continuous polarization ratio information with road surface characteristics, so as to obtain the estimated road surface shape, where the road surface shape includes the road surface inclination and width.

[0036] When the road edge detection unit fails to detect white line pixels, or when the white line pixels detected by the road edge detection unit end in the middle of the vehicle driving direction, the road edge detection unit determines the white line pixels detected in the previously generated polarization ratio information before this detection as reference pixels.

[0037] When the white line pixels detected in the previously generated polarization ratio information also end in the middle of the vehicle driving direction, the road edge detection unit uses the pixels on the extension line of the white line pixels that end in the middle as reference pixels;

[0038] When no white line pixels are detected in the previously generated polarization ratio information, the road edge detection unit uses the pixels located at the center of the road surface as reference pixels.

[0039] The specific method for the lane line detection unit in step ten to determine whether the lane line candidate points are on the lane line is as follows: The lane line detection unit divides the luminance image into an upper region and a lower region in the vehicle driving direction, sets different luminance thresholds for the upper region and the lower region, and compares the luminance level of each pixel in the upper region and the lower region with the corresponding luminance threshold one by one. If the comparison result meets the set threshold range, it is determined that the lane line candidate point is on the lane line.

[0040] Through the above design scheme, the present invention can bring the following beneficial effects:

[0041] The present invention provides an in-vehicle high-definition polarization camera imaging system. By adding a polarization element and combining corresponding optical imaging elements, it collects images of the road surface in front of the vehicle during driving, and through the mutual cooperation of various modules such as the monochromatic luminance information processing unit, the polarization ratio information processing unit, the white line detection unit, the road edge detection unit, the road surface shape estimation unit, the lane line search area determination unit, the lane line candidate point detection unit, and the lane line detection unit, it detects the polarization image of the road and obtains luminance information to make the white line have a higher luminance level. Accurately identifying the white road lines on the road and using the white road lines as a reference to guide the vehicle's progress can make the present invention have a higher recognition accuracy for the vehicle driving road in rainy, snowy, or foggy weather and better play the role of vehicle-assisted safe driving. Description of the Drawings

[0042] The following further describes the present invention in conjunction with the drawings and specific embodiments:

[0043] Figure 1 It is a block diagram of the road detection process of the device of the present invention;

[0044] Figure 2 It is a block diagram of the lane line detection process of the device of the present invention.

[0045] In the figure: 1-1 imaging unit, 3-1 first polarization image memory, 3-2 second polarization image memory, 3-3 monochromatic luminance information processing unit, 3-4 polarization ratio information processing unit, 3-5 white line detection unit, 3-6 shape information storage unit, 3-7 road surface edge detection unit, 1-2 display unit, 1-3 vehicle control unit, 4-1 road surface shape estimation unit, 4-2 lane line candidate point detection unit, 4-3 lane line search area determination unit, 4-4 lane line detection unit. Detailed implementation mode

[0046] To more clearly understand the purpose, structure, principle and function of the present invention, the following further describes in detail a device and method for road detection and polarization imaging of the present invention with reference to the attached pictures.

[0047] When light enters the interface between two substances with different refractive indices at a certain angle, the horizontal polarization component parallel to the incident plane (hereinafter referred to as the P component) and the vertical polarization component perpendicular to the incident plane (hereinafter referred to as the S component), the reflectivity of the P component is different from that of the S component. The P component decreases to zero at the Brewster angle and then increases. At the same time, the S component only increases. Since the P component and the S component have different reflection characteristics, the polarization ratio represented by the following formula (2) also changes according to the changes in the incident angle and the reflectivity.

[0048] Generally, the road surface is made of asphalt. At the same time, roadside buildings adjacent to the road surface and at a certain angle are made of materials different from asphalt, such as concrete, plants or soil. In addition, road lines such as white lines formed on the road surface are also made of materials different from asphalt.

[0049] Since different materials have different refractive indices, the polarization ratio of the road surface is different from that of the straight line or the roadside building. Different from the luminance difference, the difference in the polarization ratio is not greatly affected by the intensity of the incident light. Therefore, the boundary between the road surface and the roadside building can be detected by using the polarization ratio information, and the roadside building includes shoulders, plants or soil, etc.

[0050] The roadside building is located near the road surface and at a certain angle to the road surface. When the normal direction of the object surface is different, the incident angle from the light source to the object and the reflection angle of the light from the object to the camera will also be different. Therefore, the polarization ratio between the road surface and the adjacent roadside building will be different. The polarization rate is obtained by adding the P component and the S component and subtracting the P component and the S component. Therefore, even in a dark environment with a small luminance difference, the road surface edge can be detected by using the polarization ratio information.

[0051] This also indicates that the road shoulder, which is the edge of the roadside structure adjacent to the road surface, can be detected by using polarization ratio information. This method particularly improves the accuracy of detecting the edge of the roadside structure because, between the road surface and the roadside structure, there are polarization ratio differences not only due to material differences but also due to angular differences.

[0052] Therefore, using polarization ratio information, the boundaries between the road surface and the lane lines, as well as between the road surface and the roadside structure, can be detected based on material and angular differences.

[0053] Figure 1 It is a flowchart of the road detection process performed by the vehicle-mounted imaging system implemented according to the present invention. The high-definition polarization camera imaging system is installed on the vehicle. The imaging unit 1-1 captures the scene image in the driving direction in front of the vehicle on the road where the vehicle is traveling, and obtains the vertical polarization component and the horizontal polarization component as the original polarization image data. The obtained horizontal polarization image data is stored in the first polarization image memory 3-1, and the obtained vertical polarization image data is stored in the second polarization image memory 3-2. The horizontal polarization image data and the vertical polarization image data are sent to the monochromatic luminance information processing unit 3-3 for monochromatic luminance information processing, and the luminance information characterizing the pixel luminance level of the generated monochromatic luminance image is calculated and obtained. The horizontal polarization image data and the vertical polarization image data are also sent to the polarization ratio information processing unit 3-4, used for generating polarization ratio information, and the polarization ratio information of the polarization image pixels is calculated and obtained based on the P component and the S component.

[0054] The polarization ratio information processing unit 3-4 calculates and obtains the polarization ratio information data indicating the polarization ratio by using the following formula (1). The polarization rate represents the ratio between the polarization components and can be calculated by using the following formula (2).

[0055] Polarization ratio = P component / S component (1)

[0056] Polarization rate = (P component - S component) / (P component + S component) (2)

[0057] A wet road condition behaves like a mirror. Therefore, the reflected light from the wet road surface has polarization characteristics. When the reflectance of the vertical polarization component is defined as Rs and the reflectance of the horizontal polarization component is defined as Rp, the intensities Is and Ip of the reflected light beams with an incident light intensity of I satisfy the following relationships:

[0058] Is = Rs * I (3)

[0059] Ip = Rp * I (4)

[0060] Where, Is is the intensity of the reflected light of the vertical polarization component; Ip is the intensity of the reflected light of the horizontal polarization component;

[0061] When the incident angle of the reflected light is equal to the Brewster angle, the horizontal polarization component of the reflected light on the mirror is zero. The vertical polarization component of the reflected light is characterized in that the intensity of the reflected light gradually increases as the incident angle increases.

[0062] Since the road surface is rough when dry, specular reflection dominates. Therefore, the reflected light does not have polarization characteristics, and thus, the reflected light intensities of each polarization component are almost equal (i.e., Rs = Rp). Therefore, based on the polarization characteristics, moisture information on the road surface can be obtained from the luminance information of the horizontal polarization image and the vertical polarization image. Specifically, the ratio of the reflected light intensities is the ratio of the reflected light intensity Is of the vertical polarization component to the reflected light intensity Ip of the horizontal polarization component, that is, the image luminance ratio H is as follows:

[0063] H = Is / Ip = Rs / Rp (5)

[0064] The luminance average value of the image luminance ratio H is obtained, and the wet state of the road surface is determined according to the magnitude of the average value. For example, when the road surface is dry, the reflected light intensity Is of the vertical polarization component and the reflected light intensity Ip of the horizontal polarization component are basically the same, so the luminance ratio H is about 1. On the contrary, when the road surface is completely wet, the reflected light intensity Ip of the horizontal polarization component is much greater than the reflected light intensity Is of the vertical polarization component, so the luminance ratio is large. In addition, when the road surface is slightly wet, the luminance ratio H is between the above cases. Therefore, the wet state of the road surface can be determined according to the value of the luminance ratio H.

[0065] The white line detection unit 3-5 serves as a line detection unit and detects the white lines on the road surface based on the luminance information calculated by the monochromatic luminance information processing unit 3-3.

[0066] The road surface edge detection unit 3-7 detects and differentiates the road surface edge and roadside buildings based on the white line information obtained by the white line detection unit 3-5 and the polarization ratio information obtained by the polarization ratio information processing unit 3-4.

[0067] The road surface edge detection unit 3-7 scans the driving road surface with a light beam, segments the images on the same scan line according to the different polarization rates of the road surface and roadside buildings, and obtains the image information of the road surface edge and roadside buildings on the same scan line and their corresponding polarization ratio information;

[0068] The difference between the polarization ratio of the white line pixel information on the same scan line and the polarization ratio information of the road surface edge image is compared with a set threshold, and the data that meets the threshold condition is determined as the image data of the road surface edge and the obtained data is sent to the vehicle control unit for vehicle control;

[0069] Compare the difference between the polarization ratio of the white line pixel information on the same scan line and the polarization ratio information of the roadside building image with a set threshold. If the threshold condition is met, it is determined as the image data of the roadside building, and the obtained data is sent to the display unit 1-2 for display.

[0070] The white line detected by the white line detection unit 3-5 and the road surface edge detected by the road surface edge detection unit 3-7 are displayed on the display unit 1-2, and the display unit 1-2 is provided with a CRT or a liquid crystal display to achieve a way that is convenient for the driver to view. The data obtained by the road surface edge detection unit 3-7 can be sent to the vehicle control unit 1-3 for vehicle control. The shape information storage unit 3-6 receives the information sent by the road surface edge detection unit 3-7 and stores it.

[0071] The first polarization image memory 3-1, the second polarization image memory 3-2, the monochromatic luminance information processing unit 3-3, the polarization ratio information processing unit 3-4, the white line detection unit 3-5, and the road surface edge detection unit 3-7 constitute the image processing unit.

[0072] According to the method described later, detect the edge of the white line based on the obtained luminance information. Set the polarization ratio of the pixels within the detected white line, that is, the reference pixels, as the reference polarization ratio for scanning, and scan the polarization image based on the reference polarization ratio. Each pixel of the polarization image has a polarization ratio. The road surface edge detection unit 3-7 uses a light beam for scanning. Each row of pixels processed and generated by the polarization ratio information processing unit 3-4 is also called a scan line. The scan line represents the horizontal pixel row to be scanned by the electron beam on the display, and the pixel row goes from the left end to the right end.

[0073] The pixels on each scan line are processed in the left-right direction order. Compare the polarization ratio of the pixels on the same scan line with the corresponding reference polarization ratio as the reference pixels. If the difference between the polarization ratio of the pixel and the reference polarization ratio is less than a predetermined threshold, process the next pixel on the same scan line. At the same time, if the difference is greater than or equal to the threshold, detect the pixel as a road surface edge point.

[0074] In this example, the polarization ratio of the pixels within the white line, that is, the reference pixels, is used as the reference polarization ratio for scanning to reduce the influence of shadows generated by objects such as the vehicle in front. The roadside buildings also need to be prevented from being misrecognized as road surface edges. Alternatively, the polarization ratio of the pixels located at the center of each scan line, that is, the center of the polarization image, can be used as the reference polarization ratio to detect the road surface edge. When detecting the white line edge and the road surface edge, the scan line scans from the more reliable bottom of the image to the top of the image or in the x-axis direction or the vertical direction of the screen.

[0075] After detecting the points indicating the edges of the white line and the points indicating the edge of the road surface in a screen or an image, the approximate curves of the white line edge points and the road surface edge points are obtained by shape approximation. The approximate curves are obtained by the road surface edge detection unit 3-7, which also serves as an approximate curve acquisition unit. For example, the least squares method, the Hough transform, or a model equation can be used for shape approximation. When obtaining the approximate curves by shape approximation, higher weights are given to the reliable white line edge points and road surface edge points detected in the lower part of the road image or the screen. In this way, even if incorrect white line edge points and road surface edge points are detected in the upper region of the road image, as long as the lane line edge points are correctly detected in the lower region of the road image, the lane lines can be appropriately identified.

[0076] When detecting the white line and the road surface edge in real time, if similar white lines and similar edges are found in one or more previously acquired images or polarization images, it is determined that the detected white line and road surface edge are reliable. According to the position of the white line in the previous frame, the white line edge and the road surface edge are searched for and lines are drawn in the next frame. If the positions of the white line edge and the road surface edge are not detected in five frames of images, the search starts again from the center of the scan line below the image.

[0077] As Figure 2 shown, the flowchart of the lane line detection process performed by the vehicle-mounted imaging system according to the present invention is as follows:

[0078] The imaging unit 1-1 captures a scene image in the driving direction in front of the vehicle and obtains the vertical polarization component and the horizontal polarization component as the original polarization image data. The obtained horizontal polarization image data is stored in the first polarization image memory 3-1, and the obtained vertical polarization image data is stored in the second polarization image memory 3-2. The horizontal polarization image data and the vertical polarization image data are sent to the monochromatic luminance information processing unit 3-3 for monochromatic luminance information processing, and the luminance information characterizing the pixel luminance level of the generated monochromatic luminance image is calculated and obtained. The horizontal polarization image data and the vertical polarization image data are also sent to the polarization ratio information processing unit 3-4 for use in polarization ratio information generation, and the polarization ratio information of the polarization image pixels is calculated and obtained based on the P component and the S component.

[0079] The polarization ratio information processing unit 3-4 calculates the polarization ratio information using formula (1), thereby obtaining the polarization ratio information data. The polarization rate represents the ratio between the polarization components, can be calculated using formula (2), and the luminance information image data is generated and output using formula (3).

[0080] The road surface shape estimation unit 4-1 uses the polarization ratio information of the obtained polarization image to detect the planar alignment formed on the driving road surface, and uses a set threshold to segment the road surface and roadside buildings located on both sides of the road surface and at a certain angle to the road surface, so as to obtain the estimated road surface shape, where the road surface shape includes the road surface inclination and width;

[0081] The lane line candidate point detection unit 4-2 detects candidate points of possible lane line edges, also known as lane line candidate points, based on the polarization ratio information. The lane line can represent any type of line (such as a solid line, a dotted line or a double line) of any color (such as a white line or a yellow line) that separates a road or a tram lane. The lane line detection unit 4-4 detects the lane lines on the road surface based on the polarization ratio information. The ordinary asphalt road surface is black, and white lines are formed on the black road surface. The polarization ratio of the white line is close to zero. Therefore, the polarization ratio of the white line is sufficiently smaller than that of other parts of the road, and the white line can be detected by determining the road with a polarization ratio less than or equal to a predetermined value.

[0082] The lane line search area determination unit 4-3 obtains the lane line search area according to the road surface inclination and width in the estimated road surface shape;

[0083] Calculate the lane line width based on the detected lane line candidate points, and determine whether the calculated white line width is within a predetermined range. If the calculated white line width is within the predetermined range, the lane line candidate points are determined as the white line edges on the road surface. Since the polarization ratio contrast between the lane line in the upper part of the image and other parts of the road surface is different from the polarization ratio contrast between the lane line in the lower part of the image and other parts of the road surface. Therefore, a frame of image is divided into an upper area and a lower area, and different polarization ratio thresholds are set for the upper area and the lower area in the step of setting the polarization ratio threshold;

[0084] If the lane line edges are not detected in five frames of images, the search starts again from the center of the scan line in the lower part of the image. If the lane line edges are successfully detected, the corresponding lane line information and road surface shape information in the obtained image are stored in the shape information storage unit 3-6, and the corresponding information is synchronously displayed in image form on the display unit 1-2 for the driver to view, and is also sent to the vehicle control unit 1-3 for vehicle control.

[0085] It can be understood that the present invention is described through some embodiments. Those skilled in the art know that without departing from the spirit and scope of the present invention, various changes or equivalent replacements can be made to these features and embodiments. In addition, under the teaching of the present invention, these features and embodiments can be modified to adapt to specific situations and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application belong to the scope protected by the present invention.

Claims

1. A device for road detection and polarization imaging, comprising a vehicle-mounted high-definition polarization camera imaging system, the high-definition polarization camera imaging system including a polarization imaging unit, a first polarization image memory, a second polarization image memory, and a display unit, the polarization imaging unit being configured to acquire a first polarization image and a second polarization image of the road surface on which the vehicle travels; the first polarization image memory being configured to store the acquired first polarization image; the second polarization image memory being configured to store the acquired second polarization image; the display unit displaying each image; characterized in that: It further includes a runway pavement detection and recognition system, and the runway pavement detection and recognition system includes a white line detection unit, a pavement edge detection unit, a pavement shape estimation unit, a lane line search area determination unit, a lane line candidate point detection unit, a lane line detection unit, and a shape information storage unit; The high-definition polarization camera imaging system further includes a monochromatic luminance information processing unit and a polarization ratio information processing unit; The monochromatic luminance information processing unit is used to generate a monochromatic luminance image, and calculate and obtain the luminance information image data representing the luminance level of each pixel in the generated monochromatic luminance image based on the first polarization component and the second polarization component of the polarization image; The polarization ratio information processing unit is used to obtain polarization ratio information, and the polarization ratio information processing unit calculates and obtains the corresponding polarization ratio information data of each pixel based on the first polarization image and the second polarization image; The white line detection unit is used to detect and obtain the white line pixel information on the pavement from the luminance information image data obtained by the monochromatic luminance information processing unit; The pavement edge detection unit scans the driving pavement with a light beam to obtain an image of the predicted pavement edge, segments the images on the same scan line, initially segments the image information of the pavement edge and the image information of the roadside buildings, and then further determines the image information of the pavement edge and the image information of the roadside buildings respectively through set threshold conditions and sends the corresponding image information data to the vehicle control unit for vehicle control; The pavement shape estimation unit is used to estimate the shape of the pavement. The pavement shape estimation unit uses the polarization ratio information of each pixel to detect the plane alignment formed on the driving pavement, and separates the pavement from the roadside buildings on both sides of the pavement and at a certain angle to the pavement; The lane line search area determination unit is used to determine the lane line search area under the estimated pavement shape; The lane line candidate point detection unit is used to determine the lane line candidate points in the lane line search area; The lane line detection unit is used to determine the lane lines in the lane line search area. When there are no lane lines in the lane line search area, the lane line detection unit is used to lower the polarization ratio threshold for detecting lane lines in the lane line search area; The shape information storage unit is used to store the corresponding lane line information or pavement shape information in the images obtained by the white line detection unit, the pavement edge detection unit, and the pavement shape estimation unit, and synchronously display the corresponding information on the display unit in the form of an image for the driver to view, and also send it to the vehicle control unit for vehicle control.

2. A method for road detection and polarization imaging, which uses the device for road detection and polarization imaging according to claim 1, and is characterized in that: It includes the following steps, and the following steps are carried out sequentially: Step 1: Receive the first polarized light and the second polarized light included in the reflected light of the object in the imaging area through the polarization imaging unit of the high-definition polarization camera imaging system, and obtain the first polarization image and the second polarization image of the pavement in front of the vehicle and store them in the corresponding memories respectively. The first polarized light and the second polarized light have different polarization directions; Step 2: The monochromatic brightness information processing unit divides the first polarization image and the second polarization image into multiple processing regions one by one, respectively obtains the brightness level representation values of the first polarization image and the second polarization image in each processing region, and takes the sum of the brightness level representation values of the first polarization image and the second polarization image in the same processing region as the combined brightness level value representing the processing region; Step 3: The polarization ratio information processing unit obtains the polarization ratio information of the first polarization image and the second polarization image, and obtains the ratio of the difference between the brightness level representation values of the first polarization image and the second polarization image in the same processing region to the combined brightness level value of the processing region; Step 4: The white line detection unit extracts the values in the brightness level representation values that satisfy the brightness level values representing white line pixels and compares them with a set threshold. The pixels corresponding to the values that meet the threshold condition are determined as the white line pixel information on the road surface, and the polarization ratio of the white line pixel information is obtained; Step 5: The road surface edge detection unit uses a light beam to scan the driving road surface to obtain a predicted road surface edge image; according to the different polarization rates of the road surface and the roadside buildings, the image on the same scan line is segmented, and the image information of the road surface edge and the image information of the roadside buildings on the scan line and their corresponding polarization ratio information are initially segmented and obtained; Take the white line pixels obtained by the white line detection unit on the same scan line as reference pixels; Compare the difference between the polarization ratio of the reference pixel information on the same scan line and the polarization ratio information of the road surface edge image with a set threshold. The image data that meets the threshold condition is determined as the road surface edge and the obtained data is sent to the vehicle control unit for vehicle control; Compare the difference between the polarization ratio of the reference pixel information on the same scan line and the polarization ratio information of the roadside building image with a set threshold. The image data that meets the threshold condition is determined as the roadside building image data and the obtained data is sent to the display unit for display; Step 6: Synchronously display the white line pixel information detected by the white line detection unit and the road surface edge image data detected by the road surface edge detection unit in the form of an image on the display unit for the driver to view; Step 7: The road surface shape estimation unit uses the polarization ratio information obtained in Step 3 to detect the plane alignment formed on the driving road surface, and uses a set threshold to segment the road surface and the roadside buildings on both sides of the road surface and at a certain angle to the road surface, and obtains the estimated road surface shape, where the road surface shape includes the road surface inclination and width; Step 8: The lane line search area determination unit obtains the lane line search area according to the road surface inclination and width in the estimated road surface shape; Step 9: The lane line candidate point detection unit detects the lane lines on the road surface through the polarization ratio information in the obtained lane line search area, and the pixel points that meet the set threshold conditions are determined as lane line candidate points; Step 10: The lane line detection unit determines whether the lane line candidate points are points on the lane line, calculates the width of the line where each determined lane line point is located, and if the calculated line width is within a predetermined threshold range, determines the line as a lane line and obtains the polarization ratio information of each lane line candidate point at the edge of the lane line; If there are no lane line candidate points in the lane line search area or the width of the line where the lane line candidate points are located does not meet the set threshold, the lane line detection unit determines that no lane line is detected, and the lane line detection unit reduces the polarization ratio threshold used to detect the lane line in the lane line search area for the next detection; The shape information storage unit stores the corresponding lane line information or road surface shape information in the image obtained by the white line detection unit, the road surface edge detection unit, and the road surface shape estimation unit, and synchronously displays the corresponding information on the display unit in the form of an image for the driver to view, and also sends it to the vehicle control unit for vehicle control.

3. A method for road detection and polarization imaging according to claim 2, characterized in that: The specific steps of separating the road surface and roadside buildings on both sides of the road surface and forming a certain angle with the road surface in Step 7 are as follows: Binarize the polarization ratio information obtained in Step 3, perform a marking process on the binarized polarization ratio information according to the threshold of a predetermined parameter to obtain continuous polarization ratio information with road surface characteristics, so as to obtain the estimated road surface shape, where the road surface shape includes the road surface inclination and width.

4. A method for road detection and polarization imaging according to claim 2, characterized in that: Under the condition that the road surface edge detection unit does not detect white line pixels, or under the condition that the road surface edge detection unit detects that the white line pixels end in the middle of the vehicle driving direction, the road surface edge detection unit determines the white line pixels detected in the polarization ratio information generated previously before this detection as reference pixels.

5. A method for road detection and polarization imaging according to claim 4, characterized in that: Under the condition that the white line pixels detected in the previously generated polarization ratio information also end in the middle of the vehicle driving direction, the road surface edge detection unit uses the pixels of the extension line of the white line pixels ending in the middle as reference pixels; Under the condition that no white line pixels are detected in the previously generated polarization ratio information, the road surface edge detection unit uses the pixels located at the center of the road surface as reference pixels.

6. A method for road detection and polarization imaging according to claim 2, characterized in that: The specific method for the lane line detection unit in Step 10 to determine whether the lane line candidate points are points on the lane line is as follows: The lane line detection unit divides the luminance image into an upper region and a lower region in the vehicle driving direction, sets different luminance thresholds for the upper region and the lower region, and compares the luminance level of each pixel in the upper region and the lower region with the corresponding luminance threshold one by one. If the comparison result meets the set threshold range, it is determined that the lane line candidate point is a point on the lane line.

Images