A method and system for measuring shoulder height considering the influence of vehicle body pitch angle

Abstract

The present invention is a method and system for measuring the height of a shoulder that takes into account the influence of the pitch angle of a vehicle body. The method comprises: calibrating the camera to obtain internal and external parameters, identifying the lower edge and upper edge of the shoulder, obtaining the pixel difference of the shoulder in the vertical direction on the image plane through a multi-window anchor point detection method, and calculating the distance between the camera and the shoulder and the height of the shoulder according to the real-time pitch angle of the vehicle and the camera height. The system comprises a CAN bus information acquisition unit for the pitch angle of the vehicle body relative to the earth, as well as an on-board monocular camera, a shoulder test point calculation unit based on a multi-window anchor point detection method, an image recognition distance data and coordinate system calibration data and shoulder pixel coordinate data processing unit, a shoulder pixel coordinate and height information storage unit, and the on-board camera is installed at the center grid with an angle of γ with the horizontal plane. The present invention can improve the accuracy of shoulder height measurement, thereby achieving the purpose of preventing vehicles from colliding with shoulders and providing information for special vehicles to determine whether they can pass through shoulders.

Description

Technical Field

[0001] The invention relates to a method and system for measuring the height of a road shoulder taking into account the influence of the pitch angle of a vehicle body. Background Art

[0002] With the rapid development of the economy, material living conditions have greatly improved. As of the end of March 2022, my country's car ownership reached 307 million. While cars bring convenience, they also bring frequent traffic accidents, causing immeasurable economic and property losses and casualties. Among the numerous traffic accidents, the scratches and collisions between the car body and the shoulder trouble many drivers. The main reason is that the driver's field of vision is limited. Due to the visual blind spot, the position of the shoulder cannot be estimated. Secondly, the driver lacks experience and cannot accurately estimate the height of the shoulder to judge the vehicle's passability.

[0003] In addition, some special vehicles need to cross the shoulders, steps, etc. in front of the vehicle when operating. In order to ensure the safety of their special vehicles crossing obstacles, the height of the shoulders and steps in front of the vehicle needs to be accurately measured, and the driver determines whether the conditions for passing are met based on the measurement results.

[0004] At present, laser radar can accurately obtain the three-dimensional coordinate point cloud of objects to achieve the purpose of accurate shoulder height measurement, but the laser radar equipment has disadvantages such as high cost and long processing time. Height measurement based on monocular vision is relatively low in cost and fast in speed. The height measurement model that considers the influence of the vehicle body pitch angle and adaptive weighting can quickly and accurately measure the height of the shoulder. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a method and system for measuring the height of a road shoulder taking into account the influence of the pitch angle of a vehicle body, so as to accurately measure the height of the road shoulder.

[0006] The present invention solves the technical problem by adopting the following technical solutions:

[0007] The present invention provides a method for measuring the height of a shoulder that takes into account the influence of the pitch angle of a vehicle body. Specifically, the method comprises the following steps: after a vehicle-mounted monocular camera is calibrated by a vehicle-mounted monocular camera calibration method that takes into account vehicle suspension, the upper and lower edges of the shoulder are firstly identified by a shoulder edge detection algorithm, and the upper and lower edge anchor points of the shoulder are selected by a multi-window anchor point detection method. According to relevant information, a real-time distance measurement method that takes into account the pitch angle is used to measure the distance between the shoulder and the vehicle-mounted monocular camera. The relevant information includes internal camera parameters, camera height, and vehicle pitch angle obtained after the vehicle-mounted monocular camera is calibrated. The height of the shoulder is then measured by a height measurement model based on adaptive weighting.

[0008] The road shoulder edge detection algorithm comprises the following steps:

[0009] Step 1. Calibrate the on-board monocular camera installed in front of the vehicle to obtain the internal and external parameters of the camera. The on-board monocular camera calibration method considering the vehicle suspension is adopted. The conversion relationship from the on-board monocular camera to the vehicle suspension coordinate system is:

[0010]

[0011] In the above formula: X c ,Y c ,Z c is the coordinate system of the on-board monocular camera; β is the vehicle roll angle; θ is the vehicle pitch angle; α is the vehicle sideslip angle; X v ,Y v ,Z v is the vehicle suspension coordinate system;

[0012] Step 2: Perform Canny operator edge detection on the acquired image to obtain the edge points of the shoulder. The Canny edge detection algorithm first removes noise through Gaussian filtering, and then calculates the grayscale gradient direction of the image to find the most likely location of the edge.

[0013] Step 3: Apply Hough transform to the edge detection image to obtain the upper edge, lower edge and rear edge of the shoulder.

[0014] The multi-window anchor point detection method automatically generates 6 square windows according to the width of the entire image. Each window detects the intersection of the vertical line and the upper and lower edges of the shoulder. The width calculation formula of a single square window is:

[0015] m i =p sum / 6

[0016] Where: m i ,i=1,2,…,6 is the width of a single square window; p sum The width of the entire image, in pixels.

[0017] The real-time distance measurement method considering the pitch angle comprises the following steps:

[0018] Step 1: According to the imaging principle of the vehicle-mounted monocular camera and the relationship between the vehicle-mounted monocular camera and the ground plane, a geometric model of the distance between the image coordinate system and the earth coordinate system is established;

[0019] Step 2: According to the geometric model, the relationship expression of β is:

[0020] ε=β+γ+θ

[0021]

[0022]

[0023] The relationship between pixel coordinates and world coordinates is expressed as:

[0024]

[0025] Step 3: According to the transformation relationship between the image coordinate system and the pixel coordinate system, we can get:

[0026] x=(u-u0)dx

[0027] y=(v-v0)dy

[0028] O2B1 2 =x 2 +y 2 =[(u-u0)dx] 2 +[(v-v0)dy] 2 ;

[0029] Step 4. According to The distance calculation formula between the road shoulder and the vehicle-mounted monocular camera is obtained:

[0030]

[0031]

[0032] In the above formula: b1 represents the intersection of the bottom edge of the measured shoulder and the road surface; γ is the initial pitch angle of the vehicle-mounted monocular camera when it is installed in front of the vehicle, h is the height from the ground; f is the focal length of the vehicle-mounted monocular camera, f x 、f y are the components of the focal length in the x and y axis directions of the phase plane respectively; θ is the real-time pitch angle when the vehicle is moving; β is the angle between the projection of the on-board monocular camera and the optical axis; (u0, v0) is the position of the origin of the image coordinate system in the pixel coordinate system; L is the horizontal distance from the measured point b1 to the lens; O is the intersection of the optical axis and the image plane, which serves as the origin of the image coordinate system; O1 is the center of the on-board monocular camera lens; O2 is the focus of the optical axis and the phase plane after the pitch angle of the vehicle changes; B1(x, y) is the projection of the measured point on the image plane; O2B1 is the distance between the projection point of the optical axis on the image plane and the projection point of the target point on the image plane.

[0033] The present invention can adopt a height measurement method based on adaptive weighting to fuse the measurement values ​​of the shoulder calculated by each square window, and the height relationship formula is:

[0034]

[0035] The shoulder height fusion calculation method is as follows:

[0036]

[0037]

[0038] The total mean square error is:

[0039]

[0040] The minimum total mean square error is:

[0041]

[0042] Its corresponding weighting factor w i for:

[0043]

[0044] In the above formula: b1b2 is the length between the lower edge and the upper edge of the shoulder; y1y2 is the pixel difference of the shoulder in the vertical direction on the image plane; f y is the component of focal length in the y-axis direction of the phase plane; L is the distance between the road shoulder and the on-board monocular camera; H i is the shoulder height measured by a single square window; H is the shoulder height after fusion; w i , i = 1, 2, ..., 6 is the weight coefficient of the shoulder height measured by the corresponding square window; H0 is the shoulder pixel coordinate and the shoulder height measured at the previous moment stored in the height information storage unit; The variance of shoulder height measurement corresponding to each square window.

[0045] The present invention provides a shoulder height measurement system that takes into account the influence of the pitch angle of a vehicle body, which includes a CAN bus information acquisition unit for the pitch angle of a vehicle body relative to the ground, as well as an on-board monocular camera, a shoulder test point calculation unit based on a multi-window anchor point detection method, an image recognition distance data and coordinate system calibration data and shoulder pixel coordinate data processing unit, and a shoulder pixel coordinate and height information storage unit, wherein: the above five units are all mounted on a moving vehicle, and the image recognition distance data and coordinate system calibration data and shoulder pixel coordinate data processing unit measures and stores the height of the shoulder at the current moment based on the real-time vehicle pitch angle acquired by the CAN bus information acquisition unit for the pitch angle of the vehicle body relative to the ground, the anchor points of the upper and lower edges of the shoulder in the picture calculated by the shoulder test point calculation unit based on the multi-window anchor point detection method, the shoulder pixel coordinates, and the shoulder height at the previous moment stored in the height information storage unit.

[0046] The CAN bus information acquisition unit for the pitch angle of the vehicle body relative to the ground is mainly composed of a tilt sensor, a CAN interface, a power interface, and a vehicle-mounted shoulder height information output terminal, wherein the tilt sensor is connected to the CAN interface to transmit the vehicle pitch angle information.

[0047] The shoulder test point calculation unit based on the multi-window anchor point detection method is mainly composed of an aviation plug, a microprocessor, a CAN interface, and a power interface, wherein: the aviation plug is connected to the vehicle-mounted monocular camera, receives the collected image data and transmits it to the microprocessor, and after the microprocessor completes the calculation, it transmits the information to the image recognition distance data and coordinate system calibration data and the shoulder pixel coordinate data processing unit through the CAN interface.

[0048] The image recognition distance data, coordinate system calibration data and shoulder pixel coordinate data processing unit is mainly composed of a microprocessor, a CAN interface and a power supply interface, wherein the microprocessor receives the information required for calculation and outputs the calculation results through the CAN interface.

[0049] The shoulder pixel coordinate and height information storage unit is mainly composed of RAM memory, CAN interface, power interface and USB output interface, wherein: the CAN interface receives the calculation results of the system and transfers them to the RAM memory for storage; the USB output interface is connected to the vehicle-mounted shoulder height information output terminal to display the shoulder height information to the driver.

[0050] Compared with the prior art, the present invention has the following main advantages:

[0051] The present invention adopts the monocular vision of a camera, and the cost is greatly reduced compared to the shoulder detection of a laser radar.

[0052] The influencing factors of the vehicle in the driving state are fully considered, the real-time pitch angle of the vehicle is considered in the calculation process, and a multi-window anchor point detection method and an adaptive weighted height measurement method are proposed. Multi-point detection and fusion are performed on the shoulder in front of the vehicle, which can effectively improve the detection accuracy and fault tolerance. In this embodiment, the average relative error is only 4.32%, and the maximum relative error is 8.43%.

[0053] The system provided by the present invention can quickly and accurately detect the height of the shoulder of the road, and provide the driver with accurate shoulder height information in real time through the vehicle-mounted shoulder height information output terminal in the CAN bus information acquisition unit of the pitch angle of the vehicle body relative to the ground, thereby effectively reducing the risk of vehicle scratches and collisions, and also providing protection for the safe passage of special vehicles. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 A flowchart of a method and system for measuring shoulder height taking into account the influence of vehicle body pitch angle of the present invention;

[0055] Figure 2 It is a schematic diagram of the road shoulder detection result of the present invention;

[0056] Figure 3It is a schematic diagram of the multi-window anchor point detection method of the present invention;

[0057] Figure 4 A schematic diagram of a real-time distance measurement model considering pitch angles of the present invention;

[0058] Figure 5 A schematic diagram of a height measurement model based on adaptive weighting according to the present invention;

[0059] Figure 6 A schematic diagram of the information interaction relationship between the various units in the system of the present invention;

[0060] Figure 7 Graph showing experimental results in an embodiment of the present invention. DETAILED DESCRIPTION

[0061] The present invention discloses a method and system for measuring the height of a shoulder taking into account the influence of the pitch angle of a vehicle body. The method comprises: obtaining internal and external parameters by calibrating a camera, identifying the lower edge and upper edge of the shoulder, obtaining the pixel difference of the shoulder in the vertical direction on the image plane by a multi-window anchor point detection method, and calculating the distance between the camera and the shoulder and the height of the shoulder according to the real-time pitch angle of the vehicle and the camera height. The system mainly consists of a CAN bus information acquisition unit for the pitch angle of the vehicle body relative to the earth, a vehicle-mounted monocular camera, a shoulder test point calculation unit based on a multi-window anchor point detection method, an image recognition distance data and coordinate system calibration data and a shoulder pixel coordinate data processing unit, and a shoulder pixel coordinate and height information storage unit, wherein: the vehicle-mounted camera is installed at the center grid with an angle of γ with the horizontal plane. The present invention focuses on exploring the influence of the pitch angle of the vehicle body on the measurement of the shoulder height, which can effectively improve the accuracy of the shoulder height measurement, and achieve the purpose of preventing vehicles from colliding with the shoulder and providing information for special vehicles to determine whether they can pass through the shoulder.

[0062] The present invention is further explained and illustrated below in conjunction with the embodiments and drawings, but the present invention is not limited thereto.

[0063] The present invention provides a road shoulder height measurement system that takes into account the influence of the vehicle body pitch angle, and its structure is as follows: Figure 1 and Figure 6As shown, it includes a CAN bus information acquisition unit for the pitch angle of the vehicle body relative to the ground, as well as an on-board monocular camera, a shoulder test point calculation unit based on a multi-window anchor point detection method, an image recognition distance data and coordinate system calibration data and shoulder pixel coordinate data processing unit, and a shoulder pixel coordinate and height information storage unit, wherein: the on-board monocular camera is connected to the shoulder test point calculation unit based on the multi-window anchor point detection method to transmit image information; the shoulder test point calculation unit based on the multi-window anchor point detection method, the CAN bus information acquisition unit for the pitch angle of the vehicle body relative to the ground, and the shoulder pixel coordinate and height information storage unit transmit information to the image recognition distance data and coordinate system calibration data and shoulder pixel coordinate data processing unit respectively, wherein the transmitted information is the shoulder anchor point information, the vehicle pitch angle information and the shoulder height information measured at the previous moment; the image recognition distance data and coordinate system calibration data and shoulder pixel coordinate data processing unit calculates the shoulder height at the current moment according to the above information.

[0064] The CAN bus information acquisition unit for the pitch angle of the vehicle body relative to the ground is installed under the center console of a moving vehicle. It is mainly composed of a tilt sensor, a CAN interface, a power interface, and an on-board shoulder height information output terminal. It is used to obtain the pitch angle of the vehicle body and display the shoulder height information to the driver, wherein the tilt sensor is connected to the CAN interface to transmit the vehicle pitch angle information.

[0065] The shoulder test point calculation unit based on the multi-window anchor point detection method is installed inside the center grille of a moving vehicle. It is mainly composed of an aviation plug, a microprocessor, a CAN interface, and a power interface. It is used to process image data and calculate the test points on the shoulder edge line, wherein: the aviation plug is connected to the vehicle-mounted monocular camera, receives the collected image data and transmits it to the microprocessor. After the microprocessor completes the calculation, it transmits the information to the image recognition distance data and coordinate system calibration data and shoulder pixel coordinate data processing unit through the CAN interface.

[0066] The image recognition distance data, coordinate system calibration data and shoulder pixel coordinate data processing unit is installed under the dashboard of a moving vehicle. It is mainly composed of a microprocessor, a CAN interface, and a power interface. It is used to calculate the internal and external parameters of the on-board monocular camera, the distance between the on-board monocular camera and the shoulder, and the height of the shoulder. The microprocessor receives the information required for the calculation and outputs the calculation results through the CAN interface.

[0067] The shoulder pixel coordinate and height information storage unit is installed under the dashboard of a moving vehicle. It is mainly composed of a RAM memory, a CAN interface, a power interface and a USB output interface, and is used to store the various results calculated by this system, wherein: the CAN interface receives the calculation results of this system and transfers them to the RAM memory for storage; the USB output interface is connected to the vehicle-mounted shoulder height information output terminal to display the shoulder height information to the driver.

[0068] The shoulder height measurement system that takes into account the influence of the pitch angle of the vehicle body provided by the present invention has the following working process: the shoulder measurement point calculation unit of the multi-window anchor point detection method, the CAN bus information acquisition unit of the pitch angle of the vehicle body relative to the earth and the shoulder pixel coordinate and height information storage unit transmit the acquired information to the image recognition distance data and coordinate system calibration data and shoulder pixel coordinate data processing unit, calculate the distance from the vehicle-mounted monocular camera to the shoulder and the height of the shoulder, and transmit them to the vehicle-mounted shoulder height information output terminal through the CAN bus.

[0069] The present invention provides a method for measuring the height of a road shoulder taking into account the influence of the vehicle body pitch angle. Figure 1 , Figure 4 , Figure 5 and Figure 6 As shown, it includes: proposing a multi-window anchor point detection method for calculating the pixel difference between the upper edge and the lower edge of the shoulder at different positions in the same photo; proposing a real-time distance measurement method considering the pitch angle for calculating the distance between the camera and the shoulder in the three-dimensional world; proposing a height measurement method based on adaptive weighting for calculating the height of the shoulder.

[0070] The multi-window anchor point detection method automatically generates a square window according to the width of the image; before the anchor point detection, the shoulder edge line needs to be detected first, and the specific steps are as follows:

[0071] Step 1. Get the camera's intrinsic and extrinsic matrix and distortion parameters:

[0072] The vehicle-mounted monocular camera installed in front of the vehicle is calibrated. During calibration, the influence of the vehicle suspension needs to be considered. The vehicle will produce roll angle, pitch angle and sideslip angle under external force. These factors are fully considered during the calibration process. Therefore, the coordinate transformation from the vehicle-mounted monocular camera to the vehicle suspension is added to make the calibration result of the vehicle-mounted monocular camera more accurate.

[0073] The internal and external parameters of the vehicle-mounted monocular camera can be obtained through camera calibration. The internal parameters mainly refer to the optical and geometric characteristics of the vehicle-mounted monocular camera, including focal length, principal point coordinates, lens distortion, etc.; the external parameters include the translation and rotation of the vehicle-mounted monocular camera to the suspension system and the suspension system to the geodetic coordinates, that is, the change relative to the three-dimensional space orientation. The present invention calibrates the camera by considering the vehicle-mounted monocular camera calibration method of the vehicle suspension, obtains the camera's internal and external parameter matrices and distortion parameters, and then solves the spatial information from the image information.

[0074] The conversion relationship from the vehicle-mounted monocular camera to the vehicle suspension coordinate system is:

[0075]

[0076] In the above formula (1): X c ,Y c ,Z c is the coordinate system of the on-board monocular camera; β is the vehicle roll angle; θ is the vehicle pitch angle; α is the vehicle sideslip angle; X v ,Y v ,Z v is the vehicle suspension coordinate system.

[0077] Step 2: Find the location of the edge in the image:

[0078] The acquired image is subjected to Canny operator edge detection to obtain the edge points of the shoulder. The Canny edge detection algorithm first removes noise through Gaussian filtering, and then calculates the image grayscale gradient direction according to formula (2) to find the most likely edge location.

[0079]

[0080] In the above formula (2): G x Represents the gradient in the x direction at the image pixel (x, j); G y Represents the gradient in the y direction at the image pixel (x, j).

[0081] There may be multiple boundaries at locations where grayscale changes are concentrated, resulting in blurred edge detection results. Canny uses non-maximum suppression to search along the edge gradient direction, retaining only the points with the largest local gradient, and obtaining a clear edge that is only one pixel wide.

[0082] Step 3. Identify the upper, lower and rear edge lines of the shoulder:

[0083] Hough transform is applied to the edge detection image to obtain the upper edge, lower edge and rear edge of the shoulder. Hough transform detection detects straight lines based on the voting of edge points in the entire image, so it has strong anti-interference ability for incomplete straight lines or existing noise in the image, so it can effectively identify dotted lines or partially blocked straight lines during the detection of shoulder edges. The effect of shoulder detection after Hough transform is shown in the figure below. Figure 2 shown.

[0084] Next, see Figure 3 , a multi-window anchor point detection method is proposed, which can extract multiple anchor points from the shoulder in the image. After Hough shoulder detection, this method can obtain three non-intersecting long straight lines a1, a2, and a3, which are the bottom edge of the shoulder, the upper edge of the shoulder, and the rear edge of the shoulder. According to the width of the image, 6 square windows are automatically generated, and the width of the window is shown in formula (3). A vertical line is generated at any position in the window, and the intersection points with the lower edge a1 and the upper edge a2 of the shoulder are b1 and b2 respectively. The distance x from the vertical line to the left side of this window is [0,m i ], b1 and b2 represent the pixel difference of the shoulder in the vertical direction on the image plane. According to the number of square windows, the shoulder heights at 6 different positions can be calculated for each image.

[0085] m i =p sum / 6 (3)

[0086] In the above formula (3): m i ,i=1,2,…,6 is the width of a single square window; p sum The width of the entire image, in pixels.

[0087] The real-time distance measurement method considering the pitch angle is proposed. It takes into account that the pitch angle will be generated during the movement of the vehicle. Therefore, the real-time distance measurement method considering the pitch angle is proposed. The real-time pitch angle generated by the movement of the vehicle is added to the calculation to eliminate the influence of the pitch angle of the vehicle on the distance measurement between the vehicle-mounted monocular camera and the shoulder. Figure 4 As shown in the figure, by obtaining the installation height of the on-board monocular camera, the coordinates of the lower edge of the shoulder in the image plane and the real-time pitch angle of the vehicle, the distance L between the shoulder and the on-board monocular camera is calculated. According to the imaging principle, the geometric relationship is as follows:

[0088] ε=β+γ+θ (4)

[0089]

[0090]

[0091] According to formula (4), (5) and (6), we can get:

[0092]

[0093] O2B1 2 =x 2 +y 2 (8)

[0094] According to the transformation relationship between the image coordinate system and the pixel coordinate system, we can get:

[0095] x=(u-u0)dx (9)

[0096] y=(v-v0)dy (10)

[0097] Combining equations (8), (9) and (10), we can obtain:

[0098] O2B1 2 =[(u-u0)dx] 2 +[(v-v0)dy] 2 (11)

[0099] in Right now:

[0100]

[0101] Combining equations (7) and (12), we can obtain:

[0102]

[0103] The intrinsic parameters obtained by calibrating the on-board monocular camera and the real-time pitch angle θ of the vehicle are brought into it to calculate the distance L between the road shoulder and the on-board monocular camera.

[0104] In the above equations (4)-(13), b1 represents the intersection of the bottom edge of the measured shoulder and the road surface; γ is the initial pitch angle of the vehicle-mounted monocular camera when it is installed in front of the vehicle, h is the height from the ground; f is the focal length of the vehicle-mounted monocular camera, f x 、f y are the components of the focal length in the x and y axis directions of the phase plane respectively; θ is the real-time pitch angle when the vehicle is moving; β is the angle between the projection of the on-board monocular camera and the optical axis; (u0, v0) is the position of the origin of the image coordinate system in the pixel coordinate system; L is the horizontal distance from the measured point b1 to the lens; O is the intersection of the optical axis and the image plane, which serves as the origin of the image coordinate system; O1 is the center of the on-board monocular camera lens; O2 is the focus of the optical axis and the phase plane after the pitch angle of the vehicle changes; B1(x, y) is the projection of the measured point on the image plane; O2B1 is the distance between the projection point of the optical axis on the image plane and the projection point of the target point on the image plane.

[0105] Compared with the existing method, the proposed height measurement method based on adaptive weighting can perform adaptive weighted fusion of shoulder heights measured by multiple (six in this embodiment) square windows to obtain a final shoulder height, thereby improving the measurement accuracy of shoulder height. This method fuses the shoulder measurement values ​​calculated by each square window, and the height relationship is as follows: Figure 5 As shown, the formula is:

[0106]

[0107] In order to reduce system errors and accurately measure the height of the shoulder, an adaptive weighted fusion algorithm is designed. The six square measurement windows are used to calculate the shoulder value H. i , i=1,2,…,6, which are independent of each other. H is the shoulder height of the fused three-dimensional space. The fusion calculation method is as follows:

[0108]

[0109]

[0110] To calculate the total mean square error, it is necessary to extract the shoulder pixel coordinates and the shoulder height H0 calculated at the previous moment stored in the height information storage unit. Therefore, the total mean square error σ is derived from equations (15) and (16): 2 .

[0111]

[0112] According to the extreme value theory of multivariate function, the total mean square error σ is obtained. 2 The minimum condition is that the weight of the shoulder height measured by each square window is w i When the variance is smaller, the corresponding weight is larger, and the minimum total mean square error is for:

[0113]

[0114] Its corresponding weighting factor w i for:

[0115]

[0116] The weight w corresponding to the shoulder height obtained in each window is calculated by formula (19): i , substitute the weights corresponding to the 6 calculated shoulder heights into formula (15), and then obtain the shoulder height H in the fused three-dimensional space.

[0117] In the above equations (14)-(19): b1b2 is the distance between the lower edge and the upper edge of the shoulder; y1y2 is the pixel difference in the vertical direction of the shoulder on the image plane; fy is the component of focal length in the y-axis direction of the phase plane; L is the distance between the road shoulder and the on-board monocular camera; w i ,i=1,2,...,6 is the weight coefficient of the shoulder height measured by the corresponding square window; The variance of shoulder height measurement corresponding to each square window.

[0118] In this embodiment, a Great Wall M2 manual transmission SUV was selected for real vehicle experiments. The test vehicle was equipped with a PC7430 vehicle-mounted monocular camera and various units in the system of the present invention. The vehicle-mounted monocular camera had a pixel of 720*480 and was installed on the center net 0.86 meters above the ground, with an angle of 15° with the horizontal plane. The test site was the campus of Wuhan University of Technology, and 20 open scenes were selected to manually measure the shoulder height before conducting the experiment.

[0119] The experimental results are as follows Figure 7 As shown, the maximum relative error of the shoulder height calculation result is 8.43%, the minimum relative error is 1.22%, and the average relative error is 4.32%. Through the experimental data, it can be concluded that the method of this embodiment is accurate in shoulder height detection, and the system works stably with small relative error fluctuation.

[0120] The above is a description of the technical solution provided by the present invention. Since the content of the invention and the embodiments thereof are subject to change, for those skilled in the art, the content of the invention should not be understood as a limitation to the invention.

Claims

1. A method for measuring the height of a road shoulder taking into account the influence of the vehicle body pitch angle, characterized in that After the on-board monocular camera is calibrated by the on-board monocular camera calibration method considering the vehicle suspension, the upper and lower edges of the shoulder are first identified by the shoulder edge detection algorithm, and the upper and lower edge anchors of the shoulder are selected by the multi-window anchor detection method. According to relevant information, the distance between the shoulder and the on-board monocular camera is measured by the real-time distance measurement method considering the pitch angle. The relevant information includes the camera internal parameters, camera height and vehicle pitch angle obtained after the on-board monocular camera is calibrated. Then, the height of the shoulder is measured by the height measurement model based on adaptive weighting; The real-time distance measurement method considering the pitch angle comprises the following steps: Step 1: According to the imaging principle of the vehicle-mounted monocular camera and the relationship between the vehicle-mounted monocular camera and the ground plane, a geometric model of the distance between the image coordinate system and the earth coordinate system is established. Step 2: According to the geometric model, the relationship expression of β is: ε=β+γ+θ The relationship between pixel coordinates and world coordinates is expressed as: Step 3: According to the transformation relationship between the image coordinate system and the pixel coordinate system, we can get: x=(u-u0)dx y=(v-v0)dy O2B1 2 =x 2 +y 2 =[(u-u0)dx] 2 +[(v-v0)dy] 2 ; Step 4. According to The distance calculation formula between the road shoulder and the vehicle-mounted monocular camera is obtained: In the above formula: b1 represents the intersection of the bottom edge of the measured shoulder and the road surface; γ is the initial pitch angle of the vehicle-mounted monocular camera when it is installed in front of the vehicle, h is the height from the ground; f is the focal length of the vehicle-mounted monocular camera, f x 、f y are the components of the focal length in the x and y axis directions of the phase plane respectively; θ is the real-time pitch angle when the vehicle is moving; β is the angle between the projection of the on-board monocular camera and the optical axis; (u0, v0) is the position of the origin of the image coordinate system in the pixel coordinate system; L is the horizontal distance from the measured point b1 to the lens; O is the intersection of the optical axis and the image plane, which serves as the origin of the image coordinate system; O1 is the center of the on-board monocular camera lens; O2 is the focus of the optical axis and the phase plane after the pitch angle of the vehicle changes; B1(x, y) is the projection of the measured point on the image plane; O2B1 is the distance between the projection point of the optical axis on the image plane and the projection point of the target point on the image plane; Adopting the height measurement method based on adaptive weighting, the measurement values ​​of the shoulder calculated by each square window are fused, and the height relationship formula is: The shoulder height fusion calculation method is as follows: The total mean square error is: The minimum total mean square error is: Its corresponding weighting factor w i for: In the above formula: b1b2 is the length between the lower edge and the upper edge of the shoulder; y1y2 is the pixel difference of the shoulder in the vertical direction on the image plane; f y is the component of focal length in the y-axis direction of the phase plane; L is the distance between the road shoulder and the on-board monocular camera; H i is the shoulder height measured by a single square window; H is the shoulder height after fusion; w i , i = 1, 2, ..., 6 is the weight coefficient of the shoulder height measured by the corresponding square window; H0 is the shoulder pixel coordinate and the shoulder height measured at the previous moment stored in the height information storage unit; The variance of shoulder height measurement corresponding to each square window.

2. The method for measuring shoulder height according to claim 1, characterized in that: The road shoulder edge detection algorithm comprises the following steps: Step 1. Calibrate the on-board monocular camera installed in front of the vehicle to obtain the internal and external parameters of the camera. The on-board monocular camera calibration method considering the vehicle suspension is adopted. The conversion relationship from the on-board monocular camera to the vehicle suspension coordinate system is: In the above formula: X c ,Y c ,Z c is the coordinate system of the on-board monocular camera; β is the vehicle roll angle; θ is the vehicle pitch angle; α is the vehicle sideslip angle; X v ,Y v ,Z v is the vehicle suspension coordinate system; Step 2: Perform Canny operator edge detection on the acquired image to obtain the edge points of the shoulder. The Canny edge detection algorithm first removes noise through Gaussian filtering, and then calculates the grayscale gradient direction of the image to find the most likely location of the edge. Step 3: Apply Hough transform to the edge detection image to obtain the upper edge, lower edge and rear edge of the shoulder.

3. The method for measuring shoulder height according to claim 1, characterized in that: The multi-window anchor point detection method automatically generates 6 square windows according to the width of the entire image. Each window detects the intersection of the vertical line and the upper and lower edges of the shoulder. The width calculation formula of a single square window is: m i =p sum / 6 Where: m i ,i=1,2,…,6 is the width of a single square window; p sum The width of the entire image, in pixels.

4. A road shoulder height measurement system taking into account the influence of vehicle body pitch angle, characterized in that: The system is implemented based on the shoulder height measurement method described in any one of claims 1 to 3, and includes a CAN bus information acquisition unit for the pitch angle of the vehicle body relative to the ground, as well as an on-board monocular camera, a shoulder test point calculation unit based on a multi-window anchor point detection method, an image recognition distance data and coordinate system calibration data and shoulder pixel coordinate data processing unit, and a shoulder pixel coordinate and height information storage unit, wherein the above five units are all mounted on a moving vehicle, and the image recognition distance data and coordinate system calibration data and shoulder pixel coordinate data processing unit measures and stores the height of the shoulder at the current moment based on the real-time vehicle pitch angle obtained by the CAN bus information acquisition unit for the pitch angle of the vehicle body relative to the ground, the anchor points of the upper and lower edges of the shoulder in the picture calculated by the shoulder test point calculation unit based on the multi-window anchor point detection method, and the shoulder pixel coordinates and the shoulder height at the previous moment stored in the height information storage unit.

5. The road shoulder height measurement system considering the influence of the vehicle body pitch angle according to claim 4 is characterized in that: The CAN bus information acquisition unit for the pitch angle of the vehicle body relative to the ground is mainly composed of a tilt sensor, a CAN interface, a power interface, and an on-board shoulder height information output terminal, wherein the tilt sensor is connected to the CAN interface to transmit the vehicle pitch angle information.

6. The road shoulder height measurement system considering the influence of vehicle body pitch angle according to claim 4, characterized in that: The shoulder test point calculation unit based on the multi-window anchor point detection method is mainly composed of an aviation plug, a microprocessor, a CAN interface, and a power interface, wherein: the aviation plug is connected to the vehicle-mounted monocular camera, receives the collected image data and transmits it to the microprocessor. After the microprocessor completes the calculation, it transmits the information to the image recognition distance data and coordinate system calibration data and the shoulder pixel coordinate data processing unit through the CAN interface.

7. The road shoulder height measurement system considering the influence of vehicle body pitch angle according to claim 4, characterized in that: The image recognition distance data, coordinate system calibration data and shoulder pixel coordinate data processing unit is mainly composed of a microprocessor, a CAN interface and a power supply interface, wherein: the microprocessor receives the information required for calculation and outputs the calculation results through the CAN interface.

8. The road shoulder height measurement system considering the influence of vehicle body pitch angle according to claim 4, characterized in that: The shoulder pixel coordinate and height information storage unit is mainly composed of RAM memory, CAN interface, power interface and USB output interface, among which: the CAN interface receives the calculation results of this system and transfers them to the RAM memory for storage; the USB output interface is connected to the vehicle-mounted shoulder height information output terminal to display the shoulder height information to the driver.

Images