Pavement elevation measurement system based on stereoscopic vision and measurement method thereof

Abstract

The invention discloses a pavement elevation measurement system based on stereoscopic vision and a measurement method thereof, and belongs to the field of traffic engineering, and the pavement elevation measurement system comprises an input module, a target detection module, a size calculation module and a display module. The measurement method comprises the following steps: S21, obtaining left and right views of an obstacle by using a binocular stereo camera; s22, the improved YOLOv8 model is initialized; s23, generating a depth map by using the SGBM; s24, performing target detection by using the improved YOLOv8, and combining a target detection result with the depth map; s25, measuring the size of a front obstacle through a size calculation module; s26, calculating the height and the width of each detected object; and S27, displaying the calculated data. According to the invention, the ECA attention mechanism is integrated into the YOLOv8 model, and the SGBM is combined to carry out stereo matching so as to carry out obstacle size measurement. Compared with the effect of using the SGBM or not, the SGBM can generate a more accurate and smooth disparity map by executing global smoothing constraint in multiple directions, so that the accuracy of obstacle size measurement is improved by 12%.

Description

Technical Field

[0001] The present invention relates to the technical field of traffic engineering, and in particular to a road elevation measurement system and a measurement method thereof based on stereoscopic vision. Background Art

[0002] At present, artificial intelligence and automobile technology are constantly advancing, and the unmanned driving technology of new energy vehicles has become a product of the deep integration of the new generation of information technology, which has important application value in managing urban traffic and avoiding environmental pollution. In the process of unmanned driving technology research, the ability to detect obstacles in unknown environments is the core element to ensure the normal operation of unmanned driving technology for new energy vehicles. In order to enhance its reliability and stability, the accurate detection of obstacles in its driving environment is a research hotspot for many scientific research institutions and technicians at home and abroad.

[0003] Common dimensional measurement methods include laser scanning, active vision measurement, and passive vision measurement. The cost of laser scanning is relatively expensive, which greatly increases the cost of automobile research and development, and the cost of purchasing a car is not acceptable to the majority of ordinary income families; active vision measurement is generally measured by active light projection coding, but the equipment installation requirements are high; passive vision measurement has the advantages of low cost and easy installation. A single visual system can only obtain two-dimensional information of the target, and cannot obtain three-dimensional information of the target object, but binocular vision is more in line with the way the human eye observes things, and the accuracy is relatively high. Therefore, binocular vision has always been a hot topic of research at home and abroad, and is widely used in robot navigation, micro-operation system parameter detection, three-dimensional measurement, virtual reality and other fields.

[0004] Stereo matching is an extremely important research topic in binocular vision. The core of binocular vision is the stereo matching algorithm, which needs to find matching points on the same horizontal line in the right image for each point in the left image. The horizontal displacement deviation of the same point is the disparity of the point. The accuracy of the disparity will affect the results of 3D reconstruction and volume measurement.

[0005] At present, binocular cameras are widely used in the fields of agriculture and animal husbandry, machinery manufacturing, transportation and logistics, and are less used in the field of intelligent vehicles. This technology is of great necessity in the field of intelligent transportation. This technology can improve safety. In complex driving environments, accurate measurement of the size of obstacles is crucial to avoiding collisions. Especially in autonomous driving or assisted driving modes, accurate distance and size information can improve driving safety. In terms of reducing the probability of chassis damage, according to reports from some maintenance service providers and insurance companies, car chassis scratches are a common problem. For example, in urban parking environments, uneven ground, curbs, entrance ramps to underground parking lots, etc. can all cause chassis damage. Although there is no exact data on how many such incidents occur each year, this number cannot be ignored considering the parking density and road conditions in urban environments.

[0006] At the same time, the environmental perception capability is enhanced, and binocular vision can provide three-dimensional spatial information to help vehicles better understand the surrounding environment, which is essential for the autonomous navigation of new energy vehicles. According to the Federal Highway Administration (FHWA), about 4% of roads in the United States are considered narrow, which means that when driving on these roads, accurate measurement of obstacle size is particularly important to avoid scratches and collisions.

[0007] This technology can also adapt to a variety of scenarios. Whether it is a city road or a highway, the binocular camera can effectively detect and measure different types of obstacles. In the United States, about 20,000 traffic accidents are caused by obstacles on the road each year. Although this number includes all types of vehicles, for new energy vehicles equipped with binocular cameras, if these obstacles can be detected in advance and action can be taken, these accidents can be effectively avoided. And this technology can reduce maintenance costs. According to statistics from the National Highway Traffic Safety Administration (NHTSA), the cost of collision repairs each year is as high as tens of billions of dollars. Although this includes all types of collisions, if minor collisions caused by obstacles can be reduced, then this part of the cost will be reduced.

[0008] In summary, the application of binocular cameras for obstacle size (road elevation) measurement in new energy vehicles can not only significantly improve driving safety, but also promote the development and progress of related technologies, and have a far-reaching impact on the automotive industry. Summary of the invention

[0009] In order to solve the technical problem that passive vision measurement in the above-mentioned prior art has the advantages of low cost and easy installation, but a single vision system can only obtain two-dimensional information of the target and cannot obtain three-dimensional information of the target object, the present invention provides a road elevation measurement system based on stereo vision and a measurement method thereof.

[0010] In order to realize the above technical solution, the present invention provides a road elevation measurement system based on stereo vision, which is characterized by comprising:

[0011] An input module, wherein the input module is composed of a binocular stereo camera having a pair of parallel cameras, through which a left and right view of a scene in front of the vehicle is obtained, and a combination of the left and right views is used to confirm a scene in front;

[0012] A target detection module, wherein the target detection module is composed of a target detection network based on deep learning, wherein the target detection network is an improved YOLOv8 model, wherein the improved YOLOv8 model is integrated with the ECA attention mechanism and combined with the SGBM stereo matching algorithm to measure obstacle size;

[0013] A size calculation module, wherein the size calculation module uses the conversion relationship between the pixel coordinate system, the image coordinate system and the camera coordinate system to obtain the actual size of the target;

[0014] The display module is responsible for displaying the output target actual size value to facilitate the subsequent intelligent vehicle to adjust its performance.

[0015] Furthermore, the ECA attention mechanism is calculated by the following steps:

[0016] S1, global average pooling: perform global average pooling on the input feature map to obtain a one-dimensional vector z.

[0017] S2, one-dimensional convolution: Perform a one-dimensional convolution operation on z to obtain a new vector ω. The convolution kernel size k can be selected according to the number of channels in the feature map, using or Where b is the number of channels of the feature map and gamma is a hyperparameter that defaults to 2.

[0018] S3, activation function: Use the Sigmoid function to activate ω and obtain the attention weight s of each channel.

[0020] S4, element multiplication: perform element-by-element multiplication of the attention weight s with the original feature map to obtain the final output feature map.

[0021] Furthermore, the SGBM stereo matching algorithm includes five steps: initializing parameters, calculating disparity cost, cost aggregation, observation calculation, observation optimization and disparity visualization, wherein:

[0022] S11, initialization parameters: Use the horizontal Sobel operator for image processing, where the Sobel operator is as follows

[0023] As shown:

[0024] Sobel(x+y)=2[P(x+1,y)-P(x-1,y)]+P(x+1,y-1)-P(x-1,y-1)+P(x+1,y+1)-P(x-1,y+1)

[0025] The mapping is declared as:

[0026]

[0027] S12, calculate the disparity cost: the BT cost obtained by sampling the image gradient information preprocessed by the Sobel operator and the BT cost obtained by sampling the original image based on grayscale, and the two are weighted by the domain to obtain the local block cost.

[0028] S13, cost aggregation: replace the cost value of each pixel with the block cost, and then aggregate the matching

[0029] Cost aggregation is performed to optimize the initial cost body;

[0030] S14, disparity calculation: According to the aggregated cost map, select the disparity value corresponding to the minimum cost for each pixel.

[0031] Generate an initial disparity map;

[0032] S15, parallax optimization: uniqueness detection, sub-pixel interpolation, left-right consistency, connected area detection and

[0033] Apply methods such as median filtering or bilateral filtering to remove noise;

[0034] S16, Disparity map visualization: Convert the disparity map into a grayscale image or a color image for easy visualization, using color coding to represent different disparity values.

[0035] Furthermore, according to the imaging principle of the binocular camera, the calculation formula for the size measurement is obtained:

[0036] The parallax of obstacle p is calculated by the following formula:

[0037] d = X L -X R

[0038] P L and P R The distance between them can be expressed by B, X L and X R Indicates that, according to the principle of similar triangles, the formula can be obtained:

[0039]

[0040] The formula for the distance Z from point P to the projection plane is:

[0041]

[0042] Furthermore, the size calculation module locates the position of the camera through the world coordinate system, and the transformation of the world coordinate system is a rigid body transformation, wherein the conversion formula between the world coordinate system and the camera coordinate system is as follows:

[0043]

[0044] The conversion formula between the camera coordinate system and the image coordinate system is:

[0045]

[0046]

[0047] The coordinates of point p(x,y) in the pixel coordinate system in the image coordinate system (u,v) are:

[0048]

[0049] Furthermore, it is known that the height of the target in the pixel coordinate system is h1 and the width is w1. Assuming the conversion coefficient c from the pixel coordinate system to the image coordinate system is 1, the height value h2 and the width value w2 of the target in the image coordinate system are:

[0050] h2=c×h1

[0051] w2=c×w1

[0052] Furthermore, given that the height value of the target in the image coordinate system is h2 and the width value is w2, the depth of the target from the camera is Z, and the focal length of the camera is f, we can use the formula to obtain the actual height H and actual width W of the target:

[0053]

[0054]

[0055] Furthermore, a road elevation measurement method based on stereo vision comprises the steps of:

[0056] S21: Use a binocular stereo camera to obtain left and right views of the obstacle in front;

[0057] S22: load the image and initialize the improved YOLOv8 model;

[0058] S23: Generate depth map using SGBM;

[0059] S24: Use the improved YOLOv8 model to perform target detection and combine the target detection results with the depth map;

[0061] S25: measuring the size of the obstacle ahead through a size calculation module;

[0062] S26: converting the image coordinate system into the pixel coordinate system, and calculating the height and width of each obstacle;

[0063] S27: Output and display the calculated data to adjust the overall performance of the intelligent vehicle.

[0064] The beneficial effects of the present invention are:

[0065] The present invention integrates the Efficient Channel Attention (ECA) mechanism into the YOLOv8 model and combines it with Semi-Global Block Matching (SGBM) for stereo matching to measure the obstacle size, which can bring advantages and positive effects in many aspects.

[0066] After simulation experiment comparison, the ECA mechanism can automatically adjust the weight of each channel, thereby enhancing the model's attention to important features and improving the accuracy of obstacle detection. By dynamically adjusting the channel weights in the feature map, the model can better capture the key features of obstacles and reduce false detections and missed detections. Moreover, the improved system improves the detection performance of small targets. For the detection of small obstacles, it helps the model to better focus on the features of these small targets, and improves the detection accuracy by 4.2%. The addition of the ECA mechanism can make the model more robust when facing complex backgrounds or lighting changes, because the attention mechanism helps the model focus on the obstacles themselves rather than background interference, greatly improving the robustness of the system. Compared with other attention mechanisms, the ECA mechanism is lighter and will not significantly increase the computational burden, so it can maintain the original real-time processing capability of the YOLOv8 model, and there is no obvious delay in the calculation speed after the improvement. By learning the correlation between different channels, the ECA mechanism helps the model perform better on unseen data and enhances the generalization ability of the model.

[0067] Comparing the effects of using SGBM and not using it, SGBM can generate more accurate and smooth disparity maps by performing global smoothness constraints in multiple directions, thereby improving the accuracy of obstacle size measurement by 12%. This global optimization method helps to reduce the impact of occlusion and noise, making the disparity map more reliable and improving the disparity quality visibly. Real-time performance also has advantages, thanks to the relatively fast SGBM algorithm. SGBM provides multiple parameters for adjustment, such as disparity range, window size of cost aggregation, etc., which can be optimized according to specific application scenarios. This flexibility enables SGBM to adapt to different application scenarios and improve the performance of the overall system. Comparing the effects of using and not using SGBM under complex lighting conditions or when there is occlusion, the effect is quite different, highlighting that through cost aggregation in multiple directions, SGBM can reduce the impact of local minima and improve the robustness of disparity maps.

[0068] After the technology is implemented, it can also bring huge impact. Technological innovation has promoted the progress of computer vision algorithms, such as stereo matching algorithms and deep learning. Second, it promotes the development of autonomous driving, provides important technical support for L3 and above autonomous driving, and accelerates the commercialization process of autonomous driving technology. Third, it improves user experience by reducing accident risks and providing a smoother driving experience. The fourth is the formulation of industry standards. With the application and development of technology, it may promote the formulation and improvement of relevant industry standards to ensure the safety and consistency of technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0070] Figure 1 It is the overall framework flow chart;

[0071] Figure 2 It is a schematic diagram of the control architecture;

[0072] Figure 3 It is the projection effect diagram of the target on the left and right views;

[0073] Figure 4 This is the structural diagram of the Efficient Channel Attention (ECA) module;

[0074] Figure 5 Figure 2 is the stereo matching reasoning process diagram of SGBM;

[0075] Figure 6 This is a binocular vision model diagram;

[0076] Figure 7 Schematic diagram of the relationship between four types of coordinate systems in stereo vision;

[0077] Figure 8 Schematic diagram of the conversion relationship between the camera coordinate system and the image coordinate system;

[0078] Fig. 9 The present invention provides a flow chart of a road elevation measurement method based on stereo vision. DETAILED DESCRIPTION

[0079] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0080] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof. Unless otherwise specifically stated, the relative arrangement of the parts and steps described in these embodiments, numerical expressions and numerical values ​​do not limit the scope of the present invention.

[0081] The overall architecture of this system is as follows Figure 1 As shown. The present invention provides a road elevation measurement method system based on stereo vision, that is, a road elevation difference measurement technology using a binocular camera to measure the longitudinal height change of the road caused by road obstacles. The technology consists of four modules: an input module, a target detection module, a size calculation module, and a display module. The input module is composed of a binocular stereo camera and is responsible for image acquisition; the target detection module is responsible for detecting the target in the image and obtaining the predicted frame of the target; the size calculation module is responsible for calculating the actual height and width of the target; and the display module is responsible for displaying the road elevation measurement results.

[0082] As a preferred embodiment, Figure 2 and Figure 3 As shown in the figure, the input module is mainly composed of a binocular stereo camera. The binocular stereo camera has a pair of parallel cameras. Compared with the traditional monocular camera, the binocular stereo camera can use the left and right cameras to obtain the left and right views of the scene in front. The binocular stereo camera can calculate the depth of the target in the scene from the camera using the limit constraint theorem, thereby constructing a three-dimensional scene view.

[0083] As a preferred embodiment, Figure 4As shown, the target detection module is mainly composed of a target detection network based on deep learning. The present invention designs a target detection algorithm that only detects "obstacle" class targets, and the algorithm is mainly improved based on the YOLOv8 model.

[0084] The Efficient Channel Attention (ECA) module is a lightweight attention mechanism that aims to reduce computational complexity by replacing the fully connected layer in the traditional Squeeze-and-Excitation (SE) module with one-dimensional convolution. The EC A module dynamically adjusts the weight of each channel by learning the correlation between different channels, thereby enhancing the model's ability to capture important features.

[0085] In order to improve the performance of the YOLOv8 model in obstacle detection and size measurement, the backbone network is selected as the integration point of the ECA module. Specifically, the ECA module is inserted after the key residual block in the backbone network of the YOLOv8 model. This not only improves the distinguishing ability of the feature map, but also maintains the overall lightweight characteristics of the model.

[0086] The core idea of ​​the ECA module is to introduce an attention mechanism at the channel level to dynamically adjust the importance of each channel. It extracts global information through global average pooling, then learns the dependencies between channels through one-dimensional convolution, and finally generates attention weights.

[0087] The detailed steps are as follows:

[0088] S1, global average pooling: perform global average pooling on the input feature map to obtain a one-dimensional vector z.

[0089] S2, one-dimensional convolution: Perform a one-dimensional convolution operation on z to obtain a new vector ω. The convolution kernel size k can be selected according to the number of channels in the feature map, using or Where b is the number of channels of the feature map and gamma is a hyperparameter that defaults to 2.

[0090] S3, activation function: Use the Sigmoid function to activate ω and obtain the attention weight s of each channel.

[0092] S4, element multiplication: perform element-by-element multiplication of the attention weight s with the original feature map to obtain the final output feature map.

[0093] As a preferred embodiment, Figure 5As shown in the figure, stereo matching is to infer the depth information of each pixel from a pair of images. The traditional stereo matching algorithm converts the stereo matching problem into the process of finding the most similar pixels in the left and right images. SGBM is a classic stereo matching algorithm that optimizes the disparity map by performing global smoothness constraints in multiple directions. The core idea of ​​SGBM is to calculate the disparity cost in multiple directions and reduce the influence of occlusion effect and noise through global smoothness constraints to obtain a more accurate disparity map. This is an algorithm based on traditional computer vision. It does not rely on the low operating processor requirements required by deep learning models and can expand the scope of application. Compared with other non-deep learning stereo matching algorithms, SGBM runs faster, which is convenient for quickly identifying the size of obstacles and giving the vehicle more time to adjust the suspension or brakes. By performing cost aggregation in multiple directions, SGBM can solve the occlusion problem to a certain extent, and there are multiple parameters that can be adjusted, such as disparity range, cost aggregation window size, etc., to adapt to different application scenarios.

[0094] S11, initialization parameters: that is, in the preprocessing stage, the horizontal Sobel operator is used for image processing, where the Sobel operator is shown in the following formula:

[0095] Sobel(x+y)=2[P(x+1,y)-P(x-1,y)]+P(x+1,y-1)-P(x-1,y-1)+P(x+1,y+1)-P(x-1,y+1)

[0096] The mapping is declared as:

[0097]

[0098] Preprocessing can obtain the grayscale gradient information of the image, which will be used to calculate the matching cost later.

[0099] S12, calculate the disparity cost: the BT cost obtained by sampling the image gradient information preprocessed by the Sobel operator and the BT cost obtained by sampling the original image based on grayscale. The two are weighted and summed to obtain the local block cost, and the block cost is used to replace the cost value of each pixel, which is conducive to improving the matching accuracy.

[0100] S13, cost aggregation: After step S12 is completed, according to the idea of ​​global matching, the SGBM method aggregates the matching costs on the matching path, which can optimize the initial cost body. Cost aggregation is performed in multiple directions to reduce the influence of local minima. For each pixel and each disparity value, the cumulative cost is calculated along the defined direction. The aggregated cost reflects the global consistency of the pixel under different disparity values.

[0101] S14, disparity calculation: According to the aggregated cost map, select the disparity value corresponding to the minimum cost for each pixel to generate an initial disparity map.

[0102] S15, parallax optimization: perform uniqueness detection, sub-pixel interpolation, left-right consistency, connected area detection, and apply median filtering or bilateral filtering to remove noise.

[0103] S16, Disparity map visualization: Convert the disparity map into a grayscale image or a color image for easy visualization, using color coding to represent different disparity values.

[0104] The algorithm uses one-dimensional optimization on multiple paths to solve the high complexity problem of the global stereo matching algorithm, avoiding the direct calculation of the minimum value of the global energy function, and can ensure high disparity accuracy while improving the matching rate.

[0105] The need to convert between parallax and depth, the model of binocular stereo camera imaging is as follows Figure 6 As shown in the figure, the optical axes of the left and right cameras are parallel, and the distance between the optical centers is the baseline length. The coordinate system of the left camera is usually set to the world coordinate system. The principle of binocular imaging is to obtain the depth information of objects at different positions based on the different parallaxes at the same epipolar level of the images of the same scene at the same time obtained by the two cameras at different shooting angles.

[0106] Among them, B is the baseline, line segment X L and X R denote the pixel coordinates of the coordinate point P in the space in the left and right images, respectively. f is the focal length of the camera. Let the coordinates of point P be (X, Y, Z). Then the parallax of point P can be calculated by d = X L -X R Calculated, P L and P R The distance between them can be expressed by B, X L and X R Indicates that, according to the principle of similar triangles, the formula can be obtained:

[0107]

[0108] After sorting, we can get the formula for the distance Z from point P to the projection plane:

[0109]

[0110] From the above formula, it can be obtained that according to the known internal and external parameters of the camera, it is only necessary to calculate the value of the parallax d to obtain the depth of the object in space from the camera image through the corresponding mathematical formula, and then the three-dimensional object coordinates can be restored according to the depth of the object to measure the size.

[0111] The size calculation module is used to calculate the actual size of the target. When the target detection frame and the depth value of the target are known, the actual size of the target can be obtained by using the conversion relationship between the pixel coordinate system, the image coordinate system, and the camera coordinate system.

[0112] Starting from the image data obtained by the camera, the information of the object in the three-dimensional space can be calculated, and based on this, the object can be reconstructed and recognized. There is a close connection between the geometric position of a point on the surface of an object in the three-dimensional space and its corresponding point in the image. This connection is defined by the geometric model of camera imaging. This geometric model involves four different coordinate systems: the world coordinate system, the camera coordinate system, the image coordinate system, and the pixel coordinate system.

[0113] The world coordinate system is used to locate the position of the camera. In a binocular vision system, the origin of the world coordinate system is usually set at the optical center of the left or right camera. The camera coordinate system is a three-dimensional rectangular coordinate system centered on the camera focus and with its optical axis as the Z axis. The image coordinate system is established on the image plane, with its origin located in the lower left corner of the image, and the unit is usually millimeters. The pixel coordinate system is a two-dimensional coordinate system based on pixel units and is used for image processing.

[0114] The transformation relationship between these coordinate systems is the key to understanding the position of objects in space. The relationship between them can be Figure 7 To express it intuitively.

[0115] As a preferred method, the transformation of the world coordinate system is a rigid body transformation, that is, only the spatial position and orientation of the object are changed, without changing the shape of the object, and it is achieved through rotation and translation. The conversion formula between the world coordinate system and the camera coordinate system is as follows, where is the camera coordinate system, is the world coordinate system, R is the rotation matrix, and t is the translation matrix.

[0116]

[0117] The transformation relationship from the camera coordinate system to the image coordinate system is a perspective projection relationship, such as Figure 8 As shown, the coordinate origin O is the intersection of the camera optical axis and the image coordinate system, point P is the real target point, p is the projection point of point P on the image, and f is the focal length of the camera.

[0118] Therefore, we can use the triangular similarity principle to obtain the conversion formula between the camera coordinate system and the image coordinate system:

[0119]

[0120]

[0121] The image coordinate system and the pixel coordinate system are both plane coordinate systems, but the unit of the image coordinate system is mm, while the unit of the pixel coordinate system is pixel. When both coordinate systems take the upper left corner of the image as the origin, the coordinate value of the point p(x, y) in the pixel coordinate system in the image coordinate system (u, v) is:

[0122]

[0123] Where c is the conversion factor.

[0124] When the coordinates of point P in the pixel coordinate system are known, we can use the conversion relationship between the above-mentioned coordinate systems to finally obtain the coordinates of point P in the camera coordinate system or the world coordinate system.

[0125] As a preferred method, in the size calculation module, if the height of the target in the pixel coordinate system is known to be h1 and the width is w1, and the conversion coefficient c from the pixel coordinate system to the image coordinate system is set to 1, then we can obtain the height value h2 and the width value w2 of the target in the image coordinate system as follows:

[0126] h2=c×h1

[0127] w2=c×w1

[0128] As a preferred method, given that the height value of the target in the image coordinate system is h2 and the width value is w2, the depth of the target from the camera is Z, and the focal length of the camera is f, we can use the formula to obtain the actual height H and actual width W of the target respectively:

[0129]

[0130]

[0131] Through the above process, the size measurement of the real target can be finally achieved.

[0132] The display module is responsible for displaying the actual size value of the output target, so that the subsequent intelligent vehicle can adjust its performance.

[0133] As a preferred embodiment, Fig. 9 As shown, a road elevation measurement method based on stereo vision includes the following steps:

[0134] S21: Use a binocular stereo camera to obtain left and right views of the obstacle in front;

[0135] S22: load the image and initialize the improved YOLOv8 model;

[0136] S23: Generate depth map using SGBM;

[0137] S24: Use the improved YOLOv8 model to perform target detection and combine the target detection results with the depth map;

[0139] S25: measuring the size of the obstacle ahead through a size calculation module;

[0140] S26: converting the image coordinate system into the pixel coordinate system, and calculating the height and width of each obstacle;

[0141] S27: Output and display the calculated data to adjust the overall performance of the intelligent vehicle.

[0142] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A road elevation measurement system based on stereo vision, characterized in that: include: An input module, wherein the input module is composed of a binocular stereo camera having a pair of parallel cameras, through which a left and right view of a scene in front of the vehicle is obtained, and a combination of the left and right views is used to confirm a scene in front; A target detection module, wherein the target detection module is composed of a target detection network based on deep learning, wherein the target detection network is an improved YOLOv8 model, wherein the improved YOLOv8 model is integrated with the ECA attention mechanism and combined with the SGBM stereo matching algorithm to measure obstacle size; A size calculation module, wherein the size calculation module uses the conversion relationship between the pixel coordinate system, the image coordinate system and the camera coordinate system to obtain the actual size of the target; The display module is responsible for displaying the output target actual size value to facilitate the subsequent intelligent vehicle to adjust its performance.

2. The road elevation measurement system based on stereo vision according to claim 1 is characterized in that: The EC A attention mechanism is calculated by the following steps: S1, global average pooling: perform global average pooling on the input feature map to obtain a one-dimensional vector z; S2, one-dimensional convolution: Perform a one-dimensional convolution operation on z to obtain a new vector ω. The convolution kernel size k can be selected according to the number of channels in the feature map. Use or Where b is the number of channels of the feature map, and gamma is a hyperparameter with a default value of 2; S3, activation function: Use the Sigmoid function to activate ω and obtain the attention weight of each channel s； S4, element multiplication: perform element-by-element multiplication of the attention weight s with the original feature map to obtain the final output feature map.

3. The road elevation measurement system based on stereo vision according to claim 2 is characterized in that: The SGBM stereo matching algorithm includes five steps: initializing parameters, calculating disparity cost, cost aggregation, observation calculation, observation optimization and disparity visualization, among which: S11, initialization parameters: Use the horizontal Sobel operator for image processing, where the Sobel operator is shown in the following formula: Sobel(x+y)=2[P(x+1,y)-P(x-1,y)]+P(x+1,y-1)-P(x-1,y-1)+P(x+1,y+1)-P(x-1,y+1) mapping is expressed as: S12, calculate the disparity cost: the BT cost obtained by sampling the image gradient information preprocessed by the Sobel operator and the BT cost obtained by sampling the original image based on grayscale, and the two are weighted by the domain to obtain the local block cost. S13, cost aggregation: replace the cost value of each pixel with the block cost, and then perform cost aggregation on the matching path to optimize the initial cost volume; S14, disparity calculation: according to the aggregated cost map, select the disparity value corresponding to the minimum cost for each pixel; Generate an initial disparity map; S15, parallax optimization: uniqueness detection, sub-pixel interpolation, left-right consistency, connected area detection, and application of median filtering or bilateral filtering to remove noise; S16, Disparity map visualization: Convert the disparity map into a grayscale image or a color image for easy visualization, using color coding to represent different disparity values.

4. The road elevation measurement system based on stereo vision according to claim 3 is characterized in that: According to the imaging principle of the binocular camera, the calculation formula for the size measurement is obtained: The parallax of obstacle p is calculated by the following formula: d=X L -X R P L and P R The distance between them can be expressed by B, X L and X R Indicates that, according to the principle of similar triangles, the formula can be obtained: The formula for the distance Z from point P to the projection plane is:

5. The road elevation measurement system based on stereo vision according to claim 1 is characterized in that: The size calculation module locates the position of the camera through the world coordinate system. The transformation of the world coordinate system is a rigid body transformation. The conversion formula between the world coordinate system and the camera coordinate system is as follows: The conversion formula between the camera coordinate system and the image coordinate system is: The coordinates of point p(x,y) in the pixel coordinate system in the image coordinate system (u,v) are:

6. The road elevation measurement system based on stereo vision according to claim 5 is characterized in that: It is known that the height of the target in the pixel coordinate system is h1 and the width is w1. Assuming the conversion coefficient c from the pixel coordinate system to the image coordinate system is 1, the height value h2 and width value w2 of the target in the image coordinate system are: h2=c×h1 w2=c×w1.

7. The road elevation measurement system based on stereo vision according to claim 5 is characterized in that: It is known that the height value of the target in the image coordinate system is h2 and the width value is w2, the depth of the target from the camera is Z, and the focal length of the camera is f. We can use the formula to get the actual height H and actual width W of the target respectively:

8. A road elevation measurement method based on stereo vision, comprising the road elevation measurement system based on stereo vision according to any one of claims 1 to 7, characterized in that: The method comprises the steps of: S21: Use a binocular stereo camera to obtain left and right views of the obstacle in front; S22: load the image and initialize the improved YOLOv8 model; S23: Generate depth map using SGBM; S24: Use the improved YOLOv8 model to perform target detection and combine the target detection results with the depth map; S25: measuring the size of the obstacle ahead through a size calculation module; S26: converting the image coordinate system into the pixel coordinate system, and calculating the height and width of each obstacle; S27: Output and display the calculated data to adjust the overall performance of the intelligent vehicle.

Images