Method, apparatus and system for measuring road surface roughness, and electronic device and medium
Calculation of road flatness through binocular stereo vision technology has solved the problem of high calculation consumption in the existing technology, improved the evaluation efficiency and accuracy of road flatness, and optimized driving safety.
Patent Information
- Application Number
- PCT/CN2024/086642
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-11
- Filing Date
- 2024-04-08
- Publication Date
- 2024-11-14
AI Technical Summary
In the calculation of road flatness, the prior art requires a very large amount of calculations due to the processing of three-dimensional point cloud data, which reduces the calculation efficiency and increases the calculation consumption.
By using binocular stereoscopic vision technology, by obtaining the parallax map of the road area to be detected, dividing it into detection areas of the same size, calculating the parallax gradient vector, counting the number of direction deviations and deviation angles, predicting the level of road surface, reducing calculation consumption and improving evaluation accuracy and efficiency.
It effectively improves the calculation efficiency and evaluation accuracy of road flatness, optimizes the road perception of assisted driving, and improves driving comfort and safety.
Smart Images

Figure CN2024086642_14112024_PF_FP_ABST
Abstract
Description
Road surface smoothness detection method, device, system, electronic equipment and medium
[0001] CROSS-REFERENCE TO RELATED APPLICATIONS
[0002] This application claims priority to the Chinese patent application filed with the China Patent Office on May 11, 2023, with application number 202310531576.X, and entitled “Method, device, system, electronic equipment and medium for detecting road surface flatness”, the entire contents of which are incorporated by reference into this application. Technical Field
[0003] The present application relates to the field of automobile technology, and in particular to a method, device, electronic device, and computer-readable storage medium for detecting road surface flatness. Background Art
[0004] With the advancement of driving technology, people's demands for vehicle safety and comfort are increasing. Furthermore, with the development of deep learning technology, deep learning-based recognition methods are finding widespread application in autonomous driving, security, and industrial inspection. Urban roads and highways are key application scenarios for driving, and road conditions vary across different sections. Uneven road surfaces not only affect the driving experience but can also cause damage to the vehicle itself.
[0005] In related technologies, road surface roughness is typically determined using methods such as three-dimensional point cloud data and plane equations. Specifically, three-dimensional point cloud data is first obtained using sensors such as lidar and processed. The resulting data is then combined with the plane equation to estimate the degree of deviation at each ground point. The road surface roughness is then determined based on this deviation.
[0006] However, in related technologies, when processing three-dimensional point cloud data, each point includes complete X, Y, and Z three-dimensional floating-point data, which requires a very large amount of computing power to meet the computing requirements, thereby increasing computing consumption and reducing the efficiency of calculating road surface smoothness.
[0007] Summary of the Invention
[0008] This application provides a method, device, electronic device, and computer-readable storage medium for detecting road surface smoothness, to at least address the problem in related technologies of reduced efficiency in calculating road surface smoothness due to increased computational effort. The technical solutions of this application are as follows:
[0009] According to a first aspect of an embodiment of the present application, a method for detecting road surface flatness is provided, comprising:
[0010] Obtaining a binocular stereo vision disparity map of the road area to be detected, where the road area to be detected is divided into multiple detection areas of equal size;
[0011] Determine a disparity gradient vector corresponding to each detection area based on the disparity map of each detection area;
[0012] Count the number of directional deviations in the disparity gradient vector within each detection area, as well as the corresponding deviation angle;
[0013] The road surface smoothness in each detection area is predicted based on the number of directional deviations and the corresponding deviation angles in the corresponding detection area.
[0014] Optionally, determining a disparity gradient vector corresponding to each detection area based on the disparity map of each detection area includes:
[0015] Divide each detection area into multiple units;
[0016] Calculate the disparity gradient vector of the corresponding unit according to the disparity pixel value in the disparity map of each unit;
[0017] The disparity gradient vectors of all units in each detection area are taken as the disparity gradient vector set of the corresponding detection area.
[0018] Optionally, the number of directional deviations in the disparity gradient vector within each detection area and the corresponding deviation angles are counted, including:
[0019] Get all disparity gradient vectors in each detection area;
[0020] The number of directional deviations and the corresponding deviation angles in the disparity gradient vector in each detection area are determined according to a set statistical strategy.
[0021] Optionally, determining the number of directional deviations and corresponding deviation angles of the disparity gradient vector in each detection area according to a preset statistical strategy includes:
[0022] Calculate the directional variance of the disparity gradient vector within each detection area, compare the directional variance of the disparity gradient vector within each detection area with the predicted statistical value of the ideal flat road surface, and determine the number of directional deviations and the corresponding deviation angles of the disparity gradient vector within the corresponding detection area based on the comparison results;
[0023] or
[0024] Calculating the average value of the disparity gradient vectors in each detection area; comparing all the disparity gradient vectors in each detection area with the average value in the corresponding detection area, and determining the number of directional deviations and the corresponding deviation angles in the disparity gradient vectors in the corresponding detection area based on the comparison results;
[0025] or
[0026] Calculate the overall offset of all disparity gradient vectors in each detection area; compare the overall offset of all disparity gradient vectors in each detection area with the overall offset of disparity gradient vectors in adjacent detection areas, and determine the number of directional deviations and corresponding deviation angles in the disparity gradient vectors in the corresponding detection area based on the comparison results.
[0027] Optionally, predicting the road surface smoothness within each detection area based on the number of directional deviations and the corresponding deviation angle within the corresponding detection area includes:
[0028] When the number of directional deviations in each detection area is greater than a set threshold, and the value of each deviation angle is greater than a preset angle threshold, determining that the road surface in the corresponding detection area is rough and uneven; or
[0029] When the number of directional deviations in each detection area is greater than a set threshold and the value of each deviation angle is approximately equal to the preset angle threshold, it is determined that the road surface in the corresponding detection area is a slope road surface.
[0030] Optionally, before obtaining the binocular stereo vision disparity map of the road area to be detected, the method further includes:
[0031] Obtain the installation inclination angle and installation height of the binocular stereo camera and the road surface, as well as the binocular positioning parameters of the binocular stereo camera;
[0032] Based on the installation inclination angle, installation height and binocular positioning parameters, each detection area is controlled to be back-projected into the image field of view.
[0033] Optionally, obtaining a binocular stereo vision disparity map of the road area to be detected includes:
[0034] Obtain left and right views of the road area to be detected;
[0035] The corresponding pixel values in the left and right views are matched to obtain the disparity map of binocular stereo vision.
[0036] According to a second aspect of an embodiment of the present application, a device for detecting road surface flatness is provided, comprising:
[0037] An acquisition module is used to acquire a binocular stereoscopic disparity map of a road surface area to be detected, where the road surface area to be detected is divided into a plurality of detection areas of the same size;
[0038] a determination module, configured to determine a disparity gradient vector of a corresponding detection area based on the disparity map of each detection area;
[0039] A statistics module is used to count the number of directional deviations in the disparity gradient vector within each detection area, as well as the corresponding deviation angles;
[0040] The prediction module is used to predict the road surface smoothness in each detection area based on the number of direction deviations and the corresponding deviation angles in the corresponding detection area.
[0041] Optionally, the determination module includes:
[0042] A division module, used to divide each detection area into multiple units;
[0043] A first calculation module is used to calculate the disparity gradient vector of the corresponding unit according to the disparity pixel value in the disparity map of each unit;
[0044] The vector determination module is configured to use the disparity gradient vectors of all units in each detection area as a disparity gradient vector set of the corresponding detection area.
[0045] Optional statistics modules include:
[0046] A vector acquisition module, used to obtain all disparity gradient vectors in each detection area;
[0047] The statistical determination module is used to determine the number of directional deviations and corresponding deviation angles in the disparity gradient vector in each detection area according to a set statistical strategy.
[0048] Optionally, the statistical determination module includes: a variance calculation module, a first comparison module and a first determination module; and / or an average calculation module, a second comparison module and a second determination module; and / or an offset calculation module, a third comparison module and a third determination module, wherein:
[0049] A variance calculation module is used to calculate the directional variance value of the disparity gradient vector in each detection area;
[0050] A first comparison module is used to compare the directional variance value of the disparity gradient vector in each detection area with the predicted statistical value of the ideal flat road surface;
[0051] A first determining module is configured to determine the number of directional deviations and corresponding deviation angles of the disparity gradient vector in the corresponding detection area according to the comparison result;
[0052] An average calculation module is used to calculate the average value of the disparity gradient vector in each detection area;
[0053] A second comparison module is used to compare all disparity gradient vectors in each detection area with the average value in the corresponding detection area;
[0054] A second determining module is configured to determine the number of directional deviations and corresponding deviation angles of the disparity gradient vector in the corresponding detection area according to the comparison result;
[0055] An offset calculation module is used to calculate the overall offset of all disparity gradient vectors in each detection area;
[0056] a third comparison module, configured to compare the overall offsets of all disparity gradient vectors in each detection area with the overall offsets of the disparity gradient vectors in adjacent detection areas;
[0057] The third determining module is configured to determine the number of directional deviations and the corresponding deviation angles of the disparity gradient vector in the corresponding detection area according to the comparison result.
[0058] Optionally, the prediction module includes: a first road surface level determination module and / or a second road surface level determination module, wherein:
[0059] A first road surface degree determination module is configured to determine that the road surface in each detection area is rough and uneven when the number of directional deviations in each detection area is greater than a set threshold and the value of each deviation angle is greater than a preset angle threshold;
[0060] The second road surface degree determination module is used to determine that the road surface in each detection area is a slope road surface when the number of directional deviations in each detection area is greater than a set threshold and the value of each deviation angle is approximately equal to a preset angle threshold.
[0061] Optionally, before obtaining the binocular stereo vision disparity map of the road area to be detected, the device further includes:
[0062] A parameter acquisition module is used to obtain the installation inclination angle and installation height of the binocular stereo camera and the road surface, as well as the binocular positioning parameters of the binocular stereo camera before the acquisition module obtains the binocular stereo vision disparity map of the road surface area to be detected;
[0063] The control module is used to control the reverse projection of each detection area into the image field of view based on the installation inclination angle, the installation height and the binocular positioning parameters.
[0064] Optionally, the acquisition module includes:
[0065] A view acquisition module, used to obtain left and right views of the road area to be detected;
[0066] The processing module is used to match the corresponding pixel values in the left and right views to obtain a disparity map of binocular stereo vision.
[0067] According to a third aspect of an embodiment of the present application, a road surface flatness detection system is provided, comprising: a binocular stereo camera, and an electronic device connected to the binocular stereo camera, wherein the binocular stereo camera is connected to the electronic device, the electronic device is deployed inside a car or integrated into the car's central control platform, and the binocular stereo camera is mounted on the outer wall of the car, wherein:
[0068] A binocular stereo camera is used to capture left and right views of the road area to be inspected. The road area to be inspected is divided into multiple inspection areas of equal size.
[0069] An electronic device for processing corresponding pixel values in the left and right views captured by a binocular stereo camera to obtain a disparity map of binocular stereo vision, determining the disparity gradient vector of the corresponding detection area based on the disparity map of each detection area, counting the number of directional deviations in the disparity gradient vector within each detection area and the corresponding deviation angle, and predicting the flatness of the road surface within the corresponding detection area based on the number of directional deviations and the corresponding deviation angle within each detection area.
[0070] According to a fourth aspect of the embodiments of the present application, there is provided an electronic device, characterized by comprising:
[0071] processor;
[0072] a memory for storing processor-executable instructions;
[0073] The processor is configured to execute instructions to implement the above method for detecting road surface flatness.
[0074] According to a fifth aspect of an embodiment of the present application, a computer-readable storage medium is provided. When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device can execute the above-mentioned method for detecting the flatness of a road surface.
[0075] According to a sixth aspect of an embodiment of the present application, a computer program product is provided, comprising a computer program or instructions, which implements the above-mentioned method for detecting road surface flatness when the computer program or instructions are executed by a processor of an electronic device.
[0076] The technical solutions provided by the embodiments of this application bring at least the following beneficial effects:
[0077] In an embodiment of the present application, a binocular stereo vision disparity map of a road area to be inspected is obtained, where the road area to be inspected is divided into multiple inspection areas of equal size; a disparity gradient vector of the corresponding inspection area is determined based on the disparity map of each inspection area; the number of directional deviations in the disparity gradient vector within each inspection area and the corresponding deviation angle are counted; and the road surface flatness within the corresponding inspection area is predicted based on the number of directional deviations in the disparity gradient vector within each inspection area and the corresponding deviation angle. In other words, an embodiment of the present application utilizes a binocular stereo camera to obtain a disparity map containing depth information, determines the corresponding disparity gradient vector based on the disparity map, and determines the road surface flatness on the disparity map based on the deviation direction and angle of the disparity gradient vector. This not only reduces the computational cost of road surface flatness and effectively improves the accuracy and efficiency of road surface flatness assessment, but also optimizes road perception for assisted driving, improving driving comfort and safety.
[0078] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0079] The accompanying drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application and do not constitute an undue limitation of the present application. In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without inventive effort.
[0080] FIG1 is a flow chart of a method for detecting road surface flatness provided in an embodiment of the present application.
[0081] FIG2 is a schematic diagram of installing a binocular stereo camera on a road surface roughness detection vehicle according to an embodiment of the present application.
[0082] FIG3 is a schematic diagram of a back projection example provided in an embodiment of the present application.
[0083] FIG4 is a schematic diagram of unit segmentation provided for an embodiment of the present application.
[0084] FIG5 is a schematic diagram of photographing an undulating or rough road surface provided by an embodiment of the present application.
[0085] FIG6A is a schematic diagram of each disparity vector direction in an ideal flat road surface provided by an embodiment of the present application.
[0086] FIG6B is a schematic diagram of each disparity vector direction of a rough road surface provided by an embodiment of the present application.
[0087] FIG6C is a schematic diagram of each disparity vector direction of a road surface slope provided by an embodiment of the present application.
[0088] FIG7 is a block diagram of a road surface flatness detection device provided in an embodiment of the present application.
[0089] FIG8 is a block diagram of a determination module provided in an embodiment of the present application.
[0090] FIG9 is another block diagram of a road surface flatness detection device provided in an embodiment of the present application.
[0091] FIG10 is a block diagram of a road surface flatness detection system provided in an embodiment of the present application.
[0092] FIG11 is a block diagram of an electronic device provided in an embodiment of the present application.
[0093] FIG12 is a block diagram of a device for detecting road surface flatness based on a disparity map, provided in an embodiment of the present application. DETAILED DESCRIPTION
[0094] In order to enable ordinary people in the art to better understand the technical solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.
[0095] It should be noted that the terms "first," "second," and the like in the specification and claims of this application and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate so that the embodiments of the application described herein can be implemented in an order other than those illustrated or described herein. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.
[0096] In recent years, significant progress has been made in the research of AI-based technologies such as computer vision, deep learning, machine learning, image processing, and image recognition. Artificial Intelligence (AI) is an emerging science and technology that studies and develops theories, methods, technologies, and application systems for simulating and extending human intelligence. AI is a comprehensive discipline that involves many technologies, including chips, big data, cloud computing, the Internet of Things, distributed storage, deep learning, machine learning, and neural networks. Computer vision, as an important branch of AI, specifically enables machines to understand the world. Computer vision technologies typically include face recognition, liveness detection, fingerprint recognition and anti-counterfeiting verification, biometric recognition, face detection, pedestrian detection, object detection, pedestrian recognition, image processing, image recognition, image semantic understanding, image retrieval, text recognition, video processing, video content recognition, behavior recognition, 3D reconstruction, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), computational photography, and robot navigation and positioning. With the research and advancement of artificial intelligence technology, this technology has been applied in many fields, such as security, urban management, traffic management, building management, park management, facial access, facial attendance, logistics management, warehouse management, robots, intelligent marketing, computational photography, mobile phone imaging, cloud services, smart homes, wearable devices, unmanned driving, autonomous driving, smart medical care, facial payment, facial unlocking, fingerprint unlocking, identity verification, smart screens, smart TVs, cameras, mobile Internet, live streaming, beauty, makeup, medical beauty, smart temperature measurement and other fields.
[0097] FIG1 is a flow chart of a method for detecting road surface flatness provided in an embodiment of the present application. As shown in FIG1 , the method includes the following steps:
[0098] Step 101: Obtain a binocular stereoscopic disparity map of a road surface area to be detected. The road surface area to be detected is divided into multiple detection areas of the same size.
[0099] Step 102: Determine the disparity gradient vector corresponding to each detection area based on the disparity map of each detection area.
[0100] Step 103: Count the number of direction deviations in the disparity gradient vector within each detection area and the corresponding deviation angles.
[0101] Step 104: Predicting the road surface smoothness within each detection area based on the number of directional deviations and the corresponding deviation angles within the corresponding detection area.
[0102] The road surface flatness detection method of the present application can be applied to terminals, servers, etc., without limitation here. The terminal implementation device can be a smart phone, a laptop computer, a tablet computer, a desktop computer, a personal digital assistant (PDA, Personal Digital Assistant) and a road surface flatness detection terminal and other electronic devices. The server can be an independent server or a server cluster, or provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, intermediate services, domain name services, security services, road surface flatness detection platforms, or big data and artificial intelligence platforms, etc., without limitation here.
[0103] 1 , the specific implementation steps of a road surface flatness detection method provided in an embodiment of the present application are described in detail.
[0104] In step 101, a binocular stereoscopic disparity map of a road surface area to be detected is obtained, and the road surface area to be detected is divided into a plurality of detection areas of the same size;
[0105] In this step, a binocular stereo camera can be used to obtain left and right views of the same road area to be inspected, and the corresponding pixel values in the left and right views are matched to obtain a binocular stereo disparity map. In this embodiment, there are various methods for obtaining a binocular stereo disparity map, such as semi-global matching, structured light projection, deep learning, and other methods. These methods are well known to those skilled in the art and will not be described in detail here.
[0106] In this embodiment, the binocular stereo camera is pre-installed on a road surface roughness inspection vehicle, or on another mobile terminal for road surface roughness inspection, without limitation in this embodiment. This embodiment uses installation on a road surface roughness inspection vehicle as an example, and the corresponding schematic diagram is shown in FIG2 , which is a schematic diagram of a binocular stereo camera installed on a road surface roughness inspection vehicle provided in this embodiment. After the binocular stereo camera is installed, the installation inclination angle θ between the binocular stereo camera and the ground, as well as the installation height h, can be obtained through vehicle calibration.
[0107] In this embodiment, from the perspective of the binocular stereo camera, the road surface is usually pre-divided into MxN detection areas of equal size according to the requirements of the management department and industry standards. According to the installation inclination angle θ and the installation height h and the binocular positioning parameters of the camera (for example, including fx, fy, Q matrix, etc.), the detection area is controlled to be reversely projected into the image field of view. As shown in Figure 3, a schematic diagram of a reverse projection example provided in an embodiment of the present application is shown. In this embodiment, the detection area is taken as an example of 5x3, but in actual application, it is not limited to this. The dotted line in Figure 3 is the center line of the field of view.
[0108] Among them, an image back-projection example can establish the projection transformation of three-dimensional scenes and two-dimensional images based on the imaging principle of the camera, complete the calibration of the road scene, design the back-projection surface, reconstruct the road information on the back-projection surface, and obtain the back-projection map.
[0109] In step 102, a disparity gradient vector of a corresponding detection area is determined based on the disparity map of each detection area;
[0110] In this step, each detection area is divided into multiple units (cells), where the size of the cell can be 9x9 pixels or 11x11, etc. (the size is related to the specific gradient detection algorithm, pixel resolution and other parameters, and is not limited here. The pixel areas of two adjacent cells can partially overlap). As shown in Figure 4, a unit segmentation schematic diagram provided in an embodiment of the present application is shown; as shown in the figure, the disparity gradient vector of the corresponding unit is calculated according to the disparity pixel value in the disparity map of each unit; the disparity gradient vectors of all units in each detection area are used as the disparity gradient vector set of the corresponding detection area.
[0111] There are many ways to calculate the image gradient vector of the disparity map, such as Sobel, Scharr, Lapacian and other algorithms. The image gradient can be regarded as a two-dimensional discrete function. The image gradient is actually the derivative of this two-dimensional discrete function. This embodiment calculates the gradient vector in a simple way. The image gradient vector calculation formula is as follows:
[0112] Image gradient: G(x,y)=dx(i,j)+dy(i,j);
[0113] dx(i,j)=I(i+1,j)-I(i,j); dy(i,j)=I(i,j+1)-I(i,j).
[0114] Where I is the value of the image pixel (disparity value), and (i, j) is the coordinate of the pixel.
[0115] Of course, image gradients can generally also be calculated using median differences:
[0116] dx(i,j)=[I(i+1,j)-I(i-1,j)] / 2; dy(i,j)=[I(i,j+1)-I(i,j-1)] / 2;
[0117] Image edges are generally achieved by performing gradient operations on the image.
[0118] It should be noted that the above all use simple formulas to calculate the image gradient. In practical applications, some complex gradient formulas can also be used for calculation, and this embodiment does not impose any limitation.
[0119] In step 103 , the number of direction deviations in the disparity gradient vector within each detection area and the corresponding deviation angles are counted.
[0120] In this step, all disparity gradient vectors in each detection area are first obtained; then, the number of directional deviations and the corresponding deviation angles in each detection area are determined according to a set statistical strategy.
[0121] That is, when the ground is undulating (e.g., with a slope or dip) or encounters a rough road surface, the distance between each point on the road surface and the camera's base point differs from the distance between each point on the road surface and the base point of an ideal flat road surface, which is reflected in the disparity map, as shown in Figure 5, which is a schematic diagram of capturing an undulating or rough road surface according to an embodiment of the present application. The dotted line in the figure represents the undulating or rough road surface.
[0122] According to the principle of triangulation, the parallax of a point in the image field of view is inversely proportional to the distance from the camera base point. Based on the known installation angle and height, the parallax formula can be used to obtain the ideal parallax value for each point on an ideal flat road surface. The parallax formula is: D = bf / Distance
[0123] Where D is the disparity, b is the binocular baseline size, f is the focal length, and distance is the distance.
[0124] In other words, this step is based on analyzing the disparity map to obtain all disparity gradient vectors within each detection area. Taking a detection area as an example, Figure 6A shows a schematic diagram of the direction of each disparity vector within an ideal flat road surface provided in an embodiment of the present application. All disparity vectors within this ideal flat road surface are known. Specifically, based on the principle of triangulation, since the disparity of a point within the image field of view is inversely proportional to the distance from the camera base point, the ideal disparity value for each point on the ideal flat road surface can be obtained using the disparity formula based on the known installation angle and height.
[0125] When potholes appear on the road, due to the parallax difference, some parallax gradient vectors will deviate in direction, as shown in Figures 6B and 6C. Figure 6B is a schematic diagram of the direction of each parallax vector for a rough and uneven road surface provided by an embodiment of the present application, and Figure 6C is a schematic diagram of the direction of each parallax vector for a road surface with a slope provided by an embodiment of the present application.
[0126] In this embodiment, the number of parallax deviations and the size of the deviation angle can be statistically calculated. The statistical methods include but are not limited to: comparing with the theoretical value of an ideal flat road surface, comparing with the average value within the detection area, or comparing with the surrounding detection areas as a whole.
[0127] Specifically, the number of directional deviations and the corresponding deviation angles in each detection area are determined according to a preset statistical strategy, including any of the following methods:
[0128] In one approach, the directional variance value of the disparity gradient vector within each detection area is calculated, and the directional variance value of the disparity gradient vector within each detection area is compared with the predicted statistical value of an ideal flat road surface. Based on the comparison result, the number of directional deviations in the disparity gradient vector within the corresponding detection area and the corresponding deviation angle are determined.
[0129] For example, in the comparison results, when the directional variance values differ greatly, the gradient vector distribution is relatively dispersed.
[0130] In another method, the average value of the disparity gradient vector in each detection area is calculated; all the disparity gradient vectors in each detection area are compared with the average value in the corresponding detection area, and the number of directional deviations and the corresponding deviation angles in the disparity gradient vectors in the corresponding detection area are determined based on the comparison results.
[0131] In another method, the overall offset of all disparity gradient vectors in each detection area is calculated; the overall offset of all disparity gradient vectors in each detection area is compared with the overall offset of the disparity gradient vectors in the surrounding adjacent detection areas, and the number of directional deviations and the corresponding deviation angles in the disparity gradient vectors in the corresponding detection area are determined based on the comparison results.
[0132] In step 104 , the road surface smoothness in each detection area is predicted based on the number of directional deviations and the corresponding deviation angles in the corresponding detection area.
[0133] In this embodiment, when the number of directional deviations in each detection area is greater than a set threshold, and the value of each deviation angle is greater than a preset angle threshold, it is determined that the road surface in the corresponding detection area is a rough and uneven road surface; or
[0134] When the number of directional deviations in each detection area is greater than a set threshold and the value of each deviation angle is approximately equal to the preset angle threshold, it is determined that the road surface in the corresponding detection area is a slope road surface.
[0135] Of course, in specific applications, it is not limited to the above process, and may also include other methods, which will not be described in detail in this embodiment.
[0136] In an embodiment of the present application, a binocular stereo vision disparity map of a road area to be inspected is obtained, and the road area to be inspected is divided into multiple inspection areas of equal size; a disparity gradient vector of the corresponding inspection area is determined based on the disparity map of each inspection area; the number of directional deviations in the disparity gradient vector in each inspection area and the corresponding deviation angle are counted; and the road surface flatness in the corresponding inspection area is predicted based on the number of directional deviations in the disparity gradient vector in each inspection area and the corresponding deviation angle. In other words, the embodiment of the present application utilizes a binocular stereo camera to obtain a disparity map containing depth information, determines the corresponding disparity gradient vector based on the disparity map, and then judges the road surface flatness on the disparity map based on the deviation statistics and deviation angle of the disparity gradient vector. This not only reduces the computational cost of road surface flatness and effectively improves the accuracy and efficiency of road surface flatness assessment, but also optimizes road perception for assisted driving, improving driving comfort and safety.
[0137] It should be noted that for the method embodiments, for simplicity of description, they are all expressed as a series of action combinations. However, those skilled in the art should be aware that this disclosure is not limited by the order of the actions described, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions involved are not necessarily required by this application.
[0138] Please also refer to FIG7, which is a block diagram of a road surface flatness detection device provided in an embodiment of the present application. The device includes: an acquisition module 701, a determination module 702, a statistics module 703 and a prediction module 704, wherein:
[0139] The acquisition module 701 is used to obtain a binocular stereoscopic disparity map of a road area to be detected, where the road area to be detected is divided into multiple detection areas of the same size;
[0140] The determination module 702 is configured to determine a disparity gradient vector corresponding to each detection area based on the disparity map of each detection area;
[0141] The statistics module 703 is used to count the number of direction deviations of the disparity gradient vector in each detection area and the corresponding deviation angles;
[0142] The prediction module 704 is configured to predict the road surface smoothness within each detection area based on the number of directional deviations in the disparity gradient vector and the corresponding deviation angle within the corresponding detection area.
[0143] Optionally, in another embodiment, based on the above embodiment, the determination module 702 includes: a division module 801, a first calculation module 802 and a vector determination module 803, the structural block diagram of which is shown in FIG8 , wherein:
[0144] The division module 801 is used to divide each detection area into multiple units;
[0145] The first calculation module 802 is configured to calculate a disparity gradient vector of a corresponding unit according to a disparity pixel value in a disparity map of each unit;
[0146] The vector determination module 803 is configured to use the disparity gradient vectors of all units in each detection area as the disparity gradient vectors of the corresponding detection area.
[0147] Optionally, in another embodiment, based on the above embodiment, the statistical module includes: a vector acquisition module and a statistical determination module, wherein:
[0148] The vector acquisition module is used to obtain all disparity gradient vectors in each detection area;
[0149] The statistical determination module is used to determine the number of directional deviations and the corresponding deviation angles in each detection area according to a set statistical strategy.
[0150] Optionally, in another embodiment, based on the above embodiment, the statistical determination module includes: a variance calculation module, a first comparison module and a first determination module; and / or an average calculation module, a second comparison module and a second determination module; and / or an offset calculation module, a third comparison module and a third determination module, wherein:
[0151] A variance calculation module is used to calculate the directional variance value of the disparity gradient vector in each detection area;
[0152] A first comparison module is used to compare the directional variance value of the disparity gradient vector in each detection area with the predicted statistical value of the ideal flat road surface;
[0153] A first determining module is configured to determine the number of directional deviations and corresponding deviation angles of the disparity gradient vector in the corresponding detection area according to the comparison result;
[0154] An average calculation module is used to calculate the average value of the disparity gradient vector in each detection area;
[0155] A second comparison module is used to compare all disparity gradient vectors in each detection area with the average value in the corresponding detection area;
[0156] A second determining module is configured to determine the number of directional deviations and corresponding deviation angles of the disparity gradient vector in the corresponding detection area according to the comparison result;
[0157] An offset calculation module is used to calculate the overall offset of all disparity gradient vectors in each detection area;
[0158] a third comparison module, configured to compare the overall offsets of all disparity gradient vectors in each detection area with the overall offsets of the disparity gradient vectors in adjacent detection areas;
[0159] The third determining module is configured to determine the number of directional deviations and the corresponding deviation angles of the disparity gradient vector in the corresponding detection area according to the comparison result.
[0160] Optionally, in another embodiment, based on the above embodiment, the prediction module includes: a first road surface level determination module and / or a second road surface level determination module, wherein:
[0161] A first road surface level determination module is configured to determine that the road surface in each detection area is rough and uneven when the number of directional deviations in each detection area is greater than a set threshold and the value of each deviation angle is not greater than a preset angle threshold;
[0162] The second road surface degree determination module is used to determine that the road surface in each detection area is a slope road surface when the number of directional deviations in each detection area is greater than a set threshold and the value of each deviation angle is approximately equal to a preset angle threshold.
[0163] Optionally, in another embodiment, based on the above embodiment, the device further includes: a parameter acquisition module 901 and a control module 902, whose structural block diagram is shown in FIG9 , wherein:
[0164] The parameter acquisition module 901 is used to obtain the installation inclination angle and installation height of the binocular stereo camera and the road surface, as well as the binocular positioning parameters of the binocular stereo camera before the acquisition module 701 obtains the binocular stereo vision disparity map of the road surface area to be detected;
[0165] The control module 902 is used to control the reverse projection of each detection area into the image field of view based on the installation inclination angle, the installation height and the binocular positioning parameters.
[0166] Optionally, in another embodiment, based on the above embodiment, the acquisition module includes: a view acquisition module and a processing module, wherein:
[0167] The view acquisition module is used to obtain left and right views of the road area to be detected;
[0168] This processing module is used to match the corresponding pixel values in the left and right views to obtain a disparity map for binocular stereo vision.
[0169] Please also refer to Figure 10, which is a block diagram of a road surface flatness detection system provided in an embodiment of the present application. The system includes: a binocular stereo camera 1001, an electronic device 1002 and a car 1003. The binocular stereo camera 1001 is connected to the electronic device 1002, and the electronic device 1002 is deployed inside the car 1003 or integrated on the car's central control platform. The binocular stereo camera 1001 is installed on the outside of the car 1003, where the outside of the car can be the rear of the roof or on the door of the car trunk, etc. without limitation.
[0170] A binocular stereo camera 1001 is used to collect left and right views of the road area to be inspected. The road area to be inspected is divided into multiple inspection areas of equal size.
[0171] Electronic device 1002 is used to process corresponding pixel values in the left and right views captured by the binocular stereo camera 1001 to obtain a disparity map of binocular stereo vision, determine the disparity gradient vector of the corresponding detection area based on the disparity map of each detection area, count the number of directional deviations in the disparity gradient vector in each detection area, and the corresponding deviation angle, and predict the flatness of the road surface in the corresponding detection area based on the number of directional deviations in the disparity gradient vector in each detection area and the corresponding deviation angle.
[0172] Optionally, the electronic device 1002 includes: an acquisition module, a determination module, a statistics module and a prediction module. The functions and effects of each module are detailed above and will not be repeated here.
[0173] Optionally, an embodiment of the present application further provides an electronic device, including:
[0174] processor;
[0175] a memory for storing processor-executable instructions;
[0176] The processor is configured to execute instructions to implement the above method for detecting road surface flatness.
[0177] Optionally, an embodiment of the present application further provides a computer-readable storage medium, which, when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, enables the electronic device to perform the above-mentioned method for detecting road surface flatness.
[0178] Optionally, an embodiment of the present application further provides a computer program product, including a computer program or instructions, which implements the above-mentioned road surface flatness detection method when the computer program or instructions are executed by a processor of an electronic device.
[0179] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.
[0180] The device embodiments described above are merely illustrative. Modules described as separate components may or may not be physically separate, and components shown as modules may or may not be physical units, i.e., they may be located in one place or distributed across multiple network modules. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Those skilled in the art can understand and implement the present invention without inventive effort.
[0181] Figure 11 is a block diagram of an electronic device 1100 provided in an embodiment of the present application. For example, electronic device 1100 can be a mobile terminal or a server. The present embodiment uses a mobile terminal as an example for description. For example, electronic device 1100 can be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.
[0182] 11 , the electronic device 1100 may include one or more of the following components: a processing component 1102 , a memory 1104 , a power component 1106 , a multimedia component 1108 , an audio component 1110 , an input / output (I / O) interface 1112 , a sensor component 1114 , and a communication component 1116 .
[0183] The processing component 1102 generally controls the overall operation of the electronic device 1100, such as operations associated with display, phone calls, data communications, camera operation, and recording operations. The processing component 1102 may include one or more processors 1120 to execute instructions to perform all or part of the steps of the above-described method. In addition, the processing component 1102 may include one or more modules to facilitate interaction between the processing component 1102 and other components. For example, the processing component 1102 may include a multimedia module to facilitate interaction between the multimedia component 1108 and the processing component 1102.
[0184] The memory 1104 is configured to store various types of data to support operations on the device 1100. Examples of such data include instructions for any application or method operating on the electronic device 1100, contact data, phone book data, messages, pictures, videos, etc. The memory 1104 can be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.
[0185] The power supply component 1106 provides power to the various components of the electronic device 1100. The power supply component 1106 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 1100.
[0186] The multimedia component 1108 includes a screen that provides an output interface between the electronic device 1100 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, slides, and gestures on the touch panel. The touch sensor can not only sense the boundaries of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 1108 includes a front camera and / or a rear camera. When the device 1100 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have a focal length and optical zoom capability.
[0187] The audio component 1110 is configured to output and / or input audio signals. For example, the audio component 1110 includes a microphone (MIC), which is configured to receive external audio signals when the electronic device 1100 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 1104 or transmitted via the communication component 1116. In some embodiments, the audio component 1110 also includes a speaker for outputting audio signals.
[0188] I / O interface 1112 provides an interface between processing component 1102 and peripheral interface modules, such as a keyboard, click wheel, buttons, etc. These buttons may include but are not limited to: a home button, volume buttons, a start button, and a lock button.
[0189] The sensor assembly 1114 includes one or more sensors for providing various aspects of the status assessment of the electronic device 1100. For example, the sensor assembly 1114 can detect the open / closed state of the device 1100, the relative positioning of components, such as the display and keypad of the electronic device 1100. The sensor assembly 1114 can also detect changes in the position of the electronic device 1100 or a component of the electronic device 1100, the presence or absence of user contact with the electronic device 1100, the orientation or acceleration / deceleration of the electronic device 1100, and changes in the temperature of the electronic device 1100. The sensor assembly 1114 can include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 1114 can also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 1114 can also include an accelerometer, a gyroscope, a magnetic sensor, a pressure sensor, or a temperature sensor.
[0190] The communication component 1116 is configured to facilitate wired or wireless communication between the electronic device 1100 and other devices. The electronic device 1100 can access a wireless network based on a communication standard, such as WiFi (wireless network), an operator network (such as 2G, 3G, 4G or 5G), or a combination thereof. In an exemplary embodiment, the communication component 1116 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 1116 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.
[0191] In an embodiment, the electronic device 1100 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to execute the road surface flatness detection method shown above.
[0192] In an embodiment, a computer-readable storage medium is also provided. When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device 1100 can perform the road surface roughness detection method described above. For example, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM (Compact Disc Read Only Memory), a magnetic tape, a floppy disk, an optical data storage device, etc.
[0193] In an embodiment, a computer program product is also provided, including a computer program or instructions. When the computer program or instructions are executed by the processor 1120 of the electronic device 1100, the electronic device 1100 executes the road surface flatness detection method shown above.
[0194] FIG12 is a block diagram of an apparatus 1200 for detecting road surface roughness based on a disparity map, provided in an embodiment of the present application. For example, apparatus 1200 may be provided as a server. Referring to FIG12 , apparatus 1200 includes a processing component 1222, which further includes one or more processors, and a memory resource represented by a memory 1232 for storing instructions executable by processing component 1222, such as an application. The application stored in memory 1232 may include one or more modules, each corresponding to a set of instructions. In addition, processing component 1222 is configured to execute instructions to perform the above-described method.
[0195] The device 1200 may also include a power supply component 1226 configured to perform power management of the device 1200, a wired or wireless network interface 1250 configured to connect the device 1200 to a network, and an input / output (I / O) interface 1258. The device 1200 may operate based on an operating system stored in the memory 1232, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, or the like.
[0196] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, and the true scope and spirit of the present application are indicated by the following claims.
[0197] It should be understood that the present application is not limited to the exact structures described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.
Claims
1. A method for detecting road surface flatness, It is characterized in that include: Acquire a binocular stereoscopic disparity map of a road surface area to be detected, wherein the road surface area to be detected is divided into a plurality of detection areas of the same size; Determine a disparity gradient vector of a corresponding detection area based on the disparity map of each detection area; Counting the number of direction deviations in the disparity gradient vector in each detection area and the corresponding deviation angle; The road surface flatness in each detection area is predicted based on the number of directional deviations and the corresponding deviation angles in the corresponding detection area.
2. The method for detecting road surface flatness according to claim 1, It is characterized in that The determining the disparity gradient vector of the corresponding detection area based on the disparity map of each detection area includes: Divide each detection area into multiple units; Calculate the disparity gradient vector of the corresponding unit according to the disparity pixel value in the disparity map of each unit; The disparity gradient vectors of all units in each detection area are taken as the disparity gradient vector set of the corresponding detection area.
3. The method for detecting road surface flatness according to claim 2, It is characterized in that The counting of the number of direction deviations in the disparity gradient vector in each detection area and the corresponding deviation angles includes: Get all disparity gradient vectors in each detection area; The number of direction deviations and the corresponding deviation angles of the disparity gradient vectors in each detection area are determined according to a set statistical strategy.
4. The method for detecting road surface flatness according to claim 3, It is characterized in that Determining the number of directional deviations and corresponding deviation angles of the disparity gradient vector in each detection area according to a preset statistical strategy includes: Calculating the directional variance value of the disparity gradient vector in each detection area, comparing the directional variance value of the disparity gradient vector in each detection area with the predicted statistical value of the ideal flat road surface, and determining the number of directional deviations and the corresponding deviation angles in the disparity gradient vector in the corresponding detection area according to the comparison result; or Calculating an average value of the disparity gradient vectors in each detection area; comparing all the disparity gradient vectors in each detection area with the average value in the corresponding detection area, and determining the number of directional deviations and the corresponding deviation angles in the disparity gradient vectors in the corresponding detection area according to the comparison results; or Calculate the overall offset of all disparity gradient vectors in each detection area; compare the overall offset of all disparity gradient vectors in each detection area with the overall offset of disparity gradient vectors in surrounding adjacent detection areas, and determine the number of directional deviations and the corresponding deviation angles in the disparity gradient vectors in the corresponding detection area according to the comparison results.
5. The method for detecting road surface flatness according to claim 1, It is characterized in that The predicting of the road surface flatness in the corresponding detection area based on the number of direction deviations and the corresponding deviation angles in each detection area includes: When the number of directional deviations in each detection area is greater than a set threshold, and the value of each deviation angle is greater than a preset angle threshold, it is determined that the road surface in the corresponding detection area is a rough and uneven road surface; or The number of directional deviations in each detection area is greater than the set threshold, and the value of each deviation angle is approximately equal. When the angle threshold is preset, it is determined that the road surface in the corresponding detection area is a slope road surface.
6. The method for detecting road surface flatness according to any one of claims 1 to 5, It is characterized in that Before obtaining the disparity map of the binocular stereo vision of the road area to be detected, the method further includes: Obtain the installation inclination angle and installation height of the binocular stereo camera and the road surface, as well as the binocular positioning parameters of the binocular stereo camera; Based on the installation inclination angle, the installation height and the binocular positioning parameters, each detection area is controlled to be back-projected into the image field of view.
7. The method for detecting road surface flatness according to any one of claims 1 to 5, It is characterized in that The step of obtaining a binocular stereoscopic disparity map of the road area to be detected includes: Obtain left and right views of the road area to be detected; The corresponding pixel values in the left and right views are matched to obtain a disparity map of binocular stereo vision.
8. A device for detecting road surface flatness, It is characterized in that include: An acquisition module, used for acquiring a disparity map of binocular stereo vision of a road surface area to be detected, wherein the road surface area to be detected is divided into a plurality of detection areas of the same size; A determination module, configured to determine a disparity gradient vector of a corresponding detection area based on the disparity map of each detection area; A statistics module, used to count the number of direction deviations in the disparity gradient vector in each detection area and the corresponding deviation angles; The prediction module is used to predict the road surface flatness in the corresponding detection area based on the number of directional deviations and the corresponding deviation angles in each detection area.
9. A road surface smoothness detection system, It is characterized in that include: A binocular stereo camera, and an electronic device connected to the binocular stereo camera, the binocular stereo camera is connected to the electronic device, the electronic device is deployed inside the car or integrated on the central control platform of the car, and the binocular stereo camera is installed on the outer wall of the car, wherein, The binocular stereo camera is used to collect left and right views of the road area to be detected, and the road area to be detected is divided into multiple detection areas of the same size; The electronic device is used to process corresponding pixel values in the left and right views collected by the binocular stereo camera to obtain a disparity map of binocular stereo vision, determine the disparity gradient vector of the corresponding detection area based on the disparity map of each detection area, count the number of directional deviations in the disparity gradient vector in each detection area, and the corresponding deviation angle, and predict the road surface flatness in the corresponding detection area based on the number of directional deviations and the corresponding deviation angle in each detection area.
10. An electronic device, It is characterized in that include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the method for detecting road surface flatness as described in any one of claims 1 to 7.
11. A computer-readable storage medium, It is characterized in that When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the method for detecting road surface flatness as described in any one of claims 1 to 7.
Citation Information
Patent Citations
Passable road detection method based on binocular vision
CN106446785A
A binocular vision stereo matching method and system for road detection
CN109522833A
Smart city road pavement detection method and system based on computer vision
CN113554646A
Road surface flatness detection method, device and system, electronic equipment and medium
CN116630253A
Detection device for detecting obstacle and steep slope and vehicle including the same
JP2015031990A
Cited By
Indoor decoration flatness detection method and system
CN119197443A
Pavement flatness and compactness visual detection device carried on road roller platform
CN119779391A
