Pavement analysis method, controller, medium, product and target equipment

Abstract

The invention relates to a road surface analysis method, a controller, a medium, a product and target equipment, and the method comprises the steps: obtaining environment perception fusion data, the environment perception fusion data is obtained based on the fusion of at least two kinds of environment perception data of a target environment region, and the target environment region comprises a to-be-detected road surface; and based on the environmental perception fusion data, performing pavement analysis on the to-be-detected pavement to obtain an analysis result. On the basis, various environmental sensing data are fused, and road surface analysis is performed based on the fused sensing data, so that the wading road surface recognition accuracy under various road scenes is improved.

Description

Technical Field

[0001] The present application relates to the field of data analysis technology, and in particular to a road surface analysis method, controller, medium, product and target device. Background Art

[0002] In a vehicle driving scenario, it is necessary to determine whether the road surface along the vehicle's path is flooded, that is, whether there is water on the road surface along the vehicle's path. Current flooded road identification methods mainly use different sensors equipped on the vehicle to collect data from the road surface along the vehicle's path, and then calculate the confidence level of the data collected by different sensors to determine whether there is water on the road surface along the vehicle's path.

[0003] However, the above solution only performs a simple confidence calculation. When faced with complex road scenarios, there is still the problem of low accuracy in identifying flooded roads. Summary of the Invention

[0004] The embodiments of the present application provide a road surface analysis method, apparatus, controller, storage medium, computer program product, and target device, which integrate multiple environmental perception data and perform road surface analysis based on the fused perception data to improve the accuracy of water-related road surface recognition in various road scenarios.

[0005] The present invention provides a road surface analysis method, including:

[0006] Acquiring environmental perception fusion data, wherein the environmental perception fusion data is obtained by fusing at least two environmental perception data of a target environmental area, wherein the target environmental area includes a road surface to be tested;

[0007] Based on the environmental perception fusion data, a road surface analysis is performed on the road surface to be tested to obtain an analysis result.

[0008] Accordingly, an embodiment of the present application provides a road surface analysis device, comprising:

[0009] a data acquisition module, configured to acquire environmental perception fusion data, wherein the environmental perception fusion data is obtained by fusing at least two types of environmental perception data of a target environmental area, wherein the target environmental area includes a road surface to be tested;

[0010] The analysis module is used to perform road surface analysis on the road surface to be tested based on the environmental perception fusion data to obtain analysis results.

[0011] In addition, an embodiment of the present application also provides a controller, including one or more processors and a memory, wherein the memory stores a computer program, and the processor is used to run the computer program in the memory to implement the road surface analysis method provided in the embodiment of the present application.

[0012] In addition, an embodiment of the present application also provides a storage medium, which stores a computer program. When the computer program runs on a controller, the computer program is used to enable the controller to execute any road surface analysis method provided in the embodiment of the present application.

[0013] In addition, an embodiment of the present application also provides a computer program product, including a computer program or instructions, which, when executed by a processor, implements any road surface analysis method provided in the embodiment of the present application.

[0014] In addition, an embodiment of the present application also provides a target device, including the above-mentioned controller.

[0015] In the embodiments of the present application, fusion environmental perception data is obtained, wherein the fusion environmental perception data is obtained by fusing at least two types of environmental perception data of a target environmental region, which includes the road surface to be tested. Based on the fusion environmental perception data, a road surface analysis is performed on the road surface to obtain an analysis result. By fusing multiple types of environmental perception data and performing road surface analysis based on the fused perception data, the accuracy of water-crossing road surface recognition in various road scenarios is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0017] Figure 1 This is a schematic diagram of an implementation environment scenario of the road surface analysis method provided in an embodiment of the present application;

[0018] Figure 2 is a flow chart of the road surface analysis method provided in the embodiments of the present application;

[0019] Figure 3 is a schematic diagram of a specific process of the road surface analysis method provided in an embodiment of the present application;

[0020] Figure 4 This is another specific flow chart of the road surface analysis method provided in the embodiments of the present application;

[0021] Figure 5 This is another specific flow chart of the road surface analysis method provided in the embodiments of the present application;

[0022] Figure 6 is a schematic structural diagram of a road surface analysis device provided in an embodiment of the present application;

[0023] Figure 7 It is a schematic diagram of the structure of the controller provided in the embodiment of the present application. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0025] In addition, the term "a plurality of" in the embodiments of the present application refers to two or more than two. The terms "first" and "second" in the embodiments of the present application are used to distinguish descriptions and should not be understood to imply relative importance.

[0026] Research has found that in a vehicle driving scenario, it is necessary to determine whether the road surface on the vehicle's driving path is a flooded road surface, that is, whether there is water on the road surface on the vehicle's driving path.

[0027] The current method for identifying flooded roads mainly obtains image information and point cloud data of the road surface to be tested on the vehicle's travel path; determines a first recognition confidence level that the road surface to be tested is a flooded road surface based on the image information; determines a second recognition confidence level that the road surface to be tested is a flooded road surface based on the missing state of the point cloud data; calculates a target recognition confidence level based on the first recognition confidence level and the second recognition confidence level, and determines whether the road surface to be tested is a flooded road surface based on the target recognition confidence level.

[0028] However, the aforementioned flooded road recognition method uses a weighted approach based on the confidence levels of different sensors to determine flooded road conditions, processing image data and point cloud data separately, which can easily lead to data loss during processing. Furthermore, the potential relationships between sensor data are not fully utilized, making it impossible to fully identify complex road conditions. It only determines flooded road conditions, ignoring their shape and location. Therefore, existing flooded road recognition methods are essentially limited to the depth of the vehicle after wading, and are unable to perform advance depth detection of surface water, resulting in an inability to accurately determine whether a vehicle can pass through the flooded road.

[0029] To solve the above problems, the present invention provides a road surface analysis method, a controller, a storage medium, a computer program product, and a target device. The road surface analysis device can be integrated into a controller, which can be a server or a terminal.

[0030] Among them, the server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, network acceleration services (Content Delivery Network, CDN), as well as big data and artificial intelligence platforms.

[0031] The terminal may be a smartphone, tablet computer, laptop computer, desktop computer, smart speaker, smart watch, etc., but is not limited thereto. The terminal and the server may be connected directly or indirectly via wired or wireless communication, and this application does not impose any restrictions thereon.

[0032] See also Figure 1 , taking the road surface analysis device integrated into the controller as an example, Figure 1 This is a schematic diagram of an implementation scenario for the road surface analysis method provided in an embodiment of the present application. The controller, which can be a terminal device, acquires fused environmental perception data, derived from a fusion of at least two types of environmental perception data from a target environmental region, including the road surface to be tested. Based on the fused environmental perception data, a road surface analysis is performed on the road surface to obtain an analysis result. By fusing multiple types of environmental perception data and performing road surface analysis based on the fused perception data, the accuracy of identifying flooded roads in various road scenarios can be improved.

[0033] It should be noted that Figure 1 The schematic diagram of the implementation environment scenario of the road surface analysis method shown is merely an example. The implementation environment scenario of the road surface analysis method described in the embodiments of this application is intended to more clearly illustrate the technical solutions of the embodiments of this application and does not constitute a limitation of the technical solutions provided in the embodiments of this application. Persons skilled in the art will appreciate that with the evolution of data processing and the emergence of new business scenarios, the technical solutions provided in this application will also be applicable to similar technical problems.

[0034] The solutions provided in the embodiments of the present application are specifically described by the following embodiments. It should be noted that the description order of the following embodiments does not limit the preferred order of the embodiments.

[0035] This embodiment will be described from the perspective of a road surface analysis device. The road surface analysis device may be integrated into a controller, which may be a terminal and / or a server. This application does not impose any restrictions thereon.

[0036] See also Figure 2 , Figure 2: is a flow chart of a road surface analysis method provided in an embodiment of the present application. The road surface analysis method may include the following steps S101 to S102:

[0037] S101. Obtain environmental perception fusion data, wherein the environmental perception fusion data is obtained by fusing at least two types of environmental perception data of a target environmental area, and the target environmental area includes a road surface to be tested.

[0038] The environmental perception fusion data refers to the perception data obtained by fusing multiple environmental perception data.

[0039] Environmental perception data refers to the surrounding environment information obtained through various sensors (such as cameras, lidar, millimeter-wave radar, ultrasonic sensors, etc.) and data processing algorithms.

[0040] For example, environmental perception data may refer to two-dimensional or three-dimensional image information acquired by a camera, or distance and speed measured by laser pulse radar or millimeter wave radar.

[0041] The target environmental area refers to an environmental area that includes the road surface to be tested and where road surface analysis is required.

[0042] The specific content of the target environment area can be adjusted according to actual conditions. For example, the target environment area can refer to the environment area of ​​the target device (such as a vehicle or other device). For another example, the target environment area can refer to a specific road surface.

[0043] The road surface to be tested refers to the road surface that requires road surface analysis.

[0044] Among them, the at least two types of environmental perception data include radar detection data and visual detection data.

[0045] Radar detection data refers to data obtained by radar detecting the target environment area. Radars may include millimeter wave radars, laser radars, etc., and are not limited here.

[0046] The visual detection data refers to the data obtained by detecting the target environment area through a visual detection device. The visual detection device includes but is not limited to a camera or an infrared device, and is not limited here.

[0047] S102: Based on the environmental perception fusion data, perform road surface analysis on the road surface to be tested and obtain analysis results.

[0048] Road surface analysis includes but is not limited to water-related area analysis, road smoothness analysis, etc. Water-related area analysis includes but is not limited to the presence of water-related areas, the specific shape of water-related areas, and the location information of water-related areas.

[0049] Among them, the wading area refers to the area with accumulated water on the road surface.

[0050] As can be seen, the road surface analysis method provided in the embodiments of this application obtains environmental perception fusion data, where the environmental perception fusion data is obtained by fusing at least two types of environmental perception data from a target environmental region, which includes the road surface to be tested. Based on the environmental perception fusion data, a road surface analysis is performed on the road surface to obtain an analysis result. By fusing multiple types of environmental perception data and performing road surface analysis based on the fused perception data, the accuracy of water-related road surface recognition in various road scenarios can be improved.

[0051] In some embodiments, the target environment area is divided into a plurality of voxel grids.

[0052] A voxel grid is a discretized data structure used to represent three-dimensional space. The target environment area is divided into multiple three-dimensional grid cells (voxel grids). Each voxel grid has a fixed size and shape and can store attribute information related to the spatial area.

[0053] Based on this, the above process of obtaining environmental perception fusion data may include: obtaining environmental perception fusion data corresponding to each voxel grid based on the voxel grids contained in the target environmental area and the environmental perception data belonging to each voxel grid.

[0054] Specifically, for each voxel grid contained in the target environmental area, the environmental perception data belonging to the voxel grid is determined from each type of environmental perception data corresponding to the target environmental area. The corresponding environmental perception data belonging to the voxel grid from the multiple types of environmental perception data are fused to obtain the environmental perception fused data corresponding to the voxel grid.

[0055] In some embodiments, the target environment area is divided into voxel grids based on a target division method, and the target division method is determined based on regional characteristics of the target environment area.

[0056] The target division method refers to a voxel grid division method, which includes but is not limited to dividing the target environment area into a uniform voxel grid and dividing the target environment area into voxel grids of various sizes.

[0057] The regional characteristics of the target environment area are used to indicate the characteristics of the area where the target environment area is located. For example, the regional characteristics include the type of sensor detecting the target environment area, the environmental complexity of the target environment area, or whether the target environment area is close to a wading area.

[0058] For example, an occupancy network algorithm can be used to perform voxel gridding of the target environment area. Specifically, the target environment area can be divided into uniform grid cells (i.e., voxel grids). The grid cell shape can be square or cubic (in three-dimensional space), with the specific shape chosen based on practical application convenience and computational efficiency. During the gridding process, the grid resolution—that is, the actual size of the space represented by each grid cell—should be considered. This resolution should be selected based on a comprehensive balance of factors, including vehicle speed, sensor detection range and accuracy, and computing resources. At high vehicle speeds, relatively large grid cells can be selected to ensure real-time data processing and reduce computational complexity. For high-precision identification of water-related areas, such as when a vehicle approaches a potential water-related area, the grid cell size should be appropriately reduced to improve the resolution of road surface details. Furthermore, the grid resolution should be determined based on sensor characteristics. For example, lidar has a higher resolution, so the corresponding grid cells can be smaller to fully utilize its high-precision detection capabilities. Ultrasonic radar, on the other hand, has a shorter detection range and relatively lower accuracy, so its grid cells can be larger.

[0059] According to the actual application scenario and system requirements, uniform grid division and adaptive grid division can be used in combination. In most conventional areas where the vehicle is traveling, uniform grid division is used to ensure the stability and computational efficiency of the system; while in some key areas, such as the close distance in front of the vehicle, near possible wading areas, or areas with complex environmental changes (such as intersections, curves, etc.), adaptive grid division is enabled. By dynamically adjusting the grid resolution according to the real-time situation of the data, the recognition accuracy of these key areas can be improved. At the same time, the adaptive trigger conditions and adjustment strategies should be reasonably set to ensure that the system's recognition ability for wading roads is effectively improved without significantly increasing the computational burden.

[0060] In some embodiments, the process of performing road surface analysis on the road surface to be tested based on the environmental perception fusion data and obtaining analysis results may include: performing road surface analysis on the road surface to be tested based on the environmental perception fusion data corresponding to each voxel grid and obtaining analysis results.

[0061] In some embodiments, the road surface analysis includes a water-wading area analysis, and the analysis result includes whether the road surface to be tested contains a water-wading area.

[0062] The water-wading area analysis refers to the process of analyzing whether the road surface to be tested in the target environment area has a water-wading area.

[0063] A flooded area is an area on the road where there is accumulated water.

[0064] In some embodiments, the above-mentioned process of performing road surface analysis on the road surface to be tested based on the environmental perception fusion data corresponding to each voxel grid and obtaining the analysis results may include: determining a target voxel grid containing the road surface area of ​​the road surface to be tested from multiple voxel grids based on the environmental perception fusion data corresponding to each voxel grid; and determining whether the road surface area corresponding to the target voxel grid is a wading area based on the environmental perception fusion data corresponding to the target voxel grid to obtain the analysis results.

[0065] The target voxel grid refers to the voxel grid containing the road surface area.

[0066] There are many ways to determine the target voxel grid, which can be adjusted according to actual conditions and are not limited in the implementation of this application.

[0067] Exemplarily, the process of identifying a target voxel grid (i.e., a voxel grid containing a road surface area) among multiple voxel grids can be performed as follows: First, features that are helpful in identifying the road surface area, such as height features and reflection intensity, are extracted from each voxel grid. Then, the target voxel grid is determined from the multiple voxel grids by combining the features of the road surface area with a rule-based or model-based method. For example, a height threshold can be set, and voxel grids below this threshold can be preliminarily identified as candidate road surface areas. Assuming that the height of the vehicle sensor is 1 meter, voxel grids with a height of less than 0.5 meters can be regarded as candidate road surface areas. In addition, morphological operations or consistency detection can be used to perform subsequent processing and verification on the identified road surface areas to improve the accuracy and reliability of recognition.

[0068] There are multiple ways to determine whether the road surface area corresponding to the target voxel grid is a wading area based on the environmental perception fusion data corresponding to the target voxel grid. The specific methods can be adjusted according to actual conditions and are not limited in the embodiments of the present application.

[0069] In some embodiments, the above-mentioned process of determining whether the road surface area corresponding to the target voxel grid is a wading area based on the environmental perception fusion data corresponding to the target voxel grid may include: determining the probability that the road surface area corresponding to the target voxel grid is a wading area based on the environmental perception fusion data corresponding to the target voxel grid; and determining that the road surface area corresponding to the target voxel grid is a wading area when the probability corresponding to the target voxel grid is not less than a probability threshold.

[0070] Among them, the probability threshold can be set manually, or set according to an empirical value, or determined according to an experimental value. It can be adjusted according to actual conditions and is not limited in the embodiments of the present application.

[0071] For example, for each grid cell, a comprehensive judgment can be made based on point cloud density, image features, radar signal strength, etc. Based on the judgment result, if the road surface area corresponding to the grid cell has a high probability of being a wading area, the grid cell is assigned a probability value, such as 0.5; if the road surface area corresponding to the grid cell has a low probability of being a wading area, the grid cell is assigned a probability value, such as 0.1.

[0072] Exemplarily, for each grid cell, a preliminary occupancy probability is initialized based on the fused data. This process can be completed based on basic rules and experience. For example, if the fused data shows that there is a high probability that an object exists in the area where a certain grid cell is located (through comprehensive judgment such as point cloud density, image features, radar signal strength, etc.), then the grid cell is assigned a higher initial occupancy probability value, such as 0.5. Conversely, if there is no obvious sign of the existence of an object, a lower initial probability value, such as 0.1, is assigned. These initial values ​​are only rough estimates and will be adjusted through further calculation and optimization. In order to more accurately calculate and update the occupancy probability of each grid cell, accuracy, adaptability and interpretability can also be comprehensively considered. Preferably, a weighted average algorithm based on multi-sensor data fusion is used, combined with certain physical models and empirical rules. By reasonably allocating sensor weights and making full use of the advantages and complementarity of different sensors, this method can more accurately calculate the occupancy probability of the grid cell. At the same time, by combining physical models (such as the point cloud distribution and object reflection principles of lidar, and the signal propagation and target detection principles of millimeter-wave radar) and empirical rules (such as the typical characteristics of sensor data in different environments and the experience of judging flooded roads), probability calculations can be further optimized and adjusted to improve their accuracy and reliability.

[0073] In some embodiments, the above-mentioned process of determining whether the road surface area corresponding to the target voxel grid is a wading area based on the environmental perception fusion data corresponding to the target voxel grid may include: using a target model to analyze the environmental perception fusion data corresponding to the target voxel grid to determine whether the road surface area corresponding to the target voxel grid is a wading area.

[0074] The target model includes but is not limited to a neural network model, a deep learning model, a machine learning model, etc. The target model refers to a pre-trained model used to evaluate whether the road surface area corresponding to the voxel grid is a wading area.

[0075] For example, by aligning the LiDAR data and camera imagery to the same coordinate system, the reflection intensity and height information from the LiDAR data are extracted and combined with the color and texture features from the camera imagery. A machine learning model (such as a random forest) is used to classify the fused features and identify wading areas.

[0076] In some embodiments, the above analysis results also include regional information of the water-wading area when the road surface to be tested includes a water-wading area.

[0077] The area information is used to indicate relevant information of the water-related area.

[0078] The specific content of the regional information can be adjusted according to actual conditions and is not limited in the embodiments of the present application.

[0079] In some embodiments, the above-mentioned area information includes at least one of shape information of the water-related area, location information of the water-related area, and water depth of the water-related area.

[0080] The shape information refers to the shape of the wading area, for example, the shape information is an irregular shape or a square.

[0081] The position information refers to the position information of the wading area in the target environment area or other coordinate systems (such as the coordinate system of the vehicle).

[0082] The water depth refers to the depth information of water in the wading area, for example, the water depth is the water depth range, minimum water depth, maximum water depth, or average water depth in the wading area.

[0083] In some embodiments, when the above-mentioned analysis results include the depth of water accumulation in the wading area, the above-mentioned road surface analysis of the road surface to be tested based on the environmental perception fusion data and the process of obtaining the analysis results may include: determining the depth of water accumulation in the wading area based on the status information of the target device and the environmental perception fusion data corresponding to the wading area, wherein the environment in which the target device is located includes the target environment area.

[0084] The status information of the target device includes operation information of the target device or posture information of the target device.

[0085] The target device refers to a device that needs to perform road surface analysis on a target environment area in its environment. For example, the target device refers to a vehicle or other device.

[0086] It should be noted that there are many ways to determine the target environment area of ​​the target device, which can be adjusted according to actual conditions and are not limited in the embodiments of the present application.

[0087] In some embodiments, the environment area where the target device can be detected is directly used as the target environment area.

[0088] In some embodiments, the target environment area is determined based on a first distance value of the target device in the driving direction.

[0089] The first distance value does not exceed a maximum distance value at which the target device can be detected in the forward direction of travel.

[0090] In some embodiments, the target environment area is determined based on a first distance value of the target device in the forward direction of travel and second distance values ​​corresponding to the left and right sides of the target device.

[0091] The first distance value does not exceed a maximum distance value at which the target device can be detected in the forward direction of travel.

[0092] The second distance value does not exceed a maximum distance value at which the target device can be detected in a direction corresponding to the side thereof.

[0093] In some embodiments, the above process of determining the depth of water accumulation in the water-related area based on the status information of the target device and the environmental perception fusion data corresponding to the water-related area may also include: obtaining reference object association information of the reference object in the target environmental area; determining the depth of water accumulation in the water-related area based on the status information of the target device, the reference object association information, and the environmental perception fusion data corresponding to the water-related area.

[0094] The reference object refers to a reference object in the target environmental area that can be used to determine the depth of water accumulation in the wading area.

[0095] Reference objects may include other equipment (eg, other vehicles, other bicycles, etc.) and / or curbs.

[0096] Other devices refer to devices in the target environment area that are different from the target device.

[0097] The parameter object association information includes but is not limited to the size information of the reference object, the position information of the reference object, and the information that the reference object is submerged in water. The specific information can be adjusted according to actual conditions and is not limited in the embodiments of the present application.

[0098] For example, it is assumed that the target device is a vehicle, and the other devices are vehicles in front of the target device.

[0099] Based on this, the depth of the water can be estimated based on the wheel section height of the preceding vehicle. For example, suppose there is a vehicle in front of the vehicle (referred to as the "preceding vehicle") crossing a flooded area, and the water has submerged the lower part of the preceding vehicle's wheels. By observing the extent to which the preceding vehicle's wheel section is covered by the water, the depth of the water can be estimated. The specific steps are as follows:

[0100] 1. Obtain wheel data for the preceding vehicle: Assuming the wheel diameter of the preceding vehicle is known (for example, the wheel diameter of a typical sedan is approximately 0.6 meters, which can be obtained through wheel type database matching or vehicle recognition technology). Using image data (e.g., an image of the preceding vehicle captured by the vehicle's camera), the cross-section of the preceding vehicle's wheel is identified. Through image analysis, the height of the wheel section covered by accumulated water is measured.

[0101] 2. Calculate the depth of the water: The water depth is calculated based on the percentage of the wheels covered by water. For example, if image analysis shows that the front vehicle's wheels are covered by water for 0.15 meters, the water depth is 0.15 meters.

[0102] In some other examples, the depth of water on the road can also be obtained by combining ultrasonic radar data, vehicle body posture, and nearby obstacle information. For example, if there are no vehicles or other obstacles in front of the vehicle, the depth of water in front of the road can be calculated based on the installation position of the forward ultrasonic radar, the corresponding detection distance and detection direction angle obtained by the radar, and the road slope of the road on which the vehicle is located; or, the maximum depth of water on the road can be estimated based on the shape of the water and the slope of the road. For another example, if there are vehicles or other obstacles in front of the vehicle, a reference object height ratio calculation model can be constructed based on the wheel section height and vehicle body height data of the front vehicle. When facing an unknown waterlogged area, the surrounding reference objects (such as trees, buildings, etc.) and wheel section height data can be used to estimate the depth of water through the reference object height ratio calculation model.

[0103] In some embodiments, the above-mentioned process of determining the water depth in the water-related area based on the status information of the target device, the reference object association information, and the environmental perception fusion data corresponding to the water-related area may include: determining the water depth range of the water-related area based on the status information of the target device, the reference object association information, and the environmental perception fusion data corresponding to the water-related area; determining the water depth in the water-related area based on the water depth range.

[0104] The water depth in the wading area may be any depth value within a water depth range, or a depth value determined based on the water depth range.

[0105] In some embodiments, the ponding depth is an average depth within a range of ponding depths.

[0106] In some embodiments, the ponding depth is a maximum depth in a range of ponding depths.

[0107] In some embodiments, the water accumulation depth is an average depth determined by a preset number of depth values ​​selected randomly or from large to small in the water accumulation depth range.

[0108] In some embodiments, the above-mentioned environmental perception data includes first perception data and second perception data.

[0109] The first perception data indicates perception data that can be directly mapped to the voxel grid, and the second perception data indicates perception data that is indirectly mapped to the voxel grid via data points with the same position information in the first perception data.

[0110] Based on this, the above-mentioned process of obtaining the environmental perception fusion data corresponding to each voxel grid based on the voxel grid contained in the target environment area and the environmental perception data belonging to each voxel grid can include: for each voxel grid, based on the first perception data belonging to the voxel grid, determining the first feature of the voxel grid, and based on the second perception data belonging to the voxel grid, determining the second feature of the voxel grid; fusing the first feature and the second feature to obtain the fusion feature corresponding to the voxel grid, and using the fusion feature as the environmental perception fusion data of the voxel grid.

[0111] The first feature indicates a feature determined based on the first perception data, the second feature indicates a feature determined based on the second perception data, and the fused feature indicates a voxel feature corresponding to the voxel grid.

[0112] In some embodiments, the second perception data belonging to the voxel grid is determined based on the first perception data belonging to the voxel grid, and the correspondence between the data points corresponding to the first perception data and the data points corresponding to the second perception data.

[0113] Among them, the steps for determining the correspondence between the data points corresponding to the first perception data and the data points corresponding to the second perception data include: projecting each data point in the first perception data onto the plane corresponding to the second perception data to obtain the projection point of each data point; and constructing a correspondence for data points with the same position information and from different perception data.

[0114] For example, assume that the first perception data is based on point cloud data detected by radar, and the second perception data is based on image data captured by a camera. Divide the point cloud data into voxel grids (e.g., resolution 0.2m×0.2m×0.2m), so that each voxel grid aggregates point cloud features (including center point coordinates, density, and height variance) to generate a geometric feature grid F. LiDAR ∈R H×W×D×C1 , where C1 is the geometric feature dimension (including 3D coordinates and reflection intensity). For each lidar point P LiDAR , project the external parameters onto the image plane to obtain the visual features f of the corresponding pixels Cam (u,v). The visual feature f Cam (including RGB values, texture features, etc.) and the geometric features (X, Y, Z) of the point cloud are connected in series to form a multimodal point feature fmulti =[X,Y,Z,f Cam Finally, all points in each voxel grid are aggregated to obtain multimodal point features (including mean and maximum values), and voxel features F containing geometric and visual information are generated. fusion ∈R H×W×D×(C1+C2) Among them, C2 is the visual feature.

[0115] To better understand the above solution, a specific embodiment is explained below with a vehicle as the target device.

[0116] See also Figure 3 The specific process of the road surface analysis method may include steps S01 to S05:

[0117] Step S01: Use multiple sensors such as lidar, millimeter-wave radar, and camera to fuse, and introduce time synchronizers and space calibrators to ensure the temporal and spatial consistency of different sensors.

[0118] Specifically, sensor data is first collected. For example, a laser radar emits a laser beam and receives the reflected light signal to obtain point cloud data on the vehicle's travel path, measuring the distance to and shape of road water. A millimeter-wave radar emits millimeter-wave signals, receives reflected waves, and detects the speed and distance of objects around the vehicle. A camera captures images of the road surface to obtain information such as the color and texture of the road surface. An ultrasonic radar emits ultrasonic waves and calculates the distance to close objects around the vehicle based on the echo time. GPS / IMU receives satellite signals and measures its own acceleration, angular velocity, etc., providing the vehicle's position and attitude information.

[0119] Secondly, the above data is time synchronized. For example, based on GPS time, time calibration instructions are regularly sent to the sensor to keep the sensor's timestamp consistent with GPS time, ensuring that the data collected by different sensors at the same time can accurately correspond. Among them, the time synchronizer uses the time signal of the Network Time Protocol (NTP) or the Global Positioning System (GPS) to synchronize and calibrate the clocks of each sensor. According to the system's requirements for data accuracy, the accuracy of time synchronization is set, for example, the error is within milliseconds to ensure the consistency of data in time. Under normal circumstances, GPS time signals are preferred for time synchronization because they have the advantages of global coverage, high time accuracy and good stability. At the same time, the NTP protocol is combined to fine-tune the time within the local area network to further improve the accuracy and reliability of time synchronization.

[0120] Next, the data is spatially calibrated. For example, a calibration plate or a specific calibration scenario can be used to measure the relative position and angular deviation between sensors. For example, when the vehicle is stationary, calibration plates of known position and size are placed around the vehicle. Each sensor then detects the plates and calculates the relative position between the sensors based on the detection results. Compensation for the measured deviations is performed using pre-calibrated and pre-calibrated algorithms. In actual operation, the data collected by the different sensors is converted to the same coordinate system based on the calibration parameters to achieve spatial alignment. The allowable error range for the sensor spatial calibration is determined, such as position error within centimeters and angular error within degrees, to ensure that the fused data accurately reflects the actual conditions surrounding the vehicle. Using a calibration plate and a precise calibration algorithm for spatial calibration is a common and reliable method. By selecting a high-precision calibration plate and an optimized calibration algorithm, the deviations between sensors can be accurately measured and effectively compensated. In practice, the calibration plate material, size, and calibration algorithm can be optimized based on the vehicle type and usage environment to improve the accuracy and efficiency of spatial calibration.

[0121] Step S02: De-noise the data collected by each sensor through filters and other modules to improve data quality.

[0122] Specifically, the following processing is performed on the LiDAR point cloud data: Filtering: Using a statistical filtering algorithm, based on the statistical characteristics of the point cloud data, such as the mean and variance, noise points and outliers that are significantly different from the surrounding points are judged and removed. For example, a threshold is set. When the distance between a point and the surrounding points exceeds a certain multiple of the threshold (such as 3 times the standard deviation), it is considered an outlier and removed. Downsampling: Using methods such as voxel grid downsampling, the data density is reduced while maintaining the shape characteristics of the point cloud. The point cloud space is divided into a voxel grid of a certain size, and a representative point is retained in each voxel. For example, the center of gravity of the point cloud in the voxel can be selected as the representative point, thereby reducing the amount of point cloud data and improving the efficiency of subsequent processing. Geometric features, such as the density and distribution characteristics of the point cloud, are extracted from the LiDAR point cloud data. For example, the density distribution of the point cloud in different areas and the clustering characteristics of the point cloud can be calculated as a description of the geometric features.

[0123] The millimeter-wave radar data is processed as follows: Clutter suppression processing: Using a filtering method based on Doppler shift, the reflected clutter from stationary objects such as the ground and buildings is distinguished and removed according to the difference between the target object's moving speed and the stationary clutter. For example, a speed threshold is set, and targets with speeds below the threshold are regarded as stationary clutter and removed. Clustering processing: Clustering processing is performed on the target data detected by the millimeter-wave radar, and target points with similar distances and speeds are grouped into one cluster. A clustering algorithm based on distance and speed, such as a density clustering algorithm, can be used to divide density-connected points into a cluster for subsequent analysis and identification. Speed ​​and distance features are extracted from the millimeter-wave radar data, including information such as the relative speed, distance, and speed change rate of the target object.

[0124] The following processing is performed on the camera image data: Image enhancement processing: Use methods such as histogram equalization to enhance the contrast of the image and make the road surface features more obvious. By adjusting the grayscale histogram of the image, the grayscale distribution of the image is made more uniform, thereby improving the clarity and visual effect of the image. Denoising processing: Use algorithms such as median filtering and Gaussian filtering to remove noise in the image. Median filtering is suitable for removing salt and pepper noise, etc. It replaces the grayscale value of each pixel in the image with the median grayscale value of the pixels in the neighborhood of that point; Gaussian filtering is suitable for removing Gaussian noise. By performing Gaussian convolution operations on the image, the image is smoothed while retaining the edge and detail information of the image. Features such as the shape of the flooded road surface are extracted from the camera image data, such as extracting texture features through methods such as grayscale co-occurrence matrix, and extracting color features using methods such as color histogram.

[0125] Ultrasonic radar data is processed as follows: Due to the varying propagation speeds of ultrasound in different media and inherent sensor measurement errors, error compensation calculations are required based on the actual environment and sensor characteristics. For example, the ultrasonic propagation speed is corrected based on environmental factors such as temperature and humidity. The distance data detected by the ultrasonic wave is then calibrated to improve measurement accuracy. Features of nearby objects, such as the depth of water, are extracted from the ultrasonic radar data.

[0126] The GPS / IMU data is processed as follows: The GPS position information is integrated with the IMU's acceleration, angular velocity, and other data. Using algorithms such as the Kalman filter, the vehicle's precise position and attitude are estimated. The Kalman filter is a recursive optimal estimation method that continuously predicts and updates, fusing data from different sensors to achieve a more accurate state estimate. The IMU's bias error and scale factor error are calibrated. For example, when the vehicle is stationary, the IMU output is measured multiple times to calculate its bias error, which is then compensated for in subsequent data processing. Scale factor error can be corrected by comparing and calibrating the IMU's measurements with standard measurement equipment. Position and attitude features, such as the vehicle's latitude, longitude, altitude, heading, pitch, and roll angles, are extracted from the GPS / IMU data. At the same time, the vehicle's acceleration change characteristics can also be extracted, such as sudden acceleration, sudden deceleration, etc. These characteristics combined with other sensor information can better judge the vehicle's driving status and the impact of environmental changes on the vehicle. In some cases, it may be related to the judgment of flooded roads. For example, before entering a flooded area, the vehicle may have a special acceleration pattern due to changes in road conditions.

[0127] Step S03: construct an occupancy network model using the processed laser radar, millimeter wave radar, and camera, and use the model to obtain the shape and location of road surface water in real time.

[0128] Specifically, first, determine the range of the vehicle's surrounding environment to be monitored. This range can be set based on the vehicle's driving scenario and safety requirements. For example, it could be an area within a certain distance in front of the vehicle and within a certain angle range on either side. Then, divide this defined environmental area into uniform grid cells. The grid cell shape can be square, cubic (if three-dimensional space is considered), or other shapes. The specific shape is selected based on practical application convenience and computational efficiency. When dividing the grid, it is important to consider the grid resolution—that is, the actual size of the space represented by each grid cell. The resolution selection requires a comprehensive consideration of multiple factors, including vehicle speed, sensor detection range and accuracy, and computing resources. If the vehicle is traveling at a high speed, a relatively large grid cell can be selected to ensure real-time data processing, thereby reducing computational complexity. However, if high-precision identification of water-crossing areas is required, such as when the vehicle is approaching a potential water-crossing area, the grid cell size should be appropriately reduced to improve the resolution of road surface details. Furthermore, the grid resolution should be determined based on the sensor's characteristics. For example, lidar has a higher resolution, so the corresponding grid cell can be relatively small to fully utilize its high-precision detection capabilities. Ultrasonic radar has a shorter detection range and lower accuracy, so the corresponding grid cell can be relatively large. Based on the actual application scenario and system requirements, a combination of uniform and adaptive meshing is generally preferred, taking into account computing resources and accuracy requirements. In most common areas where the vehicle is traveling, uniform meshing is used to ensure system stability and computational efficiency. However, in critical areas, such as those close to the vehicle's front, near potential flooding, or in areas with complex environmental changes (such as intersections and curves), adaptive meshing is enabled. The mesh resolution is dynamically adjusted based on real-time data to improve recognition accuracy in these critical areas. Furthermore, appropriate adaptive triggering conditions and adjustment strategies should be set to ensure that the system's ability to identify flooded roads is effectively improved without significantly increasing the computational burden. Next, the preprocessed and fused data is used to construct an occupancy network. At this point, the fused data is a comprehensive dataset that integrates information from multiple sensors, and no specific sensor source is distinguished. For each grid cell, a preliminary occupancy probability is initialized based on the fused data. This initialization process can be based on basic rules and experience. For example, if the fused data indicates a high probability of an object existing in the area of ​​a grid cell (e.g., based on a combination of point cloud density, image features, and radar signal strength), a relatively high initial occupancy probability value, such as 0.5, can be assigned to that grid cell. Conversely, if there is no clear evidence of an object, a lower initial probability value, such as 0.1, can be assigned. These initial values ​​are only rough estimates and will be adjusted through further calculations and optimization. Finally, the probabilities are updated and optimized.Specifically, based on the fused data features, the occupancy probability of each grid cell is calculated and updated more accurately. Taking into account factors such as accuracy, adaptability, and interpretability, the weighted average algorithm based on multi-sensor data fusion combined with certain physical models and empirical rules is preferred to calculate the occupancy probability. This method can more accurately calculate the occupancy probability of grid cells by reasonably allocating sensor weights and making full use of the advantages and complementarity of different sensors. At the same time, combined with physical models (such as the point cloud distribution and object reflection principles of lidar, the signal propagation and target detection principles of millimeter-wave radar, etc.) and empirical rules (such as the typical characteristics of sensor data in different environments and the experience of judging wading roads), the probability calculation can be further optimized and adjusted to improve the accuracy and reliability of the calculation.

[0129] Step S04: Acquire the depth of water on the road by combining ultrasonic radar data, vehicle posture, and nearby obstacle information.

[0130] Specifically, if there are no vehicles or other obstacles in front of the vehicle, the depth of water in front of the road can be calculated based on the installation position of the forward ultrasonic radar, the corresponding detection distance and detection direction angle obtained by the radar, and the road slope of the vehicle. The maximum depth of water on the road can also be estimated based on the shape of the water and the slope of the road.

[0131] If there is a vehicle or other obstacle in front of the vehicle, a reference object height ratio calculation model can be constructed based on the wheel section height and body height data of the preceding vehicle. When facing an unknown flooded area, the surrounding reference objects (such as trees, buildings, etc.) and wheel section height data can be used to estimate the depth of the flooded area through the reference object height ratio calculation model.

[0132] Step S05: Based on the above content, three-dimensional modeling of road surface water is performed.

[0133] Specifically, a three-dimensional model of the flooded road surface is constructed based on the probability and depth information of the occupied grid, combined with factors such as time continuity, stability, and vehicle driving status.

[0134] In addition, based on the shape and depth of the water on the flooded road, suggestions can be given for vehicles to slow down, detour or stop.

[0135] In this way, by introducing the occupancy network to fuse radar and camera data, combined with road data and vehicle posture, the three-dimensional information in front of the vehicle is enriched, making it possible to more accurately identify the shape of the flooded road surface, such as distinguishing flooded areas of different depths and identifying special terrain such as steep slopes or deep pits, thereby more accurately judging the condition of the flooded road surface and improving the accuracy of the judgment.

[0136] Existing methods for identifying the depth of water on flooded roads are limited to the depth of the water immediately after the vehicle wades through it. This method cannot detect the depth of the water in advance, making it impossible to accurately determine whether the vehicle can pass through the flooded road surface. This invention combines information such as the height of the road curb and the vehicle in the flooded road ahead to estimate the depth of the water in advance, improving the accuracy and real-time performance of vehicle identification on flooded roads. It achieves significant benefits in terms of reliability and applicability, providing more effective protection for vehicle driving safety and possessing broader application prospects and development potential.

[0137] The above process of obtaining the road surface water depth by combining ultrasonic radar data, vehicle posture and nearby obstacle information can be as follows: Figure 4 Specifically, the following steps may be included:

[0138] Obtain information about the road ahead of the vehicle. Determine whether there are any vehicles or curbs ahead.

[0139] If it exists, a reference object height scale model is constructed based on the height of the curb. The water depth on the flooded road surface is obtained based on the reference object height scale model and the image information of the vehicle or other obstacles in front. The maximum depth of the water on the road surface is estimated based on the shape of the water and the slope of the road.

[0140] If it does not exist, the depth of water in front of the road is calculated based on the installation position of the forward ultrasonic radar, the corresponding detection distance and detection direction angle obtained by the radar, and the road slope of the vehicle.

[0141] The above-mentioned example of giving suggestions for vehicles to slow down, detour or stop based on the shape and depth of the water on the flooded road can be as follows: Figure 5 Specifically, the following steps may be included:

[0142] Obtain the shape and depth of the flooded road ahead of the vehicle.

[0143] Determine whether there is space for vehicles to pass next to the water on the road.

[0144] If there is water, back up to the front of the road.

[0145] If not, determine whether the road surface water depth reaches the wading depth threshold.

[0146] If the depth of water on the road reaches the wading depth threshold, reverse to the front of the flooded road.

[0147] If the depth of water on the road does not reach the wading depth threshold, the vehicle will be controlled to travel at a low speed according to the depth of the water, and the depth of water on the road will be updated in real time during the passage.

[0148] To facilitate better implementation of the road surface analysis method provided in the embodiment of the present application, the embodiment of the present application also provides a device based on the above road surface analysis method. The meanings of the terms herein are the same as those in the above road surface analysis method, and the specific implementation details can be referred to the description in the method embodiment.

[0149] For example, Figure 6 As shown, the road surface analysis device may include a data acquisition module 201 and an analysis module 202, specifically as follows:

[0150] A data acquisition module 201 is configured to acquire environmental perception fusion data, wherein the environmental perception fusion data is obtained by fusing at least two types of environmental perception data of a target environmental region, where the target environmental region includes a road surface to be tested;

[0151] The analysis module 202 is used to perform road surface analysis on the road surface to be tested based on the environmental perception fusion data to obtain analysis results. In some embodiments, the target environmental area is divided into multiple voxel grids, and obtaining environmental perception fusion data includes:

[0152] Based on the voxel grids contained in the target environment area and the environmental perception data belonging to each voxel grid, the environmental perception fusion data corresponding to each voxel grid is obtained.

[0153] In some embodiments, the target environment area is divided into voxel grids based on a target division method, and the target division method is determined based on regional characteristics of the target environment area.

[0154] In some embodiments, the above-mentioned road surface analysis is performed on the road surface to be tested based on the environmental perception fusion data to obtain analysis results, including:

[0155] Based on the environmental perception fusion data corresponding to each voxel grid, the road surface to be tested is analyzed to obtain the analysis results.

[0156] In some embodiments, the road surface analysis includes a water-wading area analysis, and the analysis result includes whether the road surface to be tested contains a water-wading area.

[0157] In some embodiments, the above-mentioned road surface analysis is performed on the road surface to be tested based on the environmental perception fusion data corresponding to each voxel grid, and the analysis results obtained include:

[0158] Determining a target voxel grid of a road surface area including the road surface to be tested from the plurality of voxel grids based on the environmental perception fusion data corresponding to each voxel grid;

[0159] According to the environmental perception fusion data corresponding to the target voxel grid, it is determined whether the road surface area corresponding to the target voxel grid is a wading area to obtain the analysis result.

[0160] In some embodiments, determining whether the road surface area corresponding to the target voxel grid is a wading area based on the environment perception fusion data corresponding to the target voxel grid includes:

[0161] According to the environmental perception fusion data corresponding to the target voxel grid, the probability that the road area corresponding to the target voxel grid is a wading area is determined;

[0162] When the probability corresponding to the target voxel grid is not less than the probability threshold, the road surface area corresponding to the target voxel grid is determined to be a wading area.

[0163] In some embodiments, the above analysis results also include regional information of the water-wading area when the road surface to be tested includes a water-wading area.

[0164] In some embodiments, the above-mentioned area information includes at least one of shape information of the water-related area, location information of the water-related area, and water depth of the water-related area.

[0165] In some embodiments, when the analysis result includes the depth of water in the wading area, a road surface analysis is performed on the road surface to be tested based on the environmental perception fusion data to obtain analysis results including:

[0166] The depth of accumulated water in the wading area is determined based on the status information of the target device and the environmental perception fusion data corresponding to the wading area, wherein the environment in which the target device is located includes the target environmental area.

[0167] In some embodiments, the target environment area is determined based on a first distance value of the target device in the driving direction.

[0168] In some embodiments, the target environment area is determined based on a first distance value of the target device in the forward direction of travel and second distance values ​​corresponding to the left and right sides of the target device.

[0169] In some embodiments, determining the depth of water in a wading area based on the state information of the target device and the environmental perception fusion data corresponding to the wading area includes:

[0170] Acquire reference object association information of the reference object in the target environment area;

[0171] The depth of accumulated water in the flooded area is determined based on the status information of the target device, the reference object association information, and the environmental perception fusion data corresponding to the flooded area.

[0172] In some embodiments, the reference objects include other equipment and / or curbs.

[0173] In some embodiments, determining the depth of water in a wading area based on the target device's status information, the reference object association information, and the environmental perception fusion data corresponding to the wading area includes:

[0174] Determine the depth of water in the flooded area based on the target device's status information, reference object association information, and the environmental perception fusion data corresponding to the flooded area;

[0175] Determine the depth of water in the flooded area based on the range of water depth.

[0176] In some embodiments, the above-mentioned water accumulation depth is the maximum depth in the water accumulation depth range.

[0177] In some embodiments, the environmental perception data includes first perception data and second perception data. Based on the voxel grids contained in the target environmental area and the environmental perception data belonging to each voxel grid, environmental perception fusion data corresponding to each voxel grid is obtained, including:

[0178] For each voxel grid, determining a first feature of the voxel grid based on first perception data belonging to the voxel grid, and determining a second feature of the voxel grid based on second perception data belonging to the voxel grid;

[0179] The first feature and the second feature are fused to obtain a fused feature corresponding to the voxel grid, and the fused feature is used as the environmental perception fusion data of the voxel grid.

[0180] In some embodiments, the second perception data belonging to the voxel grid is determined based on the first perception data belonging to the voxel grid, and the correspondence between the data points corresponding to the first perception data and the data points corresponding to the second perception data.

[0181] In some embodiments, the step of determining the correspondence between the data points corresponding to the first perception data and the data points corresponding to the second perception data includes:

[0182] Projecting each data point in the first perception data onto the plane corresponding to the second perception data to obtain a projection point of each data point;

[0183] Build correspondences for data points with the same location information but from different sensory data.

[0184] In some embodiments, the at least two types of environmental perception data include radar detection data and visual detection data.

[0185] As can be seen, the road surface analysis device provided in this embodiment of the application uses data acquisition module 201 to acquire fused environmental perception data. This fused environmental perception data is derived from the fusion of at least two types of environmental perception data from a target environmental region, which includes the road surface to be tested. Analysis module 202 then performs road surface analysis on the road surface to be tested based on the fused environmental perception data, generating an analysis result. By fusing multiple types of environmental perception data and performing road surface analysis based on the fused perception data, the accuracy of identifying flooded roads in various road scenarios can be improved.

[0186] During specific implementation, the above modules can be implemented as independent entities, or they can be arbitrarily combined and implemented as the same or several entities. The specific implementation methods and corresponding beneficial effects of the above modules can be found in the previous method embodiments and will not be repeated here.

[0187] The present application also provides a controller, such as Figure 7 , which shows a schematic diagram of the structure of the controller involved in the embodiment of the present application, specifically:

[0188] The controller may include one or more processing core processors 301, one or more storage media memories 302, a power supply 303, an input unit 304 and other components. Those skilled in the art will understand that Figure 7 The controller structure shown in the figure does not constitute a limitation on the controller, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0189] Processor 301 is the controller's control center, connecting all components of the controller using various interfaces and circuits. It executes computer programs and / or modules stored in memory 302 and accesses data stored in memory 302 to perform various controller functions and process data. Optionally, processor 301 may include one or more processing cores. Preferably, processor 301 integrates an application processor and a modem processor. The application processor primarily handles the operating system, user interface, and application programs, while the modem processor primarily handles wireless communications. It is understood that the modem processor may not be integrated into processor 301.

[0190] Memory 302 can be used to store computer programs and modules. Processor 301 executes various functional applications and road surface analysis by running the computer programs and modules stored in memory 302. Memory 302 may primarily include a program storage area and a data storage area. The program storage area may store an operating system, computer programs required for at least one function (such as an audio and visual prompt function, a road surface analysis function, etc.), and the data storage area may store data created based on the use of the controller. Furthermore, memory 302 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, memory 302 may also include a memory controller to provide processor 301 with access to memory 302.

[0191] The controller also includes a power supply 303 for supplying power to various components. Preferably, the power supply 303 can be logically connected to the processor 301 via a power management system, thereby enabling the power management system to manage charging, discharging, and power consumption. The power supply 303 can also include one or more DC or AC power supplies, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and other arbitrary components.

[0192] The controller may further include an input unit 304, which may be configured to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0193] Although not shown, the controller may further include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 301 in the controller will load the executable files corresponding to one or more computer program processes into the memory 302 according to the following instructions, and the processor 301 will run the computer programs stored in the memory 302 to implement various functions, such as:

[0194] Acquiring environmental perception fusion data, wherein the environmental perception fusion data is obtained by fusing at least two environmental perception data of a target environmental area, where the target environmental area includes a road surface to be tested;

[0195] Based on the environmental perception fusion data, the road surface to be tested is analyzed to obtain the analysis results.

[0196] As can be seen, the controller provided in the embodiments of the present application obtains fused environmental perception data, where the fused environmental perception data is derived from the fusion of at least two types of environmental perception data for a target environmental region, which includes the road surface to be tested. Based on the fused environmental perception data, the controller performs road surface analysis on the road surface to obtain an analysis result. By fusing multiple types of environmental perception data and performing road surface analysis based on the fused perception data, the accuracy of water-crossing road surface recognition in various road scenarios can be improved.

[0197] The specific implementation methods and corresponding beneficial effects of the above operations can be found in the detailed description of the road surface analysis method above, which will not be repeated here.

[0198] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be accomplished by a computer program, or by controlling related hardware through a computer program. The computer program may be stored in a storage medium and loaded and executed by a processor.

[0199] To this end, an embodiment of the present application provides a storage medium storing a computer program that can be loaded by a processor to execute the steps of any of the road surface analysis methods provided in the embodiments of the present application. For example, the computer program can execute the following steps:

[0200] Acquiring environmental perception fusion data, wherein the environmental perception fusion data is obtained by fusing at least two environmental perception data of a target environmental area, where the target environmental area includes a road surface to be tested;

[0201] Based on the environmental perception fusion data, the road surface to be tested is analyzed to obtain the analysis results.

[0202] As can be seen, the storage medium provided in the embodiments of the present application obtains environmental perception fusion data, where the environmental perception fusion data is obtained by fusing at least two types of environmental perception data from a target environmental region, which includes the road surface to be tested; and then performs road surface analysis on the road surface to be tested based on the environmental perception fusion data to obtain analysis results. Based on this, by fusing multiple types of environmental perception data and performing road surface analysis based on the fused perception data, the accuracy of water-crossing road surface recognition in various road scenarios can be improved.

[0203] The specific implementation methods and corresponding beneficial effects of the above operations can be found in the previous embodiments and will not be described in detail here.

[0204] The storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0205] Since the computer program stored in the storage medium can execute the steps of any pavement analysis method provided in the embodiments of the present application, the beneficial effects that can be achieved by any pavement analysis method provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0206] According to one aspect of the present application, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a storage medium. A processor of a computer device reads the computer instructions from the storage medium and executes the computer instructions, causing the computer device to perform the above-described road surface analysis method.

[0207] An embodiment of the present application also provides a target device, which includes the above-mentioned road surface analysis device, or the above-mentioned controller, or the above-mentioned computer program product.

[0208] Exemplarily, the target device includes the aforementioned controller, which acquires fused environmental perception data based on the fusion of at least two types of environmental perception data for a target environmental region, the target environmental region including the road surface to be tested. Based on the fused environmental perception data, a road surface analysis is performed on the road surface to obtain an analysis result. By fusing multiple types of environmental perception data and performing road surface analysis based on the fused perception data, the accuracy of identifying flooded roads in various road scenarios can be improved.

[0209] In some embodiments, the target device includes a vehicle.

[0210] Specifically, the vehicle may be a motor vehicle, a fuel vehicle, etc. For example, assuming that the target device is a vehicle, the vehicle may include the road surface analysis device, the controller, or the computer program product.

[0211] The specific structure of the vehicle is not limited in this application. The specific implementation of each operation of the controller and the corresponding beneficial effects are also applicable to the target device. For details, please refer to the detailed description of the road surface analysis method above, which will not be repeated here.

[0212] The above is a detailed introduction to a road surface analysis method, device, controller, storage medium, computer program product and target device provided in the embodiments of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, based on the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A road surface analysis method, characterized in that: The method comprises: Acquiring environmental perception fusion data, wherein the environmental perception fusion data is obtained by fusing at least two environmental perception data of a target environmental area, wherein the target environmental area includes a road surface to be tested; Based on the environmental perception fusion data, a road surface analysis is performed on the road surface to be tested to obtain an analysis result.

2. The road surface analysis method according to claim 1, characterized in that: The target environment area is divided into a plurality of voxel grids, and the acquiring of the environment perception fusion data includes: Based on the voxel grids contained in the target environment area and the environmental perception data belonging to each of the voxel grids, environmental perception fusion data corresponding to each of the voxel grids is obtained.

3. The road surface analysis method according to claim 2, characterized in that: The target environment area is divided into voxel grids based on a target division method, and the target division method is determined based on regional characteristics of the target environment area.

4. The road surface analysis method according to claim 2, characterized in that: The performing of road surface analysis on the road surface to be tested based on the environmental perception fusion data to obtain analysis results includes: Based on the environmental perception fusion data corresponding to each voxel grid, a road surface analysis is performed on the road surface to be tested to obtain an analysis result.

5. The road surface analysis method according to claim 4, characterized in that: The road surface analysis includes a water-wading area analysis, and the analysis result includes whether the road surface to be tested includes a water-wading area.

6. The road surface analysis method according to claim 5, characterized in that: The performing of road surface analysis on the road surface to be tested based on the environmental perception fusion data corresponding to each voxel grid to obtain an analysis result includes: Determining a target voxel grid including a road surface area of ​​the road surface to be tested from the plurality of voxel grids based on the environmental perception fusion data corresponding to each of the voxel grids; According to the environmental perception fusion data corresponding to the target voxel grid, it is determined whether the road surface area corresponding to the target voxel grid is a wading area to obtain an analysis result.

7. The road surface analysis method according to claim 6, characterized in that: The determining, based on the environmental perception fusion data corresponding to the target voxel grid, whether the road surface area corresponding to the target voxel grid is a wading area includes: Determining, based on the environmental perception fusion data corresponding to the target voxel grid, a probability that the road surface area corresponding to the target voxel grid is a wading area; When the probability corresponding to the target voxel grid is not less than a probability threshold, it is determined that the road surface area corresponding to the target voxel grid is a wading area.

8. The road surface analysis method according to claim 6, characterized in that: The analysis result further includes, when the road surface to be tested includes a wading area, area information of the wading area.

9. The road surface analysis method according to claim 8, characterized in that: The area information includes at least one of shape information of the water-wading area, location information of the water-wading area, and water depth of the water-wading area.

10. The road surface analysis method according to claim 9, characterized in that: In the case where the analysis result includes the depth of water in the wading area, the road surface analysis is performed on the road surface to be tested based on the environmental perception fusion data to obtain the analysis result, including: The water depth of the water-wading area is determined according to the status information of the target device and the environmental perception fusion data corresponding to the water-wading area, wherein the environment in which the target device is located includes the target environmental area.

11. The road surface analysis method according to claim 10, characterized in that: The target environment area is determined based on a first distance value of the target device in a traveling direction.

12. The road surface analysis method according to claim 10, characterized in that: The target environment area is determined based on a first distance value of the target device in a traveling direction and second distance values ​​corresponding to left and right sides of the target device.

13. The road surface analysis method according to claim 10, characterized in that: The determining of the water depth of the wading area according to the state information of the target device and the environmental perception fusion data corresponding to the wading area includes: Acquire reference object association information of the reference object in the target environment area; The depth of accumulated water in the water-wading area is determined according to the status information of the target device, the reference object association information, and the environmental perception fusion data corresponding to the water-wading area.

14. The road surface analysis method according to claim 13, characterized in that: The reference objects include other equipment and / or curbs.

15. The road surface analysis method according to claim 13, characterized in that: The determining of the water depth of the wading area according to the state information of the target device, the reference object association information, and the environmental perception fusion data corresponding to the wading area includes: Determining a water depth range of the wading area based on the state information of the target device, the reference object association information, and the environmental perception fusion data corresponding to the wading area; The water depth of the wading area is determined according to the water depth range.

16. The road surface analysis method according to claim 15, characterized in that: The accumulated water depth is the maximum depth in the accumulated water depth range.

17. The road surface analysis method according to claim 2, characterized in that: The environmental perception data includes first perception data and second perception data. The environmental perception fusion data corresponding to each voxel grid is obtained based on the voxel grid contained in the target environmental area and the environmental perception data belonging to each voxel grid, including: For each of the voxel grids, determining a first feature of the voxel grid based on first perception data belonging to the voxel grid, and determining a second feature of the voxel grid based on second perception data belonging to the voxel grid; The first feature and the second feature are fused to obtain a fused feature corresponding to the voxel grid, and the fused feature is used as the environment perception fusion data of the voxel grid.

18. The road surface analysis method according to claim 17, characterized in that: The second perception data belonging to the voxel grid is determined based on the first perception data belonging to the voxel grid and the corresponding relationship between the data points corresponding to the first perception data and the data points corresponding to the second perception data.

19. The road surface analysis method according to claim 18, characterized in that: The step of determining the correspondence between the data points corresponding to the first perception data and the data points corresponding to the second perception data includes: Projecting each data point in the first perception data onto a plane corresponding to the second perception data to obtain a projection point of each data point; Build correspondences for data points with the same location information but from different sensory data.

20. The road surface analysis method according to any one of claims 1 to 19, characterized in that: The at least two types of environmental perception data include radar detection data and visual detection data.

21. A controller, characterized in that: The method comprises one or more processors and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the road surface analysis method according to any one of claims 1 to 20.

22. A storage medium, characterized in that The method comprises a computer program, which is used to cause the controller to perform the steps of the road surface analysis method according to any one of claims 1 to 20 when the computer program is run on the controller.

23. A computer program product, characterized in that The method comprises a computer program or instructions, which implements the steps of the road surface analysis method according to any one of claims 1 to 20 when executed by a processor.

24. A target device, characterized in that The target device includes the controller according to claim 21.

25. The target device according to claim 24, characterized in that The target device includes a vehicle.

Images