A method, device, system and vehicle for identifying a wading surface

Abstract

The application provides a method, device and system for identifying a water-involved road surface and a vehicle. The method comprises: acquiring image information and point cloud data of a to-be-detected road surface on a driving path of the vehicle; determining a first identification confidence that the to-be-detected road surface is a water-involved road surface according to the image information; determining a second identification confidence that the to-be-detected road surface is a water-involved road surface according to a missing state of the point cloud data; calculating a target identification confidence according to the first identification confidence and the second identification confidence, and determining whether the to-be-detected road surface is a water-involved road surface according to the target identification confidence. The method can efficiently and accurately identify the water-involved road surface and improve the driving safety of the vehicle.

Description

Technical Field

[0001] This application relates to the field of path recognition technology, and in particular to a method, device, system, and vehicle for recognizing flooded road surfaces. Background Technology

[0002] Currently, both manually driven and autonomous vehicles can use image acquisition devices to collect image information along their driving path for obstacle recognition, ensuring safe driving. However, if there is standing water or the driving path is completely submerged, the waterlogged area cannot be accurately identified. If a vehicle recklessly drives into the water, it may stall, trapping occupants and potentially causing vehicle damage or personal injury.

[0003] Therefore, how to improve the accuracy of water-crossing road identification and enhance vehicle driving safety are technical problems that need to be solved by those skilled in the art. Summary of the Invention

[0004] The purpose of this application is to provide a method, device, terminal equipment, system, vehicle, and computer-readable storage medium for identifying flooded roads, aiming to improve the accuracy of flooded road identification and enhance vehicle driving safety.

[0005] Firstly, this application provides a method for identifying flooded road surfaces. The method includes:

[0006] Acquire image information and point cloud data of the road surface to be tested along the vehicle's driving path;

[0007] Based on the image information, a first confidence level is determined to identify the test road surface as a flooded road surface;

[0008] The second identification confidence level for the test road surface to be identified as the water-crossing road surface is determined based on the missing state of the point cloud data.

[0009] The target identification confidence is calculated based on the first identification confidence and the second identification confidence, and the test road surface is determined to be the water-crossing road surface based on the target identification confidence.

[0010] In one embodiment, the step of calculating the target identification confidence based on the first identification confidence and the second identification confidence, and determining whether the road surface to be tested is the wading road surface based on the target identification confidence, includes:

[0011] Obtain the current light intensity of the environment in which the vehicle is located;

[0012] Based on the current illumination intensity, determine the weight coefficients corresponding to the image information and the point cloud data, respectively.

[0013] The target identification confidence is calculated based on the first identification confidence, the second identification confidence, and each of the weighting coefficients. When the target identification confidence is greater than the determination threshold, the road surface to be tested is determined to be the water-crossing road surface.

[0014] In one embodiment, the second identification confidence level for determining the test road surface as the flooded road surface based on the missing state of the point cloud data includes:

[0015] Determine the missing locations and number of missing points in the point cloud data;

[0016] The second identification confidence level of the test road surface as the water-crossing road surface is determined based on the missing location and the number of missing parts.

[0017] In one embodiment, determining the first confidence level that the road surface to be tested is a flooded road surface based on the image information includes:

[0018] The image information is input into the image recognition model, and the image recognition model is used to output the first confidence level that the road surface to be tested is a water-eroded road surface; the image recognition model is pre-trained based on a neural network using training samples.

[0019] In one embodiment, after determining that the road surface to be tested is the flooded road surface, the method further includes:

[0020] The wading depth threshold of the vehicle is determined based on the vehicle's equipment parameters; the equipment parameters include the vehicle's maximum permissible wading depth, the amount of suspension position change caused by vehicle load, and the maximum suspension lift.

[0021] Obtain the real-time ground height of the vehicle; the real-time ground height is the distance between the center point of the rear axle of the vehicle and the horizontal ground level.

[0022] If the real-time ground altitude is less than the wading depth threshold, the vehicle is controlled to wade through water.

[0023] In one embodiment, obtaining the vehicle's real-time ground altitude includes:

[0024] Obtain the detection range and detection azimuth of the radar used to collect the point cloud data;

[0025] The radar altitude above the ground is determined based on the detection range and the detection azimuth.

[0026] The real-time ground altitude of the vehicle, corresponding to the radar's ground altitude, is determined by coordinate system transformation.

[0027] In one embodiment, if the real-time ground altitude is greater than the wading depth threshold, the method further includes:

[0028] If it is determined that there are no obstacles behind the vehicle, then the vehicle is controlled to perform the reversing tracking function.

[0029] Secondly, this application also provides a device for identifying flooded road surfaces. The device includes:

[0030] The acquisition module is used to acquire image information and point cloud data of the road surface to be tested along the vehicle's driving path;

[0031] The first identification module is used to determine the first identification confidence level of the test road surface as a water-eroded road surface based on the image information;

[0032] The second identification module is used to determine the second identification confidence level of the test road surface as the water-crossing road surface based on the missing state of the point cloud data.

[0033] The target determination module is used to calculate the target identification confidence based on the first identification confidence and the second identification confidence, and to determine whether the road surface to be tested is the water-crossing road surface based on the target identification confidence.

[0034] Thirdly, this application also provides a terminal device. The terminal device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described above.

[0035] Fourthly, this application also provides a water-crossing road surface identification system, the system comprising at least one image acquisition device, at least one radar, and a controller;

[0036] The image acquisition device is used to acquire image information of the road surface to be tested along the vehicle's driving path, and send the image information to the controller;

[0037] The radar is used to collect point cloud data of the road surface to be tested along the vehicle's driving path and send the point cloud data to the controller;

[0038] The controller is used to acquire the image information and point cloud data of the road surface to be tested on the vehicle's driving path; determine a first recognition confidence level that the road surface to be tested is a wading road surface based on the image information; determine a second recognition confidence level that the road surface to be tested is the wading road surface based on the missing state of the point cloud data; calculate a target recognition confidence level based on the first recognition confidence level and the second recognition confidence level; and determine whether the road surface to be tested is the wading road surface based on the target recognition confidence level.

[0039] Fifthly, this application also provides a vehicle, including a vehicle body and an intelligent driving controller, which performs the steps of the method described above.

[0040] Sixthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method described above.

[0041] This application provides a method for identifying flooded road surfaces. Since the point cloud data will be missing if the road surface to be tested is flooded, this method determines a second recognition confidence level for the road surface to be tested as a flooded road surface based on the missing point cloud data. Then, based on the first and second recognition confidence levels determined from the image information, the target recognition confidence level is calculated to determine whether the road surface to be tested is a flooded road surface. By comprehensively determining whether the road surface to be tested is a flooded road surface based on image information and point cloud data, this method can efficiently and accurately identify flooded road surfaces and improve vehicle driving safety.

[0042] It is understood that the water-crossing road identification device, terminal equipment, water-crossing road identification system, vehicle, and computer-readable storage medium provided in the embodiments of this application have the same beneficial effects as the water-crossing road identification method described above, and will not be repeated here. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the specific embodiments of this application or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0044] Figure 1 A flowchart illustrating a method for identifying flooded road surfaces provided in this application embodiment;

[0045] Figure 2 This is a schematic diagram illustrating the calculation process of real-time ground altitude in another embodiment of this application;

[0046] Figure 3 The diagram shown is a structural schematic of a water-eroded road surface identification device provided in an embodiment of this application;

[0047] Figure 4 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application;

[0048] Figure 5 This is a schematic diagram of the structure of a water-eroded road surface identification system provided in an embodiment of this application. Detailed Implementation

[0049] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of this application with unnecessary detail.

[0050] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0051] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0052] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0053] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0054] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized. "A plurality" means "two or more."

[0055] The present application provides a method for identifying flooded road surfaces, which can be executed by the processor of a terminal device when running a corresponding computer program.

[0056] Figure 1 The flowchart illustrates a method for identifying flooded road surfaces according to an embodiment of this application. For ease of explanation, only the parts relevant to this embodiment are shown. The method provided in this embodiment includes the following steps:

[0057] S100: Acquire image information and point cloud data of the road surface to be tested along the vehicle's driving path.

[0058] Specifically, the vehicle's driving path is first determined based on navigation information and whether a lane change is required. Then, the vehicle's image acquisition equipment collects image information of the road surface to be tested and sends it to the terminal device. Radar is then used to collect point cloud data of the road surface to be tested and sends it to the terminal device. The road surface to be tested refers to the road surface located on the vehicle's driving path and within the field of view of the image acquisition equipment and the data acquisition range of the radar.

[0059] S200: The first confidence level for determining that the road surface to be tested is a flooded road surface based on image information.

[0060] The first identification confidence level refers to the confidence level of determining that the road surface under test is a flooded road surface based on the image information.

[0061] Specifically, an image of a flooded road surface can be preset, and the similarity between the image information and the image of the flooded road surface can be calculated. The first recognition confidence level can be determined based on the calculated similarity level. Alternatively, an image recognition model can be trained in advance based on a neural network. After obtaining the image information, the image information is input into the image recognition model, and the image recognition model outputs the first recognition confidence level corresponding to the image information. This embodiment does not limit the specific method of calculating the first recognition confidence level.

[0062] S300: The second confidence level for determining whether the road surface to be tested is a flooded road surface based on the missing state of the point cloud data.

[0063] The second identification confidence level refers to the confidence level of determining that the road surface under test is a water-prone road surface based on point cloud data.

[0064] It should be noted that when a radar sends a laser signal to the surrounding environment, an echo will be generated when the laser signal hits an object. However, when the laser signal hits the water surface, most of the laser signal is absorbed by the water, and a small portion of the laser information is reflected by specular reflection. In other words, the corresponding point cloud data cannot be obtained after the laser signal hits the water surface. Therefore, this embodiment determines the missing state of the point cloud data and determines whether the road surface to be tested is a water-covered road surface based on the missing state, which is the second recognition confidence level for determining whether the road surface to be tested is a water-covered road surface.

[0065] S400: Calculate the target identification confidence based on the first identification confidence and the second identification confidence, and determine whether the road surface to be tested is a water-eroded road surface based on the target identification confidence.

[0066] Specifically, after determining the first and second recognition confidence levels, the average of the first and second recognition confidence levels can be directly calculated to obtain the target recognition confidence level. Alternatively, weight coefficients can be set for the image information and point cloud data respectively, that is, the first weight coefficient corresponding to the first recognition confidence level and the second weight coefficient corresponding to the second recognition confidence level can be determined. Then, the target recognition confidence level corresponding to the road surface to be tested can be calculated by weighting the first recognition confidence level and the first weight coefficient, as well as the second recognition confidence level and the second weight coefficient. Finally, the target recognition confidence level is used to determine whether the road surface to be tested is a flooded road surface.

[0067] In one specific implementation, a judgment threshold can be preset. After determining the target recognition confidence level, the target recognition confidence level is compared with the judgment threshold. If the target recognition confidence level is greater than or equal to the judgment threshold, the road surface to be tested is determined to be a flooded road surface. If the target recognition confidence level is less than the judgment threshold, the road surface to be tested is determined not to be a flooded road surface.

[0068] This application provides a method for identifying flooded road surfaces. Since the point cloud data will be missing if the road surface to be tested is flooded, this method determines a second recognition confidence level for the road surface to be tested as a flooded road surface based on the missing point cloud data. Then, the target recognition confidence level is calculated based on the first and second recognition confidence levels determined by the image information to determine whether the road surface to be tested is a flooded road surface. By comprehensively determining whether the road surface to be tested is a flooded road surface based on the image information and point cloud data, the method can efficiently and accurately identify flooded road surfaces and improve vehicle driving safety.

[0069] Based on the above embodiments, this embodiment further explains and optimizes the technical solution. Specifically, in this embodiment, the target identification confidence is calculated based on the first identification confidence and the second identification confidence, and the determination of whether the road surface to be tested is a water-eroded road surface is based on the target identification confidence, including:

[0070] Obtain the current light intensity of the environment in which the vehicle is located;

[0071] The weighting coefficients corresponding to the image information and point cloud data are determined based on the current illumination intensity.

[0072] The target identification confidence is calculated based on the first identification confidence, the second identification confidence, and each weight coefficient. When the target identification confidence is greater than the judgment threshold, the road surface to be tested is determined to be a water-eroded road surface.

[0073] The system can collect the current light intensity of the vehicle's environment using a light intensity sensor pre-installed on the vehicle, and then send the acquired light intensity to the terminal device via a gateway or other short-range transmission devices such as Bluetooth or Zigbee.

[0074] In this embodiment, at least two illumination intensity ranges are preset, and for each illumination intensity range, weight coefficients corresponding to image information and point cloud data are set respectively. After obtaining the current illumination intensity of the vehicle's environment, the weight coefficients corresponding to image information and point cloud data are determined according to the correspondence between the current illumination intensity and the illumination intensity range.

[0075] In another specific implementation, a pre-defined correspondence between illumination intensity ranges and time intervals is established, and for each time interval, weight coefficients corresponding to image information and point cloud data are set respectively. After obtaining the current illumination intensity of the vehicle's environment, the target time interval corresponding to the current illumination intensity is determined based on the correspondence between the current illumination intensity and the illumination intensity range, as well as the correspondence between the illumination intensity range and the time interval; then, based on the pre-defined weight coefficients corresponding to the target time interval, the weight coefficients corresponding to the image information and point cloud data are determined respectively.

[0076] In one specific embodiment, a light intensity threshold is preset. If the light intensity is greater than or equal to the light intensity threshold, the corresponding time interval is determined to be daytime; if the light intensity is less than the light intensity threshold, the corresponding time interval is determined to be nighttime. In this embodiment, the weighting coefficients for determining daytime and nighttime are as follows:

[0077] Weighting coefficient Image information Point cloud data daytime 0.6 0.4 night 0.3 0.7

[0078] After determining the weight coefficients corresponding to the image information and point cloud data respectively, the target recognition confidence is calculated by weighting the first recognition confidence, the second recognition confidence, and the weight coefficients corresponding to the image information and point cloud data respectively. Then, the target recognition confidence is compared with the judgment threshold. If the target recognition confidence is greater than or equal to the judgment threshold, the road surface to be tested is determined to be a flooded road surface. If the target recognition confidence is less than the judgment threshold, the road surface to be tested is determined not to be a flooded road surface.

[0079] In this embodiment, considering that the accuracy of identifying wading roads using image information and point cloud data varies under different lighting conditions, weighting coefficients corresponding to the image information and point cloud data are determined based on the current lighting intensity of the vehicle's environment, and then the target recognition confidence score is calculated. Therefore, determining the target recognition confidence score according to this embodiment can further improve the accuracy of identifying wading roads.

[0080] Based on the above embodiments, this embodiment further explains and optimizes the technical solution. Specifically, in this embodiment, determining the second identification confidence level of the road surface to be tested as a flooded road surface based on the missing state of point cloud data includes:

[0081] Determine the location and number of missing points in the point cloud data;

[0082] The second identification confidence level for determining whether the road surface to be tested is a flooded road surface is determined based on the location and number of missing parts.

[0083] Specifically, the radar emits laser signals into its data acquisition range and determines point cloud data based on the received reflected laser signals. Therefore, the missing point cloud data location refers to the location of the missing point cloud data within the radar's data acquisition range, which is the location where the reflected laser signal was not received; the missing quantity refers to the number of missing point cloud data, which is the number of times the reflected laser signal was not received.

[0084] In this embodiment, the second identification confidence level of the road surface to be tested as a water-eroded road surface is determined based on whether the coverage diameter corresponding to the missing location of the point cloud data is greater than or equal to a preset distance threshold, and whether the number of missing points in the point cloud data is greater than or equal to a preset number threshold.

[0085] As can be seen, the method of this embodiment can efficiently and accurately obtain the second recognition confidence level of image information.

[0086] Based on the above embodiments, this embodiment further explains and optimizes the technical solution. Specifically, in this embodiment, determining the first confidence level of identifying the road surface to be tested as a flooded road surface based on image information includes:

[0087] Image information is input into an image recognition model, which then outputs the first confidence level that the road surface under test is a flooded road surface. The image recognition model is pre-trained using training samples based on a neural network.

[0088] Specifically, a large number of training samples are pre-set; these samples include wading images and information labels. The large number of training samples are input into a neural network for training. When the model's accuracy reaches the desired value, an image recognition model is output. The input to the trained image recognition model is image information, and the output is the confidence level that the road surface corresponding to that image information is a wading road surface, i.e., the first recognition confidence level.

[0089] The neural network can be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a long short-term memory network (LSTM), a feed-forward neural network (FNN), or an attention mechanism (Transformer), etc. This embodiment does not limit the specific type of neural network.

[0090] This embodiment utilizes an image recognition model to output the first recognition confidence level corresponding to the image information, which can efficiently and accurately obtain the first recognition confidence level of the image information.

[0091] Based on the above embodiments, this embodiment further explains and optimizes the technical solution. Specifically, in this embodiment, after determining that the road surface to be tested is a flooded road surface, the method further includes:

[0092] The vehicle's wading depth threshold is determined based on the vehicle's equipment parameters; these parameters include the vehicle's maximum permissible wading depth, the amount of suspension position change caused by vehicle load, and the maximum suspension lift.

[0093] Obtain the vehicle's real-time ground height; the real-time ground height is the distance between the center point of the vehicle's rear axle and the horizontal ground level.

[0094] If the real-time ground altitude is less than the wading depth threshold, control the vehicle to drive through the water.

[0095] The maximum permissible wading depth H1 for a vehicle is generally determined at the time the vehicle leaves the factory.

[0096] The heavier the vehicle load, the greater the suspension reduction. Therefore, the suspension position change H2 is determined based on the vehicle load. Specifically, it can be determined by taking the average of the suspension changes collected multiple times within a preset driving time after the vehicle is powered on. The preset time can be 10 seconds.

[0097] If the vehicle has a variable suspension, meaning the suspension can be adjusted to lift while the vehicle is driving through water, then the maximum suspension lift H3 is obtained.

[0098] Among them, the wading depth threshold, i.e. the maximum allowable wading depth, is the maximum allowable wading depth of the vehicle H1 - the suspension position change H2 + the maximum suspension lift H3.

[0099] Wherein, the real-time ground height H of the vehicle is the horizontal distance between the center point of the rear axle of the vehicle and the ground; when the real-time ground height H of the vehicle is greater than H1-H2+H3, it means that the vehicle can pass through the water-crossing road surface, so the vehicle can be controlled to drive through the water; otherwise, that is, H≤H1-H2+H3, it means that the vehicle cannot pass through the water-crossing road surface, and the vehicle can be controlled to stop driving.

[0100] This embodiment can control the vehicle to safely wade through water after determining that the road surface to be tested is a water-crossing road surface, when the vehicle's real-time ground height is less than the water depth threshold.

[0101] Based on the above embodiments, this embodiment further explains and optimizes the technical solution. Specifically, in this embodiment, obtaining the real-time ground altitude of the vehicle includes:

[0102] Obtain the detection range and detection azimuth of the radar used to collect point cloud data;

[0103] The radar's ground altitude is determined based on the detection range and azimuth.

[0104] The real-time ground altitude of the vehicle, corresponding to the radar's ground altitude, is determined by coordinate system transformation.

[0105] Here, detection range refers to the distance the radar reaches the edge of the ground; detection azimuth refers to the angle between the edge of the detection viewing angle and the direction of gravity. For example... Figure 2 The diagram shows the calculation process of real-time altitude above the ground; as shown. Figure 2 As shown, in one specific implementation, the radar acquires the corresponding detection range S and detection azimuth angle α, and sends the detection range S and detection azimuth angle α to the terminal device. The terminal device calculates the radar's height relative to the ground based on the detection range S and detection azimuth angle α, that is, the radar's height above the ground h = s·cosα.

[0106] It's understandable that the radar's ground altitude is its height relative to the ground in a coordinate system with the radar as the origin. Therefore, after calculating the radar's ground altitude *h*, the vehicle's real-time ground altitude is obtained through coordinate transformation. This involves converting to a coordinate system with the center point of the vehicle's rear axle as the origin to determine the vehicle's real-time ground altitude. This coordinate transformation includes translation and rotation. When the vehicle is wading through water, this decrease in real-time ground altitude corresponds to the vehicle's wading depth.

[0107] As can be seen, the method of this embodiment can efficiently and accurately obtain the real-time ground altitude of the vehicle, thereby efficiently and accurately controlling the vehicle's wading through water.

[0108] In one specific embodiment, if the real-time ground altitude is greater than the wading depth threshold, the method further includes:

[0109] If it is determined that there are no obstacles behind the vehicle, then the vehicle will be controlled to perform the reversing tracking function.

[0110] Specifically, if the real-time ground clearance exceeds the wading depth threshold, the vehicle cannot continue wading. Therefore, the system uses the vehicle's camera and side radars to identify obstacles in the adjacent lane or within a preset distance (e.g., 5 meters). Obstacles include vulnerable road users (VRUs). If no obstacles are found behind the vehicle, the system activates the reversing trajectory function. This function involves pre-recording the vehicle's previous driving route and automatically reversing a specified distance without requiring driver steering.

[0111] In one specific embodiment, when the road surface to be tested is determined to be a flooded road surface, when the driver activates Automatic Cruise Control (ACC) or Cruise Pilot (CP) function, the vehicle speed is reduced to 5 kph in advance based on the current speed; when the vehicle begins to drive through the water, inertial measurement unit (IMU) is used to... The vehicle's position and orientation (pitch, raw angle) are acquired in real time by an IMU (Integrated Vehicle Unit). If the vehicle's position and orientation are within a preset range, the vehicle's camera and side radars identify whether there are vulnerable road users in the adjacent lane or within a preset distance (e.g., 5m). If not, the vehicle can be slowly accelerated to 10kph (to handle gentle wading and avoid vulnerable road users who may be splashed). When the vehicle's position and orientation exceed the preset range, the vehicle slows down to 5kph and stops when the maximum wading depth is reached. During the wading process, the suspension height is increased based on the real-time ground clearance until the increase reaches the maximum suspension height. If the real-time ground clearance is greater than the wading depth threshold, and if there are no obstacles behind the vehicle, the vehicle is controlled to reverse and slowly reverse out of the wading area, and the user is notified on the head unit (HUT) to take over. If there are obstacles behind the vehicle, the vehicle slowly reverses to 2m in front of the obstacle and stops, and the user is notified on the head unit (HUT) to take over. According to the method of this embodiment, the vehicle can automatically reverse and drive out of the flooded road when it cannot pass through the flooded road, which can further improve driving safety.

[0112] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0113] Figure 3The diagram shown is a structural schematic of a water-eroded road surface identification device provided in an embodiment of this application. Figure 3 As shown, the water-crossing road surface identification device of this embodiment includes an acquisition module 310, a first identification module 320, a second identification module 330, and a target determination module 340; wherein,

[0114] The acquisition module 310 is used to acquire image information and point cloud data of the road surface to be tested along the vehicle's driving path;

[0115] The first identification module 320 is used to determine the first identification confidence level of the test road surface as a water-crossing road surface based on the image information;

[0116] The second identification module 330 is used to determine the second identification confidence level of the test road surface as a water-crossing road surface based on the missing state of the point cloud data.

[0117] The target determination module 340 is used to calculate the target identification confidence based on the first identification confidence and the second identification confidence, and to determine whether the road surface to be tested is a water-eroded road surface based on the target identification confidence.

[0118] The water-eroded road surface identification device provided in this application embodiment has the same beneficial effects as the water-eroded road surface identification method described above.

[0119] In one embodiment, the target determination module includes:

[0120] The light intensity acquisition submodule is used to acquire the current light intensity of the environment in which the vehicle is located;

[0121] The weight coefficient determination submodule is used to determine the weight coefficients corresponding to the image information and point cloud data respectively based on the current illumination intensity.

[0122] The target determination submodule is used to calculate the target identification confidence level based on the first identification confidence level, the second identification confidence level, and each weight coefficient, and to determine the road surface to be tested as a water-eroded road surface when the target identification confidence level is greater than the judgment threshold.

[0123] In one embodiment, the second identification module includes:

[0124] The missing information determination submodule is used to determine the location and number of missing data in point cloud data;

[0125] The second identification confidence determination submodule is used to determine the second identification confidence of the road surface to be tested as a water-prone road surface based on the missing location and the number of missing items.

[0126] In one embodiment, determining a first confidence level that the road surface to be tested is a flooded road surface based on image information includes:

[0127] Image information is input into an image recognition model, which then outputs the first confidence level that the road surface under test is a flooded road surface. The image recognition model is pre-trained using training samples based on a neural network.

[0128] In one embodiment, a device for identifying flooded road surfaces further includes:

[0129] The threshold determination module is used to determine the wading depth threshold of the vehicle based on the vehicle's equipment parameters; the equipment parameters include the vehicle's maximum permissible wading depth, the amount of suspension position change caused by vehicle load, and the maximum suspension lift.

[0130] The altitude acquisition module is used to obtain the vehicle's real-time ground altitude; the real-time ground altitude is the distance between the center point of the vehicle's rear axle and the horizontal ground.

[0131] The driving control module is used to control the vehicle to drive through water if the real-time ground altitude is less than the wading depth threshold.

[0132] In one embodiment, the height acquisition module includes:

[0133] The detection data acquisition submodule is used to acquire the detection range and detection azimuth of the radar used to collect point cloud data;

[0134] The first calculation submodule is used to determine the radar's ground altitude based on the detection range and detection azimuth.

[0135] The second calculation submodule is used to determine the real-time ground altitude of the vehicle corresponding to the radar's ground altitude through coordinate system transformation.

[0136] In one embodiment, a device for identifying flooded road surfaces further includes:

[0137] If the real-time ground clearance is greater than the wading depth threshold, and if it is determined that there are no obstacles behind the vehicle, then the vehicle will be controlled to perform the reversing tracking function.

[0138] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0139] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0140] Figure 4 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application. Figure 4 As shown, the terminal device 400 of this embodiment includes a memory 401, a processor 402, and a computer program 403 stored in the memory 401 and executable on the processor 402; when the processor 402 executes the computer program 403, it implements the steps in the above-described embodiments of the water-crossing road identification methods; or when the processor 402 executes the computer program 403, it implements the functions of each module / unit in the above-described device embodiments.

[0141] For example, computer program 403 can be divided into one or more modules / units, one or more of which are stored in memory 401 and executed by processor 402 to implement the method of this application embodiment. One or more modules / units can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of computer program 403 in terminal device 400. For example, computer program 403 can be divided into an acquisition module, a first identification module, a second identification module, and a target determination module, with the specific functions of each module as follows:

[0142] The acquisition module is used to acquire image information and point cloud data of the road surface to be tested along the vehicle's driving path;

[0143] The first identification module is used to determine the first identification confidence level of the road surface to be tested as a flooded road surface based on the image information;

[0144] The second identification module is used to determine the second identification confidence level of the test road surface as a water-prone road surface based on the missing state of the point cloud data.

[0145] The target determination module is used to calculate the target identification confidence based on the first identification confidence and the second identification confidence, and to determine whether the road surface to be tested is a water-eroded road surface based on the target identification confidence.

[0146] In applications, terminal device 400 may be an Intelligent Driving Controller (IDC) or a Vehicle Controller Unit (VCU). Terminal device 400 may include, but is not limited to, memory 401 and processor 402. Those skilled in the art will understand that... Figure 4 This is merely an example of a terminal device and does not constitute a limitation on the terminal device. It may include more or fewer components than shown, or combine certain components, or different components. For example, a terminal device may also include input / output devices, network access devices, buses, etc.; among which, input / output devices may include cameras, audio acquisition / playback devices, displays, etc.; network access devices may include communication modules for wireless communication with external devices.

[0147] In applications, the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0148] In applications, memory can be an internal storage unit of a terminal device, such as its hard drive or RAM; it can also be an external storage device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card; or it can include both internal and external storage units. Memory is used to store operating systems, applications, boot loaders, data, and other programs, such as computer program code. Memory can also be used to temporarily store data that has been output or will be output.

[0149] Figure 5This is a schematic diagram of a water-eroded road surface identification system provided in an embodiment of this application. Figure 5 As shown, a water-crossing road identification system 500 of this embodiment includes at least one image acquisition device 510, at least one radar 520 and a controller 530.

[0150] Image acquisition device 510 is used to acquire image information of the road surface to be tested along the vehicle's driving path and send the image information to controller 530;

[0151] Radar 520 is used to collect point cloud data of the road surface to be tested along the vehicle's driving path and send the point cloud data to controller 530;

[0152] The controller 530 is used to acquire image information and point cloud data of the road surface to be tested along the vehicle's driving path; determine a first recognition confidence level that the road surface to be tested is a water-crossing road surface based on the image information; determine a second recognition confidence level that the road surface to be tested is a water-crossing road surface based on the missing state of the point cloud data; calculate the target recognition confidence level based on the first recognition confidence level and the second recognition confidence level; and determine whether the road surface to be tested is a water-crossing road surface based on the target recognition confidence level.

[0153] In this configuration, at least one image acquisition device 510 and at least one radar 520 are respectively connected to the controller 530 via a network.

[0154] The radar 520 can be either a lidar or an ultrasonic radar; this embodiment does not limit the choice. When installing the radar 520 on a vehicle, the installation height, the angle between the mounting surface and the ground, the vertical detection angle of the probe, the actual required detection distance, and the upward angle of the probe bracket need to be considered. If the radar hits the ground after installation, this problem can be solved by raising the installation position, increasing the upward angle of the probe, or reducing the detection distance.

[0155] The image acquisition device 510 can be a vehicle body camera or other pre-installed image acquisition devices, and this embodiment does not limit it.

[0156] This application provides a system for identifying flooded roads. Since point cloud data is missing if the road surface to be tested is flooded, the terminal device determines a second confidence level that the road surface is flooded based on the missing point cloud data. Then, based on the first and second confidence levels determined from the image information, a target recognition confidence level is calculated to determine whether the road surface is flooded. By comprehensively determining whether the road surface is flooded based on both image information and point cloud data, this system can efficiently and accurately identify flooded roads, improving vehicle driving safety.

[0157] This application also provides a vehicle, which includes a vehicle body and an intelligent driving controller that performs the steps in the above-described method embodiments.

[0158] The vehicles include various motor vehicles, such as cars, trucks and buses. This embodiment does not limit the type of vehicle.

[0159] The vehicle provided in this application embodiment has the same beneficial effects as the above-described method for identifying flooded road surfaces.

[0160] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can implement the steps in the above-described method embodiments.

[0161] The computer-readable storage medium provided in this application embodiment has the same beneficial effects as the above-described method for identifying flooded road surfaces.

[0162] This application implements all or part of the processes in the methods of the above embodiments, which can be accomplished by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium can include at least: any entity or device capable of carrying the computer program code to a terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, such as a USB flash drive, a portable hard drive, a magnetic disk, or an optical disk.

[0163] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0164] Those skilled in the art will recognize that the device and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0165] In the embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interface, or the device may be indirectly coupled or communicated, and may be electrical, mechanical, or other forms.

[0166] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for identifying flooded road surfaces, characterized in that, The method includes: Acquire image information and point cloud data of the road surface to be tested along the vehicle's driving path; Based on the image information, a first confidence level is determined to identify the test road surface as a flooded road surface; The second identification confidence level for determining the test road surface as the water-crossing road surface is determined based on the missing state of the point cloud data. Obtain the current light intensity of the environment in which the vehicle is located; Determining weight coefficients corresponding to the image information and the point cloud data based on the current illumination intensity includes: pre-setting at least two illumination intensity ranges, and setting weight coefficients corresponding to the image information and the point cloud data for each illumination intensity range; and determining weight coefficients corresponding to the image information and the point cloud data based on the correspondence between the current illumination intensity and the illumination intensity ranges. Alternatively, a correspondence between illumination intensity range and time interval is pre-defined, and for each time interval, a weight coefficient corresponding to the image information and the point cloud data is set respectively; based on the correspondence between the current illumination intensity and the illumination intensity range and the correspondence between the illumination intensity range and the time interval, a target time interval corresponding to the current illumination intensity is determined; based on the pre-defined weight coefficient corresponding to the target time interval, weight coefficients corresponding to the image information and the point cloud data are determined respectively. The target identification confidence is calculated based on the first identification confidence, the second identification confidence, and each of the weighting coefficients. When the target identification confidence is greater than the determination threshold, the road surface to be tested is determined to be the water-crossing road surface.

2. The method according to claim 1, characterized in that, The second identification confidence level for determining the test road surface as the flooded road surface based on the missing state of the point cloud data includes: Determine the missing locations and number of missing points in the point cloud data; The second identification confidence level of the test road surface as the water-crossing road surface is determined based on the missing location and the number of missing parts.

3. The method according to claim 1, characterized in that, The first confidence level for determining the test road surface as a flooded road surface based on the image information includes: The image information is input into the image recognition model, and the image recognition model is used to output the first confidence level that the road surface to be tested is a water-eroded road surface; the image recognition model is pre-trained based on a neural network using training samples.

4. The method according to any one of claims 1 to 3, characterized in that, After determining that the road surface to be tested is the flooded road surface, the method further includes: The wading depth threshold of the vehicle is determined based on the vehicle's equipment parameters; the equipment parameters include the vehicle's maximum permissible wading depth, the amount of suspension position change caused by vehicle load, and the maximum suspension lift. Obtain the real-time ground height of the vehicle; the real-time ground height is the distance between the center point of the rear axle of the vehicle and the horizontal ground level. If the real-time ground altitude is less than the wading depth threshold, the vehicle is controlled to wade through water.

5. The method according to claim 4, characterized in that, The process of obtaining the vehicle's real-time altitude above the ground includes: Obtain the detection range and detection azimuth of the radar used to collect the point cloud data; The radar altitude above the ground is determined based on the detection range and the detection azimuth. The real-time ground altitude of the vehicle, corresponding to the radar's ground altitude, is determined by coordinate system transformation.

6. The method according to claim 4, characterized in that, If the real-time ground altitude is greater than the wading depth threshold, the method further includes: If it is determined that there are no obstacles behind the vehicle, then the vehicle is controlled to perform the reversing tracking function.

7. A device for identifying flooded road surfaces, characterized in that, The device includes: The acquisition module is used to acquire image information and point cloud data of the road surface to be tested along the vehicle's driving path; The first identification module is used to determine the first identification confidence level of the test road surface as a water-eroded road surface based on the image information; The second identification module is used to determine the second identification confidence level of the test road surface as the water-crossing road surface based on the missing state of the point cloud data. A target determination module is used to calculate a target recognition confidence score based on the first recognition confidence score and the second recognition confidence score, and to determine whether the road surface to be tested is the wading road surface based on the target recognition confidence score. The target determination module includes: a light intensity acquisition submodule, used to acquire the current light intensity of the environment in which the vehicle is located; and a weight coefficient determination submodule, used to determine weight coefficients corresponding to the image information and the point cloud data respectively based on the current light intensity; including: pre-setting at least two light intensity ranges, and setting weight coefficients corresponding to the image information and the point cloud data respectively for each light intensity range; and determining the weight coefficients corresponding to the image information and the point cloud data respectively based on the correspondence between the current light intensity and the light intensity range. Alternatively, a pre-defined correspondence between light intensity range and time interval is established, and for each time interval, weight coefficients corresponding to the image information and the point cloud data are set respectively; based on the correspondence between the current light intensity and the light intensity range, and the correspondence between the light intensity range and the time interval, a target time interval corresponding to the current light intensity is determined; based on the pre-defined weight coefficients corresponding to the target time interval, weight coefficients corresponding to the image information and the point cloud data are determined respectively; the target determination submodule is used to calculate the target recognition confidence based on the first recognition confidence, the second recognition confidence, and each of the weight coefficients, and determine the road surface to be tested as the wading road surface when the target recognition confidence is greater than the judgment threshold.

8. A system for identifying flooded road surfaces, characterized in that, The system includes at least one image acquisition device, at least one radar, and a controller; The image acquisition device is used to acquire image information of the road surface to be tested along the vehicle's driving path, and send the image information to the controller; The radar is used to collect point cloud data of the road surface to be tested along the vehicle's driving path and send the point cloud data to the controller; The controller is used to acquire the image information and point cloud data of the road surface to be tested on the vehicle's driving path; Based on the image information, a first confidence level is determined to identify the test road surface as a flooded road surface; The second identification confidence level for determining the test road surface as the water-crossing road surface is determined based on the missing state of the point cloud data. Obtain the current light intensity of the environment in which the vehicle is located; Determining weight coefficients corresponding to the image information and the point cloud data based on the current illumination intensity includes: pre-setting at least two illumination intensity ranges, and setting weight coefficients corresponding to the image information and the point cloud data for each illumination intensity range; determining weight coefficients corresponding to the image information and the point cloud data based on the correspondence between the current illumination intensity and the illumination intensity ranges; or, pre-setting the correspondence between illumination intensity ranges and time intervals, and setting weight coefficients corresponding to the image information and the point cloud data for each time interval; determining a target time interval corresponding to the current illumination intensity based on the correspondence between the current illumination intensity and the illumination intensity ranges and the correspondence between the illumination intensity ranges and the time intervals; determining weight coefficients corresponding to the image information and the point cloud data based on the preset weight coefficients corresponding to the target time intervals; calculating a target recognition confidence level based on the first recognition confidence level, the second recognition confidence level, and each of the weight coefficients, and determining the test road surface as the wading road surface when the target recognition confidence level is greater than a judgment threshold.

9. A vehicle, comprising a vehicle body, characterized in that, It also includes an intelligent driving controller that performs the steps of the method as described in any one of claims 1 to 6.

Images