Road adhesion coefficient identification method, electronic equipment, vehicle and storage medium

By using drones to capture road surface images and combining them with dynamic models, the problem of inaccurate road surface adhesion coefficient estimation caused by blind spots in camera vision has been solved, achieving more accurate and timely road surface adhesion coefficient identification and improving vehicle driving safety.

CN121789160APending Publication Date: 2026-04-03BYD CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202510273580.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-03-07
Publication Date
2026-04-03

AI Technical Summary

Technical Problem

In existing technologies, cameras have blind spots when shooting steep mountain roads or rocky surfaces in outdoor off-road environments, making it impossible to obtain timely image information of the road surface ahead, resulting in inaccurate estimates of the road surface adhesion coefficient.

Method used

By using drones to capture images of the road surface in front of the vehicle, the visual estimate of the road surface adhesion coefficient is determined through road surface type identification and preset correspondence. The dynamic estimate is then determined by combining the Kalman filter algorithm and the vehicle dynamics model, thereby improving the estimation accuracy.

Benefits of technology

It improves the accuracy and timeliness of road surface adhesion coefficient identification, thereby enhancing the safety and stability of vehicle operation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN121789160A_ABST
    Figure CN121789160A_ABST
Patent Text Reader

Abstract

The invention discloses a pavement adhesion coefficient identification method, electronic equipment, a vehicle and a storage medium, relates to the technical field of pavement identification, and aims to solve the problem of low accuracy of a pavement adhesion coefficient. The road adhesion coefficient identification method comprises the following steps: acquiring a road type identification result of a road in front of a vehicle, wherein the road type is obtained by identifying a road image of the road in front of the vehicle shot by unmanned aerial vehicle equipment flying in front of the vehicle; determining a visual estimation value of a road adhesion coefficient of a road in front of the vehicle based on the road type recognition result and a preset corresponding relation; wherein the preset corresponding relation is used for indicating pavement adhesion coefficients corresponding to different pavement types; and determining the road adhesion coefficient of the road in front of the vehicle based on the visual estimation value of the road adhesion coefficient.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of road surface recognition technology, and more particularly to a method for recognizing road surface adhesion coefficient, electronic equipment, vehicles, and storage media. Background Technology

[0002] Accurate estimation of the road surface adhesion coefficient (BPCC) is crucial for ensuring vehicle safety and performance during driving. BPCC directly affects vehicle stability during braking and cornering. Current technologies detect BPCC by acquiring images of the road surface ahead using onboard cameras, and then estimating the BPCC based on these images. However, in off-road driving, especially on steep mountain roads or rocky surfaces, cameras have blind spots and cannot acquire images of the road surface ahead in a timely manner, leading to inaccurate and untimely estimates of the BPCC. Summary of the Invention

[0003] The purpose of this invention is to provide a method, electronic device, vehicle, and storage medium for identifying road surface adhesion coefficient, aiming to solve the problem of low accuracy in estimating road surface adhesion coefficient.

[0004] To achieve the above objectives, the present invention adopts the following technical solution:

[0005] In a first aspect, the present invention provides a method for identifying road surface adhesion coefficient, comprising: acquiring a road surface type identification result of the road surface in front of a vehicle, wherein the road surface type is identified based on a road surface image of the road surface in front of the vehicle captured by a drone device flying alongside the vehicle; determining a visual estimate of the road surface adhesion coefficient of the road surface in front of the vehicle based on the road surface type identification result and a preset correspondence; wherein the preset correspondence is used to indicate the road surface adhesion coefficient corresponding to different road surface types; and determining the road surface adhesion coefficient of the road surface in front of the vehicle based on the visual estimate of the road surface adhesion coefficient.

[0006] The road surface adhesion coefficient identification method provided in this application determines the road surface type by using images of the road surface in front of the vehicle captured by a drone. The drone's camera is less prone to obstruction, resulting in smaller blind spots in the acquired images. This reduces the limitations of the road surface adhesion coefficient identification method and increases its flexibility. Furthermore, the drone's position in front of the vehicle allows for more timely identification of the road surface type. Therefore, the visual estimate of the road surface adhesion coefficient in front of the vehicle has higher accuracy, and the accuracy of the determined road surface adhesion coefficient is also higher, contributing to improved vehicle driving safety.

[0007] In some embodiments, determining the road surface adhesion coefficient of the road surface in front of the vehicle based on a visual estimate of the road surface adhesion coefficient includes: determining a dynamic estimate of the road surface adhesion coefficient of the road surface in front of the vehicle; and determining the road surface adhesion coefficient of the road surface in front of the vehicle based on the visual estimate of the road surface adhesion coefficient and the dynamic estimate of the road surface adhesion coefficient.

[0008] In some embodiments, determining the road surface adhesion coefficient of the road surface in front of the vehicle based on the visual estimate and the dynamic estimate of the road surface adhesion coefficient includes: when the weight of the visual estimate of the road surface adhesion coefficient is greater than or equal to a preset threshold and the dynamic estimate of the road surface adhesion coefficient is within a preset range, using the dynamic estimate of the road surface adhesion coefficient as the road surface adhesion coefficient of the road surface in front of the vehicle; wherein, the weight of the visual estimate of the road surface adhesion coefficient is used to reflect the reliability of the visual estimate of the road surface adhesion coefficient; the preset range is the range of values ​​of the road surface adhesion coefficient corresponding to the road surface type of the road surface in front of the vehicle; when the weight of the visual estimate of the road surface adhesion coefficient is greater than or equal to the preset threshold, but the dynamic estimate of the road surface adhesion coefficient is not within the preset range, using the visual estimate of the road surface adhesion coefficient as the road surface adhesion coefficient of the road surface in front of the vehicle; when the weight of the visual estimate of the road surface adhesion coefficient is less than the preset threshold, using the dynamic estimate of the road surface adhesion coefficient as the road surface adhesion coefficient of the road surface in front of the vehicle.

[0009] In some embodiments, the weight of the visual estimate of the road surface adhesion coefficient is determined based on the reliability of the road surface type identification result; wherein the reliability of the road surface type identification result is determined based on the distance between the image region that determines the road surface type identification result and the center point of the road surface image.

[0010] In some embodiments, determining the dynamic estimate of the road surface adhesion coefficient of the road surface in front of the vehicle includes: using a Kalman filter algorithm, based on the visual estimate of the road surface adhesion coefficient, vehicle operating information, vehicle dynamics model, and tire model, to determine the dynamic estimate of the road surface adhesion coefficient of the road surface in front of the vehicle.

[0011] In some embodiments, a Kalman filter algorithm is used to determine the dynamic estimate of the road surface adhesion coefficient in front of the vehicle based on the visual estimate of the road surface adhesion coefficient, vehicle operating information, vehicle dynamics model, and tire model. This includes: determining the measurement equation of the estimation system for the road surface adhesion coefficient based on the vehicle dynamics model and tire model; using the road surface adhesion coefficients of the four wheels of the vehicle as the state variables of the estimation system, the visual estimate of the road surface adhesion coefficient as the initial value of the state variables, the front wheel steering angle of the vehicle as the input parameter of the estimation system, and the vehicle operating information as the observation of the estimation system to construct the state equation of the estimation system; and obtaining the dynamic estimate of the road surface adhesion coefficient based on the measurement equation and the state equation using the Kalman filter algorithm.

[0012] In some embodiments, the time for collecting vehicle operation information is determined based on the distance between the UAV and the vehicle and the vehicle's speed when the UAV takes road images.

[0013] In some embodiments, obtaining the road surface type identification result of the road surface in front of the vehicle includes: acquiring a road surface image of the road surface in front of the vehicle taken by a drone device; and obtaining the road surface type identification result of the road surface in front of the vehicle based on the road surface image and the road surface identification model.

[0014] In some embodiments, obtaining the road surface type identification result of the road surface in front of the vehicle includes: receiving the road surface type identification result of the road surface in front of the vehicle sent by the UAV device; wherein the road surface type identification result of the road surface in front of the vehicle is obtained by the UAV device based on the road surface image and road surface identification model of the road surface in front of the vehicle.

[0015] In some embodiments, the road surface type identification results for the road surface in front of the vehicle include multiple results, each determined based on a different image region in the road surface image of the road surface in front of the vehicle; based on the road surface type identification results and a preset correspondence, a visual estimate of the road surface adhesion coefficient for the road surface in front of the vehicle is determined, including: based on the road surface type identification result with the highest reliability among the multiple road surface type identification results and the preset correspondence, a visual estimate of the road surface adhesion coefficient for the road surface in front of the vehicle is determined; wherein, the reliability of the road surface type identification results is determined based on the distance between the image region that determines the road surface type identification result and the center point of the road surface image.

[0016] In some embodiments, the visual estimate of the road surface adhesion coefficient of the road surface in front of the vehicle is determined based on the road surface type identification result and a preset correspondence, including: determining the range of values ​​for the road surface adhesion coefficient corresponding to the road surface type based on the road surface type indicated by the road surface type identification result and the preset correspondence; and determining the visual estimate of the road surface adhesion coefficient of the road surface in front of the vehicle based on the range of values ​​for the road surface adhesion coefficient corresponding to the road surface type.

[0017] In some embodiments, determining a visual estimate of the road surface adhesion coefficient of the road surface in front of the vehicle based on the range of values ​​of the road surface adhesion coefficient corresponding to the road surface type includes: determining a visual estimate of the road surface adhesion coefficient of the road surface in front of the vehicle based on the average value of the endpoint values ​​of the range of values ​​of the road surface adhesion coefficient.

[0018] In some embodiments, after determining the road surface adhesion coefficient of the road surface in front of the vehicle based on a visual estimate of the road surface adhesion coefficient, the method further includes: adjusting the vehicle's driving strategy based on the road surface adhesion coefficient of the road surface in front of the vehicle.

[0019] In some embodiments, the road surface adhesion coefficient includes the road surface adhesion coefficient of the vehicle's left wheel and the road surface adhesion coefficient of the vehicle's right wheel; determining the visual estimate of the road surface adhesion coefficient of the road surface in front of the vehicle based on the road surface type recognition result and a preset correspondence includes: determining the visual estimate of the road surface adhesion coefficient of the vehicle's left wheel based on the left road surface type recognition result and a preset correspondence; wherein the left road surface type recognition result is obtained based on the left region image recognition of the road surface image of the road surface in front of the vehicle; determining the visual estimate of the road surface adhesion coefficient of the vehicle's right wheel based on the right road surface type recognition result and a preset correspondence; wherein the right road surface type recognition result is obtained based on the right region image recognition of the road surface image of the road surface in front of the vehicle.

[0020] In some embodiments, the method further includes: in response to a trigger command issued by a user for a road surface adhesion coefficient recognition task, sending instruction information to a drone device, the instruction information being used to instruct the drone device to capture a road surface image of the road surface in front of the vehicle.

[0021] In some embodiments, the method further includes: predicting the road conditions ahead based on navigation routes and map information; and, if the road conditions ahead of the vehicle are complex, sending instruction information to the drone device, the instruction information being used to instruct the drone device to capture images of the road surface ahead of the vehicle.

[0022] In some embodiments, the flight altitude of the drone device is determined based on the vehicle's speed to balance the resolution and coverage of the images captured by the drone device.

[0023] In some embodiments, the data collection frequency of the drone device is determined based on the vehicle's speed and / or the road conditions ahead of the vehicle.

[0024] In some embodiments, the flight attitude of the drone device is determined based on the road conditions in front of the vehicle, so that the camera device of the drone device remains parallel to the road surface.

[0025] In a second aspect, this application provides an electronic device comprising: a processor and a memory; the memory storing processor-executable instructions; when the processor is configured to execute the instructions, causing the electronic device to implement the method of the first aspect described above.

[0026] Thirdly, this application provides a vehicle that includes the electronic equipment described in the second aspect.

[0027] Fourthly, this application provides a computer-readable storage medium comprising: computer software instructions; which, when executed in an electronic device, cause the electronic device to implement the method described in the first aspect.

[0028] Fifthly, this application provides a computer program product comprising a computer program; when the computer program is run in an electronic device, it causes the electronic device to implement the method described in the first aspect.

[0029] The beneficial effects of the second to fifth aspects mentioned above are described in the corresponding description of the first aspect and will not be repeated here. Attached Figure Description

[0030] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 This is a schematic diagram of the composition of a road surface adhesion coefficient identification system provided in an embodiment of this application;

[0032] Figure 2 A schematic flowchart illustrating a method for identifying road surface adhesion coefficient provided in an embodiment of this application;

[0033] Figure 3 A flowchart illustrating another method for identifying road surface adhesion coefficient provided in this application embodiment;

[0034] Figure 4 A flowchart illustrating another method for identifying road surface adhesion coefficient provided in this application embodiment;

[0035] Figure 5 A flowchart illustrating another method for identifying road surface adhesion coefficient provided in this application embodiment;

[0036] Figure 6 A flowchart illustrating the use of road surface adhesion coefficient is provided in this application embodiment;

[0037] Figure 7 A scenario diagram illustrating the transmission of instruction information provided in an embodiment of this application;

[0038] Figure 8 A flowchart illustrating another method for identifying road surface adhesion coefficient provided in this application embodiment;

[0039] Figure 9 This is a schematic diagram of the composition of a road surface adhesion coefficient identification device provided in an embodiment of this application;

[0040] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0041] Reference numerals: UAV device 100, vehicle 200, sensor 210, processor 220, gateway device 230, display module 240, drive motor controller 250, brake controller 260, traction controller 270, stability controller 280. Detailed Implementation

[0042] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0043] In the description of this invention, it should be understood that the terms "upper," "lower," "left," "right," "front," "rear," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or relative positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Unless otherwise specified, the above-described orientation can be flexibly set in practical applications, provided that the relative positional relationship shown in the accompanying drawings is satisfied.

[0044] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "communication" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection. They can refer to a direct connection or an indirect connection through an intermediate medium, or a communication between the internal components of two elements. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0045] In embodiments of the invention, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, article, or apparatus that includes that element.

[0046] In embodiments of the present invention, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" or "for example" in embodiments of the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Rather, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0047] In the description of this specification, specific features, structures, materials, or characteristics may be combined in any suitable manner in one or more embodiments or examples.

[0048] Accurate estimation of the road surface adhesion coefficient (GFCC) is crucial for ensuring vehicle safety and performance during driving, as it directly affects vehicle stability during braking and cornering. Current technologies estimate the GFCC by acquiring images of the road surface ahead using a camera and employing deep learning algorithms to identify the road surface type, thereby matching the road surface type with the GFCC to obtain a visual estimate. Next, combining vehicle motion information, a dynamic model, and a tire model, an unscented Kalman filter algorithm is used to calculate a dynamic estimate. Finally, through a spatiotemporal synchronization fusion strategy and a probability density function truncation method, the visual and dynamic estimates are combined to obtain the final GFCC estimate.

[0049] The above method combines visual estimation from a camera with dynamic estimation from unscented Kalman filtering, and finally proposes a road adhesion coefficient estimation mode decision method. However, in outdoor off-road driving, for steep mountain roads or rocky surfaces, the camera has blind spots and cannot acquire road surface image information in a timely manner, resulting in inaccurate and untimely estimation of the road adhesion coefficient.

[0050] To address the aforementioned technical problems, this application provides a method for identifying the road surface adhesion coefficient, comprising: acquiring a road surface type identification result for the road surface in front of the vehicle, wherein the road surface type is identified based on an image of the road surface in front of the vehicle captured by a drone device flying alongside the vehicle; the drone's camera is less likely to be obstructed, resulting in a smaller blind spot in the acquired image, thus reducing the limitations of the road surface adhesion coefficient identification method and increasing its flexibility. Furthermore, the drone's location in front of the vehicle allows for more timely identification of the road surface type. Based on this, the visual estimate of the road surface adhesion coefficient in front of the vehicle has higher accuracy, and the accuracy of the determined road surface adhesion coefficient is also higher, contributing to improved vehicle driving safety.

[0051] The embodiments provided in this application will now be described in detail with reference to the accompanying drawings.

[0052] The road surface adhesion coefficient identification method provided in this application can be applied to, for example... Figure 1The road surface adhesion coefficient identification system shown. (For example...) Figure 1 As shown, the road surface adhesion coefficient recognition system includes: unmanned aerial vehicle (UAV) equipment 100 and vehicle 200.

[0053] In some embodiments, vehicle 200 includes: sensor 210, processor 220, and gateway device 230.

[0054] For example, sensor 210 is connected to processor 220, and drone device 100 is connected to processor 220 through gateway device 230.

[0055] In some embodiments, the drone device 100 and the gateway device 230 are connected via a wireless network.

[0056] For example, a wireless network connection can be a Wi-Fi connection, a cellular network connection, a long-range low-power radio (LoRa) connection, a narrowband Internet of Things (NB-IoT) connection, or a Bluetooth connection.

[0057] In some embodiments, the drone device 100 is used to capture road surface images of the road surface in front of the vehicle. For example, the drone device 100 captures road surface images of the road surface in front of the vehicle in response to an instruction message and sends them to the processor 220.

[0058] Optionally, the drone device 100 can also be used to perform image preprocessing and feature extraction operations on the road surface images captured in front of the vehicle.

[0059] In some embodiments, the gateway device 230 is used to realize the communication connection between the drone device 100 and the processor 220. For example, the gateway device 230 may be a central gateway, a local area gateway, a dedicated short-range communication gateway, a multi-protocol gateway, etc.

[0060] In some embodiments, sensor 210 is used to acquire vehicle operation information.

[0061] For example, vehicle operating information includes: the speed and acceleration of each tire.

[0062] For example, sensor 210 may be a wheel speed sensor, an acceleration sensor, etc.

[0063] In some embodiments, the processor 220 is used to determine the road surface adhesion coefficient of the road surface in front of the vehicle. For example, the processor 220 obtains the road surface type identification result of the road surface in front of the vehicle, and determines the visual estimate of the road surface adhesion coefficient of the road surface in front of the vehicle based on the road surface type identification result and a preset correspondence; and determines the road surface adhesion coefficient of the road surface in front of the vehicle based on the visual estimate of the road surface adhesion coefficient.

[0064] For example, the processor 220 can be a processor in a vehicle, such as a central processing unit (CPU), etc., but this application embodiment does not limit it.

[0065] In some embodiments, such as Figure 1 As shown, the vehicle 200 also includes a display module 240, and the processor 220 is communicatively connected to the display module 240.

[0066] For example, the display module 240 is used to implement human-computer interaction. For instance, the display module 240 is used to receive user operation commands and send them to the processor 220. The operation commands include at least one of the following: a trigger command for the road surface adhesion coefficient recognition task, a take-off command for the UAV device 100, a landing command for the UAV device 100, and a flight path planning command for the UAV device 100.

[0067] For example, processor 220 sends instruction information to drone device 100 in response to user operation commands.

[0068] The instruction information is used to instruct the drone device 100 to take a picture of the road surface in front of the vehicle; or, the instruction information is used to instruct the drone device 100 to take off; or, the instruction information is used to instruct the drone device 100 to land; or, the instruction information is used to instruct the drone device 100 to plan its flight path.

[0069] In some embodiments, such as Figure 1 As shown, the vehicle 200 also includes a drive motor controller 250, a brake controller 260, a traction controller 270, and a stability controller 280. The processor 220 can also communicate with the drive motor controller 250, the brake controller 260, the traction controller 270, and the stability controller 280.

[0070] For example, the drive motor controller 250, brake controller 260, traction controller 270, and stability controller 280 are used to execute corresponding control commands based on the driving strategy.

[0071] It should be noted that the system architecture described in the embodiments of this application is for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and does not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of system architecture, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0072] See Figure 2 This is a schematic flowchart illustrating a method for identifying road surface adhesion coefficients provided in an embodiment of this application. Figure 2As shown, the road surface adhesion coefficient identification method provided in this application can be applied to the processor shown in the figure, and can be implemented by the above-mentioned road surface adhesion coefficient identification system, specifically including the following steps S201 to S203:

[0073] S201. Obtain the road surface type recognition result of the road surface in front of the vehicle.

[0074] The road surface type is determined by road surface image recognition based on images of the road surface in front of the vehicle taken by a drone that flies alongside the vehicle.

[0075] For example, road surface type identification is based on analyzing factors such as color and texture, lighting and reflection, temperature and moisture, material and structure in road surface images.

[0076] For example, the road surface type includes at least one of the following: dry cement road surface, dry asphalt road surface, wet cement road surface, wet asphalt road surface, cement road surface in rainy weather, asphalt road surface in rainy weather, and ice surface in snowy weather.

[0077] In some embodiments, the flight altitude of the drone device is determined based on the vehicle's speed to balance the resolution and coverage of the images captured by the drone device.

[0078] For example, the vehicle's speed is negatively correlated with the drone's flight altitude; for instance, the faster the vehicle travels, the lower the drone's flight altitude.

[0079] For example, the resolution of an image is used to represent the amount of information stored in the image, and the resolution of an image is the number of pixels that the image has in the horizontal and vertical directions.

[0080] For example, the shooting coverage area is used to represent the range of ground or space that the drone equipment can capture when shooting.

[0081] In some embodiments, the data collection frequency of the drone device is determined based on the vehicle's speed and / or the road conditions ahead of the vehicle.

[0082] For example, the vehicle's speed is positively correlated with the data collection frequency of the drone equipment. For instance, the faster the vehicle travels and the higher the rate of change of the road surface type the vehicle is traveling on, the higher the data collection frequency of the drone equipment.

[0083] For example, the complexity of the road conditions ahead of the vehicle is positively correlated with the data collection frequency of the drone equipment. For instance, the more complex the road conditions ahead of the vehicle and the higher the rate of change of the road type the vehicle is traveling on, the higher the data collection frequency of the drone equipment.

[0084] In some embodiments, the flight attitude of the drone device is determined based on the road conditions in front of the vehicle, so that the camera device of the drone device remains parallel to the road surface.

[0085] Understandably, keeping the drone's camera parallel to the road surface ensures that the images captured accurately reflect road conditions. For example, when the road ahead of a vehicle is a curve or slope, the drone will fly at an angle to keep its camera parallel to the road surface.

[0086] S202. Based on the road surface type identification results and the preset correspondence, determine the visual estimate of the road surface adhesion coefficient of the road surface in front of the vehicle.

[0087] The preset correspondence is used to indicate the road adhesion coefficient corresponding to different road surface types.

[0088] For example, the coefficient of friction is the static friction coefficient between the tire and the road surface. For instance, the coefficient of friction is highest on dry asphalt or concrete surfaces in good condition because such surfaces provide a good friction surface; when the road surface is wet, the coefficient of friction decreases significantly. Water lubricates the road surface, reducing friction between the tire and the road; the coefficient of friction is lowest on roads covered by ice and snow because ice and snow reduce the contact area between the tire and the road, increasing lubrication.

[0089] In some embodiments, the preset correspondence can be found in Table 1.

[0090] Table 1 Preset Correspondence

[0091] Road surface type Road surface adhesion coefficient Dry cement road surface 0.8-0.9 Dry asphalt pavement 0.8-0.9 damp cement road 0.5-0.7 wet asphalt road 0.6-0.8 cement road surface in rainy weather 0.4-0.6 Asphalt road surface in rainy weather 0.5-0.7 Snowy Ice Surface 0.2-0.4

[0092] In some embodiments, the visual estimate of the road surface adhesion coefficient is determined based on the range of values ​​of the road surface adhesion coefficient corresponding to the road surface type. For a detailed description, please refer to step S2022 below, which will not be repeated here.

[0093] In some embodiments, the road surface adhesion coefficient includes the road surface adhesion coefficient of the left wheel of the vehicle and the road surface adhesion coefficient of the right wheel of the vehicle.

[0094] It should be noted that for vehicles with independent distributed drive, the road adhesion coefficients of the left and right wheels may differ.

[0095] As one possible approach, a visual estimate of the road surface adhesion coefficient of the vehicle's left wheel is determined based on the left-side road surface type identification results and a preset correspondence.

[0096] The left-side road surface type recognition result is obtained based on the left-side region image of the road surface image in front of the vehicle.

[0097] For example, the road surface type identification result on the left is determined based on the left region image and the road surface identification model. The road surface identification model is used to identify the road surface type based on the road surface image.

[0098] For example, the road surface recognition model is introduced in step S2012 below, and will not be described in detail here.

[0099] As another possible implementation, a visual estimate of the road surface adhesion coefficient of the vehicle's right wheel is determined based on the right-side road surface type identification results and a preset correspondence.

[0100] The right-side road surface type recognition result is obtained based on the right-side region image of the road surface image in front of the vehicle.

[0101] For example, the right-side road surface type identification result is determined based on the right-side region image and the road surface identification model.

[0102] S203. Determine the road surface adhesion coefficient in front of the vehicle based on the visual estimate of the road surface adhesion coefficient.

[0103] In some embodiments, step S203 can be implemented as follows: determining a dynamic estimate of the road surface adhesion coefficient; and determining the road surface adhesion coefficient of the road surface in front of the vehicle based on the visual estimate and the dynamic estimate of the road surface adhesion coefficient. Specific implementation steps are described in steps S2031 to S2032 below, and will not be repeated here.

[0104] It is understood that the road surface adhesion coefficient identification method provided in this application determines the road surface type by using images of the road surface in front of the vehicle captured by a drone. The drone's camera is less prone to obstruction, resulting in smaller blind spots in the acquired images. This reduces the limitations of the road surface adhesion coefficient identification method and increases its flexibility. Furthermore, the drone's location in front of the vehicle allows for more timely identification of the road surface type. Based on this, the visual estimate of the road surface adhesion coefficient in front of the vehicle has higher accuracy, and the accuracy of the determined road surface adhesion coefficient is also higher, contributing to improved vehicle driving safety.

[0105] In some embodiments, such as Figure 3 As shown, the above step S201 can be specifically implemented as the following steps S2011 to S2012:

[0106] S2011. Acquire road surface images of the road ahead of the vehicle taken by the drone equipment.

[0107] S2012. Based on road surface images and road surface recognition models, the road surface type recognition results of the road surface in front of the vehicle are obtained.

[0108] In some embodiments, the road surface recognition model is built on a lightweight convolutional neural network (MobileNetV2).

[0109] For example, the MobileNetV2 lightweight convolutional neural network has higher computational efficiency and a smaller road surface recognition model, enabling high-precision image recognition tasks on vehicles or drones.

[0110] For example, the road surface recognition model is based on deep learning and training on an existing road surface image dataset.

[0111] Understandably, road type identification results obtained based on road surface recognition models can more accurately determine road surface types, thereby improving vehicle driving safety. Road surface recognition models have higher computational efficiency, which can improve the working efficiency of road surface adhesion coefficient recognition systems and enhance the timeliness of road surface adhesion coefficient recognition.

[0112] In some embodiments, step S201 above can also achieve: receiving the road surface type identification result of the road surface in front of the vehicle sent by the drone device.

[0113] The road surface type identification result in front of the vehicle is obtained by the UAV equipment based on the road surface image and road surface recognition model.

[0114] Understandably, the MobileNetV2 lightweight convolutional neural network has higher computational efficiency and a smaller road surface recognition model, so it can be deployed on drone devices to identify road surface types.

[0115] Understandably, using drone equipment to determine the road surface type in front of the vehicle reduces the workload of the processor, thereby improving the efficiency of road surface adhesion coefficient recognition.

[0116] In some embodiments, the road surface type identification results of the road surface in front of the vehicle include multiple results, which are determined based on different image regions in the road surface image of the road surface in front of the vehicle. In this case, step S202 can be specifically implemented as follows: based on the road surface type identification result with the highest reliability among the multiple road surface type identification results and the preset correspondence, determine the visual estimate of the road surface adhesion coefficient of the road surface in front of the vehicle.

[0117] For example, determining the visual estimate of the road surface adhesion coefficient includes: dividing the road surface image into multiple image regions based on multiple straight lines perpendicular to the vehicle's driving direction; obtaining the road surface type recognition result corresponding to each image region based on the multiple image regions and the road surface recognition model; adding the reliability of image regions with the same road surface type recognition result to obtain the reliability of multiple road surface type recognition results; comparing the magnitude of the reliability of multiple road surface type recognition results to determine the road surface type recognition result with the highest reliability among the multiple road surface type recognition results; and determining the visual estimate of the road surface adhesion coefficient of the road surface in front of the vehicle based on the road surface type recognition result with the highest reliability among the multiple road surface type recognition results and a preset correspondence.

[0118] The reliability of the road surface type identification result is determined based on the distance between the image region used to identify the road surface type and the center point of the road surface image. For example, the closer the distance between the image region and the center point of the road surface image, the higher the reliability of the corresponding road surface type identification result.

[0119] For example, the preset correspondence can be referred to Table 1, which will not be elaborated here.

[0120] Understandably, identifying road surface types in different image regions can integrate the road surface type identification results from multiple image regions, reduce the impact of errors in identifying a single image region on the road surface type identification results, and increase the flexibility of the road surface adhesion coefficient identification method.

[0121] In some embodiments, such as Figure 4 As shown, step S202 above can be specifically implemented as the following steps S2021 to S2022:

[0122] S2021. Based on the pavement type indicated by the pavement type identification result and the preset correspondence, determine the range of values ​​for the pavement adhesion coefficient corresponding to the pavement type.

[0123] For example, a preset correspondence is used to indicate the road adhesion coefficient corresponding to different road surface types.

[0124] For example, assuming the road type identification result indicates that the road type is dry cement road, the range of the road adhesion coefficient in Table 1 is [0.8-0.9].

[0125] S2022. Based on the range of values ​​for the road surface adhesion coefficient corresponding to the road surface type, determine the visual estimate of the road surface adhesion coefficient in front of the vehicle.

[0126] In some embodiments, the visual estimate of the road surface adhesion coefficient in front of the vehicle is determined based on the average of the endpoint values ​​of the range of road surface adhesion coefficient values.

[0127] For example, the endpoints of the range of values ​​for the road surface adhesion coefficient include: the upper limit of the range of values ​​for the road surface adhesion coefficient and the lower limit of the range of values ​​for the road surface adhesion coefficient.

[0128] For example, assuming the range of the road surface adhesion coefficient is [ab], the visual estimate of the road surface adhesion coefficient can satisfy the following formula (1):

[0129]

[0130] Where u1 represents the visual estimate of the road surface adhesion coefficient, a represents the upper limit of the range of values ​​based on the road surface adhesion coefficient, and b represents the lower limit of the range of values ​​based on the road surface adhesion coefficient.

[0131] Understandably, the road surface type identification result indicates the road surface type and the preset correspondence, which can more accurately determine the range of the road surface adhesion coefficient corresponding to the road surface type, and then determine the visual estimate of the road surface adhesion coefficient. Based on the road surface adhesion coefficient, the vehicle's driving strategy can be adjusted in real time to provide users with a safe and comfortable driving experience.

[0132] In some embodiments, such as Figure 5 As shown, step S203 above can be specifically implemented as the following steps S2031 to S2032:

[0133] S2031. Determine the dynamic estimate of the road surface adhesion coefficient in front of the vehicle.

[0134] In some embodiments, the Kalman filter algorithm is used to determine the dynamic estimate of the road surface adhesion coefficient in front of the vehicle based on the visual estimate of the road surface adhesion coefficient, vehicle operation information, vehicle dynamics model, and tire model.

[0135] For example, the time for collecting vehicle operation information is determined based on the distance between the drone and the vehicle and the vehicle's speed when the drone takes road images.

[0136] For example, the acquisition time is the time interval between the time it takes for the drone equipment to capture images of the road ahead and the time it takes for the vehicle to acquire operational information.

[0137] It should be noted that, in order to ensure that the dynamic estimate and the visual estimate of the road surface adhesion coefficient are calculated for the same area, after the UAV takes a picture of the road surface, the vehicle needs to arrive at the area where the UAV took the picture after a certain period of time. This certain period of time is the time for collecting vehicle operation information.

[0138] For example, vehicle operating information includes the speed and acceleration of each tire, and this vehicle operating information is the basis for estimating the road surface adhesion coefficient.

[0139] For example, vehicle dynamics models are used to describe the motion of wheels on various road surfaces, including longitudinal motion, lateral motion, and yaw motion.

[0140] For example, a tire model is used to describe the interaction between the tire and the road, as well as the forces acting on the tire under behaviors such as rolling, lateral slippage, and longitudinal slippage. For instance, a tire model could be a Dugoff tire model.

[0141] S2032. Determine the road surface adhesion coefficient in front of the vehicle based on the visual estimate and the dynamic estimate of the road surface adhesion coefficient.

[0142] In some embodiments, the road surface adhesion coefficient of the road surface in front of the vehicle is determined based on the weights of the visual estimate of the road surface adhesion coefficient and the dynamic estimate of the road surface adhesion coefficient.

[0143] It should be noted that when a vehicle is traveling in a straight line, since the rear tires follow the same path as the front tires, the road adhesion coefficient of the rear tires is determined based on the road adhesion coefficient of the front tires, the vehicle's speed, and the distance between the front and rear tires.

[0144] Understandably, combining visual and dynamic estimates of the road surface adhesion coefficient to determine the road surface adhesion coefficient allows for a more in-depth analysis of road surface types, timely identification of potential factors affecting vehicle safety, and improvement of vehicle driving safety.

[0145] In some embodiments, step S2031 can be specifically implemented as the following steps:

[0146] a1. Measurement equations for determining the road adhesion coefficient estimation system based on vehicle dynamics and tire models.

[0147] In some embodiments, the measurement equation is used to describe the relationship between the vehicle's motion state and the road surface adhesion coefficient.

[0148] In some embodiments, the measurement equation (h(x,u)) of the road surface adhesion coefficient estimation system can satisfy the following formula (2):

[0149]

[0150] Where h1(x,y) represents the longitudinal acceleration of the vehicle, and h2(x,y) represents the lateral acceleration of the vehicle.

[0151] For example, the longitudinal acceleration (h1(x,u)) and lateral acceleration (h2(x,u)) of the vehicle can satisfy the following formula (3):

[0152]

[0153] in, and The three-degree-of-freedom dynamic model of the vehicle is determined by the dynamic model of the vehicle in three directions: x (longitudinal), y (lateral), and z (vertical). The x-direction degree of freedom is used to represent the motion of the vehicle along the direction of travel, the y-direction degree of freedom is used to represent the motion of the vehicle perpendicular to the direction of travel, and the z-direction degree of freedom is used to represent the rotational motion of the vehicle about its vertical axis.

[0154] For example, the dynamic model of a vehicle in the three degrees of freedom of x, y, and z can satisfy the following formula (4):

[0155]

[0156] Where m represents the vehicle's mass, γ represents the vehicle's yaw rate, δ represents the vehicle's front wheel steering angle, fl represents the front left tire, fr represents the front right tire, rl represents the rear left tire, rr represents the rear right tire, and F... x F is used to represent the longitudinal force acting on a tire. y V is used to represent the lateral force acting on a tire. x Used to represent the longitudinal acceleration of a tire, I Z Used to represent the moment of inertia of a vehicle about the Z-axis, l f Used to indicate the distance from the front axle to the center of gravity of a vehicle, l r t is used to represent the distance from the rear axle of a vehicle to its center of gravity. w Used to indicate the distance between the front and rear tires of a vehicle.

[0157] For example, the dynamic model of each tire in the vehicle can satisfy the following formula (5):

[0158]

[0159] Among them, J w The moment of inertia ω is used to represent the rotational inertia of a tire. i T is used to represent the angular velocity of a tire. di R is used to represent the driving torque of the tire, and R is used to represent the radius of the tire.

[0160] For example, the longitudinal force (F) experienced by each tire in the tire model x ) and lateral force (Fy It can satisfy the following calculation formula (6):

[0161]

[0162] Where μ represents the road surface adhesion coefficient, F z C is used to represent the vertical force acting on a tire. x C is used to represent the longitudinal stiffness of a tire. y The parameter λ represents the lateral stiffness of the tire, α represents the slip ratio of the tire, α represents the slip angle of the tire, and L represents the parameters of the tire slip characteristics.

[0163] In some embodiments, f(L) in formula (6) can satisfy the following formula (7):

[0164]

[0165] Where L is used to represent the parameter of tire slip characteristics, which can satisfy the following formula (8):

[0166]

[0167] Where ε is used to represent the velocity influence factor, v x C is used to indicate the longitudinal speed of a tire. x C is used to represent the longitudinal stiffness of a tire. y The lateral stiffness of a tire is represented by λ, the slip ratio of a tire is represented by α, and the slip angle of a tire is represented by α.

[0168] For example, based on the force on the tire in formula (6) converted into a relationship with the road surface adhesion coefficient, the relationship between the measurement equation (h(x,u)) and the road surface adhesion coefficient is determined, and then the measurement equation is determined.

[0169] a2. Using the road surface adhesion coefficients of the four wheels of the vehicle as the state variables of the estimation system, the visual estimate of the road surface adhesion coefficients as the initial values ​​of the state variables, the front wheel steering angle of the vehicle as the input parameter of the estimation system, and the vehicle operation information as the observation of the estimation system, the state equation of the estimation system is constructed.

[0170] For example, the estimated system state variable (x) satisfies the following formula (9):

[0171] x = [μ fl ,μ fr ,μ rl ,μ rr ] Formula (9)

[0172] Where, μ fl The coefficient of friction (μ) used to represent the road surface adhesion of the tires on the left front of the vehicle. frThe coefficient of friction (μ) used to represent the road surface adhesion of the tires on the right front side of a vehicle. rl The coefficient of friction (μ) used to represent the road surface adhesion of the tires on the left rear of the vehicle. rr The coefficient of friction for the tires on the right rear of the vehicle.

[0173] For example, the input parameter (u) of the estimation system is equal to the front wheel steering angle (δ) of the vehicle.

[0174] For example, the observations of the estimation system include: the vehicle's longitudinal acceleration (h1(x,u)), lateral acceleration (h2(x,u)), and yaw rate (γ).

[0175] For example, the estimated state equation of the system can satisfy the following formula (10):

[0176]

[0177] in, The function is used to represent the change of state variable s with time t; f(x,u) is used to represent the relationship between state variable x and input parameter u; w(t) is used to represent process noise (which can be considered as a random variable with a mean of 0); y is used to represent the observation of the estimated system; h(x,u) is used to represent the relationship between the observation and state variable x, which is determined based on the above formula (2); v(t) is used to represent measurement noise (which can be considered as a random variable with a mean of 0).

[0178] It should be noted that the initial values ​​of the state variables are visual estimates of the road surface adhesion coefficient.

[0179] a3. Based on the Kalman filter algorithm, and based on the measurement equation and the state equation, the dynamic estimate of the road surface adhesion coefficient is obtained.

[0180] For example, the unscented Kalman filter algorithm uses the unscented transform (UT) to select a representative set of sample points, and combines this with weighted statistical linear regression to form an effective estimate of the nonlinear system. Through an iterative process, the unscented Kalman filter algorithm continuously updates the dynamic estimate (u2) of the road surface adhesion coefficient using the calculation results of the state equation and the observation equation.

[0181] Understandably, combining vehicle dynamics models and tire models with the Kalman filter algorithm provides a reliable method for estimating the dynamic value of real-time road surface adhesion coefficient, which helps improve vehicle driving safety and the working efficiency of the road surface adhesion coefficient identification system.

[0182] In some embodiments, step S2032 can be implemented in the following ways:

[0183] Method 1: When the weight of the visual estimate of the road surface adhesion coefficient is greater than or equal to a preset threshold, and the dynamic estimate of the road surface adhesion coefficient is within a preset range, the dynamic estimate of the road surface adhesion coefficient is used as the road surface adhesion coefficient of the road surface in front of the vehicle.

[0184] In some embodiments, the weights of the visual estimates of the road surface adhesion coefficient are used to reflect the reliability of the visual estimates of the road surface adhesion coefficient.

[0185] For example, the weight of the visual estimate of the road surface adhesion coefficient is determined based on the reliability of the road surface type identification result. For instance, the weight of the visual estimate of the road surface adhesion coefficient is the highest reliability of the road surface type identification result.

[0186] For example, the reliability of the road surface type identification result is determined based on the distance between the image region that determines the road surface type identification result and the center point of the road surface image. For instance, the closer the distance between the image region that determines the road surface type identification result and the center point of the road surface image, the higher the reliability of the road surface type identification result.

[0187] In some embodiments, the preset range is the range of values ​​for the road adhesion coefficient corresponding to the road surface type of the road surface ahead.

[0188] For example, the preset range is the range of values ​​for the road surface adhesion coefficient corresponding to the road surface type in Table 1. For example, the preset range is [mn].

[0189] In some embodiments, the preset threshold is a visual recognition confidence threshold P, which is the weighted lower limit of the visual estimate of the road adhesion coefficient. The preset threshold changes according to the vehicle tire operating information and is not limited here. For example, the preset threshold is 80%.

[0190] For example, the dynamic estimate of the road surface adhesion coefficient is used as the road surface adhesion coefficient in front of the vehicle. The weight (Q) of the visual estimate of the road surface adhesion coefficient and the dynamic estimate (u2) of the road surface adhesion coefficient can satisfy the following formula (11):

[0191]

[0192] Where P represents the preset threshold, m represents the lower limit of the range of values ​​for the road surface adhesion coefficient, and n represents the upper limit of the range of values ​​for the road surface adhesion coefficient.

[0193] Method 2: When the weight of the visual estimate of the road surface adhesion coefficient is greater than or equal to the preset threshold, but the dynamic estimate of the road surface adhesion coefficient is not within the preset range, the visual estimate of the road surface adhesion coefficient is used as the road surface adhesion coefficient of the road surface in front of the vehicle.

[0194] It should be noted that when the dynamic estimated value of the road surface adhesion coefficient is not within the preset range due to sensor errors or environmental factors, but the weight of the visual estimated value of the road surface adhesion coefficient is greater than or equal to the preset threshold, the visual estimated value of the road surface adhesion coefficient is used as the road surface adhesion coefficient of the road surface in front of the vehicle.

[0195] Exemplarily, the weight (Q) of the visual estimated value of the road surface adhesion coefficient and the dynamic estimated value (u2) of the road surface adhesion coefficient used as the visual estimated value of the road surface adhesion coefficient in front of the vehicle may satisfy the following formula (12):

[0196]

[0197] Where, P is used to represent the preset threshold, m is used to represent the lower limit value in the value range of the road surface adhesion coefficient, and n is used to represent the upper limit value in the value range of the road surface adhesion coefficient.

[0198] Mode 3: When the weight of the visual estimated value of the road surface adhesion coefficient is less than the preset threshold, the dynamic estimated value of the road surface adhesion coefficient is used as the road surface adhesion coefficient of the road surface in front of the vehicle.

[0199] It should be noted that in an overly complex section or when the pixel of the image captured by the UAV device is too low, the weight of the visual estimated value of the road surface adhesion coefficient is less than the preset threshold, and the dynamic estimated value of the road surface adhesion coefficient is used as the road surface adhesion coefficient of the road surface in front of the vehicle.

[0200] Exemplarily, when the dynamic estimated value of the road surface adhesion coefficient is used as the road surface adhesion coefficient of the road surface in front of the vehicle, the weight (Q) of the visual estimated value of the road surface adhesion coefficient may satisfy the following formula (13):

[0201] Q < P Formula (13)

[0202] Where, P is used to represent the preset threshold.

[0203] It can be understood that by comprehensively analyzing various situations of the visual estimated value and the dynamic estimated value of the road surface adhesion coefficient to determine the road surface adhesion coefficient of the road surface in front of the vehicle, the accuracy of the road surface adhesion coefficient is improved, and at the same time, the comprehensiveness and reliability of the road surface adhesion coefficient recognition system are also enhanced.

[0204] In some embodiments, after step S203, the road surface adhesion coefficient recognition method provided by the present application further includes the following steps: adjusting the driving strategy of the vehicle based on the road surface adhesion coefficient of the road surface in front of the vehicle.

[0205] Exemplarily, adjusting the driving strategy of the vehicle includes: dynamically adjusting the torque distribution of each tire.

[0206] For example, in off-road terrain, the road surface requires extremely high grip from the vehicle's tires. Real-time estimation of the road adhesion coefficient and adjustment of the vehicle's driving strategy are crucial to ensure sufficient traction even on slippery, soft, or uneven surfaces. Off-road terrain includes various complex road surfaces such as mud, gravel, rocks, and slopes.

[0207] For example, during vehicle acceleration, the torque of tires with higher road adhesion coefficients is increased to provide the traction needed for acceleration. During emergency obstacle avoidance or high-speed cornering, torque distribution is flexibly adjusted based on the road adhesion coefficients of each tire to ensure the vehicle remains smoothly on its intended path. This is achieved by increasing the torque of the inner tires to provide centripetal force while reducing the torque of the outer tires to reduce centrifugal force, effectively reducing the risk of rollover. During braking, the torque of tires with lower road adhesion coefficients is reduced (i.e., braking force), while the torque of the remaining tires is increased (i.e., braking force) to achieve smooth deceleration. In general, in off-road terrain, such as... Figure 6 As shown, the road adhesion coefficient of each tire is obtained, and the torque distribution of each tire is dynamically adjusted to control traction, enhance stability, and optimize braking performance.

[0208] Understandably, by using the real-time road adhesion coefficient, the vehicle's driving strategy can be adjusted in real time according to different road surface types, improving the vehicle's adaptability to complex terrain and the flexibility of driving strategy. Adjusting the vehicle's driving strategy by using the road adhesion coefficient can effectively reduce the possibility of vehicle rollover, thereby increasing the safety of the vehicle during driving.

[0209] In some embodiments, the road surface adhesion coefficient identification method provided in this application further includes the following steps: in response to a trigger command issued by a user for a road surface adhesion coefficient identification task, sending instruction information to a drone device.

[0210] The instruction information is used to instruct the drone equipment to take images of the road surface in front of the vehicle.

[0211] For example, such as Figure 7 As shown, the processor can be the central processing unit in the vehicle, which sends instruction information to the drone device. After receiving the instruction information, the drone device takes a picture of the road surface in front of the vehicle.

[0212] Optionally, the triggering command may also include a take-off command for the drone device, with indication information used to instruct the drone device to take off; or, the triggering command may also include a landing command for the drone device, with indication information used to instruct the drone device to land.

[0213] Understandably, sending instruction information to the drone based on the user's trigger command can enable image capture based on the user's needs, thus improving the user's driving experience. At the same time, the user's trigger command can activate the road adhesion coefficient recognition method when the road conditions ahead are unpredictable, increasing the flexibility of the road adhesion coefficient recognition method.

[0214] In some embodiments, the road surface adhesion coefficient identification method provided in this application further includes the following steps: predicting the road conditions ahead based on navigation routes and map information; and sending instruction information to the drone device when the road conditions ahead of the vehicle are complex.

[0215] The instruction information is used to instruct the drone equipment to take images of the road surface in front of the vehicle.

[0216] For example, the navigation route includes: the vehicle's preset driving route.

[0217] For example, map information includes: road layout, building locations, etc.

[0218] For example, complex road conditions are used to describe road sections with complex and variable road conditions, uncertain traffic conditions, or potential safety risks. For instance, complex road conditions can include slippery roads, icy roads, damaged roads, sharp bends, steep slopes, etc.

[0219] Understandably, navigation routes and map information can more accurately predict road conditions ahead and obtain road adhesion coefficients in a timely manner, thereby improving vehicle driving safety.

[0220] like Figure 8 The image shows a specific implementation of the road surface adhesion coefficient identification method provided in this application.

[0221] c1. Obtain a road surface image in front of the vehicle.

[0222] For example, a drone device captures an image of the road surface in front of the vehicle and sends it to a processor.

[0223] c2. Obtain the road surface type recognition result of the road surface in front of the vehicle.

[0224] For example, the processor obtains the road surface type identification result of the road surface in front of the vehicle based on the road surface image and the road surface recognition model.

[0225] c3. Determine the range of values ​​for the road adhesion coefficient corresponding to the road surface type.

[0226] For example, the processor determines the range of values ​​for the road adhesion coefficient corresponding to the road type based on the road type identification result and the preset correspondence.

[0227] c4. Determine the weight of the visual estimate of the road surface adhesion coefficient.

[0228] For example, the processor determines the weight of the visual estimate of the road surface adhesion coefficient based on the reliability of the road surface type identification results.

[0229] The reliability of the road surface type identification result is determined based on the distance between the image region that determines the road surface type identification result and the center point of the road surface image.

[0230] c5. Determine the visual estimate of the road surface adhesion coefficient in front of the vehicle.

[0231] For example, the processor determines a visual estimate of the road surface adhesion coefficient in front of the vehicle based on the average of the endpoint values ​​of the range of road surface adhesion coefficient values.

[0232] c6. Obtain vehicle operation information.

[0233] For example, sensors collect vehicle operating information and send it to a processor.

[0234] The timing of vehicle operation information collection is determined based on the distance between the drone and the vehicle and the vehicle's speed when the drone takes road images.

[0235] c7. Determine the measurement equations for the estimation system of the road surface adhesion coefficient.

[0236] For example, the processor determines the measurement equations of the estimation system for the road adhesion coefficient based on the vehicle dynamics model and the tire model.

[0237] c8. Construct the state equations of the estimated system.

[0238] For example, the processor uses the road adhesion coefficients of the four wheels of the vehicle as the state variables of the estimation system, the visual estimate of the road adhesion coefficients as the initial values ​​of the state variables, the front wheel steering angle of the vehicle as the input parameters of the estimation system, and the vehicle operation information as the observations of the estimation system to construct the state equation of the estimation system.

[0239] c9. Obtain the dynamic estimate of the road surface adhesion coefficient.

[0240] For example, the processor uses a Kalman filter algorithm to obtain a dynamic estimate of the road surface adhesion coefficient based on the measurement equation and the state equation.

[0241] c10. Based on the time of vehicle operation information collection, the visual estimate and dynamic estimate of the road surface adhesion coefficient are synchronized in time and space.

[0242] For example, the time for collecting vehicle operation information is determined based on the distance between the drone and the vehicle and the vehicle's speed when the drone takes road images.

[0243] For example, after the drone equipment captures an image of the road surface in front of the vehicle, the vehicle's operation data is collected during the vehicle operation information collection time.

[0244] c11. Determine whether the weight of the visual estimate of the road surface adhesion coefficient is greater than or equal to a preset threshold.

[0245] For example, the weights of the visual estimates of the road surface adhesion coefficient are used to reflect the reliability of the visual estimates of the road surface adhesion coefficient.

[0246] For example, if the weight of the visual estimate of the road surface adhesion coefficient is greater than or equal to a preset threshold, proceed to step c12; if the weight of the visual estimate of the road surface adhesion coefficient is less than the preset threshold, proceed to step c13.

[0247] c12. Determine whether the dynamic estimate of the road surface adhesion coefficient is within the preset range.

[0248] For example, the preset range is the range of values ​​for the road adhesion coefficient corresponding to the road surface type ahead.

[0249] For example, if the dynamic estimate of the road surface adhesion coefficient is within a preset range, proceed to step c13; if the dynamic estimate of the road surface adhesion coefficient is not within a preset range, proceed to step c14.

[0250] c13. Use the dynamic estimate of the road surface adhesion coefficient as the road surface adhesion coefficient in front of the vehicle.

[0251] c14. Use the visual estimate of the road surface adhesion coefficient as the road surface adhesion coefficient in front of the vehicle.

[0252] For example, Figure 9 This is a schematic diagram illustrating the composition of a road surface adhesion coefficient identification device provided in an embodiment of this application. Figure 9 As shown, the road surface adhesion coefficient identification device 800 includes a communication module 801 and a processing module 802. The communication module 801 is used to acquire the road surface type identification result of the road surface in front of the vehicle. The road surface type is identified based on the road surface image of the road surface in front of the vehicle captured by a drone device flying in front of the vehicle. The processing module 802 is used to determine the visual estimate of the road surface adhesion coefficient of the road surface in front of the vehicle based on the road surface type identification result and a preset correspondence. The preset correspondence is used to indicate the road surface adhesion coefficient corresponding to different road surface types. The road surface adhesion coefficient of the road surface in front of the vehicle is determined based on the visual estimate of the road surface adhesion coefficient.

[0253] In some embodiments, the processing module 802 is specifically used to determine a dynamic estimate of the road surface adhesion coefficient of the road surface in front of the vehicle; and to determine the road surface adhesion coefficient of the road surface in front of the vehicle based on the visual estimate of the road surface adhesion coefficient and the dynamic estimate of the road surface adhesion coefficient.

[0254] In some embodiments, the processing module 802 is specifically configured to, when the weight of the visual estimate of the road surface adhesion coefficient is greater than or equal to a preset threshold and the dynamic estimate of the road surface adhesion coefficient is within a preset range, use the dynamic estimate of the road surface adhesion coefficient as the road surface adhesion coefficient of the road ahead of the vehicle; wherein, the weight of the visual estimate of the road surface adhesion coefficient is used to reflect the reliability of the visual estimate of the road surface adhesion coefficient; the preset range is the range of values ​​of the road surface adhesion coefficient corresponding to the road surface type of the road ahead; when the weight of the visual estimate of the road surface adhesion coefficient is greater than or equal to the preset threshold, but the dynamic estimate of the road surface adhesion coefficient is not within the preset range, use the visual estimate of the road surface adhesion coefficient as the road surface adhesion coefficient of the road ahead of the vehicle; when the weight of the visual estimate of the road surface adhesion coefficient is less than the preset threshold, use the dynamic estimate of the road surface adhesion coefficient as the road surface adhesion coefficient of the road ahead of the vehicle.

[0255] In some embodiments, the weight of the visual estimate of the road surface adhesion coefficient is determined based on the reliability of the road surface type identification result; wherein the reliability of the road surface type identification result is determined based on the distance between the image region that determines the road surface type identification result and the center point of the road surface image.

[0256] In some embodiments, the processing module 802 is specifically used to employ a Kalman filter algorithm to determine the dynamic estimate of the road surface adhesion coefficient in front of the vehicle based on the visual estimate of the road surface adhesion coefficient, vehicle operation information, vehicle dynamics model, and tire model.

[0257] In some embodiments, the processing module 802 is specifically used to determine the measurement equation of the estimation system for the road surface adhesion coefficient based on the vehicle dynamics model and the tire model; using the road surface adhesion coefficients of the four wheels of the vehicle as the state variables of the estimation system, the visual estimate of the road surface adhesion coefficient as the initial value of the state variables, the front wheel steering angle of the vehicle as the input parameter of the estimation system, and the vehicle operation information as the observation of the estimation system, to construct the state equation of the estimation system; and based on the Kalman filter algorithm, and based on the measurement equation and the state equation, to obtain the dynamic estimate of the road surface adhesion coefficient.

[0258] In some embodiments, the time for collecting vehicle operation information is determined based on the distance between the UAV and the vehicle and the vehicle's speed when the UAV takes road images.

[0259] In some embodiments, the communication module 801 is specifically used to acquire road surface images of the road surface in front of the vehicle captured by the drone equipment; and to obtain road surface type identification results based on the road surface images and the road surface recognition model.

[0260] In some embodiments, the communication module 801 is specifically used to receive the road surface type identification result of the road surface in front of the vehicle sent by the UAV device; wherein, the road surface type identification result of the road surface in front of the vehicle is obtained by the UAV device based on the road surface image and road surface identification model of the road surface in front of the vehicle.

[0261] In some embodiments, the road surface type identification results of the road surface in front of the vehicle include multiple results, which are determined based on different image regions in the road surface image of the road surface in front of the vehicle. The processing module 802 is specifically used to determine the visual estimate of the road surface adhesion coefficient of the road surface in front of the vehicle based on the road surface type identification result with the highest reliability among the multiple road surface type identification results and a preset correspondence. The reliability of the road surface type identification result is determined based on the distance between the image region that determines the road surface type identification result and the center point of the road surface image.

[0262] In some embodiments, the processing module 802 is specifically used to determine the range of values ​​for the road surface adhesion coefficient corresponding to the road surface type based on the road surface type indicated by the road surface type identification result and the preset correspondence; and to determine the visual estimate of the road surface adhesion coefficient of the road surface in front of the vehicle based on the range of values ​​for the road surface adhesion coefficient corresponding to the road surface type.

[0263] In some embodiments, the processing module 802 is specifically used to determine a visual estimate of the road surface adhesion coefficient in front of the vehicle based on the average value of the endpoints of the range of values ​​of the road surface adhesion coefficient.

[0264] In some embodiments, after determining the road surface adhesion coefficient of the road surface in front of the vehicle based on the visual estimate of the road surface adhesion coefficient, the processing module 802 is further used to adjust the vehicle's driving strategy based on the road surface adhesion coefficient of the road surface in front of the vehicle.

[0265] In some embodiments, the road surface adhesion coefficient includes the road surface adhesion coefficient of the vehicle's left wheel and the road surface adhesion coefficient of the vehicle's right wheel; the processing module 802 is specifically used to determine the visual estimate of the road surface adhesion coefficient of the vehicle's left wheel based on the left road surface type identification result and a preset correspondence; wherein, the left road surface type identification result is obtained based on the left region image recognition of the road surface image of the road surface in front of the vehicle; and the visual estimate of the road surface adhesion coefficient of the vehicle's right wheel is determined based on the right road surface type identification result and a preset correspondence; wherein, the right road surface type identification result is obtained based on the right region image recognition of the road surface image of the road surface in front of the vehicle.

[0266] In some embodiments, the communication module 801 is further configured to send instruction information to the drone device in response to a trigger command issued by the user for a road surface adhesion coefficient identification task. The instruction information is used to instruct the drone device to capture a road surface image of the road surface in front of the vehicle.

[0267] In some embodiments, the communication module 801 is further configured to predict the road conditions ahead based on the navigation route and map information; and, if the road conditions ahead of the vehicle are complex, to send instruction information to the drone device, the instruction information being used to instruct the drone device to capture images of the road surface ahead of the vehicle.

[0268] In some embodiments, the flight altitude of the drone device is determined based on the vehicle's speed to balance the resolution and coverage of the images captured by the drone device.

[0269] In some embodiments, the data collection frequency of the drone device is determined based on the vehicle's speed and / or the road conditions ahead of the vehicle.

[0270] In some embodiments, the flight attitude of the drone device is determined based on the road conditions in front of the vehicle, so that the camera device of the drone device remains parallel to the road surface.

[0271] When implementing the functions of the integrated modules described above in hardware, this embodiment of the invention provides a possible structural diagram of the electronic device involved in the above embodiments. For example... Figure 10 As shown, the electronic device 900 includes: a processor 902, a communication interface 903, and a bus 904. Optionally, the electronic device 900 may also include a memory 901.

[0272] Processor 902 may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 902 may be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 902 may also be a combination that implements computing functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0273] The communication interface 903 is used to connect to other devices via a communication network. This communication network can be Ethernet, wireless access network, wireless local area network (WLAN), etc.

[0274] The memory 901 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto.

[0275] In one possible implementation, the memory 901 can exist independently of the processor 902. The memory 901 can be connected to the processor 902 via a bus 904 and is used to store instructions or program code. When the processor 902 calls and executes the instructions or program code stored in the memory 901, it can implement the vehicle driving method provided in this embodiment of the invention.

[0276] In another possible implementation, the memory 901 can also be integrated with the processor 902.

[0277] The 904 bus can be an extended industry standard architecture (EISA) bus, etc. The 904 bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 10 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0278] Through the above description of the implementation methods, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the service calling device can be divided into different functional modules to complete all or part of the functions described above.

[0279] This application also provides a computer-readable storage medium. All or part of the processes in the above method embodiments can be executed by computer instructions instructing related hardware. The program can be stored in the aforementioned computer-readable storage medium, and when executed, it can include the processes of the above method embodiments. The computer-readable storage medium can be any of the foregoing embodiments or memory. The aforementioned computer-readable storage medium can also be an external storage device of the aforementioned service invocation device, such as a pluggable hard drive, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the aforementioned service invocation device. Further, the aforementioned computer-readable storage medium can include both internal storage units of the aforementioned service invocation device and external storage devices. The aforementioned computer-readable storage medium is used to store the aforementioned computer program and other programs and data required by the aforementioned service invocation device. The aforementioned computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0280] This application also provides a computer program product, which includes a computer program that, when run on a computer, causes the computer to execute any of the road surface adhesion coefficient identification methods provided in the above embodiments.

[0281] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for identifying road surface adhesion coefficient, characterized in that, The method includes: The road surface type identification result of the road surface in front of the vehicle is obtained, and the road surface type is identified based on the road surface image of the road surface in front of the vehicle taken by the drone equipment flying in front of the vehicle. Based on the road surface type identification results and the preset correspondence, a visual estimate of the road surface adhesion coefficient of the road surface in front of the vehicle is determined; wherein, the preset correspondence is used to indicate the road surface adhesion coefficient corresponding to different road surface types; The road surface adhesion coefficient in front of the vehicle is determined based on the visual estimate of the road surface adhesion coefficient.

2. The method according to claim 1, characterized in that, Determining the road surface adhesion coefficient in front of the vehicle based on the visual estimate of the road surface adhesion coefficient includes: Determine the dynamic estimate of the road surface adhesion coefficient in front of the vehicle; The road surface adhesion coefficient in front of the vehicle is determined based on the visual estimate and the dynamic estimate of the road surface adhesion coefficient.

3. The method according to claim 2, characterized in that, Determining the road surface adhesion coefficient in front of the vehicle based on the visual estimate and the dynamic estimate of the road surface adhesion coefficient includes: If the weight of the visual estimate of the road surface adhesion coefficient is greater than or equal to a preset threshold, and the dynamic estimate of the road surface adhesion coefficient is within a preset range, the dynamic estimate of the road surface adhesion coefficient is used as the road surface adhesion coefficient of the road surface in front of the vehicle; wherein, the weight of the visual estimate of the road surface adhesion coefficient is used to reflect the reliability of the visual estimate of the road surface adhesion coefficient; the preset range is the range of values ​​of the road surface adhesion coefficient corresponding to the road surface type of the road surface in front. If the weight of the visual estimate of the road surface adhesion coefficient is greater than or equal to a preset threshold, but the dynamic estimate of the road surface adhesion coefficient is not within the preset range, the visual estimate of the road surface adhesion coefficient shall be used as the road surface adhesion coefficient of the road surface in front of the vehicle. If the weight of the visual estimate of the road surface adhesion coefficient is less than a preset threshold, the dynamic estimate of the road surface adhesion coefficient is used as the road surface adhesion coefficient of the road surface in front of the vehicle.

4. The method according to claim 3, characterized in that, The weight of the visual estimate of the road surface adhesion coefficient is determined based on the reliability of the road surface type identification result; The reliability of the road surface type identification result is determined based on the distance between the image region that determines the road surface type identification result and the center point of the road surface image.

5. The method according to claim 2, characterized in that, The dynamic estimate of the road surface adhesion coefficient in front of the vehicle includes: Using the Kalman filter algorithm, based on the visual estimate of the road surface adhesion coefficient, vehicle operation information, vehicle dynamics model, and tire model, the dynamic estimate of the road surface adhesion coefficient in front of the vehicle is determined.

6. The method according to claim 5, characterized in that, Using the Kalman filter algorithm, based on the visual estimate of the road surface adhesion coefficient, vehicle operating information, vehicle dynamics model, and tire model, the dynamic estimate of the road surface adhesion coefficient in front of the vehicle is determined, including: The measurement equations of the estimation system for the road adhesion coefficient are determined based on the vehicle dynamics model and the tire model. The road adhesion coefficients of the four wheels of the vehicle are used as the state variables of the estimation system, the visual estimates of the road adhesion coefficients are used as the initial values ​​of the state variables, the front wheel steering angle of the vehicle is used as the input parameters of the estimation system, and the vehicle operation information is used as the observations of the estimation system to construct the state equation of the estimation system. Based on the Kalman filter algorithm, and based on the measurement equation and the state equation, the dynamic estimate of the road surface adhesion coefficient is obtained.

7. The method according to claim 5, characterized in that, The time for collecting the vehicle operation information is determined based on the distance between the UAV and the vehicle and the vehicle's speed when the UAV takes the road image.

8. The method according to claim 1, characterized in that, The acquisition of road surface type identification results in front of the vehicle includes: Acquire road surface images of the road surface in front of the vehicle captured by the drone equipment; Based on the road surface image and the road surface recognition model, the road surface type recognition result of the road surface in front of the vehicle is obtained.

9. The method according to claim 1, characterized in that, The acquisition of road surface type identification results in front of the vehicle includes: The system receives the road surface type identification result of the road surface in front of the vehicle sent by the UAV device; wherein the road surface type identification result of the road surface in front of the vehicle is obtained by the UAV device based on the road surface image and road surface identification model of the road surface in front of the vehicle.

10. The method according to claim 1, characterized in that, The road surface type identification results for the road surface in front of the vehicle include multiple results, each determined based on a different image region in the road surface image. The step of determining a visual estimate of the road surface adhesion coefficient for the road surface in front of the vehicle based on the road surface type identification results and a preset correspondence includes: Based on the road type identification result with the highest reliability among multiple road type identification results and the preset correspondence, a visual estimate of the road adhesion coefficient of the road surface in front of the vehicle is determined. The reliability of the road surface type identification result is determined based on the distance between the image region that determines the road surface type identification result and the center point of the road surface image.

11. The method according to claim 1, characterized in that, The step of determining a visual estimate of the road surface adhesion coefficient in front of the vehicle based on the road surface type identification result and a preset correspondence includes: Based on the road surface type identification result and the preset correspondence, the range of values ​​for the road surface adhesion coefficient corresponding to the road surface type is determined. Based on the range of values ​​for the road surface adhesion coefficient corresponding to the road surface type, a visual estimate of the road surface adhesion coefficient in front of the vehicle is determined.

12. The method according to claim 11, characterized in that, The step of determining a visual estimate of the road surface adhesion coefficient in front of the vehicle based on the range of values ​​for the road surface adhesion coefficient corresponding to the road surface type includes: Based on the average value of the endpoints of the range of the road surface adhesion coefficient, a visual estimate of the road surface adhesion coefficient in front of the vehicle is determined.

13. The method according to claim 1, characterized in that, After determining the road surface adhesion coefficient of the road surface in front of the vehicle based on the visual estimate of the road surface adhesion coefficient, the method further includes: The driving strategy of the vehicle is adjusted based on the road surface adhesion coefficient in front of the vehicle.

14. The method according to claim 1, characterized in that, The road surface adhesion coefficient includes the road surface adhesion coefficient of the left wheel of the vehicle and the road surface adhesion coefficient of the right wheel of the vehicle. Based on the road surface type identification results and the preset correspondence, a visual estimate of the road surface adhesion coefficient of the road surface in front of the vehicle is determined, including: Based on the left-side road surface type recognition result and the preset correspondence, the visual estimate of the road surface adhesion coefficient of the vehicle's left wheel is determined; wherein, the left-side road surface type recognition result is obtained based on the left-side region image of the road surface image in front of the vehicle. Based on the right-side road surface type identification result and the preset correspondence, the visual estimate of the road surface adhesion coefficient of the vehicle's right wheel is determined; wherein, the right-side road surface type identification result is obtained based on the right-side region image of the road surface image in front of the vehicle.

15. The method according to claim 1, characterized in that, The method further includes: In response to a user-issued trigger command for a road surface adhesion coefficient recognition task, an instruction message is sent to the drone device, which instructs the drone device to capture an image of the road surface in front of the vehicle.

16. The method according to claim 1, characterized in that, The method further includes: Predict road conditions ahead based on navigation routes and map information; If the road conditions ahead of the vehicle are complex, an instruction message is sent to the drone device, which instructs the drone device to take a picture of the road surface ahead of the vehicle.

17. The method according to claim 1, characterized in that, The flight altitude of the drone is determined based on the vehicle's speed to balance the resolution and coverage of the images captured by the drone.

18. The method according to claim 1, characterized in that, The acquisition frequency of the drone equipment is determined based on the vehicle's speed and / or the road conditions ahead of the vehicle.

19. The method according to claim 1, characterized in that, The flight attitude of the drone is determined based on the road conditions in front of the vehicle, so that the camera device of the drone remains parallel to the road surface.

20. An electronic device, characterized in that, It includes a processor and a memory, the processor being coupled to the memory; the memory is used to store computer instructions, which are loaded and executed by the processor to enable the computer device to implement the road surface adhesion coefficient identification method as described in any one of claims 1 to 19.

21. A vehicle, characterized in that, Including the electronic device as described in claim 20.

22. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes computer-executable instructions that, when executed on a computer, cause the computer to perform the road surface adhesion coefficient identification method according to any one of claims 1 to 19.

23. A computer program product, characterized in that, The computer program product includes a computer program that, when run on an electronic device, causes the electronic device to perform the road surface adhesion coefficient identification method as described in any one of claims 1 to 19.