Road surface friction coefficient prediction method and device, computer equipment and storage medium

Abstract

The invention relates to a pavement friction coefficient prediction method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring an initial image of a to-be-detected area and self-vehicle acquisition data; segmenting the initial image to obtain a segmented image comprising a plurality of segmented regions; identifying the segmented image to obtain road surface data corresponding to each segmented region; and determining a friction coefficient prediction index based on the data collected by the vehicle and the road surface data corresponding to each segmented region, and inputting the friction coefficient prediction index into the prediction model to obtain a friction coefficient prediction result of the region to be detected. By adopting the method, the friction coefficients of pavements made of different materials can be respectively predicted, and the method has relatively high prediction precision and accuracy.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device, computer equipment, and storage medium for predicting road friction coefficient. Background Art

[0002] During the driving process of the vehicle, analyzing the road conditions in front of the vehicle can effectively increase the user or the self-driving vehicle's understanding of the road conditions ahead and assist the user or the vehicle in making decisions.

[0003] The road friction coefficient is an important element in road condition analysis. Existing methods for analyzing road friction coefficient usually use image recognition to collect images of the area in front of the vehicle through cameras and other equipment, and predict the road friction coefficient based on the image analysis results. However, existing image recognition methods can only predict the friction coefficient of the entire area in front, and the prediction results are of low accuracy. Summary of the invention

[0004] Based on this, it is necessary to provide a road friction coefficient prediction method, device, computer equipment, computer-readable storage medium and computer program product that can predict the friction coefficient of the road surface in front of the vehicle in different areas in order to address the above technical problems.

[0005] In a first aspect, the present application provides a method for predicting a road friction coefficient, comprising:

[0006] Obtain the initial image of the area to be tested and the data collected by the vehicle;

[0007] Segmenting the initial image to obtain a segmented image including a plurality of segmented regions;

[0008] Recognizing the segmented image to obtain road surface data corresponding to each segmented area;

[0009] Based on the self-vehicle collected data and the road surface data corresponding to each of the segmented areas, a friction coefficient prediction index is determined, and the friction coefficient prediction index is input into a prediction model to obtain a friction coefficient prediction result of the area to be tested.

[0010] In one embodiment, segmenting the initial image to obtain a segmented image including a plurality of segmented regions includes:

[0011] Obtaining a plurality of segmentation masks of the initial image based on the segmentation model, wherein each segmentation mask corresponds to a segmentation region;

[0012] Decoding the segmentation mask to obtain segmentation area contour coordinates corresponding to each segmentation area;

[0013] The segmented image is acquired based on the segmentation mask, the coordinates of the segmentation area contour and the initial image.

[0014] In one embodiment, the identifying the segmented image to obtain the road surface data corresponding to each segmented area includes:

[0015] Based on the recognition model, obtaining label information of each segmented area;

[0016] Based on the label information of each of the segmented areas, road surface data corresponding to each of the segmented areas is determined.

[0017] In one embodiment, the vehicle-collected data is collected by the vehicle's own sensors; the vehicle's own sensors include at least one of a temperature and humidity sensor, a tire pressure sensor or a camera; wherein, when the vehicle's own sensors include the temperature and humidity sensor, the vehicle-collected data includes temperature and humidity data; when the vehicle's own sensors include the tire pressure sensor, the vehicle-collected data includes tire pressure; when the vehicle's own sensors include a camera, the vehicle-collected data includes weather data.

[0018] In one embodiment, the step of inputting the friction coefficient prediction index into a prediction model to obtain a friction coefficient prediction result of the area to be measured includes:

[0019] The friction coefficient prediction index is input into the prediction model to obtain the segmentation prediction results of each segmented area, and all the segmented areas and the corresponding segmentation prediction results are used as the friction coefficient prediction results of the area to be measured.

[0020] In one embodiment, the method further comprises:

[0021] If the initial image of the area to be tested cannot be obtained, determining at least one target vehicle in front of the vehicle;

[0022] Determining the motion state of the target vehicle;

[0023] Based on the motion state of the target vehicle, a friction coefficient prediction result of the area to be measured is determined.

[0024] In one embodiment, determining the motion state of the target vehicle includes:

[0025] Obtaining a vehicle identification of each of the target vehicles;

[0026] Based on the vehicle identification, obtaining vehicle behavior video frames corresponding to each of the target vehicles;

[0027] The vehicle behavior video frame is input into a video understanding model to obtain the motion state of the corresponding target vehicle.

[0028] In a second aspect, the present application also provides a road friction coefficient prediction device, comprising:

[0029] The acquisition module is used to obtain the initial image of the area to be tested and the self-vehicle acquisition data;

[0030] A segmentation module, used for segmenting the initial image to obtain a segmented image including a plurality of segmented regions;

[0031] A recognition module, used to recognize the segmented image and obtain road surface data corresponding to each segmented area;

[0032] The prediction module is used to determine a friction coefficient prediction index based on the self-vehicle collected data and the road surface data corresponding to each of the segmented areas, input the friction coefficient prediction index into a prediction model, and obtain a friction coefficient prediction result of the area to be tested.

[0033] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0034] Obtain the initial image of the area to be tested and the data collected by the vehicle;

[0035] Segmenting the initial image to obtain a segmented image including a plurality of segmented regions;

[0036] Recognizing the segmented image to obtain road surface data corresponding to each segmented area;

[0037] Based on the self-vehicle collected data and the road surface data corresponding to each of the segmented areas, a friction coefficient prediction index is determined, and the friction coefficient prediction index is input into a prediction model to obtain a friction coefficient prediction result of the area to be tested.

[0038] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0039] Obtain the initial image of the area to be tested and the data collected by the vehicle;

[0040] Segmenting the initial image to obtain a segmented image including a plurality of segmented regions;

[0041] Recognizing the segmented image to obtain road surface data corresponding to each segmented area;

[0042] Based on the self-vehicle collected data and the road surface data corresponding to each of the segmented areas, a friction coefficient prediction index is determined, and the friction coefficient prediction index is input into a prediction model to obtain a friction coefficient prediction result of the area to be tested.

[0043] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0044] Obtain the initial image of the area to be tested and the data collected by the vehicle;

[0045] Segmenting the initial image to obtain a segmented image including a plurality of segmented regions;

[0046] Recognizing the segmented image to obtain road surface data corresponding to each segmented area;

[0047] Based on the self-vehicle collected data and the road surface data corresponding to each of the segmented areas, a friction coefficient prediction index is determined, and the friction coefficient prediction index is input into a prediction model to obtain a friction coefficient prediction result of the area to be tested.

[0048] The above-mentioned road friction coefficient method, device, computer equipment, computer-readable storage medium and computer program product segment the image of the area to be tested according to the road material to obtain a segmented image, identify the segmented image to obtain the identification results of each segmented area, and predict the friction coefficient for each segmented area in the area to be tested based on the data collected by the vehicle sensor and the identification results of the segmented area. Segmenting the area to be tested can avoid the omission of some smaller areas caused by the recognition of the entire image in the traditional method, and the output friction coefficient prediction result is more accurate than the prediction value of the entire image. In addition, the environment in which the vehicle is located and the situation of the vehicle itself are also taken into account, and the influence of the data collected by the vehicle is added to the prediction. A more precise and accurate friction coefficient prediction result can be obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0050] Figure 1 A schematic diagram of a road friction coefficient method in one embodiment;

[0051] Figure 2 is a schematic diagram of a sub-flow chart of step 102 in one embodiment;

[0052] Figure 3 1 is a schematic diagram of a sub-flow chart of step 103 in one embodiment;

[0053] Figure 4 A schematic diagram of a flow chart of another implementation of a road friction coefficient method in an embodiment;

[0054] Figure 5 A schematic diagram of the structure of a model framework used in an embodiment;

[0055] Figure 6 A schematic diagram of the structure of another model framework applied in one embodiment;

[0056] Figure 7 is a schematic diagram of a practical application in one embodiment;

[0057] Figure 8 is a schematic structural diagram of a road friction coefficient device in one embodiment;

[0058] Fig. 9 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0060] In an exemplary embodiment, in one embodiment, as Figure 1 As shown, a method for predicting road friction coefficient is provided. This embodiment uses the method applied to a computer device as an example for illustration. It can be understood that the method can also be applied to a server, and can also be applied to a system including a computer device and a server, and is implemented through the interaction between the computer device and the server. In this embodiment, the method includes the following steps:

[0061] Step 101, obtaining an initial image of the area to be tested and data collected by the vehicle;

[0062] During the driving process of the vehicle, monitoring the friction coefficient of the road ahead is of great significance to driving safety. Therefore, this embodiment uses the area within a preset range in front of the vehicle as the area to be tested. In some embodiments, the initial image can be an image of the area to be tested taken by the vehicle camera; the self-vehicle collected data can be data collected by some sensors of the vehicle, such as cameras, temperature and humidity sensors, tire pressure detectors, etc. The initial image and the self-vehicle collected data can be obtained only through the vehicle's own sensors, without the need for external data, and can run even without networking, with a wide range of applications and higher robustness. Exemplarily, the current vehicle can be a manned vehicle or an unmanned vehicle, which is not specifically limited here.

[0063] Step 102, segmenting the initial image to obtain a segmented image including a plurality of segmented regions;

[0064] Different road materials have different friction coefficients, and during driving, the road material in front of the vehicle may not be of the same type. Conventional image recognition methods have problems with difficult distinction or inaccurate recognition. For example, if it is rainy or snowy, there may be water or ice on the road ahead; if you want to judge the friction coefficient of the road ahead, you need to consider the area of ​​the original road material and the ice and water areas. At this time, the initial image segmentation can achieve good results. For another example, if there is a very small piece of oil on the road ahead, it may be missed in conventional image recognition, but segmentation can separate this area separately. The segmented image can display multiple segmented areas for subsequent recognition.

[0065] Step 103, identifying the segmented image to obtain road surface data corresponding to each segmented area;

[0066] Each segmented area of ​​the segmented image is identified in turn to obtain the identification result of each segmented area, and the corresponding label and confidence are output, so that the road material of each segmented area can be identified.

[0067] Step 104 , based on the self-vehicle collected data and the road surface data corresponding to each segmented area, a friction coefficient prediction index is determined, and the friction coefficient prediction index is input into a prediction model to obtain a friction coefficient prediction result of the area to be tested.

[0068] The data collected by the vehicle and the road surface data corresponding to each segmented area are used as friction coefficient prediction indicators, and the friction coefficient indicators are input into the prediction model to obtain the final prediction results.

[0069] In the above-mentioned road friction coefficient prediction method, the image of the area to be tested is segmented according to the road material to obtain a segmented image, the segmented image is identified to obtain the identification results of each segmented area, and the friction coefficient is predicted for each segmented area in the area to be tested based on the data collected by the vehicle sensor and the identification results of the segmented area. Segmenting the area to be tested can avoid the omission of some smaller areas caused by the recognition of the entire image in the traditional method, and the output friction coefficient prediction result is more accurate than the prediction value of the entire image. In addition, the environment in which the vehicle is located and the situation of the vehicle itself are also taken into account, and the influence of the data collected by the vehicle is added to the prediction. A more precise and accurate friction coefficient prediction result can be obtained.

[0070] In an exemplary embodiment, Figure 2 As shown, the initial image is segmented to obtain a segmented image including multiple segmented areas, including:

[0071] Step 201, obtaining multiple segmentation masks of the initial image based on the segmentation model, wherein each segmentation mask corresponds to a segmentation area;

[0072] Step 202, decoding the segmentation mask to obtain the segmentation area contour coordinates corresponding to each segmentation area;

[0073] Step 203, obtaining a segmented image based on the segmentation mask, the coordinates of the segmentation area contour and the initial image.

[0074] In one of the embodiments, a SAM (Semgent Anything) model is used as a segmentation model to achieve segmentation of the initial image. The SAM model can accurately segment areas of different road materials. The SAM model has a strong zero-sample prediction capability, so it can be directly used in practical applications without a large amount of migration training. The output of the SAM model has two, one is a mask of different segmented areas (a group of pixels that draw the outline of an object), and the other is the confidence of each segmented area.

[0075] The SAM model includes an image encoder, a prompt encoder and a mask decoder, wherein the image encoder is used to encode the input initial image to obtain image features; the prompt encoder includes a prompt word encoder and a mask encoder, and the prompt word encoder can output a feature vector and add it to the features output by the image encoder. The mask encoder uses a downsampled image of the labeled object contour coordinates, and then extracts the image features through a convolutional neural network and adds them to the output of the image encoder; the mask decoder is used to decode the features passed through the prompt encoder and the image encoder, and output the contour coordinates and confidence of the segmented area after segmentation. The final initial image including the segmentation mask and the contour coordinates of the segmentation area is used as the segmented image.

[0076] Previous road detection solutions can only vaguely estimate the friction coefficient of the entire area ahead and cannot accurately segment it, and can only provide a warning message. In this embodiment, by accurately segmenting the initial image, different areas of the road ahead can be accurately detected, and then the friction coefficient of each segmented area can be predicted, thereby achieving accurate control of the vehicle to avoid dangerous areas.

[0077] In an exemplary embodiment, Figure 3 As shown, the segmented image is identified to obtain the road surface data corresponding to each segmented area, including:

[0078] Step 301, based on the recognition model, obtain label information of each segmented area;

[0079] Step 302: determining the road surface data corresponding to each segmented area based on the label information of each segmented area.

[0080] Exemplarily, the recognition model can use the CLIP (Contrastive Language-Image Pre-Training) model. Since only the segmentation mask is obtained in step 102, but there is no label information of the segmentation mask, the recognition model is added to classify different areas. Specifically, the input of the recognition model is the segmentation image, and the output is the label and confidence of each segmentation mask. This step can identify the road surface material of each segmented area and determine the identified road surface material as road surface data.

[0081] In an exemplary embodiment, the friction coefficient prediction index is input into the prediction model to obtain the friction coefficient prediction result of the area to be measured, including:

[0082] The friction coefficient prediction index is input into the prediction model to obtain the segmentation prediction results of each segmented area, and all the segmented areas and the corresponding segmentation prediction results are used as the friction coefficient prediction results of the area to be tested.

[0083] For example, the prediction model can use a classifier, such as a multilayer perceptron (MLP), which can map the vehicle state and road material in different environments to the friction coefficient of the road surface. In practical applications, the friction coefficient prediction index needs to be encoded and converted into a vector, and then input into the prediction model for prediction to obtain the friction coefficient of each segmented area. The output result can be a labeled image or a table, etc., which is not specifically limited here.

[0084] In an exemplary embodiment, the data collected by the vehicle itself is collected by the vehicle's own sensors; the vehicle's own sensors include at least one of a temperature and humidity sensor, a tire pressure sensor or a camera; wherein, when the vehicle's own sensors include a temperature and humidity sensor, the data collected by the vehicle itself includes temperature and humidity data; when the vehicle's own sensors include a tire pressure sensor, the data collected by the vehicle itself includes tire pressure; when the vehicle's own sensors include a camera, the data collected by the vehicle itself includes weather data.

[0085] Specifically, temperature and humidity data can reflect the environment around the vehicle, tire pressure can reflect the vehicle's own status, and weather data collected by the camera can reflect weather conditions. These data are collected by the vehicle's own sensors and do not rely on external data. Therefore, data can be obtained and analyzed without networking, which is more convenient to use and has a wider range of applications. It can also operate without a network in extreme environments.

[0086] In an exemplary embodiment, the inference time can be further reduced by adding a text encoder to the original recognition model to obtain a new recognition classification model. Specifically, taking the recognition model using the CLIP model as an example, the input of the original CLIP model is a segmented image and the output is text.

[0087] In this embodiment, historical data is first obtained, and the segmented image with a mask in the historical data is input into the image decoder of the recognition classification model to obtain the historical image features of the segmented image; the corresponding self-vehicle collected data at this time is input into the text encoder to obtain the historical text features; the true value of the road friction coefficient in the historical data is obtained, and it is also encoded through the text encoder to obtain the historical friction coefficient features; the recognition classification model is trained based on the historical image features, historical text features, and historical friction coefficient features, so that the feature spaces of these three feature vectors are closer, maximizing the similarity of matching image and text pairs, and minimizing the similarity between images and unmatched text pairs. Exemplarily, the text encoder can be the LLaMA3 model, and the image decoder can be the ViT model.

[0088] In practical applications, the trained recognition and classification model can be used to input the segmented image and the data collected by the vehicle to directly output the friction coefficient prediction result. Compared with directly using unsupervised recognition models and prediction models, this embodiment is slightly more difficult to implement and requires pre-training of the recognition and classification model, but it can further reduce the reasoning time and improve the accuracy of the prediction.

[0089] In an exemplary embodiment, Figure 4 As shown, the road friction coefficient prediction method provided by the present application also includes:

[0090] Step 401, if the initial image of the area to be measured cannot be obtained, then determine at least one target vehicle in front of the vehicle;

[0091] Step 402, determining the motion state of the target vehicle;

[0092] Step 403: Determine the friction coefficient prediction result of the area to be measured based on the motion state of the target vehicle.

[0093] If the road surface information of the blocked area ahead cannot be obtained in time due to the occlusion of the vehicle ahead, the friction coefficient of the area where the vehicle is located can be inferred by identifying the action of the vehicle ahead. Based on this, the target vehicle ahead of the vehicle can be obtained through the target detection network, and the frame and label of the target vehicle can be output at the same time.

[0094] In an exemplary embodiment, determining the motion state of the target vehicle includes:

[0095] Obtaining the vehicle identification of each target vehicle;

[0096] Based on the vehicle identification, the vehicle behavior video frame corresponding to each target vehicle is obtained;

[0097] The vehicle behavior video frames are input into the video understanding model to obtain the motion state of the corresponding target vehicle.

[0098] See also Figure 5 , Figure 5 A schematic diagram of the model framework of the present embodiment is shown. Before obtaining the vehicle identification of the target vehicle, the multi-frame output of the target detection network is first sent to the target tracking network, and the vehicle identification and frame of each target vehicle are calculated by optical flow method, Kalman filtering or other methods. Based on the vehicle identification, the vehicle behavior video frames with the target vehicle are screened out and sent to the video understanding model. The video understanding model can extract the key frames in the vehicle behavior video frames, perform feature extraction on the key frames, and expand the high-dimensional vector of the features into a one-dimensional vector, add position encoding, and then perform global spatiotemporal feature modeling to extract video features, and finally output the motion state of the corresponding target vehicle through a classification head. The video understanding model can analyze all target vehicles. For example, if there are 3 vehicles in front, the video understanding model will analyze the three vehicles separately.

[0099] In this way, even when the road ahead is blocked, the friction coefficient of the road ahead can be analyzed through the motion state of the target vehicle ahead. For example, if the vehicle ahead skids, it means that the friction coefficient of the road ahead is small, which makes up for the deficiency of the prior art that only the road surface can be analyzed.

[0100] To explain the road friction coefficient method and effect in detail, the following is a most detailed embodiment:

[0101] See also Figure 6 , Figure 6 The schematic diagram of the structure of the model framework used in this embodiment is shown, as well as the framework of the segmentation model, recognition model and prediction model used in this embodiment. Exemplarily, the SAM model provided in this embodiment consists of three parts, namely, an image encoder (ImageEncoder), a prompt encoder (PromptEncoder), and a mask decoder (MaskDecoder). The input of the segmentation model is the initial image, and the output is a segmented image with multiple groups of segmentation masks (coordinate points) after segmentation.

[0102] Specifically, the role of the image encoder is to encode the input image and output the image features. The image encoder uses the ViT model trained by MAE (Mask Auto Encoder). The specific workflow of the ViT (Visison Transformer) model is that the input image is scaled and padded to a uniform size of 1024x1024 pixels, and then the image is downsampled by 16 times, that is, the image is split into 64x64 patches, each of which is 16x16 in size, and expanded into a one-dimensional vector. Then the 64x64 vector is sent to the encoder for image encoding after position encoding to extract the image features. Using ViT, the image can be directly divided into blocks and encoded, and then the global information is modeled using the self-attention mechanism. This allows the ViT model to better capture the long-range dependencies and overall structure in the image, and perform better for image tasks with complex global relationships. Self-supervised training of the image encoder using the MAE (Mask Auto Encoder) method can enhance the feature representation ability of the image encoder.

[0103] The prompt encoder consists of two parts, one is a sparse prompt encoder (Prompt Encoder) that provides points, boxes, and text descriptions (text), and the other is a dense mask encoder (Mask Encoder). The prompt encoder will input the corresponding target point coordinates, as well as the box (the coordinates of the point in the upper left corner and the point in the lower right corner), and a sentence describing the object. The prompt encoder here uses a trained CLIP model. The prompt encoder outputs a feature vector and adds it to the features output by the image encoder. The mask encoder uses a 4-fold downsampled image with labeled object contour coordinates. Then a convolutional neural network is used to extract the image features and add them to the output of the image encoder.

[0104] The function of the mask decoder is to decode the features passed through the prompt encoder + image encoder, and output the coordinates of the segmented area contour after segmentation, as well as the confidence. In addition, the coordinates of the segmented area contour can also be converted to calculate the position of the object in 3D space and output it.

[0105] See also Figure 7 In a possible implementation, the initial image is segmented into two regions: region 701 and region 702 which is the remaining portion of the initial image excluding region 701. This means that region 701 and region 702 belong to roads of different materials and can be represented by segmentation masks with different parameters when outputting.

[0106] Exemplarily, this embodiment uses the CLIP model as a recognition model. The CLIP model has powerful multimodal understanding capabilities, excellent zero-sample learning capabilities, and efficient training and reasoning capabilities. The CLIP model can have high reasoning capabilities, and even if some objects have not been seen in the training stage, a correct result may be output. The CLIP model is divided into two parts, one is an image encoder, and the other is a text encoder. The input of the recognition model is a picture with segmentation masks of each segmented area. The output of the model is the label and confidence of the masked part, which can be used to identify the material of different road surfaces. Taking the above-mentioned area 701 and area 702 as an example, if area 702 is an asphalt road surface and area 701 is an ice surface, the output label is: area 702 corresponds to asphalt (asphalt) and the corresponding confidence, area 701 corresponds to ice (ice) and the corresponding confidence.

[0107] Finally, the segmented image, the road surface data of each segmented area, and the self-vehicle collected data are converted into vectors through variable encoding. The encoding method can be One Hot encoding, and the friction coefficient of each segmented area is output. For example, when the self-vehicle collected data is specific, the friction coefficient of dry asphalt road is 0.8, the friction coefficient of wet asphalt road is 0.5, and the friction coefficient of ice surface is 0.1. Then, the final output result is Figure 7 On this basis, labels and friction coefficients of each segmented area are added; it can also be output in the form of tables, text, etc., according to parameters such as area numbers, labels, friction coefficients, etc.

[0108] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0109] Based on the same inventive concept, the embodiment of the present application also provides a road friction coefficient prediction device for implementing the road friction coefficient prediction method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more road friction coefficient prediction device embodiments provided below can refer to the limitations of the road friction coefficient prediction method above, and will not be repeated here.

[0110] In an exemplary embodiment, Figure 8 As shown, a road friction coefficient prediction device 800 is provided, comprising: a collection module 801, a segmentation module 802, a recognition module 803 and a prediction module 804, wherein:

[0111] The acquisition module 801 is used to obtain the initial image of the area to be tested and the self-vehicle acquisition data;

[0112] The segmentation module 802 is used to segment the initial image to obtain a segmented image including a plurality of segmented regions;

[0113] The recognition module 803 is used to recognize the segmented image and obtain the road surface data corresponding to each segmented area;

[0114] The prediction module 804 is used to determine the friction coefficient prediction index based on the self-vehicle collected data and the road surface data corresponding to each segmented area, input the friction coefficient prediction index into the prediction model, and obtain the friction coefficient prediction result of the area to be tested.

[0115] In an exemplary embodiment, the segmentation module 802 is further configured to:

[0116] Based on the segmentation model, a plurality of segmentation masks of the initial image are obtained, wherein each segmentation mask corresponds to a segmentation region;

[0117] Decode the segmentation mask to obtain the segmentation area contour coordinates corresponding to each segmentation area;

[0118] A segmented image is obtained based on the segmentation mask, the coordinates of the segmentation region contour and the initial image.

[0119] In an exemplary embodiment, the identification module 803 is further configured to:

[0120] Based on the recognition model, obtain the label information of each segmented area;

[0121] Based on the label information of each segmented area, the road surface data corresponding to each segmented area is determined.

[0122] In an exemplary embodiment, the prediction module 804 is further configured to:

[0123] The friction coefficient prediction index is input into the prediction model to obtain the segmentation prediction results of each segmented area, and all the segmented areas and the corresponding segmentation prediction results are used as the friction coefficient prediction results of the area to be tested.

[0124] In an exemplary embodiment, the road friction coefficient prediction device 800 further includes:

[0125] Vehicle detection module for:

[0126] If the initial image of the area to be measured cannot be obtained, at least one target vehicle in front of the vehicle is determined;

[0127] Determine the motion state of the target vehicle;

[0128] Based on the motion state of the target vehicle, the friction coefficient prediction result of the area to be tested is determined.

[0129] The vehicle detection module is also used to:

[0130] Obtaining the vehicle identification of each target vehicle;

[0131] Based on the vehicle identification, the vehicle behavior video frame corresponding to each target vehicle is obtained;

[0132] The vehicle behavior video frames are input into the video understanding model to obtain the motion state of the corresponding target vehicle.

[0133] Each module in the above-mentioned road friction coefficient prediction device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0134] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Fig. 9As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store road friction coefficient prediction data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a road friction coefficient prediction method is implemented.

[0135] Those skilled in the art will understand that Fig. 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0136] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0137] Obtain the initial image of the area to be tested and the data collected by the vehicle;

[0138] Segmenting the initial image to obtain a segmented image including a plurality of segmented regions;

[0139] Recognize the segmented image to obtain the road surface data corresponding to each segmented area;

[0140] Based on the data collected by the vehicle and the road surface data corresponding to each segmented area, the friction coefficient prediction index is determined, and the friction coefficient prediction index is input into the prediction model to obtain the friction coefficient prediction result of the area to be tested.

[0141] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0142] Based on the segmentation model, multiple segmentation masks of the initial image are obtained, wherein each segmentation mask corresponds to a segmentation area; the segmentation masks are decoded to obtain the segmentation area contour coordinates corresponding to each segmentation area; based on the segmentation masks, the segmentation area contour coordinates and the initial image, a segmentation image is obtained.

[0143] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0144] Based on the recognition model, label information of each segmented area is obtained; based on the label information of each segmented area, road surface data corresponding to each segmented area is determined.

[0145] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0146] The friction coefficient prediction index is input into the prediction model to obtain the segmentation prediction results of each segmented area, and all the segmented areas and the corresponding segmentation prediction results are used as the friction coefficient prediction results of the area to be tested.

[0147] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0148] If the initial image of the area to be measured cannot be obtained, at least one target vehicle in front of the vehicle is determined; the motion state of the target vehicle is determined; and based on the motion state of the target vehicle, a friction coefficient prediction result of the area to be measured is determined.

[0149] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0150] Obtain the vehicle identification of each target vehicle; based on the vehicle identification, obtain the vehicle behavior video frame corresponding to each target vehicle; input the vehicle behavior video frame into the video understanding model to obtain the motion state of the corresponding target vehicle.

[0151] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0152] Obtain the initial image of the area to be tested and the data collected by the vehicle;

[0153] Segmenting the initial image to obtain a segmented image including a plurality of segmented regions;

[0154] Recognize the segmented image to obtain the road surface data corresponding to each segmented area;

[0155] Based on the data collected by the vehicle and the road surface data corresponding to each segmented area, the friction coefficient prediction index is determined, and the friction coefficient prediction index is input into the prediction model to obtain the friction coefficient prediction result of the area to be tested.

[0156] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0157] Based on the segmentation model, multiple segmentation masks of the initial image are obtained, wherein each segmentation mask corresponds to a segmentation area; the segmentation masks are decoded to obtain the segmentation area contour coordinates corresponding to each segmentation area; based on the segmentation masks, the segmentation area contour coordinates and the initial image, a segmentation image is obtained.

[0158] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0159] Based on the recognition model, label information of each segmented area is obtained; based on the label information of each segmented area, road surface data corresponding to each segmented area is determined.

[0160] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0161] The friction coefficient prediction index is input into the prediction model to obtain the segmentation prediction results of each segmented area, and all the segmented areas and the corresponding segmentation prediction results are used as the friction coefficient prediction results of the area to be tested.

[0162] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0163] If the initial image of the area to be measured cannot be obtained, at least one target vehicle in front of the vehicle is determined; the motion state of the target vehicle is determined; and based on the motion state of the target vehicle, a friction coefficient prediction result of the area to be measured is determined.

[0164] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0165] Obtain the vehicle identification of each target vehicle; based on the vehicle identification, obtain the vehicle behavior video frame corresponding to each target vehicle; input the vehicle behavior video frame into the video understanding model to obtain the motion state of the corresponding target vehicle.

[0166] It should be noted that the data involved in this application (including but not limited to data used for analysis, stored data, displayed data, etc.) are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0167] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0168] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0169] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for predicting road friction coefficient, characterized in that: The method comprises: Obtain the initial image of the area to be tested and the data collected by the vehicle; Segmenting the initial image to obtain a segmented image including a plurality of segmented regions; Recognizing the segmented image to obtain road surface data corresponding to each segmented area; Based on the self-vehicle collected data and the road surface data corresponding to each of the segmented areas, a friction coefficient prediction index is determined, and the friction coefficient prediction index is input into a prediction model to obtain a friction coefficient prediction result of the area to be tested.

2. The method according to claim 1, characterized in that The step of segmenting the initial image to obtain a segmented image including a plurality of segmented regions includes: Obtaining a plurality of segmentation masks of the initial image based on the segmentation model, wherein each segmentation mask corresponds to a segmentation region; Decoding the segmentation mask to obtain segmentation area contour coordinates corresponding to each segmentation area; The segmented image is acquired based on the segmentation mask, the coordinates of the segmentation area contour and the initial image.

3. The method according to claim 1, characterized in that: The identifying the segmented image to obtain road surface data corresponding to each segmented area includes: Based on the recognition model, obtaining label information of each segmented area; Based on the label information of each of the segmented areas, road surface data corresponding to each of the segmented areas is determined.

4. The method according to claim 1, characterized in that The self-vehicle collected data is collected by the vehicle's own sensors; the vehicle's own sensors include at least one of a temperature and humidity sensor, a tire pressure sensor or a camera; wherein, when the vehicle's own sensors include the temperature and humidity sensor, the self-vehicle collected data includes temperature and humidity data; when the vehicle's own sensors include the tire pressure sensor, the self-vehicle collected data includes tire pressure; when the vehicle's own sensors include a camera, the self-vehicle collected data includes weather data.

5. The method according to claim 1, characterized in that The step of inputting the friction coefficient prediction index into a prediction model to obtain a friction coefficient prediction result of the area to be measured includes: The friction coefficient prediction index is input into the prediction model to obtain the segmentation prediction results of each segmented area, and all the segmented areas and the corresponding segmentation prediction results are used as the friction coefficient prediction results of the area to be measured.

6. The method according to claim 1, characterized in that The method further comprises: If the initial image of the area to be tested cannot be obtained, determining at least one target vehicle in front of the vehicle; Determining the motion state of the target vehicle; Based on the motion state of the target vehicle, a friction coefficient prediction result of the area to be measured is determined.

7. The method for predicting road friction coefficient according to claim 6, characterized in that: Determining the motion state of the target vehicle includes: Obtaining a vehicle identification of each of the target vehicles; Based on the vehicle identification, obtaining vehicle behavior video frames corresponding to each of the target vehicles; The vehicle behavior video frame is input into a video understanding model to obtain the motion state of the corresponding target vehicle.

8. A road friction coefficient prediction device, characterized in that: The device comprises: The acquisition module is used to obtain the initial image of the area to be tested and the self-vehicle acquisition data; A segmentation module, used for segmenting the initial image to obtain a segmented image including a plurality of segmented regions; A recognition module, used to recognize the segmented image and obtain road surface data corresponding to each segmented area; The prediction module is used to determine a friction coefficient prediction index based on the self-vehicle collected data and the road surface data corresponding to each of the segmented areas, input the friction coefficient prediction index into a prediction model, and obtain a friction coefficient prediction result of the area to be tested.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images