Image recognition method and related device

Abstract

The embodiment of the invention provides an image recognition method and a related device. The method comprises the steps of splicing images collected from N directions in the driving process of a vehicle into a to-be-recognized image, inputting the to-be-recognized image into a classification model, outputting a recognition result, and determining the pavement type of a to-be-driven road in the driving process of the vehicle according to the recognition result. According to the embodiment of the invention, since the images collected from N directions are adopted, the recognition effect of the image collected from any direction is poor. The vehicle can also determine the road surface type of the road to be driven according to the images collected from other directions, so that the accuracy of identifying the road surface type is improved.

Description

Technical Field

[0001] This application relates to the field of road detection technology, and in particular to an image recognition method and related apparatus. Background Technology

[0002] With the development of intelligent driving technology, vehicles can adjust themselves based on road surface types. For example, during driving, due to differences in road terrain and road surface materials, it is often necessary to adjust the vehicle's suspension, traction, or friction between the wheels and the ground to provide users with a more comfortable driving experience.

[0003] Typically, vehicles determine the road surface type based on vibration signals from the suspension. However, because this requires the vehicle to be actually driving on the road to collect these vibration signals, the road surface type determined is only the road the vehicle is currently driving on or has already traveled. When the local terrain changes, the vehicle cannot determine the terrain change based on the current road surface type. This results in a certain lag in road surface type recognition and a relatively low accuracy rate. Summary of the Invention

[0004] This application provides an image recognition method and related apparatus. By determining the road surface type of the road to be traveled based on images collected from N directions while the vehicle is in motion, the accuracy of road surface type recognition is improved.

[0005] In a first aspect, embodiments of this application provide an image recognition method, the method comprising:

[0006] The image to be identified is determined, wherein the image to be identified is an image obtained by stitching together multiple road surface images, the multiple road surface images including images collected from N directions during the vehicle's travel, where N is a positive integer greater than 1;

[0007] The image to be identified is input into the classification model, and the recognition result is output.

[0008] The road surface type of the road to be traveled by the vehicle during its driving process is determined based on the recognition results.

[0009] In the above method, images acquired from N directions during vehicle travel are used to determine the road surface type of the road to be traveled. This is because the images acquired from N directions include images of the road ahead of the vehicle and images of the road the vehicle is currently traveling on. Since the road surface is continuous, combining images of the road ahead and the road the vehicle is currently traveling on allows for identification of the road surface type. This improves the reliability of the determined road surface type and increases the accuracy of road surface type identification. Compared to existing solutions that determine the road surface type based on images acquired by a forward-facing camera, this embodiment uses images acquired from multiple directions. Even if the camera in any direction is affected by sunlight, resulting in poor image recognition, the vehicle can still identify the road surface type based on images acquired from other directions, ensuring the reliability of the road surface type identification.

[0010] In one alternative embodiment of the first aspect, determining the image to be identified includes:

[0011] The multiple road surface images are stitched together to obtain the first image;

[0012] The image to be identified is obtained by stitching the second image and the first image together. The second image is an image acquired from the forward-looking direction among the multiple road surface images.

[0013] In the above method, multiple road surface images are used to effectively utilize the terrain information around the vehicle, which can assist in identifying the road surface type of the road to be driven and provide a reliable visual perception environment. If the recognition effect of the second image is poor, due to the continuity of the ground, the vehicle can use the road surface type determined based on the first image as the road surface type of the road to be driven, thus improving the feasibility of road type identification.

[0014] In one alternative embodiment of the first aspect, the step of stitching the second image and the first image together to obtain the image to be identified includes:

[0015] The target area is determined from the second image based on the steering wheel angle of the vehicle;

[0016] The target region and the first image are stitched together to obtain the image to be identified.

[0017] In the above method, since the steering wheel angle is used to indicate the vehicle's direction of travel, the target area can be extracted from the second image based on the steering wheel angle. This is equivalent to reducing the road surface information in the second image to the road surface information included in the target area, reducing the computational load on the vehicle and increasing the computational speed. This allows the vehicle to identify a more accurate road surface type based on more precise road surface information, making the results more practically meaningful.

[0018] In one alternative embodiment of the first aspect, the step of stitching the target region and the first image together to obtain the image to be identified includes:

[0019] The target region and the first image are superimposed in the color channel direction of the target region and the first image to obtain the image to be identified.

[0020] In the above method, images are stitched together by superimposing them along their color channel directions, simplifying the image stitching process. Simultaneously, it preserves the target area and road surface information from the first image, effectively improving vehicle recognition speed.

[0021] In one alternative embodiment of the first aspect, determining the road surface type of the road to be driven based on the identification result includes:

[0022] The vehicle's mileage is determined based on the vehicle's speed and steering wheel angle, wherein the vehicle's mileage includes the distance the vehicle travels from the first moment to the second moment.

[0023] The road surface type of the road to be traveled is determined based on the vehicle's mileage, the vehicle's historical road surface type, and the identification result. The historical road surface type includes the road surface type corresponding to the road from the first time point to the second time point.

[0024] In the above method, the accuracy of the identification result is determined based on the vehicle's mileage and historical road surface types. If correct, the road surface type of the road to be traveled is output. This improves the robustness of road surface type identification and further enhances the accuracy of the identification results.

[0025] In one alternative embodiment of the first aspect, determining the road surface type of the road to be traveled based on the vehicle's mileage, the vehicle's historical road surface types, and the identification result includes:

[0026] If the vehicle's mileage is less than a threshold, and the identification result is the same as the historical road surface type, the identification result is taken as the road surface type of the road to be driven.

[0027] In the above method, due to the continuity of the road surface, when the vehicle's mileage is less than a threshold, the road surface type of the road to be traveled can be consistent with the historical road surface type. Therefore, introducing the vehicle's mileage allows the vehicle to further judge the accuracy of the identification results based on historical road surface types. This greatly improves the robustness and accuracy of the vehicle's road surface type identification.

[0028] In one alternative embodiment of the first aspect, the step of inputting the image to be identified into a classification model and outputting a recognition result includes:

[0029] The image to be identified is processed by multiple backbone networks in the trained classification model to obtain the processing result;

[0030] Based on the processing results, the probabilities corresponding to multiple types of results are obtained through the branch structure in the trained classification model, wherein the multiple types of results are used to represent the road surface type of the road to be driven.

[0031] The recognition result is determined based on the probabilities corresponding to the various types of results.

[0032] In the above method, by adding multiple backbone networks and corresponding branch structures to the classification model, the model gains strong feature extraction capabilities, enabling it to extract abstract information from the image to be recognized. This improves the accuracy and recall of the classification model.

[0033] Secondly, embodiments of this application provide an image recognition device, the device comprising:

[0034] A determining unit is used to determine an image to be identified, wherein the image to be identified is an image obtained by stitching together multiple road surface images, the multiple road surface images including images collected from N directions during the vehicle's travel, where N is a positive integer greater than 1;

[0035] The processing unit is used to input the image to be identified into the classification model and output the recognition result;

[0036] The determining unit is further configured to determine the road surface type of the road to be traveled by the vehicle during the driving process based on the identification result.

[0037] In an alternative embodiment of the second aspect, the determining unit, specifically used for determining the image to be identified, includes:

[0038] The multiple road surface images are stitched together to obtain the first image;

[0039] The image to be identified is obtained by stitching the second image and the first image together. The second image is an image acquired from the forward-looking direction among the multiple road surface images.

[0040] In an alternative embodiment of the second aspect, the processing unit, specifically used for stitching the second image and the first image to obtain the image to be identified, includes:

[0041] The target area is determined from the second image based on the steering wheel angle of the vehicle;

[0042] The target region and the first image are stitched together to obtain the image to be identified.

[0043] In an alternative embodiment of the second aspect, the processing unit, specifically used for stitching the target region and the first image to obtain the image to be identified, includes:

[0044] The target region and the first image are superimposed in the color channel direction of the target region and the first image to obtain the image to be identified.

[0045] In an alternative embodiment of the second aspect, the determining unit, specifically used for determining the road surface type of the road to be driven based on the identification result, includes:

[0046] The vehicle's mileage is determined based on the vehicle's speed and steering wheel angle, wherein the vehicle's mileage includes the distance the vehicle travels from the first moment to the second moment.

[0047] The road surface type of the road to be traveled is determined based on the vehicle's mileage, the vehicle's historical road surface type, and the identification result. The historical road surface type includes the road surface type corresponding to the road from the first time point to the second time point.

[0048] In an alternative embodiment of the second aspect, the determining unit is specifically used to determine the road surface type of the road to be traveled based on the vehicle's mileage, the vehicle's historical road surface types, and the identification result, including:

[0049] If the vehicle's mileage is less than a threshold, and the identification result is the same as the historical road surface type, the identification result is taken as the road surface type of the road to be driven.

[0050] In an alternative embodiment of the second aspect, the processing unit, specifically used for inputting the image to be recognized into the classification model and outputting the recognition result, includes:

[0051] The image to be identified is processed by multiple backbone networks in the trained classification model to obtain the processing result;

[0052] Based on the processing results, the probabilities corresponding to multiple types of results are obtained through the branch structure in the trained classification model, wherein the multiple types of results are used to represent the road surface type of the road to be driven.

[0053] The recognition result is determined based on the probabilities corresponding to the various types of results.

[0054] Thirdly, embodiments of this application provide a vehicle including a processor and a memory, the processor being coupled to the memory for storing a computer program, and the processor for calling and running the computer program, causing the vehicle to perform the method described in any of the preceding first aspects.

[0055] Fourthly, embodiments of this application provide a computing device including a processor and a memory; the processor is coupled to the memory, the memory is used to store a computer program, and the processor is used to invoke and run the computer program so that the computing device performs the method as described in any of the preceding first aspects.

[0056] Optionally, the computing device further includes a communication interface for receiving and / or sending data, and / or for providing input and / or output to the processor.

[0057] It should be noted that the above embodiments are illustrated using a processor (or general-purpose processor) that executes the method by invoking a computer-specified instruction. In practice, the processor can also be a dedicated processor, in which case the computer instructions have been pre-loaded into the processor. Optionally, the processor can include both dedicated and general-purpose processors.

[0058] Optionally, the processor and memory may be integrated into a single device, meaning they can be combined.

[0059] Fifthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when run on a computer or processor, implements the method described in any of the first aspects above.

[0060] The beneficial effects of the technical solutions provided in the second to fifth aspects of this application can be referred to the beneficial effects of the technical solutions in the first aspect, and will not be repeated here. Attached Figure Description

[0061] The accompanying drawings used in the description of the embodiments will be briefly introduced below.

[0062] Figure 1 This is a schematic diagram of the architecture of a vehicle provided in an embodiment of this application;

[0063] Figure 2 This is a schematic flowchart of an image recognition method provided in an embodiment of this application;

[0064] Figure 3 This is a schematic diagram illustrating the determination of a target region provided in an embodiment of this application;

[0065] Figure 4 This is a schematic diagram of a classification model provided in an embodiment of this application;

[0066] Figure 5 This is a schematic diagram of a training classification model provided in an embodiment of this application;

[0067] Figure 6 This is a schematic diagram of another image recognition method provided in the embodiments of this application;

[0068] Figure 7 This is a block diagram of the functional units of an image recognition device provided in an embodiment of this application;

[0069] Figure 8 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application. Detailed Implementation

[0070] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0071] The terms "first," "second," "third," and "fourth," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such processes, methods, products, or apparatus.

[0072] With the development of intelligent driving technology, vehicles are equipped with various sensors to assist drivers and provide a more comfortable and safer driving experience. These sensors include, but are not limited to, onboard cameras (panoramic cameras, forward-facing cameras, etc.), inertial measurement units (IMUs), and global positioning systems (GPS). Simultaneously, with the increasing prevalence of these sensors in vehicles, the methods for identifying road surface types are constantly improving. In addition to identifying the road surface type based on suspension vibration signals, vehicles can also use cameras to capture images of the road ahead, and then perform image processing and model inference classification to determine the road surface type. However, when the vehicle's camera is exposed to direct sunlight or affected by light intensity, the road surface information in the captured images can be affected, leading to recognition errors or instability. Furthermore, in situations with complex road terrain ahead, the vehicle may also make misjudgments. Therefore, existing solutions have relatively low robustness and accuracy in identifying road surface types, which is detrimental to assisting drivers.

[0073] In summary, this application provides an image recognition method in which images acquired by a vehicle from N directions during its journey are stitched together to form an image to be recognized. This image is then input into a classification model, which outputs the recognition result. The vehicle then determines the road surface type of the road it is traveling on based on the recognition result. Because road surface images are acquired from N directions, even if the camera in any direction is affected by lighting conditions, the vehicle can still determine the road surface type based on images acquired from other directions. This improves the accuracy of road surface type recognition and enhances the user's driving experience.

[0074] The system architecture used in the embodiments of this application is described below. It should be noted that the system architecture and business scenarios described in this application are for the purpose of more clearly illustrating the technical solutions of this application, and do not constitute a limitation on the technical solutions provided in this application. As those skilled in the art will know, with the evolution of system architecture and the emergence of new business scenarios, the technical solutions provided in this application are also applicable to similar technical problems.

[0075] Please see Figure 1 , Figure 1 This is a schematic diagram of the architecture of a vehicle provided in an embodiment of this application. Figure 1 As shown, vehicle 10 includes multiple camera devices for acquiring road surface images from N directions. The multiple camera devices include... Figure 1The front-view camera 101, front surround-view camera 102, left surround-view camera 103, right surround-view camera 104, and rear surround-view camera 105 are shown. It should be noted that the types, number, and locations of the multiple camera devices include, but are not limited to, those shown below. Figure 1 The situation shown is not limited here.

[0076] Vehicle 10 can be a vehicle powered by electricity, a vehicle powered by gasoline, or a vehicle powered by a new energy hybrid powertrain. For example, when vehicle 10 is powered by electricity, it can be a new energy vehicle, such as a pure electric vehicle, a range-extended electric vehicle, a hybrid electric vehicle, or a fuel cell electric vehicle. When vehicle 10 is powered by gasoline, it can be a car, an agricultural transport vehicle, a tractor, or a trailer. When vehicle 10 is a car, it can be a sedan, an SUV, a truck, a bus, or a van.

[0077] The forward-view camera 101 is generally installed on the windshield, rearview mirror, or roof of the vehicle 10 to capture images of the area in front of the vehicle 10. The forward-view camera 101 includes, but is not limited to, monocular and binocular cameras, etc., which are not limited here.

[0078] For example, since the forward-view camera 101 is used to capture images at a distance, the images captured by the forward-view camera 101 by the vehicle 10 contain road surface information of the road to be driven.

[0079] The front surround view camera 102 is generally installed on or near the front emblem of the vehicle 10 to capture images of the front of the vehicle 10. The front surround view camera 102 includes, but is not limited to, fisheye cameras or wide-angle cameras, which are not limited here.

[0080] For example, since the front surround view camera 102 is used to capture close-up images, the images captured by the vehicle 10 through the front surround view camera 102 contain road surface information of the road to be driven.

[0081] The left-side surround view camera 103 is generally installed at the left-side rearview mirror of the vehicle 10, or at the lower part of the vehicle body below the left-side rearview mirror, to capture images of the left side of the vehicle 10. The left-side surround view camera 103 includes, but is not limited to, a fisheye camera or a wide-angle camera, which is not limited here.

[0082] For example, since the left surround view camera 103 is used to capture close-up images, the images captured by the vehicle 10 through the left surround view camera 103 contain road surface information of the road to the left of the vehicle 10.

[0083] The right-side surround view camera 104 is generally installed at the right-side rearview mirror of the vehicle 10, or at the lower part of the vehicle body below the right-side rearview mirror, to capture images of the right side of the vehicle 10. The right-side surround view camera 104 includes, but is not limited to, fisheye cameras or wide-angle cameras, which are not limited here.

[0084] For example, since the right surround view camera 104 is used to capture close-up images, the images captured by the vehicle 10 through the right surround view camera 104 contain road surface information of the road to the right of the vehicle 10.

[0085] The rear surround view camera 105 is generally installed on or near the rear emblem of the vehicle 10 to capture images behind the vehicle 10. The rear surround view camera 105 includes, but is not limited to, fisheye cameras or wide-angle cameras, which are not limited here.

[0086] For example, since the rear surround view camera 105 is used to capture close-up images, the images captured by the rear surround view camera 105 of the vehicle 10 contain road surface information of the road behind the vehicle 10.

[0087] Since the images captured by the front surround-view camera 102 contain road surface information of the road to be traveled, the images captured by the left surround-view camera 103 contain road surface information of the road to the left of the vehicle 10, the images captured by the right surround-view camera 104 contain road surface information of the road to the right of the vehicle 10, and the images captured by the rear surround-view camera 105 contain road surface information of the road behind the vehicle 10, the vehicle 10 can determine the road surface type of the road it is traveling on based on the images captured by the front surround-view camera 102, the left surround-view camera 103, the right surround-view camera 104, and the rear surround-view camera 105.

[0088] Since the image captured by the forward-view camera 101 contains road surface information of the road to be traveled, the vehicle 10 can determine the road surface type of the road to be traveled based on the images captured by the forward-view camera 101 and / or the front surround-view camera 102. Even if the forward-view camera 101 or the front surround-view camera 102 is affected by lighting, due to the continuity of the road surface, the vehicle 10 can regard the road surface type determined by the images captured by the front surround-view camera 102, the left surround-view camera 103, the right surround-view camera 104, and the rear surround-view camera 105—that is, the road surface type of the road that the vehicle 10 is currently traveling on—as the road surface type of the road to be traveled. This improves the feasibility of road surface type identification.

[0089] Furthermore, vehicle 10 stitches together road surface images acquired by the front-view camera 101, front surround-view camera 102, left surround-view camera 103, right surround-view camera 104, and rear surround-view camera 105 into a single image to be identified. This image is then input into a trained classification model, which outputs the recognition result. Finally, the road surface type of the road to be traveled is determined based on the recognition result.

[0090] Please see Figure 2 , Figure 2 This is a schematic flowchart of an image recognition method provided in an embodiment of this application. The method is applied to, for example... Figure 1 The vehicle shown. (As shown) Figure 2 As shown, the method includes, but is not limited to, the following steps:

[0091] Step S201: Determine the image to be recognized.

[0092] The image to be identified is a stitched image composed of multiple road surface images. These multiple road surface images include images captured by the vehicle from N directions during its movement, where N is a positive integer greater than 1. For example, such as... Figure 1 In this system, the vehicle uses a forward-facing camera to capture images of the road ahead. Since the forward-facing camera captures images of a long distance, the images it captures include road surface information of the road being traveled. The vehicle then uses a front surround-view camera to capture images of the road ahead, a left surround-view camera to capture images of the road to the left, a right surround-view camera to capture images of the road to the right, and a rear surround-view camera to capture images of the road behind. Because the front, left, right, and rear surround-view cameras are used to capture images of close distances, the images they capture include road surface information of the road the vehicle is currently traveling on.

[0093] In one possible implementation, the vehicle stitches together multiple road surface images to obtain a first image, and then stitches together a second image and the first image to obtain an image to be identified.

[0094] The second image is an image captured from the forward-looking direction among multiple road surface images, such as an image of a vehicle passing by. Figure 1 Images captured by the forward-facing camera.

[0095] Specifically, the vehicle stitches together multiple road surface images to obtain the first image. These multiple road surface images can be obtained through... Figure 1The vehicle captures images from the front, left, right, and rear surround-view cameras. For example, the vehicle stitches together four road surface images captured by these cameras into a first image. This first image contains overall road surface information around the vehicle, facilitating subsequent road type identification. Furthermore, the second image is captured from the forward-looking direction and contains road surface information of the road to be traveled. Therefore, the vehicle stitches the second and first images together to create a new image that includes road surface information of the road to be traveled and the road surface information around the vehicle, i.e., the road surface information of the road the vehicle is currently traveling on. This provides rich road surface information for subsequent road type identification.

[0096] In one possible implementation, the vehicle determines the target region from the second image based on the steering wheel angle, and stitches the target region and the first image together to obtain the image to be identified.

[0097] Specifically, since a vehicle may continue traveling in its original direction or turn while in motion, the vehicle can determine its intended direction based on the steering wheel angle. Then, a corresponding target region is extracted from the second image based on this intended direction. By treating the road surface information contained within the target region as the road surface information of the road to be traveled, the large amount of road surface imagery in the second image can be reduced to the road surface imagery contained within the target region. This reduces the computational load for vehicle road surface type identification and improves vehicle identification accuracy.

[0098] For example, please refer to Figure 3 , Figure 3 This is a schematic diagram illustrating the determination of a target region according to an embodiment of this application. For example... Figure 3 As shown, the second image is a view of the vehicle's front taken while it is in motion, so y1 represents the vehicle's current direction of travel, and x1 is perpendicular to y1. The angle between y2 and y1 is the steering wheel angle, as shown below. Figure 3 The angle between y2 and y1 indicates that the vehicle will turn left. x2 is perpendicular to y2, so the vehicle can determine the direction of x2 based on the direction of y2, and thus determine the target area based on x2 and y2. For example, the target area is 672 × 672 pixels. Since the threshold angle for a vehicle turning left or right is 3 to 30 degrees, the angle between y2 and y1 is also 3 to 30 degrees. If the steering wheel angle is less than 3 degrees, it is calculated as 0 degrees. If the steering wheel angle is greater than 30 degrees, it is calculated as 30 degrees.

[0099] In one possible implementation, the vehicle uses an Around View Monitor (AVM) system to stitch together multiple road surface images to obtain the first image.

[0100] Specifically, the vehicle will stitch together the road images captured by the front surround view camera, the left surround view camera, the right surround view camera, and the rear surround view camera using AVM, so that the resulting first image contains information about the surrounding road surface near the vehicle.

[0101] For example, the vehicle captures four raw images with a resolution of 1920×1300 from the front, left, right, and rear surround-view cameras, in YUV format. The vehicle first performs color gamut conversion on the YUV road surface images, specifically converting them to RGB color mode. Then, it crops the Region of Interest (ROI) from the road surface images. Specifically, it preserves the area within two meters of the vehicle in the road surface images as the ROI. Finally, the vehicle stitches the ROIs corresponding to the four road surface images using AVM (Aspect-Visible Vector Machine) to obtain the first image.

[0102] For example, the second image captured by the vehicle's forward-facing camera is raw data with a resolution of 3860×2160, and the format of the second image is YUV. The vehicle can first perform color gamut conversion on the YUV format second image, specifically converting it to RGB format.

[0103] In one possible implementation, the vehicle superimposes the target area and the first image in the color channel direction of the target area and the first image to obtain the image to be identified.

[0104] Specifically, first, the vehicle converts the target area and the first image to the same size. For example, the specific size could be 672×672×3. Here, 672×672 represents the spatial dimensions of the image, indicating that the image has 672 pixels horizontally and 672 pixels vertically. 3 represents the number of image channels, typically RGB color channels (red, green, and blue). Then, the vehicle directly superimposes the target area and the first image along the color channel direction. The resulting image to be recognized is 672×672×6. Since the target area and the first image include length, width, and height, the color channel direction is the height direction, and the height of the target area and the first image is 3. Furthermore, to ensure that the image information contained in the stitched image to be recognized was acquired at the same time, the target area and the first image at the same timestamp can be stitched together.

[0105] In one possible implementation, after the vehicle stitches the target region and the first image together, it needs to perform an image resizing operation before inputting it into the classification model. Since the target region and the first image are stitched directly along the color channel direction, both the target region and the first image before stitching are three-dimensional images, while the resulting image is a six-dimensional image. To allow the classification model to better predict the image to be recognized, a resizing operation is needed to reduce the dimensionality, transforming the stitched six-dimensional image into a three-dimensional image.

[0106] Specifically, this embodiment employs an autoencoder to reduce the dimensionality of the stitched image. The original 3D image is self-supervised and trained using a Variational Autoencoder (VAE) algorithm to obtain a trained VAE model. The vehicle can input the stitched image into the VAE model, which outputs a dimensionality-reduced image to be recognized. The VAE model outputs a feature map of size 448×448×3, which can be used as input to a classification model.

[0107] Step S202: Input the image to be recognized into the classification model and output the recognition result.

[0108] Specifically, the vehicle inputs the image to be recognized into the classification model, and obtains the recognition result output by the classification model. The recognition result includes the road surface type of the road to be driven, where the type of road surface can be determined according to the actual road conditions, such as including but not limited to: snow, mud, sand, grass, and ordinary roads, etc., which are not limited here.

[0109] In one possible implementation, the vehicle processes the image to be recognized through multiple backbone networks in a pre-trained classification model, obtaining processing results. Then, based on the processing results, the probabilities corresponding to multiple types of results are obtained through the branching structure in the pre-trained classification model. The recognition result is determined based on the probabilities corresponding to the multiple types of results.

[0110] Specifically, the classification model includes a convolutional head, an attention mechanism, multiple backbone networks, a fully connected structure, and a branched structure. The vehicle initially extracts features from the image to be recognized using the convolutional head. Then, the vehicle inputs the feature map information extracted by the convolutional head into the attention mechanism module. This module automatically weights the feature map information output by the convolutional head, allowing the model to focus its attention on more valuable regions in subsequent processing. These more valuable regions are those that can represent road surface types.

[0111] The vehicle inputs the results processed by the attention mechanism model into multiple backbone networks, obtaining multiple results from each. Then, the vehicle inputs these multiple results into a fully connected structure for concatenation. Since different backbone networks can be used to extract features at different scales, and their computational costs vary, integrating multiple backbone networks allows for simultaneous processing of the image to be recognized. This combines the advantages of multiple backbone networks, leveraging their strengths and improving the feature extraction capability and accuracy of the classification model's output.

[0112] Then, the vehicle inputs the vector concatenated through the fully connected structure into the branching structure, obtaining probabilities corresponding to multiple types of results. These multiple types represent the road surface type to be driven. For example, if the road surface type includes snow, mud, sand, grass, and ordinary roads, then the type results are also snow, mud, sand, grass, and ordinary roads. The probabilities corresponding to each of these five types can be obtained through the branching structure. The vehicle can use a normalized exponential function (Softmax) to balance the probabilities of these multiple types, obtaining the final recognition result, which is one of the aforementioned types. Therefore, the branching structure in the classification model can, to some extent, mitigate the impact of imbalance among categories (road surface types) on the recognition results output by the classification model during training.

[0113] For example, please refer to Figure 4 , Figure 4 This is a schematic diagram of a classification model provided in an embodiment of this application. For example... Figure 4 As shown, the classification model includes a convolutional head, an attention mechanism module, backbone network 1, backbone network 2, backbone network 3, a fully connected structure, and a five-branch structure.

[0114] like Figure 4As shown, the vehicle input feature map is fed into the convolution head, where the feature map is the image to be recognized, with a size of 448×448×3. Since the image to be recognized contains a lot of abstract information, to more effectively utilize this abstract information, the convolution head can first perform preliminary feature extraction from the image to be recognized. For example, the convolution head includes a depthwise separable convolution with a 5×5 kernel, a dilated convolution with a 3×3 kernel and a dilation rate of 2, a batch normalization (BN) layer, and a global average pooling layer. The BN layer is applied before the activation function to ensure that the output of the previous layer is distributed with a mean of 0 and a variance of 1, effectively normalizing the input of the next layer. The feature map output by the convolution head module has a size of 224×224×3.

[0115] For example, the attention mechanism module includes a GAM attention mechanism, an FcaNet attention mechanism, and a Coordinate attention mechanism. By combining multiple attention mechanisms, the strengths of each can be leveraged to achieve more effective weighting of the feature maps.

[0116] For example, three well-known backbone networks are used: ResNet50, EfficientNet_b0, and Inception V4. These three backbone networks have good classification performance and each has different characteristics. For example, their computational costs are medium, low, and high, respectively. Integrating these three backbone networks helps to combine their advantages and leverage the strengths of each network, improving the feature extraction ability of the classification model. Since the feature map output from the attention mechanism module is 224×224×3 in size, it can be directly input into backbone network 1, backbone network 2, and backbone network 3. The output layer size of backbone network 1, backbone network 2, and backbone network 3 is set to 100, and then they are directly concatenated in a fully connected structure.

[0117] For example, the fully connected structure concatenates the outputs of backbone network 1, backbone network 2, and backbone network 3 into a 300-dimensional vector, followed by a 100-dimensional vector in the next layer. This is then connected to a 20-dimensional fully connected layer, and subsequently to a five-branch structure. The fully connected structure allows for a more abstract representation of features.

[0118] For example, if there are five road surface types: snow, mud, sand, grass, and ordinary road, then the classification model's branching structure would be a five-branch structure. Vehicles use this five-branch structure to determine the recognition result corresponding to the image to be recognized. The five-branch structure can, to some extent, mitigate the impact of imbalance among categories (road surface types) on the recognition results output by the classification model during training.

[0119] Step S203: Determine the road surface type of the road to be traveled by the vehicle during the driving process based on the recognition results.

[0120] Specifically, during vehicle operation, the system can determine the road surface type of the road ahead based on the recognition results. If the recognition result is correct, then the road surface type is the recognized result. If the recognition result is incorrect, then the road surface type is not the recognized result. Therefore, after obtaining the recognition result through the classification model, the vehicle can further verify the accuracy of the recognition result to ultimately obtain the road surface type of the road to be identified.

[0121] In one possible implementation, the vehicle determines its mileage based on its speed and steering wheel angle, for example, using its odometer. The vehicle then determines the road surface type of the road to be traveled based on the mileage, the vehicle's historical road surface types, and the identification results.

[0122] The vehicle's mileage includes the distance traveled from the first moment to the second moment. The vehicle's historical road surface type includes the road surface type corresponding to the roads traveled from the first moment to the second moment.

[0123] Specifically, because road surface types are continuous, the road surface type traveled by a vehicle over a short distance may be the same. For example, a vehicle might travel on grass for a certain distance without immediately traveling to snow. Furthermore, the road surface type corresponding to the vehicle's journey from one moment to the next is the same. The historical road surface type determined by the classification model from the first moment to the second moment is also the same. Therefore, the vehicle can first calculate the specific mileage. Then, based on the mileage, the accuracy of the identification result is judged using historical road surface types. If the identification result is accurate, it is used as the road surface type for the road to be traveled. If the identification result is inaccurate, due to the continuity of road surface types, the historical road surface type can be used as the road surface type for the road to be traveled.

[0124] In one possible implementation, if the vehicle's mileage is less than a threshold, and the identification result is the same as the historical road surface type, the identification result is taken as the road surface type of the road to be driven.

[0125] Specifically, to prevent abrupt changes in the classification model's identification results, the accuracy of the identification results is judged by the vehicle's mileage and historical road surface types, which can improve the reliability of vehicle road type identification. An abrupt change in the classification model's identification result refers to a significant difference between the identified result and historical road surface types. For example, multiple historical road surface types are sand, but the current output identification result is snow. Due to the significant differences between sand and snow, a vehicle is unlikely to travel from sand to snow in a short period. Therefore, the snow identification result is an abrupt change. The vehicle needs to further determine the accuracy of the snow identification result based on its odometer reading. If the vehicle's mileage is less than a threshold, the vehicle is unlikely to travel from sand to snow. In this case, the vehicle's identification result output by the classification model is incorrect, and the vehicle uses the historical road surface type of sand as the road surface type to be traveled. If the vehicle's mileage is greater than or equal to the threshold, the vehicle may travel from sand to snow. In this case, the vehicle's identification result output by the classification model is correct, and the vehicle uses the snow identification result as the road surface type to be traveled.

[0126] In one possible implementation, after determining the road surface type of the road to be traveled, the vehicle can be adjusted accordingly. For example, the vehicle's suspension, traction, and wheel-to-ground friction can be adjusted based on the road surface type. When the vehicle is traveling on the designated road, this can provide a more comfortable driving experience and enhance the overall driving experience.

[0127] The training methods for classification models will be introduced in detail below.

[0128] In one possible implementation, multiple road surface images corresponding to different road surface types are first acquired. Then, these multiple road surface images, acquired simultaneously, are stitched together to obtain training images. The training images are then labeled based on the terrain information displayed within them, yielding corresponding labels. These multiple training images and their corresponding labels are used as a training set to train a classification model, resulting in a trained classification model.

[0129] For example, please refer to Figure 5 , Figure 5 This is a schematic diagram of a training classification model provided in an embodiment of this application.

[0130] like Figure 5 As shown, the specific process includes one or more steps in S501-S506, as detailed below:

[0131] S501, Stitching Training Images. The acquired road surface images are stitched together into a training image according to the acquisition time. Specifically, firstly, four road surface images acquired by the front, left, right, and rear surround view cameras are stitched together to form the first image. Then, the second image acquired by the front-view camera is stitched together with the first image to form the training image. The acquisition time of the second image is the same as the acquisition time of the first image.

[0132] S502, perform a resize operation on the stitched training image. Since the second and first images are stitched together along the color channel direction, both the first and second images before stitching are three-dimensional images, while the resulting training image is a six-dimensional image. To better train the classification model, a resize operation is needed to reduce the dimensionality of the training image. The stitched six-dimensional image is transformed into a three-dimensional image, resulting in the dimensionality-reduced training image.

[0133] S503, Expanding the Training Set. This involves expanding the training set through dataset augmentation to improve the model's generalization ability and performance. Regardless of whether the road images are captured by the front-view, front-view, left-view, right-view, or rear-view cameras, dataset augmentation is performed. For example, color transformation and random erasure can be used to expand the training set. These augmentation operations effectively improve the model's robustness and generalization ability, enabling it to better adapt to different scenarios and changes.

[0134] S504, Data Labeling. The training images are labeled based on the terrain information displayed in them to obtain corresponding labels. For example, the data labeling uses JSON format, which includes the road surface type (e.g., grassland, mud, snow, sand, ordinary road, etc.) from the training data.

[0135] S505, Training the classification model. Multiple training images and their corresponding labels are used as the training set to train the classification model, resulting in a trained classification model.

[0136] For example, the structure of a classification model is as follows: Figure 4 As shown, the backbone networks 1, 2, and 3 in the classification model are pre-trained first. Specifically, based on the ImageNet public dataset, backbone networks 1, 2, and 3 are trained unsupervised using the contrastive learning SimCLR v2 method. After ensuring that each backbone network has strong feature extraction capabilities, the entire classification model is then trained on the training set.

[0137] S506, Deploy the classification model. The trained classification model is first converted to TensorRT format, then to Engine format, and finally deployed to the vehicle's onboard domain controller for vehicle road type identification.

[0138] Please see Figure 6 , Figure 6 This is a schematic flowchart of another image recognition method provided in an embodiment of this application. Figure 6 As shown, the specific process includes one or more steps in S601-S610, as detailed below:

[0139] S601, Acquire road surface images. The vehicle acquires road surface images in front of it using a forward-facing camera. Since the forward-facing camera is used to acquire images at a distance, the images acquired by the forward-facing camera contain road surface information of the road to be traveled. The vehicle acquires road surface images in front of it using a front surround-view camera, road surface images on the left side of the vehicle using a left surround-view camera, road surface images on the right side of the vehicle using a right surround-view camera, and road surface images behind it using a rear surround-view camera. Since the front, left, right, and rear surround-view cameras are used to acquire images at close range, the images acquired by the front, left, right, and rear surround-view cameras contain road surface information of the road on which the vehicle is currently traveling.

[0140] In addition, the vehicle also needs to obtain the steering wheel angle and vehicle speed.

[0141] S602, determine the ROI region of the road surface image.

[0142] For example, for images captured by the front surround view camera, left surround view camera, right surround view camera and rear surround view camera, the vehicle can retain the area within two meters of the vehicle in the road image as the ROI area.

[0143] S603 determines the vehicle's mileage. The vehicle's mileage is determined based on its speed and steering wheel angle, for example, by the vehicle's odometer.

[0144] S604, multiple road surface images are stitched together to form a first image. The vehicle stitches together road surface images captured by the front surround-view camera, the left surround-view camera, the right surround-view camera, and the rear surround-view camera using AVM (Adaptive Dynamic Model), with a zero-pixel filled area in the middle. This ensures that the resulting first image contains road surface information around the vehicle.

[0145] S605, determine the target area of ​​the second image. For example, for the second image captured by the forward-facing camera, the vehicle can determine its intended direction of travel based on the steering wheel angle, and then extract the corresponding target area from the second image according to that direction. By treating the road surface information contained in the target area as the road surface information of the road to be traveled, the large amount of road surface imagery contained in the second image can be reduced to the road surface imagery contained in the target area.

[0146] S606, stitch the first image and the second image together. The vehicle overlays the target area of ​​the second image and the first image along the color channel direction.

[0147] Specifically, first, the vehicle converts the target region and the first image to the same size. Then, the vehicle directly overlays the first image of the target region along the color channel direction. Since the target region and the first image include three directions—length, width, and height—the color channel direction is the height direction, and the height of the target region and the first image is 3.

[0148] S607, Dimensionality reduction is performed on the stitched image. After stitching the target region and the first image, the vehicle needs to perform an image resizing operation. Specifically, this embodiment uses an autoencoder to reduce the dimensionality of the stitched image. The original 3D image is self-supervised trained using a Variational Autoencoder (VAE) algorithm to obtain a trained VAE model. The vehicle can input the stitched image into the VAE model and output the dimensionality-reduced image to be recognized.

[0149] S608, Input the image to be recognized into the classification model. The vehicle inputs the image to be recognized into the classification model and obtains the recognition result output by the classification model. The recognition result includes the road surface type of the road to be driven, where the type of road surface can be determined according to the actual road conditions, such as including but not limited to: snow, mud, sand, grass, and ordinary roads, etc., which are not limited here.

[0150] S609 determines the road surface type of the road to be driven based on mileage and historical road surface types. If the vehicle's mileage is less than a threshold, and the identification result is the same as the historical road surface type, the identification result is taken as the road surface type of the road to be driven.

[0151] Specifically, to prevent abrupt changes in the classification model's identification results, the accuracy of the identification results is judged by the vehicle's mileage and historical road surface types, which can improve the reliability of the vehicle's road type identification. If the vehicle's mileage is less than a threshold, then the identification result output by the classification model is incorrect, and the vehicle uses the historical road surface type as the road surface type for the road to be driven. If the vehicle's mileage is greater than or equal to the threshold, then the identification result output by the classification model is correct, and the vehicle uses the identified result as the road surface type for the road to be driven.

[0152] S610 outputs the road surface type of the road to be driven. After determining the road surface type, the vehicle can adjust itself accordingly. For example, the vehicle can adjust the suspension, traction, and wheel-to-ground friction based on the road surface type. When the vehicle is driving on the designated road, this provides a more comfortable driving experience and enhances the overall driving experience.

[0153] The methods of the embodiments of this application have been described in detail above. The apparatus of the embodiments of this application is provided below.

[0154] Please see Figure 7 , Figure 7 This is a block diagram of the functional units of an image recognition device 70 provided in an embodiment of this application. The image recognition device 70 may include a determining unit 701 and a processing unit 702. The image recognition device 70 is used to implement the aforementioned image recognition method, for example... Figure 2 The image recognition method shown.

[0155] It should be noted that the above division of multiple units is merely a logical division based on function and does not constitute a limitation on the specific structure of the image recognition device 70. In actual implementation, some functional modules may be further subdivided into more smaller functional modules, and some functional modules may be combined into a single functional module.

[0156] In one possible implementation, the determining unit 701 is used to determine the image to be identified, wherein the image to be identified is an image obtained by stitching together multiple road surface images, the multiple road surface images including images collected from N directions during the vehicle's travel, where N is a positive integer greater than 1;

[0157] Processing unit 702 is used to input the image to be recognized into the classification model and output the recognition result;

[0158] The determining unit 701 is also used to determine the road surface type of the road to be traveled on during the vehicle's driving process based on the recognition result.

[0159] In one possible implementation, the determining unit 701 is specifically used to determine the image to be identified, including:

[0160] The first image is obtained by stitching together multiple road surface images;

[0161] The second image and the first image are stitched together to obtain the image to be identified. The second image is an image collected from the forward-looking direction among multiple road surface images.

[0162] In one possible implementation, the processing unit 702 is specifically configured to stitch the second image and the first image together to obtain the image to be recognized, including:

[0163] The target area is determined from the second image based on the vehicle's steering wheel angle;

[0164] The target region and the first image are stitched together to obtain the image to be identified.

[0165] In one possible implementation, the processing unit 702 is specifically configured to stitch the target region and the first image together to obtain the image to be identified, including:

[0166] The target region and the first image are superimposed in the color channel direction of the target region and the first image to obtain the image to be identified.

[0167] In one possible implementation, the determining unit 701 is specifically configured to determine the road surface type of the road to be traveled based on the identification result, including:

[0168] The vehicle's mileage is determined based on the vehicle's speed and steering wheel angle, including the distance the vehicle travels from the first moment to the second moment.

[0169] The road surface type of the road to be traveled is determined based on the vehicle's mileage, the vehicle's historical road surface type, and the identification results. The historical road surface type includes the road surface type corresponding to the road from the first moment to the second moment.

[0170] In one possible implementation, the determining unit 701 is specifically configured to determine the road surface type of the road to be traveled based on the vehicle's mileage, the vehicle's historical road surface types, and the identification results, including:

[0171] If the vehicle's mileage is less than the threshold, and the identification result is the same as the historical road surface type, the identification result will be used as the road surface type of the road to be driven.

[0172] In one possible implementation, the processing unit 702 is specifically configured to input the image to be recognized into a classification model and output a recognition result, including:

[0173] The image to be recognized is processed by multiple backbone networks in the trained classification model to obtain the processing result.

[0174] Based on the processing results, the probabilities corresponding to multiple types of results are obtained through the branch structure in the trained classification model. These multiple types of results are used to represent the road surface type of the road to be driven.

[0175] The identification result is determined based on the probability corresponding to each of the multiple types of results.

[0176] It should be noted that, in the embodiments of this application, the specific implementation and technical effects of each unit can also be referred to accordingly. Figure 2 The corresponding description of the method embodiments shown.

[0177] This application also provides a vehicle including a processor and a memory, the processor being coupled to the memory, the memory storing a computer program, and the processor calling and running the computer program to cause the vehicle to perform the aforementioned image recognition method, for example... Figure 2 The method.

[0178] Please see Figure 8 , Figure 8 This is a schematic diagram of the structure of a computing device provided in an embodiment of this application. Figure 8 As shown, the computing device 80 may include one or more processors 801, one or more memories 802, and one or more communication interfaces 803. These components may be connected via a bus 804 or other means. Figure 8 Taking a connection via bus 804 as an example. Where:

[0179] The communication interface 803 can be used by the computing device 80 to communicate with other communication devices, such as other computing devices. Specifically, the communication interface 803 can be a wired interface.

[0180] The memory 802 can be coupled to the processor 801 via a bus 804 or an input / output port, or the memory 802 can be integrated with the processor 801. The memory 802 is used to store various software programs and / or multiple sets of instructions or data. Specifically, the memory 802 can be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or it can be an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited thereto. Memory 802 may include high-speed random access memory and may also include non-volatile memory, such as one or more disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. Memory 802 may store an operating system (hereinafter referred to as the system), such as uCOS, VxWorks, RTLinux, or other embedded operating systems. Memory 802 may also store network communication programs that can be used to communicate with one or more additional devices, one or more user devices, or one or more terminals. Memory 802 may exist independently and be connected to processor 801 via bus 804. Memory 802 may also be integrated with processor 801.

[0181] The memory 802 stores the application code for executing the above scheme, and its execution is controlled by the processor 801. The processor 801 executes the application code stored in the memory 802.

[0182] Processor 801 may be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field-programmable gate array, or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in connection with this disclosure. Processor 801 may also be a combination that implements a specific function, such as a combination of one or more microprocessors, a combination of a digital signal processor and a microprocessor, etc.

[0183] This application also provides a computer-readable storage medium storing instructions that, when executed on at least one processor, implement the aforementioned image recognition method, for example... Figure 2 The method.

[0184] This application also provides a computer program product including computer instructions that, when executed by a computing device, implement the aforementioned image recognition method, for example... Figure 2 The method.

[0185] In this application, the terms "for example" or "for instance" are used to indicate that something is being described as an example, illustration, or illustration. Any embodiment or design described as "for example" or "for instance" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Rather, the use of terms such as "for example" or "for instance" is intended to present the relevant concepts in a specific manner.

[0186] In this application, "at least one" in the embodiments refers to one or more items, and "more than one" refers to two or more items. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c can represent: a, b, c, (a and b), (a and c), (b and c), or (a and b and c), where a, b, and c can be single or multiple. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship.

[0187] Furthermore, unless otherwise stated, the use of ordinal numbers such as "first" and "second" in the embodiments of this application is for distinguishing multiple objects and is not for limiting the order, sequence, priority, or importance of multiple objects. For example, "first device" and "second device" are only for ease of description and do not indicate that the first device and the second device are different in structure, importance, etc. In some embodiments, the first device and the second device may also be the same device.

[0188] In the above embodiments, the term "when..." can be interpreted, depending on the context, as meaning "if...", "after...", "in response to determining...", or "in response to detecting...". The above are merely optional embodiments of this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the concept and principles of this application should be included within the protection scope of this application.

[0189] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0190] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An image recognition method characterized by, The method comprises: determining a to-be-recognized image, wherein the to-be-recognized image is an image obtained by splicing a plurality of road surface images, the plurality of road surface images comprising images collected from N directions during driving of a vehicle, N being a positive integer greater than 1; inputting the to-be-recognized image into a classification model to output a recognition result; determining a road surface type of a to-be-traveled road of the vehicle during driving according to the recognition result.

2. The method of claim 1, wherein, The determination of the to-be-recognized image comprises: splicing the plurality of road surface images to obtain a first image; splicing a second image and the first image to obtain the to-be-recognized image, the second image being an image collected from a front direction among the plurality of road surface images.

3. The method of claim 2, wherein, The splicing of the second image and the first image to obtain the to-be-recognized image comprises: determining a target region from the second image based on a steering wheel angle of the vehicle; splicing the target region and the first image to obtain the to-be-recognized image.

4. The method of claim 3, wherein, The splicing of the target region and the first image to obtain the to-be-recognized image comprises: superimposing the target region and the first image in a color channel direction of the target region and the first image to obtain the to-be-recognized image.

5. The method according to any one of claims 1 to 4, characterized in that, The determination of the road surface type of the to-be-traveled road according to the recognition result comprises: determining a mileage of the vehicle according to a vehicle speed and a steering wheel angle of the vehicle, wherein the mileage of the vehicle comprises a mileage of the vehicle from a first time to a second time; determining the road surface type of the to-be-traveled road according to the mileage of the vehicle, a historical road surface type of the vehicle, and the recognition result, the historical road surface type comprising a road surface type corresponding to a road on which the vehicle travels from the first time to the second time.

6. The method of claim 5, wherein, The determination of the road surface type of the to-be-traveled road according to the mileage of the vehicle, the historical road surface type of the vehicle, and the recognition result comprises: in a case where the mileage of the vehicle is less than a threshold value, when the recognition result is the same as the historical road surface type, regarding the recognition result as the road surface type of the to-be-traveled road.

7. The method according to any one of claims 1 to 6, characterized in that, The inputting of the to-be-recognized image into the classification model to output the recognition result comprises: processing the to-be-recognized image through a plurality of backbone networks in the trained classification model to obtain processing results; obtaining probabilities respectively corresponding to a plurality of type results through a branch structure in the trained classification model based on the processing results, wherein the plurality of type results are used to represent the road surface type of the to-be-traveled road; determining the recognition result based on the probabilities respectively corresponding to the plurality of type results.

8. A vehicle characterized by comprising: A computing device comprising a processor and a memory, the processor being coupled to the memory, the memory being configured to store a computer program, and the processor being configured to invoke and run the computer program, so that the computing device performs the method according to any one of claims 1-7.

9. A computing device, comprising: A computing device comprising a processor, the processor being coupled to a memory, the memory being configured to store a computer program, and the processor being configured to invoke and run the computer program, so that the computing device performs the method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium has stored therein a computer program comprising instructions for performing the method of any of claims 1-7.

Images