Vehicle control method and related apparatus

Abstract

A vehicle control method and a related apparatus. The method comprises: obtaining a pavement image captured when a vehicle travels on a pavement; determining the distance from the vehicle to a preset area on the basis of the pavement image, wherein the preset area includes an area comprising a preset feature on the pavement; and controlling a preset function of the vehicle on the basis of the distance from the vehicle to the preset area to implement safe traveling of the vehicle, wherein the preset function includes a function used when the vehicle travels in the preset area.

Description

Vehicle control method and related device

[0001] This application claims priority to Chinese Patent Application No. 202411048581.6, filed on July 31, 2024, the entire contents of which are incorporated herein by reference. TECHNICAL FIELD

[0002] The present disclosure relates to the technical field of vehicle control, and in particular, to a vehicle control method and related device. BACKGROUND

[0003] When a vehicle travels on a road surface with a preset feature, a function corresponding to the preset feature can be turned on. The preset feature includes but is not limited to water, snow, sand, etc., and the function corresponding to the preset feature is mainly used to assist the vehicle to travel on the road surface containing the preset feature and avoid damage to the vehicle. For example, the function corresponding to the preset feature includes but is not limited to turning off at least one of the engine or the electric motor, locking the vehicle door, closing the vehicle window, opening the sunroof, and closing part of the auxiliary driving function. For example, taking the preset feature as water as an example, through the function corresponding to the preset feature, the vehicle can help deal with a large amount of accumulated water during a typhoon or heavy rain, and realize the forward driving, turning, and other operations of the vehicle on the accumulated water road surface. SUMMARY

[0004] The present disclosure provides a vehicle control method and related device. According to the distance between the vehicle and the preset area, the preset function is controlled, and the safety of the vehicle driving through the preset area is improved.

[0005] In a first aspect, a vehicle control method is provided, the method comprising:

[0006] obtaining a road surface image, the road surface image comprising an image collected by a vehicle traveling on a road surface;

[0007] determining a distance from the vehicle to a preset area according to the road surface image, the preset area comprising an area containing a preset feature on the road surface;

[0008] controlling a preset function of the vehicle according to the distance from the vehicle to the preset area to achieve safe driving of the vehicle, the preset function comprising a function used by the vehicle when traveling on the preset area.

[0009] In the method, the distance between the vehicle and the preset area is determined according to the image collected when the vehicle travels on the road surface. Then, the preset function is controlled based on the distance. Since the preset function is a function used by the vehicle when traveling on the preset area, and the preset area includes a region on the road surface containing a preset feature, the preset function is a function related to the vehicle involving the preset feature, which is different from the normal driving of the vehicle. If the vehicle starts the preset function on the road surface without the preset area, the driving feeling or driving performance of the vehicle may be reduced. Therefore, according to the distance between the vehicle and the preset area, it is determined when to start the preset function, so that the preset function can be used in a more accurate position. Thus, the preset function is started or stopped in time to ensure the safety of the vehicle driving through the preset area on the basis of ensuring the performance of the vehicle in normal driving.

[0010] In some embodiments, the distance between the vehicle and the preset area is determined according to the road surface image, comprising:

[0011] The presence of the preset area on the road surface is determined by an identification model based on the road surface image.

[0012] In the case where the preset area exists on the road surface, the distance between the vehicle and the preset area is determined by at least one segmentation model.

[0013] In the method, first, the identification model is used to determine whether the preset area exists on the road surface. If the preset area exists, the segmentation model is further used to determine the distance between the vehicle and the preset area. This is because the identification model requires less computing power and has a faster calculation speed. First, it is determined whether the preset area exists on the road surface, which can save computing power and improve the time for the vehicle to calculate the distance between the preset area and the vehicle.

[0014] In some embodiments, the vehicle includes a first camera device and a second camera device, the segmentation model includes a first segmentation model and a second segmentation model, and the distance between the vehicle and the preset area is determined by the segmentation model, comprising:

[0015] The first road surface image is input into the trained first segmentation model to output a first segmentation result image, the first road surface image is collected by the first camera device, and the first camera device is configured to collect images of a first distance;

[0016] The second road surface image is input into the trained second segmentation model to output a second segmentation result image, the second road surface image is collected by the second camera device, and the second camera device is configured to collect images of a second distance;

[0017] The distance between the vehicle and the preset area is determined according to the first segmentation result image and the second segmentation result image.

[0018] In the above method, the first camera is used to collect images at a long distance, so the first road surface image includes road surface information at a long distance from the vehicle, enabling the vehicle to measure distances at a long distance. The second camera is used to collect images at a short distance. Therefore, the second road surface image includes road surface information at a short distance from the vehicle, which can solve the problem of the blind area of the first camera at a short distance, enabling the vehicle to measure distances at a short distance. The fusion of the first segmentation result image and the second segmentation result image can respectively take advantage of the fact that the first camera can measure the distance of a preset region at a long distance and the second camera can more easily measure the distance of a preset region at a short distance, thereby achieving an effective fusion ranging scheme with a wide and stable ranging range.

[0019] In some embodiments, the determining the distance from the vehicle to the preset region according to the first segmentation result image and the second segmentation result image comprises:

[0020] determining a grid image according to the first segmentation result image and the second segmentation result image, the grid image comprising a plurality of grids;

[0021] determining one or more preset grids from the plurality of grids, the one or more preset grids containing information of the preset region;

[0022] determining a target grid from the one or more preset grids, in the case where the one or more preset grids comprise one preset grid, the one preset grid being the target grid; in the case where the one or more preset grids comprise a plurality of preset grids, the target grid being closer to the vehicle than the other preset grids in the plurality of preset grids;

[0023] determining the distance from the vehicle to the preset region according to the coordinates of the target grid in the grid image.

[0024] In the above method, the information in the road surface image is divided into a plurality of grids, and first, the preset grids in which the information of the preset region exists in the plurality of grids are determined. Then, the grid closest to the vehicle in the preset grids is determined as the target grid. The distance between the vehicle and the preset region is determined according to the coordinates of the target grid. Since the size of the grid can be adjusted, the distance between the vehicle and the preset region can be calculated more accurately, ensuring the accuracy of the result.

[0025] In some embodiments, the determining the distance from the vehicle to the preset region by the segmentation model comprises:

[0026] inputting the road surface image into a plurality of branches of the trained segmentation model respectively, and outputting a plurality of feature maps;

[0027] The distance of the vehicle to the preset area is determined according to the plurality of feature maps.

[0028] For example, the segmentation model used by some embodiments of the present disclosure is improved based on a deep learning semantic segmentation network (BiseNet). The plurality of branches of the segmentation model includes a deep learning classification network (Inception) branch, a spatial branch, and a context branch. The Inception branch simultaneously captures spatial information at different scales through different sizes of convolution kernels. The Inception branch is characterized by combining convolution kernels together to establish a multi-branch structure, enabling the network to calculate in parallel. By allocating different computing resources, the best performance can be obtained. The spatial branch is used to retain semantic information to generate higher resolution activation values (feature maps) and reduce the number of down-sampling. The context branch uses a fast down-sampling strategy to obtain sufficient receptive fields. The feature maps output by the Inception branch, the spatial branch, and the context branch are spliced together and output to a feature fusion module to obtain a segmentation result. Since the spliced feature maps have the same size as the feature maps of the original BiseNet input feature fusion module, the original feature fusion module does not need to be modified.

[0029] In the above method, the distance is measured by the plurality of branches in the segmentation model, which can improve the accuracy of segmenting the road surface image. Since different branches can play different dimensional roles, the feature maps obtained by integrating the plurality of branches can make the determined result have higher reliability. The accuracy of vehicle distance measurement is improved.

[0030] In some embodiments, one branch of the plurality of branches includes a plurality of convolution channels, and the plurality of convolution channels are respectively used to determine features of different scales of the road surface image.

[0031] One branch used by some embodiments of the present disclosure is an Inception branch. One branch includes four convolution channels with different morphologies. Features are input to the four convolution channels respectively. After global average pooling, the size of the output feature map is 64x64x32. After splicing the feature maps output by the four convolution channels, a feature map with a size of 64x64x128 is finally output. The first convolution channel is a 1x1 convolution. The second convolution channel is a 3x3 dilated convolution with an expansion rate of 2 and a step size of 1, and a 3x3 convolution with a step size of 2. The third convolution channel is a 5x5 convolution with a step size of 2, and a 5x5 convolution with a step size of 2. The fourth convolution channel is a 1x1 convolution, a 3x3 convolution with a step size of 2, and a 3x3 convolution with a step size of 2.

[0032] In the method, one branch includes multiple convolution channels, different scales of features of the road surface image can be extracted respectively, and the determined result can have higher reliability.

[0033] In some embodiments, the determining, by the at least one recognition model, that the road surface has the preset area based on the road surface image comprises:

[0034] inputting the multiple road surface images into the trained recognition model, and outputting multiple recognition results, the recognition results including a road surface type and a probability value corresponding to the road surface type;

[0035] determining, according to a collection time of the multiple road surface images, a weight of the probability value corresponding to each of the multiple road surface images;

[0036] determining, according to the weight of the probability value corresponding to each of the multiple road surface images, that the road surface has the preset area.

[0037] For example, the input feature map first passes through the convolution head of the recognition model, and then sequentially passes through four levels of processing stages (for example, stage1, stage2, stage3 and stage4), and finally enters the processing of the head (head). The input feature map is a region of interest (ROI) region of an image collected by a camera device, and the size can be 996x996. The stage1, stage2, stage3, stage4 and head are components of a deep learning classification network model (G-GhostNet). The embodiments of the present application add a convolution head and an attention mechanism 1 and an attention mechanism 2 on the basis of the G-GhostNet model. The convolution head includes one convolution with a 5x5 convolution kernel, one convolution with a 3x3 convolution kernel and a dilation rate of 2, one BN layer, and one global average pooling layer. The attention mechanism 1 is a spatial attention mechanism, and the attention mechanism 2 is a channel attention mechanism.

[0038] The knowledge distillation method is used to lighten the identification model after training the identification model. The reason for selecting the G-GhostNet model as the backbone network is that the G-GhostNet is a lightweight network, which has less calculation amount and faster calculation speed, and can improve the speed of judging whether the preset area exists on the road surface. For example. Existing lightweight inference networks (EfficientNet, MobileNet) are generally based on depth separable convolution. Compared with neural networks based on ordinary convolution (ResNet, InceptionNet), these networks have less calculation amount, but the actual measurement speed on electronic devices is slower than ordinary convolution. Therefore, some embodiments of the present disclosure select the G-GhostNet model as the backbone network to improve the calculation speed.

[0039] In the above method, the road surface type output by the identification model can be determined according to the actual road surface. Different road surface types require the vehicle to start different preset functions, so determining the road surface type by the identification model can provide a basis for subsequent vehicle starting of the preset function. At the same time, judging whether the preset area exists on the road surface according to the plurality of road surface images improves the accuracy of the identification result. According to the collection time of the road surface image, the probability value is given a weight, which ensures the coherence of the road surface image.

[0040] In some embodiments, the controlling the preset function of the vehicle according to the distance of the vehicle to the preset area to achieve safe driving of the vehicle comprises:

[0041] determining a distance threshold according to the vehicle speed of the vehicle;

[0042] in the case that the distance of the vehicle to the preset area is less than the distance threshold, starting the preset function of the vehicle.

[0043] In the above method, the vehicle can first determine the distance threshold according to the vehicle speed. For example, if the vehicle speed is fast, the vehicle reaches the preset area in a short time. The vehicle needs to turn on the preset function at a position far from the preset area, so the corresponding distance threshold is larger. If the vehicle speed is slow, the vehicle reaches the preset area in a long time. The vehicle can turn on the preset function at a position close to the preset area, so the corresponding distance threshold is smaller. So that the vehicle can get a more accurate position to start the preset function.

[0044] In a second aspect, a vehicle control device is provided, which includes a communication unit and a processing unit. The communication unit is configured to acquire a road surface image, the road surface image including an image captured by a vehicle driving on a road surface; and the processing unit is configured to determine a distance from the vehicle to a preset area based on the road surface image, the preset area including an area on the road surface containing a preset feature; and the processing unit is further configured to control a preset function of the vehicle to achieve safe driving of the vehicle based on the distance from the vehicle to the preset area, the preset function including a function used by the vehicle when driving on the preset area.

[0045] In some embodiments, the processing unit is configured to determine the distance from the vehicle to the preset area based on the road surface image by:

[0046] determining, based on the road surface image, that the road surface contains the preset area by using a recognition model;

[0047] determining the distance from the vehicle to the preset area by using a segmentation model in a case where the road surface contains the preset area.

[0048] In some embodiments, the vehicle includes a first camera device and a second camera device, the segmentation model includes a first segmentation model and a second segmentation model, and the processing unit is configured to determine the distance from the vehicle to the preset area by using the segmentation model by:

[0049] inputting a first road surface image into the trained first segmentation model to output a first segmentation result image, the first road surface image being captured by the first camera device, the first camera device being configured to capture a long-distance image;

[0050] inputting a second road surface image into the trained second segmentation model to output a second segmentation result image, the second road surface image being captured by the second camera device, the second camera device being configured to capture a short-distance image;

[0051] determining the distance from the vehicle to the preset area based on the first segmentation result image and the second segmentation result image.

[0052] In some embodiments, the processing unit is configured to determine the distance from the vehicle to the preset area based on the first segmentation result image and the second segmentation result image by:

[0053] determining a grid image based on the first segmentation result image and the second segmentation result image, the grid image including a plurality of grids;

[0054] determining a preset grid from the plurality of grids, the preset grid containing information of the preset area;

[0055] determine a target grid from the preset grids, the target grid being closer to the vehicle than other preset grids;

[0056] determine the distance from the vehicle to the preset area according to coordinates of the target grid in the grid map.

[0057] In some embodiments, the processing unit is configured to determine the distance from the vehicle to the preset area by the segmentation model, including:

[0058] input the road surface image into each branch of the trained segmentation model respectively, and output a plurality of feature maps;

[0059] determine the distance from the vehicle to the preset area according to the plurality of feature maps.

[0060] In some embodiments, one branch of the plurality of branches includes a plurality of convolution channels, each of which is configured to determine features of different scales of the road surface image.

[0061] In some embodiments, the processing unit is configured to determine that the road surface has the preset area based on the road surface image by the recognition model, including:

[0062] input the plurality of road surface images into the trained recognition model, and output a plurality of recognition results, the recognition results including a road surface type and a probability value corresponding to the road surface type;

[0063] determine weights of the probability values corresponding to the plurality of road surface images respectively according to collection times of the plurality of road surface images;

[0064] determine that the road surface has the preset area according to the weights of the probability values corresponding to the plurality of road surface images respectively.

[0065] In some embodiments, the processing unit is configured to control a preset function of the vehicle according to the distance from the vehicle to the preset area to achieve safe driving of the vehicle, including:

[0066] determine a distance threshold according to a vehicle speed of the vehicle;

[0067] turn on the preset function of the vehicle when the distance from the vehicle to the preset area is less than the distance threshold.

[0068] In a third aspect, a vehicle is provided, including a processor and a memory, the processor being coupled to the memory, the memory being configured to store a computer program, and the processor being configured to invoke and run the computer program, so that the vehicle executes the method described in the foregoing first aspect.

[0069] In a fourth aspect, a computing device is provided, comprising a processor and a memory; the processor is coupled with the memory, the memory is configured to store a computer program, and the processor is configured to invoke and run the computer program to cause the computing device to perform the method according to the first aspect described above.

[0070] In some embodiments, the computing device further comprises a communication interface configured to receive and / or send data, and / or the communication interface is configured to provide input and / or output for the processor.

[0071] It should be noted that the above embodiments are described by way of example with a processor (or general-purpose processor) that invokes a computer program to perform the method. In the implementation process, the processor can also be a special-purpose processor, and at this time the computer program has been preloaded in the processor. In some embodiments, the processor can also include both special-purpose processors and general-purpose processors.

[0072] In some embodiments, the processor and the memory can also be integrated into one device, that is, the processor and the memory can also be integrated together.

[0073] In a fifth aspect, a computer-readable storage medium is provided, and the computer-readable storage medium stores a computer program, when the computer program is run on a computer or a processor, the method according to the first aspect described above is implemented. BRIEF DESCRIPTION OF DRAWINGS

[0074] The drawings needed to be used in the following embodiment description will be briefly introduced.

[0075] FIG. 1 is a schematic diagram of an architecture of a vehicle according to some embodiments of the present disclosure;

[0076] FIG. 2 is a flowchart of a vehicle control method according to some embodiments of the present disclosure;

[0077] FIG. 3A is a schematic diagram of data labeling according to some embodiments of the present disclosure;

[0078] FIG. 3B is another schematic diagram of data labeling according to some embodiments of the present disclosure;

[0079] FIG. 4 is a schematic diagram of a recognition model according to some embodiments of the present disclosure;

[0080] FIG. 5 is a schematic diagram of a segmentation model according to some embodiments of the present disclosure;

[0081] FIG. 6 is a schematic diagram of a one-branch according to some embodiments of the present disclosure;

[0082] FIG. 7 is a schematic diagram of a grid map according to some embodiments of the present disclosure;

[0083] FIG. 8 is a flowchart of another vehicle control method according to some embodiments of the present disclosure;

[0084] FIG. 9 is a block diagram of functional units of a vehicle control apparatus according to some embodiments of the present disclosure;

[0085] FIG. 10 is a block diagram of a computing device according to some embodiments of the present disclosure. DETAILED DESCRIPTION

[0086] The embodiments of the present disclosure will be described in detail below with reference to the drawings.

[0087] The terms "first", "second", "third", and "fourth" and the like in the description and claims of the present disclosure and the accompanying drawings are used to distinguish between similar objects, not to describe a particular sequential order. In addition, the terms "include" and "have" and any variations thereof are intended to cover a non-exclusive inclusion. For example, a process, method, system, product, or device that includes a list of steps or units is not limited to the listed steps or units, but can optionally include additional steps or units not listed or can optionally include other steps or units inherent to such processes, methods, products, or devices.

[0088] The system architecture to which the embodiments of the present disclosure are applied will be described below. It should be noted that the system architecture and business scenarios described in the present disclosure are intended to more clearly illustrate the technical solutions of the present disclosure, and do not constitute a limitation on the technical solutions provided by the present disclosure. Those skilled in the art can know that, as the system architecture evolves and new business scenarios appear, the technical solutions provided by the present disclosure are also applicable to similar technical problems.

[0089] In the related art, the functions corresponding to the preset features on the vehicle need to be actively opened by the user. If the user does not actively open the functions corresponding to the preset features, it may cause the vehicle to be in danger, for example, it may cause the vehicle to slide sideways, anchor, lose control of the steering wheel, water in the vehicle, and the like. This will affect the service life of the vehicle and bring serious safety hazards to the people in the vehicle.

[0090] To solve the above problems, some embodiments of the present disclosure provide a vehicle control method and related apparatus.

[0091] Please refer to FIG. 1, which is a schematic diagram of an architecture of a vehicle according to some embodiments of the present disclosure. As shown in FIG. 1, the vehicle 10 includes a first camera device 101 and a second camera device 102.

[0092] The vehicle 10 can be a vehicle driven by electric energy, a vehicle driven by fuel, or a vehicle driven by new energy hybrid power. For example, when the vehicle 10 is a vehicle driven by electric energy, it can be a new energy vehicle, such as a pure electric vehicle, a range-extended electric vehicle, a hybrid vehicle, a fuel cell electric vehicle, etc. When the vehicle 10 is a vehicle driven by fuel, it can be a car, an agricultural transport vehicle, a tractor, or a trailer, etc. When the vehicle 10 is a car, it can be a sedan, a SUV, a van, a bus, or a minivan, etc.

[0093] The first camera 101 is installed on the roof of the vehicle 10 and is configured to capture images at a long distance (first distance). For example, the first camera 101 can be a front-view camera, including but not limited to a monocular camera and a binocular camera, etc., which are not limited here.

[0094] The vehicle 10 can identify a preset area at a long distance from the vehicle 10 according to the first image captured by the first camera 101, thereby achieving long-distance ranging. The preset area includes an area on the road surface containing a preset feature, and the preset feature includes but is not limited to water, snow, sand, mud, etc., and the type of the preset feature is not limited in the present disclosure.

[0095] The second camera 102 is installed on the front bumper of the vehicle 10 and is configured to capture images at a short distance (second distance). For example, the second camera 102 can be a front surround-view camera, including but not limited to a fisheye camera, a wide-angle camera, etc., which are not limited here.

[0096] The vehicle 10 can identify a preset area at a short distance from the vehicle 10 according to the second image captured by the second camera 102, thereby solving the problem of the blind area of the first camera 101 at a short distance from the vehicle 10 and achieving short-distance ranging of the vehicle 10.

[0097] The vehicle 10 inputs multiple road surface images captured by at least one of the first camera 101 or the second camera 102 into the trained recognition model, and outputs multiple recognition results. According to the multiple recognition results, it is determined whether there is a preset area on the road surface on which the vehicle 10 travels.

[0098] In the case where there is a preset area on the road surface, the vehicle 10 inputs the first road surface image captured by the first camera 101 into the trained first segmentation model, and outputs a first segmentation result map. The vehicle 10 inputs the second road surface image captured by the second camera 102 into the trained second segmentation model, and outputs a second segmentation result map. Finally, the vehicle 10 determines the distance from the vehicle 10 to the preset area according to the first segmentation result map and the second segmentation result map.

[0099] In summary, the vehicle 10 determines the distance from the vehicle 10 to the preset area according to the first road surface image collected by the first camera 101 and the second road surface image collected by the second camera 102. In this way, the advantages of measuring the distance of the preset area far away by the first camera 101 and measuring the distance of the preset area close by the second camera 102 can be achieved, and a fusion ranging scheme with a wide measurement range and stable measurement can be achieved.

[0100] Referring to FIG. 2, FIG. 2 is a flowchart of a vehicle control method according to some embodiments of the present disclosure, which is applied to the vehicle shown in FIG. 1. As shown in FIG. 2, the method includes but is not limited to the following steps S201-S203.

[0101] In step S201, a road surface image is obtained.

[0102] For example, when the vehicle is driving on the road surface, the road surface image can be collected by the camera. Since some embodiments of the present disclosure are mainly used to control the preset function according to the distance between the vehicle and the preset area, the preset area includes an area on the road surface containing a preset feature. Therefore, the vehicle can obtain the road surface image in front of the vehicle during driving.

[0103] For example, the road surface image in front of the vehicle can be collected by a front-view camera or a front surround-view camera. The front-view camera is a camera installed on the roof of the vehicle. The front-view camera includes but is not limited to a monocular camera or a binocular camera, which is not limited herein. The front surround-view camera is a camera installed on the front bumper of the vehicle. The front surround-view camera includes but is not limited to a fisheye camera or a wide-angle camera, which is not limited herein. In an implementation, the road surface image collected by the vehicle through the front-view camera is a three-channel RGB color mode image with a pixel of 3680x2160. The road surface image collected by the vehicle through the front surround-view camera is a three-channel RGB image with a pixel of 1920x1300.

[0104] In step S202, the distance from the vehicle to the preset area is determined according to the road surface image.

[0105] The preset area includes an area on the road surface containing a preset feature, and the preset feature includes but is not limited to water, snow, sand, mud, etc. The type of the preset feature is not limited herein. In some embodiments, the vehicle can determine whether the preset area exists on the road surface on which the vehicle travels according to the collected road surface image. In the case where the preset area exists, the distance from the vehicle to the preset area is determined according to the road surface image. The distance from the vehicle to the preset area can be understood as the distance that the vehicle needs to travel from the current position to the preset area. Since the vehicle generally travels in a straight line, the distance from the vehicle to the preset area can be the shortest distance from the vehicle to the preset area.

[0106] In an implementation, the vehicle determines that the preset area exists on the road surface based on the road surface image through the identification model. In the case where the preset area exists on the road surface, the distance from the vehicle to the preset area is determined through the segmentation model.

[0107] In some embodiments, in order to save computing power, the vehicle can first determine whether the preset area exists on the road surface. If the preset area does not exist, the vehicle continues to collect the road surface image to determine whether the preset area exists on the road surface. If the preset area exists, the vehicle determines the distance from the vehicle to the preset area. Since the identification model requires less computing power and has a faster operation speed, dividing the determination of whether the preset area exists and the calculation of the distance between the vehicle and the preset area into two steps with a sequence can save computing power and improve the operation speed of the vehicle.

[0108] In an implementation, the vehicle inputs multiple road surface images into the trained identification model and outputs multiple identification results. The identification result includes the road surface type and the probability value corresponding to the road surface type. Then, the vehicle determines the weight of the probability value corresponding to each of the multiple road surface images according to the collection time of the multiple road surface images. The vehicle determines that the preset area exists on the road surface based on the weight of the probability value corresponding to each of the multiple road surface images.

[0109] For example, in order to improve the accuracy of the identification result, the vehicle can determine whether the preset area exists on the road surface by synthesizing the identification results of the multiple road surface images. Since the collection times of the multiple road surface images are different, for example, the vehicle travels from a position far away from the preset area to the direction of the preset area, and the collection times of the multiple road surface images collected by the vehicle at intervals during the travel are different. Therefore, the sizes of the preset areas in the collected road surface images are also different. For example, the size of the preset area in the road surface image collected at the first time can be smaller than the size of the preset area in the road surface image collected at the second time, and the second time is after the first time.

[0110] Since the size of the preset area in the plurality of road surface images is different, the probability value of the road surface type output by the identification model is also different. For example, since the preset area of the road surface image at the first time is smaller than the preset area of the road surface image at the second time. Therefore, the probability value corresponding to the road surface image at the first time is smaller than the probability value corresponding to the road surface image at the second time.

[0111] In order to balance the plurality of identification results output by the identification model, the vehicle can determine the weight of the probability value corresponding to the plurality of road surface images in the order of the collection time of the plurality of road surface images. For example, the more recent the collection time, the higher the accuracy of the identification result obtained by the identification model. Therefore, the road surface image with the more recent collection time is given a greater weight. For example, the weight of the probability value corresponding to the road surface image at the second time is greater than the weight of the probability value corresponding to the road surface image at the first time, and the second time is after the first time.

[0112] For example, the vehicle inputs the N+1 frames of road surface images into the trained identification model, and outputs N+1 identification results, that is, N+1 road surface types and probability values corresponding to the road surface types. The N+1 frames of road surface images include the current frame of road surface image and the N frames of continuous road surface images before the current frame. The probability values in the identification results are given different weights according to the principle that the closer to the current frame, the greater the weight. Then the probability values in the N+1 identification results are multiplied by the corresponding weights and summed to output a weighted probability. The final road surface type is determined according to the weighted probability as the determination result of whether the preset area exists on the road surface.

[0113] In an implementation manner, the road surface type can be determined according to actual conditions. For example, the road surface type can be two types of road surface with preset area and road surface without preset area, which are used to determine whether the preset area exists on the road surface. Further, the road surface type can also be divided according to the type of the preset feature in the preset area. Taking water as an example, the road surface type includes but is not limited to water-involved road surface, water-floating road surface and other road surface, etc., which are not limited here. The other road surface means that the road surface does not have the preset area. The water-involved road surface and the water-floating road surface both mean that the road surface has the preset area, and the water range or water depth of the water-floating road surface is greater than that of the water-involved road surface. For example, the water-involved road surface is used to indicate that the vehicle can drive through the preset area, and the water-floating road surface indicates that the vehicle needs to drive through the preset area in the water-floating mode, that is, in the floating state. Therefore, the road surface type obtained by the vehicle through the identification model can also be used to indicate which preset functions the vehicle needs to start, that is, which preset functions the vehicle needs to start to drive through the preset area.

[0114] The above is the method for the vehicle to determine whether the preset area exists on the road surface through the identification model. The training method of the identification model will be introduced below.

[0115] In an implementation, before training the identification model, a training set needs to be determined first. For example, a plurality of road surface images are obtained, and then the plurality of road surface images are labeled to determine labels. The training set is constructed based on the plurality of road surface images and the corresponding labels.

[0116] For example, the road surface images are labeled by means of soft labels. The labeled information includes the road surface type of the road surface image and the probability value (between 0 and 1) corresponding to the road surface type. For example, taking water as a preset feature, the road surface image can be labeled as: the road surface type is a wading road surface, and the corresponding probability value is 0.7.

[0117] For example, please refer to FIG. 3A and FIG. 3B, which are both schematic diagrams of data labeling according to some embodiments of the present disclosure. As shown in FIG. 3A and FIG. 3B, the size of the preset area in road surface image 1 accounts for 70% of road surface image 1, so the label of road surface image 1 is a road surface containing a preset feature and the probability value is 0.7. The size of the area not being the preset area in road surface image 2 accounts for 60% of road surface image 2, so the label of road surface image 2 is a road surface not containing a preset feature and the probability value is 0.6.

[0118] By assigning probability values to the road surface images in the training set, the identification model can learn the corresponding probability values in the learning process, thereby strengthening the understanding of the road surface information by the identification model, making the identification result output by the identification model less likely to jump, and ensuring the accuracy of the identification result. The jump of the identification result refers to the case that in a plurality of identification results output by the identification model for a plurality of continuous road surface images, there is an identification result that is significantly different from other identification results. The accuracy of the identification result that jumps is relatively low, which affects the judgment of the vehicle on the road surface type.

[0119] In an implementation, the identification model is trained according to the training set described above to obtain a trained identification model. The identification model is used to determine whether a preset area exists on the road surface on which the vehicle travels.

[0120] For example, please refer to FIG. 4, which is a schematic diagram of an identification model according to some embodiments of the present disclosure. As shown in FIG. 4, the input feature map is first processed by the convolution head of the identification model, and then sequentially processed by four levels of processing stages, such as stage1, stage2, stage3 and stage4, and finally processed by the head. The input feature map is the region of interest (ROI) region of the road surface image, and the size can be 996x996. Stage1, stage2, stage3, stage4 and head are components of the deep learning classification network model (G-GhostNet). Some embodiments of the present disclosure add a convolution head and attention mechanism 1 and attention mechanism 2 to the G-GhostNet model. The convolution head includes a convolution with a 5x5 convolution kernel, a 3x3 convolution kernel, a dilated convolution with an expansion rate of 2, a BN layer, and a global average pooling layer. Attention mechanism 1 is a spatial attention mechanism, and attention mechanism 2 is a channel attention mechanism. After training the identification model, the identification model is lightened using the knowledge distillation method. The reason for choosing G-GhostNet model as the backbone network is that G-GhostNet is a lightweight network, which has less calculation amount and faster calculation speed, and can improve the speed of determining whether there is a preset area on the road surface. For example. The lightweight inference network in the related art (such as EfficientNet, MobileNet) is generally based on depthwise separable convolution. Compared with the neural network based on ordinary convolution (such as ResNet, InceptionNet), these networks have less calculation amount, but the actual measurement speed on the electronic device is slower than the ordinary convolution. Therefore, some embodiments of the present disclosure select the G-GhostNet model as the backbone network to improve the calculation speed.

[0121] In an implementation, the vehicle inputs the road surface image into multiple branches of the trained segmentation model respectively, and outputs multiple feature maps. Then, the distance from the vehicle to the preset area is determined according to the multiple feature maps.

[0122] For example, the segmentation model is a semantic segmentation model, which includes different branches for extracting features of different scales, so that the result output by the segmentation model is more accurate.

[0123] For example, please refer to FIG. 5, which is a schematic diagram of a segmentation model according to some embodiments of the present disclosure. The segmentation model used by some embodiments of the present disclosure is a model improved on the basis of a deep learning semantic segmentation network (e.g., BiseNet). As shown in FIG. 5, the multiple branches of the segmentation model include a deep learning classification network (e.g., Inception) branch, a spatial branch, and a context branch. First, the road surface image is input into the Inception branch, the spatial branch, and the context branch respectively. The Inception branch simultaneously captures spatial information at different scales through different sizes of convolution kernels. The Inception branch is characterized by combining convolution kernels together to establish a multi-branch structure, so that the network can calculate in parallel. By allocating different computing resources, the best performance can be obtained. The spatial branch is used to retain semantic information to generate higher resolution activation values (e.g., feature maps) and reduce the number of down-sampling. The context branch uses a fast down-sampling strategy to obtain sufficient receptive field. The feature maps output by the Inception branch, the spatial branch, and the context branch are spliced together and output to a feature fusion module to obtain a segmentation result. Since the spliced feature maps have the same size as the feature maps of the original BiseNet input feature fusion module, the original feature fusion module does not need to be modified.

[0124] In an implementation, one branch of the multiple branches, i.e., the Inception branch, includes multiple convolution channels. The multiple convolution channels are respectively used to determine the features of different scales of the road surface image.

[0125] For example, please refer to FIG. 6, which is a schematic diagram of one branch according to some embodiments of the present disclosure. The one branch used by some embodiments of the present disclosure is the Inception branch. As shown in FIG. 6, the one branch includes four convolution channels with different morphologies. The features (e.g., the input road surface image) are input into the four convolution channels respectively. After global average pooling, the feature map output by each convolution channel has a size of 64x64x32. After splicing the feature maps output by the four convolution channels, a feature map with a size of 64x64x128 is finally output. The first convolution channel is a 1x1 convolution. The second convolution channel is a 3x3 dilated convolution with an expansion rate of 2 and a step size of 1, and a 3x3 convolution with a step size of 2. The third convolution channel is a 5x5 convolution with a step size of 2, and a 5x5 convolution with a step size of 2. The fourth convolution channel is a 1x1 convolution, a 3x3 convolution with a step size of 2, and a 3x3 convolution with a step size of 2.

[0126] In an embodiment, the vehicle comprises a first camera and a second camera, and the segmentation model comprises a first segmentation model and a second segmentation model. The vehicle inputs the first road surface image into the trained first segmentation model to output a first segmentation result image. The vehicle inputs the second road surface image into the trained second segmentation model to output a second segmentation result image. Then, the vehicle determines the distance from the vehicle to the preset area according to the first segmentation result image and the second segmentation result image.

[0127] The first road surface image is captured by the first camera, and the first camera is used to capture images at a long distance. For example, the first camera can be a front-view camera installed on the roof of the vehicle to capture images at a long distance. The front-view camera includes but is not limited to a monocular camera and a binocular camera, etc., which are not limited here.

[0128] The second road surface image is captured by the second camera, and the second camera is used to capture images at a short distance. For example, the second camera can be a front surround-view camera installed on the front bumper of the vehicle to capture images at a short distance. The front surround-view camera includes but is not limited to a fisheye camera, a wide-angle camera, etc., which are not limited here.

[0129] The vehicle can identify the preset area far away from the vehicle according to the first image captured by the first camera, thereby realizing long-distance ranging. The vehicle can identify the preset area close to the vehicle according to the second image captured by the second camera, thereby solving the problem of the blind area of the first camera in the vicinity of the vehicle and realizing short-distance ranging of the vehicle. Therefore, the vehicle determines the distance from the vehicle to the preset area according to the first segmentation result image and the second segmentation result image, which can take advantage of the first camera to measure the distance of the preset area far away and the second camera to measure the distance of the preset area close. By fusing the first segmentation result image and the second segmentation result image, a fusion ranging scheme with wide measurement range and stable measurement is realized.

[0130] In an embodiment, the vehicle determines a grid map according to the first segmentation result image and the second segmentation result image. The grid map comprises a plurality of grids. A preset grid is determined from the plurality of grids, and the preset grid comprises information of the preset area. A target grid is determined from the preset grid, and the distance between the target grid and the vehicle is less than the distance between other preset grids and the vehicle. Then, the vehicle determines the distance from the vehicle to the preset area according to the coordinates of the target grid in the grid map.

[0131] For example, the vehicle converts the first segmentation result map and the second segmentation result map into the same grid map. Since the first segmentation map includes the road surface information of a long distance, and the second segmentation map includes the road surface information of a short distance. Therefore, the grid map determined according to the first segmentation result map and the second segmentation result map contains the road surface information of a long distance and a short distance. The grid map divides the image containing the road surface into different parts by multiple grids. The vehicle can first determine a preset grid containing the information of the preset area from the multiple grids. For example, if the road surface image contains the preset area, the road surface image is divided by multiple grids, so the preset area can be composed of one or more preset grids. The vehicle can first determine one or more preset grids from the multiple grids, and then determine a target grid from the one or more preset grids. For example, the preset grid closest to the vehicle in the one or more preset grids is taken as the target grid. Then, the distance from the vehicle to the preset area is determined based on the coordinates of the target grid in the grid map, that is, the shortest distance from the vehicle to the preset area.

[0132] It should be noted that, in the case of one or more preset grids including one preset grid, the preset grid is the target grid.

[0133] In an implementation manner, the first segmentation model and the second segmentation model are both semantic segmentation models, and the output first segmentation result map and the second segmentation result map are mask maps. The vehicle can project the first segmentation result map and the second segmentation result map into the same probability-occupied grid map according to the intrinsic and extrinsic parameters of the first camera and the intrinsic and extrinsic parameters of the second camera.

[0134] For example, the road surface image in front of the vehicle is cut by a grid, and a probability is used to represent the possibility of whether each grid contains the information of the preset area. Each grid stores a probability value between 0 and 1. The larger the probability value, the greater the possibility that the grid contains the information of the preset area. The smaller the probability value, the smaller the possibility that the grid contains the information of the preset area.

[0135] Further, in order to improve the accuracy of vehicle ranging, the vehicle can range based on multiple grid maps. For example, the vehicle respectively collects multiple first road surface images through the first camera and multiple second road surface images through the second camera at different times. Then, multiple first segmentation result maps are determined based on the multiple first road surface images, and multiple second segmentation result maps are determined based on the multiple second road surface images. Multiple grid maps are determined based on the multiple first segmentation result maps and the multiple second segmentation result maps. Therefore, the same grid in the multiple grid maps is in different states at different times. For example, the grid does not contain the information of the preset area at the first time, and may contain the information of the preset area at the second time. Therefore, the state of the grid at time t may be related to the sensor observation at time t and the state of the grid at time t-1.

[0136] For example, if the probability of any grid containing information of the preset area is denoted as p(my, x = 1), and the probability of any grid not containing information of the preset area is denoted as p(my, x = 0). Then the probability ratio of whether the grid contains information of the preset area is expressed as the following expression (1):

[0137] where my, x represents a grid in the grid map, and x and y are the coordinates of the grid in the grid map.

[0138] At time t, according to the Bayes update formula, the following expression (2) and expression (3) can be obtained:

[0139] where, is the observation value of the perception algorithm, and the perception algorithm is used to represent the above segmentation model. is the probability of the observation value, is the probability of the grid containing information of the preset area under the condition of the observation value at time t. is the probability of the grid not containing information of the preset area under the condition of the observation value at time t. is the probability of the observation value under the condition that the grid contains information of the preset area at time t-1. is the probability of the observation value under the condition that the grid does not contain information of the preset area at time t-1. is the probability of the grid containing information of the preset area at time t-1. is the probability of the grid not containing information of the preset area at time t-1.

[0140] Then, according to expression (1), expression (2) and expression (3), the update formula of Occy, x is obtained as the following expression (4):

[0141] where, represents the probability ratio of whether the grid contains information of the preset area at time t-1.

[0142] In order to avoid the probability ratio Occy, x from accumulating near 0 or 1, the log of both sides of expression (4) is taken to obtain the following expression (5):

[0143] According to expression (5), ​​The probability is related to the state of the grid at time t-1 and the sensor observation at time t. Since the state of the grid is related to the probability of the grid containing the information of the preset feature, for example, if the probability of the grid containing the information of the preset feature is 0.9, it can be determined that the state of the grid is that the grid contains the information of the preset feature. For example, the vehicle can determine the correspondence between the probability of the grid and the state of the grid according to the relationship table between the leakage and false detection probability of the segmentation model and the confidence. Therefore, according to the simple addition and subtraction in the above expression (5), the state of the grid at time t can be determined according to the probability of the grid, that is, the preset grid containing the information of the preset feature is determined from the multiple grids of the grid map.

[0144] Further, after the vehicle determines one or more preset grids from the multiple grids of the grid map, the vehicle can determine a target grid from the one or more preset grids. The distance between the target grid and the vehicle is less than the distance between the other preset grids and the vehicle. The vehicle can convert the coordinates (x, y) of the target grid into the longitudinal distance L meters between the vehicle and the front preset feature in the vehicle coordinate system according to the relationship between the pre-projection grid map and the real-world coordinate system.

[0145] For example, please refer to FIG. 7, which is a schematic diagram of a grid map according to some embodiments of the present disclosure. As shown in FIG. 7, the grid map is determined by the vehicle according to the first segmentation result map and the second segmentation result map, and the grid map includes multiple grids. The gray area in the grid map is the preset area. As can be seen, the multiple preset grids in the grid map all contain the information of the preset area. The vehicle determines the preset grid closest to the vehicle from the multiple preset grids as the target grid. Then, the distance between the vehicle and the preset area is determined according to the coordinates of the target grid in the grid map.

[0146] In step S203, the preset function of the vehicle is controlled according to the distance between the vehicle and the preset area to achieve safe driving of the vehicle.

[0147] The preset function includes the function used by the vehicle when driving on the preset area. For example, the preset function includes but is not limited to turning off the engine and / or motor, locking the vehicle door, closing the window, opening the sunroof, closing part of the auxiliary driving function, etc., to enable the vehicle to safely drive through the road surface containing the preset feature, for example, taking the preset feature as water as an example, through the preset function, the vehicle can help deal with a large amount of water during a typhoon or heavy rain, and realize the forward driving, turning, etc. of the vehicle on the waterlogged road surface.

[0148] In an implementation manner, the vehicle determines the distance threshold according to the vehicle speed, and turns on the preset function of the vehicle when the distance between the vehicle and the preset area is less than the distance threshold.

[0149] For example, the vehicle can first determine the distance threshold according to the vehicle speed. For example, if the vehicle speed is fast, the vehicle takes a short time to reach the preset area. The vehicle needs to open the preset function at a position far from the preset area, so the corresponding distance threshold is large. If the vehicle speed is slow, the vehicle takes a long time to reach the preset area. The vehicle can open the preset function at a position close to the preset area, so the corresponding distance threshold is small.

[0150] The vehicle automatically calculates the distance threshold for opening the preset function as M meters in combination with the current vehicle speed. The distance threshold serves as the basis for whether to automatically open the preset function. If the real-time calculation result of the vehicle is greater than M meters, the vehicle does not send a signal to open the preset function. If the real-time calculation result of the vehicle is less than M meters, the vehicle sends a signal to automatically open the preset function.

[0151] In an implementation manner, after the vehicle opens the preset function, if the vehicle judges that there is no preset area on the road surface on which the vehicle travels, the vehicle closes the preset function.

[0152] For example, the vehicle inputs multiple frames of continuous road surface images into the trained recognition model to obtain multiple recognition results. When multiple results all show that there is no preset area, the vehicle determines that there is no preset area in front of the vehicle, and the vehicle closes the preset function and switches to a normal driving mode. If there is a recognition result in the multiple recognition results indicating that there is a preset area on the road surface, the vehicle continues to calculate the distance between the preset area and the vehicle until there is no preset area on the road surface.

[0153] Please refer to FIG. 8, which is a flowchart of another vehicle control method according to some embodiments of the present disclosure. As shown in FIG. 8, the method includes at least one step in S801-S809.

[0154] In S801, a road surface image is acquired.

[0155] When the vehicle travels on the road surface, the road surface image can be collected by the camera. Since some embodiments of the present disclosure are mainly used to control the preset function according to the distance between the vehicle and the preset area, the preset area includes an area on the road surface containing a preset feature. Therefore, the vehicle can acquire the road surface image in front of the vehicle during the travel process.

[0156] For example, the road surface image in front of the vehicle can be collected by a front-view camera, a front surround-view camera, or the like, but is not limited thereto. The front-view camera is a camera installed on the roof of the vehicle. The front-view camera includes a monocular camera, a binocular camera, or the like, but is not limited thereto. The front surround-view camera is a camera installed on the front bumper of the vehicle. The front surround-view camera includes a fisheye camera, a wide-angle camera, or the like, but is not limited thereto. In an implementation, the road surface image collected by the front-view camera is a three-channel RGB color mode (RGB) image with a pixel of 3680x2160. The road surface image collected by the front surround-view camera is a three-channel RGB image with a pixel of 1920x1300.

[0157] In S802, the road surface image is input into the trained recognition model.

[0158] To save computing power, the vehicle can first determine whether the preset area exists on the road surface. The vehicle inputs multiple road surface images into the trained recognition model and outputs multiple recognition results. The recognition result includes a road surface type and a probability value corresponding to the road surface type. Then, the vehicle determines the weights of the probability values respectively corresponding to the multiple road surface images according to the collection time of the multiple road surface images. The vehicle determines whether the preset area exists on the road surface based on the weights of the probability values respectively corresponding to the multiple road surface images.

[0159] In S803, it is determined whether the preset area exists on the road surface.

[0160] The vehicle inputs N+1 frames of road surface images into the trained recognition model and outputs N+1 recognition results, i.e., N+1 road surface types and probability values corresponding to the road surface types. The N+1 frames of road surface images include a current frame of road surface image and N continuous frames of road surface images before the current frame. The probability values in the recognition results are given different weights according to the principle that the closer to the current frame, the greater the weight. Then, the probability values in the N+1 recognition results are multiplied by the corresponding weights and summed to output a weighted probability. The final road surface type is determined according to the weighted probability as the determination result of whether the preset area exists on the road surface. If the preset area exists on the road surface, S804 is entered, and if the preset area does not exist on the road surface, S801 is returned.

[0161] In S804, the road surface image is input into the trained segmentation model.

[0162] The segmentation model includes a first segmentation model and a second segmentation model. The vehicle inputs a first road surface image into the trained first segmentation model and outputs a first segmentation result. The vehicle inputs a second road surface image into the trained second segmentation model and outputs a second segmentation result. Then, the vehicle determines the distance from the vehicle to the preset area according to the first segmentation result and the second segmentation result.

[0163] The vehicle can identify the preset area far away from the vehicle according to the first image collected by the first camera, so as to realize distance measurement at a long distance. The vehicle can identify the preset area close to the vehicle according to the second image collected by the second camera, so as to solve the problem of the blind area of the first camera in the vicinity of the vehicle, and realize distance measurement at a short distance. Therefore, the vehicle determines the distance from the vehicle to the preset area according to the first segmentation result image and the second segmentation result image, so as to exert the advantages of the first camera in measuring the distance of the preset area far away and the second camera in measuring the distance of the preset area close. By fusing the first segmentation result image and the second segmentation result image, a fusion distance measurement scheme with a wide measurement range and stable measurement is realized.

[0164] In S805, it is judged whether the distance between the vehicle and the preset area is less than a distance threshold.

[0165] The vehicle automatically calculates the distance threshold for starting the preset function as M meters in combination with the current vehicle speed, and the distance threshold is used as the basis for automatically starting the preset function. If the real-time distance measurement result of the vehicle is greater than M meters, a signal for starting the preset function is not sent. If the real-time distance measurement result of the vehicle is less than M meters, a signal for automatically starting the preset function is sent, and S806 is entered.

[0166] In S806, the preset function is started.

[0167] When the vehicle receives the signal for starting the preset function, the corresponding preset function is started. The preset function includes a function used by the vehicle when driving on the preset area. For example, the preset function includes, but is not limited to, at least one of turning off the engine or the electric motor, locking the vehicle door, closing the vehicle window, opening the sunroof, and closing part of the auxiliary driving function, etc., so as to enable the vehicle to safely drive over the road surface containing the preset feature. Taking water as an example, the preset function can help the vehicle deal with a large amount of water during a typhoon or heavy rain, and realize the forward driving, turning, etc. of the vehicle on the waterlogged road surface.

[0168] In S807, it is judged whether the front road surface still has the preset area. After the vehicle starts the preset function, if the vehicle judges that there is no preset area on the road surface on which the vehicle drives, the vehicle closes the preset function.

[0169] For example, the vehicle inputs a plurality of continuous road surface images into the trained recognition model to obtain a plurality of recognition results. When the plurality of results all show that the preset area does not exist, the vehicle determines that there is no preset area in front of the vehicle, and enters S808. If there is a recognition result in the plurality of recognition results indicating that the road surface has the preset area, S809 is entered.

[0170] In S808, the vehicle closes the preset function and switches to the normal driving mode.

[0171] In S809, a distance from the vehicle to the preset area is calculated. The vehicle continuously calculates the distance from the preset area to the vehicle until the preset area is not present on the road surface.

[0172] The above describes the method of some embodiments of the present disclosure in detail, and the following provides the device of some embodiments of the present disclosure.

[0173] Please refer to FIG. 9, which is a block diagram of functional units of a vehicle control device according to some embodiments of the present disclosure. The vehicle control device 90 can include a communication unit 901 and a processing unit 902. The vehicle control device 90 is configured to implement the vehicle control method described above, such as the vehicle control method shown in FIG. 2.

[0174] It should be noted here that the division of the above-mentioned units is only a logical division according to functions, and does not limit the structure of the vehicle control device 90. In some implementations, some functional modules of the vehicle control device 90 can be subdivided into more detailed functional modules, and some functional modules can be combined into one functional module.

[0175] In an embodiment, the communication unit 901 is configured to acquire a road surface image, the road surface image including an image collected when the vehicle travels on a road surface;

[0176] The processing unit 902 is configured to determine a distance from the vehicle to a preset area according to the road surface image, the preset area including an area on the road surface containing a preset feature;

[0177] The processing unit 902 is further configured to control a preset function of the vehicle according to the distance from the vehicle to the preset area to achieve safe driving of the vehicle, the preset function including a function used by the vehicle when traveling on the preset area.

[0178] In an embodiment, the processing unit 902 is configured to determine the distance from the vehicle to the preset area according to the road surface image, including:

[0179] determining, based on the road surface image, that the road surface has the preset area by using a recognition model;

[0180] In the case where the road surface has the preset area, determining the distance from the vehicle to the preset area by using a segmentation model.

[0181] In an embodiment, the vehicle includes a first camera device and a second camera device, the segmentation model includes a first segmentation model and a second segmentation model, and the processing unit 902 is configured to determine the distance from the vehicle to the preset area by using the segmentation model, including:

[0182] The first road surface image is input into the trained first segmentation model, and a first segmentation result image is output. The first road surface image is collected by a first camera device configured to collect images at a long distance.

[0183] The second road surface image is input into the trained second segmentation model, and a second segmentation result image is output. The second road surface image is collected by a second camera device configured to collect images at a short distance.

[0184] The distance of the vehicle to the preset area is determined according to the first segmentation result image and the second segmentation result image.

[0185] In an embodiment, the processing unit 902 is configured to determine the distance of the vehicle to the preset area according to the first segmentation result image and the second segmentation result image, including:

[0186] A grid image is determined according to the first segmentation result image and the second segmentation result image. The grid image includes a plurality of grids.

[0187] A preset grid is determined from the plurality of grids. The preset grid contains information of the preset area.

[0188] A target grid is determined from the preset grid. The distance between the target grid and the vehicle is less than the distance between other preset grids and the vehicle.

[0189] The distance of the vehicle to the preset area is determined according to the coordinates of the target grid in the grid image.

[0190] In an embodiment, the processing unit 902 is configured to determine the distance of the vehicle to the preset area by using the segmentation model, including:

[0191] The road surface image is input into a plurality of branches of the trained segmentation model, and a plurality of feature maps are output.

[0192] The distance of the vehicle to the preset area is determined according to the plurality of feature maps.

[0193] In an embodiment, one branch of the plurality of branches includes a plurality of convolution channels. The plurality of convolution channels are respectively configured to determine features of different scales of the road surface image.

[0194] In an embodiment, the processing unit 902 is configured to determine, based on the road surface image, whether the preset area exists on the road surface by using the recognition model, including:

[0195] A plurality of road surface images are input into the trained recognition model, and a plurality of recognition results are output. The recognition result includes a road surface type and a probability value corresponding to the road surface type.

[0196] The weights of the probability values corresponding to the plurality of road surface images are determined according to the collection times of the plurality of road surface images.

[0197] The preset region is determined according to the weights of the probability values respectively corresponding to the plurality of road surface images.

[0198] In an implementation, the processing unit 902 is configured to control the preset function of the vehicle according to the distance between the vehicle and the preset region to achieve safe driving of the vehicle, including:

[0199] determining the distance threshold according to the vehicle speed of the vehicle;

[0200] turning on the preset function of the vehicle when the distance between the vehicle and the preset region is less than the distance threshold.

[0201] It should be noted that in some embodiments of the present disclosure, the implementation and technical effects of each unit can also be referred to the corresponding description of the method embodiments shown in FIG. 2.

[0202] Some embodiments of the present disclosure also provide a vehicle 10, which comprises a processor and a memory, the processor is coupled with the memory, the memory is configured to store a computer program, and the processor is configured to invoke and run the computer program, so that the vehicle 10 executes the vehicle control method described above, for example, the method of FIG. 2.

[0203] Please refer to FIG. 10, which is a schematic diagram of a computing device according to some embodiments of the present disclosure. As shown in FIG. 10, the computing device 100 can include one or more processors 1001, one or more memories 1002, and one or more communication interfaces 1003. These components can be connected through a bus 1004 or other means, and FIG. 10 takes the example of being connected through the bus 1004.

[0204] The communication interface 1003 can be used for the computing device 100 to communicate with other communication devices, such as other computing devices. For example, the communication interface 1003 can be a wired interface.

[0205] The memory 1002 can be coupled to the processor 1001 via the bus 1004 or input / output port, and can also be integrated into the processor 1001. The memory 1002 is configured to store various software programs and / or sets of instructions or data. For example, the memory 1002 can be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, and can also be an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disk storage, a magnetic disk storage or other magnetic storage devices, or any other medium capable of storing desired program codes in the form of instructions or data structures and that can be accessed by a computer, but is not limited thereto. The memory 1002 can include a high-speed random access memory, and can also include a non-volatile memory, such as one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid-state storage devices.

[0206] The memory 1002 can store an operating system (hereinafter referred to as a system), such as an embedded operating system, Micro Controller Operating Systems (uCOS), VxWorks, A Real-Time Linux (RTLinux), and the like. The memory 1002 can also store a network communication program, which can be used to communicate with one or more additional devices, one or more user devices, and one or more terminals. The memory 1002 can exist independently and be connected to the processor 1001 via the bus 1004. The memory 1002 can also be integrated with the processor 1001.

[0207] The memory 1002 is configured to store application program codes for executing the above solutions, and the processor 1001 is configured to control the execution. The processor 1001 is configured to execute the application program codes stored in the memory 1002.

[0208] The processor 1001 can be a central processing unit, a general purpose processor, a digital signal processor, an application specific integrated circuit, a field programmable gate array, or other programmable logic device, transistor logic, hardware components, or any combination thereof. It can implement or execute various example logical blocks, modules, and circuits described in connection with the embodiments of the present disclosure. The processor 1001 can also be a combination of implementing determining functions, such as a combination of one or more microprocessors, a combination of a digital signal processor and a microprocessor, and the like.

[0209] Some embodiments of the present disclosure further provide a computer readable storage medium, which stores instructions, when the instructions are executed on at least one processor, implement the vehicle control method described above, such as the method of FIG. 2.

[0210] Some embodiments of the present disclosure further provide a computer program product, which includes computer instructions, when executed by a computing device, implement the vehicle control method described above, such as the method of FIG. 2.

[0211] In some embodiments of the present disclosure, the word "for example," "e.g.," or "like" is used to indicate an example, an instance, or an illustration. Any embodiment or design presented as "for example" or "like" in the present disclosure should not be construed as being more preferred or advantageous than other embodiments or designs. Rather, the use of "for example" or "like" is intended to present concepts in a concrete manner.

[0212] In the present disclosure, "at least one" means one or more, and "multiple" means two or more. "At least one of the following (a)" or the like means any combination of these items, including any combination of single item (a) or multiple items (a). For example, at least one of a, b, or c can mean a, b, c, (a and b), (a and c), (b and c), or (a and b and c), and a, b, and c can be single or multiple. "And / or" describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects.

[0213] Also, unless otherwise stated, the ordinal numbers such as "first", "second", etc. used in the embodiments of the present disclosure are used to distinguish the multiple objects, and are not used to represent the order, time sequence, priority or importance of the multiple objects. For example, the first device and the second device are merely used for the convenience of description, and do not represent the difference in structure, importance, etc. of the first device and the second device. In some embodiments, the first device and the second device can also be the same device.

[0214] In the above embodiments, the term "when" can be interpreted as meaning "if", "after" or "in response to determining" or "in response to detecting" according to the context. The above is only an optional embodiment of the present application, and is not used to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the concept and principle of the present application shall be included in the protection scope of the present application.

[0215] A person of ordinary skill in the art can understand that all or part of the steps of the above-mentioned embodiments can be completed by hardware, or by program instructing relevant hardware to complete, and the program can be stored in a computer readable storage medium. The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0216] The above is only a specific embodiment of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any person skilled in the art can easily think of various equivalent modifications or replacements within the technical range disclosed by the present disclosure, and these modifications or replacements shall be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims.

Claims

1. A vehicle control method, comprising: obtaining a road surface image, wherein the road surface image comprises an image collected when a vehicle travels on a road surface; determining a distance from the vehicle to a preset area according to the road surface image, wherein the preset area comprises an area containing a preset feature on the road surface; controlling a preset function of the vehicle to achieve safe driving of the vehicle according to the distance from the vehicle to the preset area, the preset function comprising a function used by the vehicle when traveling on the preset area.

2. The method of claim 1, wherein, The determining of the distance from the vehicle to the preset area according to the road surface image comprises: determining, based on the road surface image, that the road surface contains the preset area by a recognition model; in a case where the road surface contains the preset area, determining the distance from the vehicle to the preset area by at least one segmentation model.

3. The method of claim 2, wherein, The vehicle comprises a first camera device and a second camera device, and the at least one segmentation model comprises a first segmentation model and a second segmentation model, and the determining of the distance from the vehicle to the preset area by the at least one segmentation model comprises: inputting a first road surface image into the trained first segmentation model to output a first segmentation result image, wherein the first road surface image is collected by the first camera device configured to collect images of a first distance; inputting a second road surface image into the trained second segmentation model to output a second segmentation result image, wherein the second road surface image is collected by the second camera device configured to collect images of a second distance; determining the distance from the vehicle to the preset area according to the first segmentation result image and the second segmentation result image.

4. The method of claim 3, wherein, The determining of the distance from the vehicle to the preset area according to the first segmentation result image and the second segmentation result image comprises: determining a grid image according to the first segmentation result image and the second segmentation result image, wherein the grid image comprises a plurality of grids; determining at least one preset grid from the plurality of grids, the at least one preset grid containing information of the preset area; determining a target grid from the at least one preset grid, in a case where the at least one preset grid comprises one preset grid, the preset grid being the target grid; in a case where the at least one preset grid comprises a plurality of preset grids, the target grid having a distance from the vehicle smaller than that of the remaining preset grids in the plurality of preset grids; determining the distance from the vehicle to the preset area according to coordinates of the target grid in the grid image.

5. The method of any one of claims 2-4, wherein, The determining of the distance from the vehicle to the preset area by the at least one segmentation model comprises: inputting the road surface image into a plurality of branches of the trained at least one segmentation model respectively to output a plurality of feature maps; determining the distance from the vehicle to the preset area according to the plurality of feature maps.

6. The method of claim 5, wherein, One branch of the plurality of branches comprises a plurality of convolution channels respectively configured to determine features of different scales of the road surface image.

7. The method of any one of claims 2-6, wherein, The determining, by the recognition model, that the road surface has the preset region based on the road surface image includes: inputting a plurality of road surface images into the trained recognition model to output a plurality of recognition results, wherein the recognition results include a road surface type and a probability value corresponding to the road surface type; determining weights of the probability values corresponding to the plurality of road surface images respectively according to collection times of the plurality of road surface images; determining that the road surface has the preset region according to the weights of the probability values corresponding to the plurality of road surface images respectively.

8. The method of any one of claims 1-7, wherein, The controlling, according to the distance of the vehicle to the preset region, of a preset function of the vehicle to achieve safe driving of the vehicle includes: determining a distance threshold according to a vehicle speed of the vehicle; in a case where the distance of the vehicle to the preset region is less than the distance threshold, starting the preset function of the vehicle. 9.A vehicle control apparatus, comprising: a communication unit configured to acquire a road surface image, wherein the road surface image includes an image collected while a vehicle is driving on a road surface; and a processing unit configured to determine a distance of the vehicle to a preset region according to the road surface image, wherein the preset region includes a region including a preset feature on the road surface; wherein the processing unit is further configured to control a preset function of the vehicle according to the distance of the vehicle to the preset region to achieve safe driving of the vehicle, the preset function including a function used by the vehicle when driving on the preset region. 10.A vehicle, comprising a processor and a memory, the processor being coupled to the memory, the memory being configured to store a computer program, and the processor being configured to invoke and run the computer program, so that the vehicle performs the method according to any one of claims 1-8. 11.A computing device, comprising a processor, the processor being coupled to a memory, the memory being configured to store a computer program, and the processor being configured to invoke and run the computer program, so that the computing device performs the method according to any one of claims 1-8.

12. A computer readable storage medium, wherein, The computer readable storage medium stores a computer program, and the computer program includes instructions for performing the method according to any one of claims 1-8.

Images