Control method of vehicle, electronic device, vehicle, and storage medium

Abstract

The application discloses a control method of a vehicle, a control device of the vehicle, an electronic device, the vehicle and a computer readable storage medium. The application relates to the technical field of automatic control of vehicles, and the control method comprises the following steps: acquiring pit information according to a road condition image; determining wheel torque according to driving information of the vehicle and the pit information; and controlling the vehicle to drive according to the wheel torque. The application can improve the smoothness during driving of the vehicle.

Description

Technical Field

[0001] This application relates to the field of vehicle control technology, and more specifically, to a vehicle control method, electronic device, vehicle, and computer-readable storage medium. Background Technology

[0002] The vehicle's controller sends a torque request to the motor to instruct it to output actual torque, thereby controlling the vehicle's movement. Currently, the controller typically determines the requested torque based on various driving parameters acquired by sensors within the vehicle, such as accelerator pedal opening, vehicle speed, steering wheel angle, driving mode, and drive mode. However, this method of determining the requested torque based on vehicle driving parameters assumes the vehicle is traveling on a smooth road surface. When the vehicle is traveling on potholes or bumpy roads, the lack of advance awareness of road conditions results in a less smooth driving experience. Summary of the Invention

[0003] This application provides a vehicle control method, an electronic device, a vehicle, and a computer-readable storage medium.

[0004] This application provides a vehicle control method, comprising: determining wheel torque based on pothole information and vehicle driving information; and controlling vehicle driving based on the wheel torque; wherein the pothole information is obtained based on road condition information.

[0005] In some implementations, the road condition information includes road condition images, and the pothole information is obtained based on the road condition information, including: obtaining pothole information based on the road condition images and a preset pothole detection model.

[0006] In some embodiments, the preset pothole detection model includes a backbone network, a neck network, and a head network. The step of obtaining pothole information based on the road condition image and the preset pothole detection model includes: acquiring the road condition image and inputting it into the backbone network for feature extraction processing to obtain a feature-extracted image; inputting the feature-extracted image into the neck network for feature fusion processing to obtain a feature-fused image; and inputting the feature-fused image into the head network for computation processing to determine the pothole information.

[0007] In some embodiments, the feature fusion image includes a left-view feature map and a right-view feature map, and the pit information includes the pit location. The step of inputting the feature fusion image into a head network for calculation and processing to determine the pit information includes: inputting the left-view feature map and the right-view feature map into the head network, subtracting the gray value of the right-view feature map from the gray value of the left-view feature map to obtain a left disparity map; and determining the pit location based on the left disparity map.

[0008] In some embodiments, the feature fusion image includes a left-view feature map and a right-view feature map, and the pit information includes the pit depth. The step of inputting the feature fusion image into a head network for computational processing to determine the pit information includes: inputting the left-view feature map and the right-view feature map into the head network, and subtracting the grayscale value of the right-view feature map from the grayscale value of the left-view feature map to obtain a left disparity map; fitting a preset road surface equation based on the left disparity map to obtain a target road surface equation; and determining the pit depth based on the target road surface equation.

[0009] In some embodiments, the feature fusion image includes a left-view feature map and a right-view feature map, and the pit information includes the pit location. The step of inputting the feature fusion image into a head network for calculation and processing to determine the pit information includes: inputting the left-view feature map and the right-view feature map into the head network, subtracting the gray value of the left-view feature map from the gray value of the right-view feature map to obtain a right disparity map; and determining the pit location based on the right disparity map.

[0010] In some embodiments, the feature fusion image includes a left-view feature map and a right-view feature map, and the pit information includes the pit depth. The step of inputting the feature fusion image into a head network for computational processing to determine the pit information includes: inputting the left-view feature map and the right-view feature map into the head network, and subtracting the grayscale value of the left-view feature map from the grayscale value of the right-view feature map to obtain a right disparity map; fitting a preset road surface equation based on the right disparity map to obtain a target road surface equation; and determining the pit depth based on the target road surface equation.

[0011] In some embodiments, determining the pothole depth based on the target road surface equation includes: substituting the target road surface equation into a preset pothole depth formula to determine the pothole depth.

[0012] In some embodiments, determining the wheel torque based on the driving information and the pothole information includes: determining an expected time based on the driving information and the pothole information; determining the wheel torque based on the driving information, the current time, and the expected time; wherein the expected time is later than the current time.

[0013] In some embodiments, the driving information includes driving speed, the pothole information includes pothole location, and determining the expected time based on the driving information and the pothole information includes: determining the expected time based on the driving speed and the pothole location, wherein the expected time is the arrival time of the vehicle when it reaches the pothole location.

[0014] In some embodiments, determining the wheel torque based on the driving information, the current time, and the expected time includes: determining the total required torque for the pothole based on the driving information when the current time equals the expected time; allocating the total required torque for the pothole according to a preset allocation coefficient to determine the required wheel torque for the pothole and the yaw torque for the pothole; wherein the driving information includes at least one of motor speed, accelerator pedal opening, wheel track, effective wheel radius, steering wheel angle, and brake pedal opening; the allocation coefficient includes at least one of the front wheel torque coefficient, the left rear wheel torque coefficient, and the right rear wheel torque coefficient; and the required wheel torque for the pothole includes at least one of the front axle torque for the pothole, the left rear wheel torque for the pothole, and the right rear wheel torque for the pothole.

[0015] In some embodiments, controlling vehicle movement based on the wheel torque includes controlling vehicle movement based on the required wheel torque for the pothole and the yaw torque of the pothole.

[0016] In some embodiments, determining the wheel torque based on the driving information, the current time, and the expected time includes: when the current time is not equal to the expected time, determining the total planar demand torque based on the driving information, wherein the total planar demand torque includes the planar demand wheel torque and the planar yaw torque; wherein the current driving information includes at least one of motor speed, accelerator pedal opening, wheel track, effective wheel radius, steering wheel angle, and brake pedal opening; the planar demand wheel torque includes at least one of the planar front axle torque, the planar left rear wheel torque, and the planar right rear wheel torque; the planar front axle torque is the sum of the planar left rear wheel torque and the planar right rear wheel torque; and the planar left rear wheel torque is equal to the planar right rear wheel torque.

[0017] In some embodiments, controlling vehicle movement based on the wheel torque includes controlling vehicle movement based on the planar required wheel torque and the planar yaw torque.

[0018] In some embodiments, the vehicle is equipped with a binocular camera, and the control method further includes: using the binocular camera to capture road condition images to obtain the road condition information.

[0019] In some embodiments, the vehicle is equipped with a binocular camera, and the control method further includes calibrating the binocular camera according to a preset camera calibration algorithm.

[0020] This application also provides an electronic device, which includes a memory and a processor. The memory is configured to store a computer program, and the processor, when executing the computer program, implements the control method of any of the above embodiments.

[0021] This application also provides a vehicle that includes the control device of any of the above embodiments, or includes the electronic device of any of the above embodiments.

[0022] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the control method in any of the above embodiments.

[0023] The vehicle control method, control device, electronic device, vehicle, and computer-readable storage medium provided in this application acquire pothole information through road condition images, thereby determining pothole information such as location and depth on the road ahead of the vehicle. Then, based on the vehicle's driving information and the pothole information, this application adjusts the wheel torque and controls the vehicle's movement accordingly. This application can detect road conditions in advance and adjust wheel torque in a timely manner when the vehicle passes over potholes to reduce vehicle bumps and optimize the smoothness of the vehicle's driving process.

[0024] Additional aspects and advantages of embodiments of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of embodiments of this application. Attached Figure Description

[0025] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, wherein:

[0026] Figure 1 This is a flowchart illustrating a vehicle control method according to some embodiments of this application;

[0027] Figure 2 This is a schematic diagram of the structure of a vehicle control device according to some embodiments of this application;

[0028] Figure 3 This is a flowchart illustrating a vehicle control method according to some embodiments of this application;

[0029] Figure 4 This is a schematic diagram of the structure of a preset pit detection model according to some embodiments of this application;

[0030] Figure 5 This is a flowchart illustrating a vehicle control method according to some embodiments of this application;

[0031] Figure 6 This is a flowchart illustrating a vehicle control method according to some embodiments of this application;

[0032] Figure 7 This is a schematic diagram of the control logic of a vehicle control method according to some embodiments of this application;

[0033] Figure 8 This is a schematic flowchart of a vehicle control method according to some embodiments of this application;

[0034] Figure 9 This is a flowchart illustrating a vehicle control method according to some embodiments of this application;

[0035] Figure 10 This is a schematic diagram of the structure of a vehicle control device according to other embodiments of this application;

[0036] Figure 11 This is a flowchart illustrating a vehicle control method according to other embodiments of this application;

[0037] Figure 12 This is a structural schematic diagram of a vehicle according to certain embodiments of this application;

[0038] Figure 13 This is a schematic diagram illustrating the connection state of a computer-readable storage medium and a processor according to certain embodiments of this application.

[0039] Explanation of key component symbols:

[0040] 100 vehicles;

[0041] Vehicle control device 10;

[0042] Acquisition module 11; Determination module 12; Control module 13;

[0043] Processor 20;

[0044] 200; computer-readable storage medium; 202; computer program;

[0045] 30 electronic devices;

[0046] 40-degree binocular camera. Detailed Implementation

[0047] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the embodiments of this application, and should not be construed as limiting the embodiments of this application.

[0048] The vehicle's controller controls the motor by sending a torque request, thereby adjusting the motor's output torque and precisely controlling the vehicle's driving state. The controller determines the requested torque based on driving parameters provided by various sensors within the vehicle, such as accelerator pedal opening, vehicle speed, steering wheel angle, driving mode, and drive mode. However, current methods that determine the requested torque based on current vehicle driving parameters typically assume the vehicle is traveling on a smooth road surface, ignoring situations where the vehicle is traveling on uneven surfaces. When the vehicle is traveling on a bumpy road with potholes, the controller cannot anticipate and adapt to the road conditions, potentially resulting in significant vibrations or bumps, leading to a poor ride smoothness for the driver and passengers. How to solve the problem of poor vehicle ride smoothness caused by the inability to anticipate road conditions has become a pressing issue for those skilled in the art. To address this problem, this application provides a vehicle control method (such as...). Figure 1 As shown), vehicle control devices (such as...) Figure 2 and Figure 10 As shown), electronic equipment 30 (such as...) Figure 12 As shown), vehicle 100 (e.g.) Figure 12 (as shown) and computer-readable storage medium 200 (e.g.) Figure 13 (As shown).

[0049] Please see Figure 1 as well as Figure 2 The vehicle control method according to the embodiments of this application includes:

[0050] 05: Determine the wheel torque based on the vehicle's driving information and dent information;

[0051] 07: Control vehicle movement based on wheel torque;

[0052] 03: Among them, pothole information is obtained based on road condition information.

[0053] The above-described vehicle control method can be applied to a vehicle control device 10. The vehicle control device 10 according to this embodiment includes an acquisition module 11, a determination module 12, and a control module 13. The acquisition module 11 is used to acquire pothole information based on road condition information. The determination module 12 is used to determine wheel torque based on vehicle driving information and pothole information. The control module 13 is used to control vehicle driving based on wheel torque.

[0054] Specifically, the vehicle control device 10 is one of the core control components installed inside the vehicle, responsible for managing and coordinating the operation of various parts of the vehicle. The vehicle control device 10 collects, processes, and analyzes data from various sensors, such as cameras, ranging radar, and infrared ranging devices, to control the operating status of various components of the vehicle, including the engine, transmission, braking system, and steering system, to ensure the safe and efficient operation of the vehicle. The vehicle control device 10 controls the overall performance of the vehicle by adjusting parameters inside the vehicle. In this application, the vehicle control device 10 acquires road condition images through the acquisition module 11, which represent the smoothness of the road surface ahead of the vehicle. Then, the vehicle control device 10 further processes the road condition images through the determination module 12 to determine whether there are potholes on the road surface ahead of the vehicle. If potholes are present, the determination module 12 determines the location and depth of the potholes, and then determines the wheel torque based on the vehicle's driving information, the location of the potholes, and the depth of the potholes. Finally, the vehicle control unit 10 is used to control the vehicle's movement based on wheel torque via the control module 13, thereby improving the smoothness of the vehicle's movement.

[0055] More specifically, the vehicle control device 10 includes an acquisition module 11, a determination module 12, and a control module 13. The acquisition module 11 executes method 03, the determination module 12 executes methods 03 and 05, and the control module 13 executes method 07. The acquisition module 11 is responsible for collecting and processing various driving parameters and status information of the vehicle within the vehicle control device 10. The acquisition module 11 can acquire real-time data through various sensors on the vehicle and transmit this data to the control system to make corresponding decisions based on this information. The determination module 12 is the core module of the vehicle control device 10. The determination module 12 can make decisions and generate the data required by the control module 13 to execute control commands based on the vehicle's driving information and road condition images provided by the acquisition module 11. The determination module 12 analyzes various vehicle information to determine how to adjust key components such as the motor, braking system, and steering system to optimize the vehicle's driving performance and response. The control module 13 is the core module of the vehicle control device 10 responsible for making decisions based on the data provided by the determination module 12 and the acquisition module 11, and controlling the operating status of various systems and components of the vehicle. In this application, the control module 13 is used to control the vehicle's movement based on the wheel torque.

[0056] Furthermore, in method 03, road condition images are first captured using an onboard camera or other visual sensors, and then input into a pre-defined pothole detection model. The pothole detection model is trained based on deep learning or image processing algorithms (such as convolutional neural networks, CNNs) and is capable of identifying pothole features in the image. The pothole detection model analyzes the input road condition image to locate pothole regions. It can extract the shape, location, and other relevant information of potholes from complex road surface images. If the pothole detection model identifies a pothole in the road condition image, it determines the specific location of the pothole (i.e., the coordinates of the corresponding pixel in the image) and the depth of the pothole. The pothole location can be represented by pixel coordinates in the image, while the pothole depth can be estimated using a depth camera, stereo vision technology, or by combining a known road height model. The vehicle control device 10 calculates wheel torque by combining the vehicle's driving information, pothole location, and pothole depth. The vehicle's driving information may include current vehicle speed, accelerator pedal opening, suspension system status, steering angle, and other data. The location and depth of potholes are particularly important for the vehicle's control unit 10, as their size and location directly affect the vibration and bump experience when the vehicle passes over them. The vehicle's control unit 10 needs to adjust the wheel torque output based on the vehicle's driving information, pothole location, and pothole depth to maintain a relatively stable driving state when passing over potholes, thereby improving the ride smoothness. For example, when the vehicle approaches a pothole, the control module 13 of the vehicle's control unit 10 can adjust the wheel torque in advance. By adjusting the torque output of the motor, it can prevent the vehicle from experiencing severe vibrations due to the impact of the pothole, ensuring a smooth ride.

[0057] It is understood that the vehicle control method provided in this application obtains pothole information from road condition images to determine the location and depth of potholes on the road ahead of the vehicle. Then, based on the vehicle's driving information and the pothole information, this application adjusts the wheel torque and controls the vehicle's movement accordingly. This application can detect road conditions in advance and adjust wheel torque in a timely manner when the vehicle passes over potholes to reduce vehicle bumps and optimize the smoothness of the driving process.

[0058] In some implementations, please refer to Figure 2 Road condition information includes road condition images; 03: Pothole information is obtained based on road condition information and includes:

[0059] 031: Based on the road condition image and the preset pothole detection model, obtain pothole information, including pothole location and pothole depth.

[0060] The above-mentioned vehicle control method can be applied to the vehicle control device 10. The acquisition module 11 is used to acquire pothole information based on the road condition image and the preset pothole detection model. The pothole information includes the pothole location and pothole depth.

[0061] Specifically, pothole detection models are trained based on deep learning or image processing algorithms (such as convolutional neural networks, CNNs) and are capable of identifying pothole features in images. The model analyzes the input road condition image to locate pothole regions. It can extract the shape, location, and other relevant information of potholes from complex road surface images. If the model identifies a pothole in the image, it determines its specific location (i.e., the coordinates of the corresponding pixel in the image) and its depth. The pothole location can be represented by pixel coordinates in the image, while the depth can be estimated using depth cameras, stereo vision techniques, or by combining known road height models.

[0062] In some implementations, please refer to Figure 2 , Figure 3 and Figure 4 The preset pothole detection model includes a backbone network, a neck network, and a head network. 031: Based on the road condition image and the preset pothole detection model, pothole information is obtained, including:

[0063] 0311: Acquire road condition images and input them into the backbone network for feature extraction processing to obtain feature-extracted images;

[0064] 0312: Input the feature-extracted image into the neck network for feature fusion processing to obtain the feature-fused image;

[0065] 0313: Input the feature fusion image into the head network for computation and processing to determine whether there are potholes in the road condition image, and if there are potholes in the road condition image, determine the pothole information.

[0066] The aforementioned vehicle control method can be applied to the vehicle control device 10. The acquisition module 11 is used to acquire road condition images. The determination module 12 is used to: input the road condition image into the backbone network for feature extraction processing to obtain a feature-extracted image; input the feature-extracted image into the neck network for feature fusion processing to obtain a feature-fused image; input the feature-fused image into the head network for calculation processing to determine whether there are potholes in the road condition image, and if there are potholes in the road condition image, determine the pothole information.

[0067] Specifically, please combine Figure 4The backbone network module is primarily used for feature extraction from the preview images of the left and right cameras of a stereo camera. Backbone network types include, but are not limited to, lightweight network models such as MobileNet, SqueezeNet, EfficientNet, ShuffleNet, and GhostNet. The backbone network is the core of the pre-defined pit detection model, mainly used to extract basic and effective features from the input data. In image processing, the backbone network typically employs a classic Convolutional Neural Network (CNN) architecture (such as ResNet and VGG). Road condition images are first processed by the backbone network, which performs a series of convolutional operations on the image, transforming the raw pixel information into a more abstract feature representation (such as edges, textures, and shapes). These extracted features contain important information from the image, enabling subsequent steps to perform deeper analysis.

[0068] Specifically, the neck network module is mainly used to collect feature maps from different stages of the backbone network for multi-scale feature fusion. Types of neck networks include, but are not limited to, Spatial Pyramid Pooling (SPP), Dilated Spatial Convolutional Pooling Pyramid (ASPP), Feature Pyramid Network (FPN), and Path Aggregation Network (PAN). The neck network is typically used to further process, integrate, or fuse features extracted by the backbone network. In multi-stage detection models, the neck network's role is to improve feature representation by combining feature information from different levels. The neck network may use structures such as FPN (Feature Pyramid Network) or PAN (Path Aggregation Network) to fuse features at different scales to capture richer details. The neck network fuses features from different levels obtained from the backbone network, ensuring the model can handle image information at different scales and capture more complex image details. For example, shallower layers capture edge or local information, while deeper layers capture more complex structural or contextual information. By fusing this information, the model can more accurately identify features such as pits in the image.

[0069] Specifically, the head network is used to predict the category of an object and its location. In the pit detection model, the head network is responsible for calculating specific tasks from the fused features. In pit detection, the head network's task is to make the final decision output based on the fused feature information, such as determining whether pits exist in the image, their location, and their depth. The head network determines whether an image contains pits through classification or regression operations. This process involves the final analysis of the image to determine the presence, location, and depth of pits. The depth of a pit can be predicted using regression methods, while its location is typically determined through coordinate regression. Once a pit is identified, the pit detection model further analyzes its specific location within the image. The pit location is represented by the pixel coordinates of the image. Besides the pit location, the extraction of pit depth is more complex, and this process will be explained in detail below.

[0070] Please see Figure 2 and Figure 4 In some implementations, 031: Before acquiring a road condition image and inputting the road condition image into the backbone network for feature extraction processing to obtain a feature-extracted image, the control method further includes processing the road condition image using a UV parallax algorithm.

[0071] The above-mentioned vehicle control method can be applied to the vehicle control device 10, and the determination module 12 is used to process road condition images using the UV parallax algorithm.

[0072] Understandably, the UV parallax algorithm can be used to remove large obstacles on the road surface in road condition images, including but not limited to vehicles and pedestrians. This can improve the accuracy of subsequent image processing.

[0073] In some implementations, please refer to Figure 2 , Figure 4 as well as Figure 5 The feature fusion image includes left-view feature maps and right-view feature maps. The pit information includes pit locations. 0313: The feature fusion image is input into the head network for computation and processing to determine the pit information, including:

[0074] 03131: Input the left and right view feature maps into the head network, and subtract the gray value of the left view feature map from the gray value of the right view feature map to obtain the left disparity map;

[0075] 03132: Determine the location of the pit based on the left parallax diagram.

[0076] The above-mentioned vehicle control method can be applied to the vehicle control device 10. The determination module 12 is used to input the left-view feature map and the right-view feature map into the head network, and subtract the gray value of the right-view feature map from the gray value of the left-view feature map to obtain the left disparity map; the position of the pit is determined according to the left disparity map.

[0077] Specifically, in a stereo vision system, two cameras (or sensors, or binocular cameras) are used to capture images of the same scene from different angles, resulting in two viewpoints, referred to as the "left view image" and the "right view image." These two viewpoints have a certain spatial deviation, thus, the depth information of the scene can be calculated by comparing these images.

[0078] More specifically, the head network in deep learning is typically the part that performs specific tasks (such as classification, regression, etc.). Here, the head network takes the features of road condition images from two perspectives as input and processes them further. By comparing the road condition images from these two perspectives, the head network can obtain the depth information of the road condition images. The determination module 12 obtains the left disparity map by subtracting the gray values ​​of the right-view feature map from the gray values ​​of the left-view feature map. Each pixel in the left disparity map represents the disparity at the corresponding position in the left and right views, i.e., the distance difference between the object and the camera. The larger the disparity, the closer the object; the smaller the disparity, the farther away the object.

[0079] In some implementations, please refer to Figure 2 , Figure 4 as well as Figure 5 The feature fusion image includes left-view feature maps and right-view feature maps. The pit information includes pit depth. 0313: The feature fusion image is input into the head network for computation and processing to determine the pit information, including:

[0080] 03131: Input the left and right view feature maps into the head network, and subtract the gray value of the left view feature map from the gray value of the right view feature map to obtain the left disparity map;

[0081] 03133: Fit the preset road surface equation based on the left disparity map to obtain the target road surface equation;

[0082] 03134: Determine the pothole depth based on the target road surface equation.

[0083] The above-mentioned vehicle control method can be applied to the vehicle control device 10. The determination module 12 is used to: input the left-view feature map and the right-view feature map into the head network, and subtract the gray value of the right-view feature map from the gray value of the left-view feature map to obtain the left disparity map; fit the preset road surface equation according to the left disparity map to obtain the target road surface equation; and determine the pothole depth according to the target road surface equation.

[0084] Specifically, the road surface equation is used to describe the geometry of the road surface. The road surface equation can be used to express the shape of the ground or road surface in space. In this application, the preset road surface equation is Ax + By + Cz + D = 0 (C ≠ 0). The determination module 12 arbitrarily selects three pixels from the left disparity map and substitutes them into the preset road surface equation to obtain the fitted road surface equation, i.e., the target road surface equation. The determination module analyzes the left disparity map to identify areas with deeper depth values ​​in the image, thereby determining the location and depth of the potholes. This process will be explained in detail below.

[0085] In some implementations, please refer to Figure 2 , Figure 4 and Figure 6 The feature fusion image includes left-view feature maps and right-view feature maps. The pit information includes pit locations. 0313: The feature fusion image is input into the head network for computation and processing to determine the pit information, including:

[0086] 03135: Input the left and right view feature maps into the head network, and subtract the gray value of the right view feature map from the gray value of the left view feature map to obtain the right disparity map;

[0087] 03136: Determine the location of the pit based on the right parallax diagram.

[0088] The above-mentioned vehicle control method can be applied to the vehicle control device 10. The determination module 12 is used to: input the left-view feature map and the right-view feature map into the head network, and subtract the gray value of the left-view feature map from the gray value of the right-view feature map to obtain the right disparity map; and determine the position of the pit based on the right disparity map.

[0089] Specifically, in a stereo vision system, two cameras (or sensors, or binocular cameras) are used to capture images of the same scene from different angles, resulting in two viewpoints, referred to as the "left view image" and the "right view image." These two viewpoints have a certain spatial deviation, thus, the depth information of the scene can be calculated by comparing these images.

[0090] More specifically, the head network in deep learning is typically the part that performs specific tasks (such as classification, regression, etc.). Here, the head network takes the features of road condition images from two perspectives as input and processes them further. By comparing the road condition images from these two perspectives, the head network can obtain the depth information of the road condition images. The determination module 12 obtains the right disparity map by subtracting the gray values ​​of the left-view feature map from the gray values ​​of the right-view feature map. Each pixel in the right disparity map represents the disparity at the corresponding position in the left and right views, i.e., the distance difference between the object and the camera. The larger the disparity, the closer the object; the smaller the disparity, the farther away the object.

[0091] In some implementations, please refer to Figure 2 , Figure 4 and Figure 6 The feature fusion image includes left-view feature maps and right-view feature maps. The pit information includes pit depth. The feature fusion image is input into the head network for computation and processing to determine the pit information, including:

[0092] 03135: Input the left and right view feature maps into the head network, and subtract the gray value of the right view feature map from the gray value of the left view feature map to obtain the right disparity map;

[0093] 03137: Fit the preset road surface equation based on the right disparity map to obtain the target road surface equation;

[0094] 03138: Determine the pothole depth based on the target road surface equation.

[0095] The above-mentioned vehicle control method can be applied to the vehicle control device 10. The determination module 12 is used to input the left-view feature map and the right-view feature map into the head network, and subtract the gray value of the left-view feature map from the gray value of the right-view feature map to obtain the right disparity map; fit the preset road surface equation according to the right disparity map to obtain the target road surface equation; and determine the pothole depth according to the target road surface equation.

[0096] Specifically, the road surface equation is used to describe the geometry of the road surface. The road surface equation can be used to express the shape of the ground or road surface in space. In this application, the preset road surface equation is Ax + By + Cz + D = 0 (C ≠ 0), which can be transformed into z = a0x + a1y + a2, where a0 = -A / C, a1 = -B / C, and a2 = -D / C. The determining module 12 arbitrarily selects three pixels (e.g., (x...)) from the left disparity map. i ,y i ,z i Substituting the values ​​of i (i = 0, 1, ..., n-1, and n ≥ 3) into the preset road surface equation for fitting, and minimizing the sum of the squared differences between the Z values ​​read from the pixels and the Z values ​​obtained from the road surface equation transformation, the fitted road surface equation, i.e., the target road surface equation, is obtained. The determination module identifies areas with deeper depth values ​​in the image by analyzing the right disparity map, thereby determining the location and depth of the potholes. This process will be explained in detail below.

[0097] Please see Figure 2 and Figure 4 In some implementations, 03138: determining the pothole depth based on the target road surface equation includes:

[0098] 031381: Substitute the target road surface equation into the preset pothole depth formula to determine the pothole depth.

[0099] The above-mentioned vehicle control method can be applied to the vehicle control device 10. The determination module 12 is used to substitute the target road surface equation into the preset pothole depth formula to determine the pothole depth.

[0100] Specifically, the formula for pit depth in, Where H is the pit depth, (X,Y,Z) are the pixel coordinates, D is the image depth, f is the focal length, b is the baseline distance, and DSI is the gray value of the corresponding pixel in the left or right disparity map.

[0101] Please see Figure 2 , Figure 7 and Figure 8 In some implementations, the driving information includes vehicle speed; 05: Based on the driving information and the pothole information, determine the wheel torque, including:

[0102] 051: Determine the estimated time based on driving information and dent information;

[0103] 053: Determine wheel torque based on driving information, current time, and expected time;

[0104] The expected time is later than the current time.

[0105] The above-mentioned vehicle control method can be applied to the vehicle control device 10. The determining module 12 is used to: determine the expected time based on driving information and dent information; determine the wheel torque based on driving information, current time and expected time; wherein the expected time is later than the current time.

[0106] Specifically, please combine Figure 7 , Figure 7 The binocular pre-aiming control system is the vehicle control device 10 in this application. After determining the location of the pothole, the determining module 12 can determine the distance between the wheel and the pothole based on the pothole location. Then, based on the vehicle speed, the time required for the vehicle to travel to the pothole location can be obtained, i.e., the arrival time mentioned above. The arrival time is... Figure 7 In Chinese, t is used to represent... Figure 7 In this context, τ represents the time required for advance acquisition of road surface information, i.e., the system delay time of the vehicle's control device 10. When the vehicle's travel time equals its arrival time, the determination module 12 determines the wheel torque based on the travel information and the depth of the pothole, thereby enabling control of the wheels through the control module 13.

[0107] In some implementations, please refer to Figure 2 Driving information includes driving speed, and pothole information includes the location of the pothole. Based on the driving information and pothole information, the estimated time is determined, including:

[0108] 0511: Determine the expected time based on the driving speed and the location of the pothole. The expected time is the arrival time when the vehicle reaches the location of the pothole.

[0109] The above-mentioned vehicle control method can be applied to the vehicle control device 10. The determining module 12 is used to: determine the expected time based on the driving speed and the position of the pothole. The expected time is the arrival time when the vehicle travels to the position of the pothole.

[0110] Understandably, if the vehicle is traveling on a normal, flat road surface and there is a pothole in front of it, the determining module 12 will determine the distance between the pothole and the vehicle based on the location of the pothole, and then determine the expected time required for the vehicle to come into contact with the pothole based on the vehicle's speed.

[0111] In some implementations, please refer to Figure 2 and Figure 9 When the current time equals the expected time, 053: Determine the wheel torque based on driving information, the current time, and the expected time, including:

[0112] 0531: Given that the current time equals the expected time, determine the total torque required for the pothole based on driving information;

[0113] 0532: Distribute the total torque required for the dent according to the preset distribution coefficient to determine the wheel torque required for the dent and the yaw torque of the dent.

[0114] The driving information includes at least one of the following: motor speed, accelerator pedal opening, wheel track, effective wheel radius, steering wheel angle, and brake pedal opening. The distribution coefficient includes at least one of the following: front wheel torque coefficient, left rear wheel torque coefficient, and right rear wheel torque coefficient. The required wheel torque for the pothole includes at least one of the following: pothole front axle torque, pothole left rear wheel torque, and pothole right rear wheel torque.

[0115] The above-mentioned vehicle control method can be applied to the vehicle control device 10. The determining module 12 is used to: determine the total torque required for the pothole based on the driving information when the current time is equal to the expected time; and allocate the total torque required for the pothole according to the preset allocation coefficient to determine the wheel torque required for the pothole and the yaw torque of the pothole. The driving information includes at least one of the following: motor speed, accelerator pedal opening, wheel track, effective wheel radius, steering wheel angle, and brake pedal opening. The allocation coefficient includes at least one of the following: front wheel torque coefficient, left rear wheel torque coefficient, and right rear wheel torque coefficient. The wheel torque required for the pothole includes at least one of the following: front axle torque for the pothole, left rear wheel torque for the pothole, and right rear wheel torque for the pothole.

[0116] Specifically, if the current time equals the expected time, it indicates that the vehicle is driving over a pothole. This application can take a three-motor vehicle as an example. A three-motor vehicle includes a front axle motor, a left rear wheel motor, and a right rear wheel motor. Therefore, it is necessary to distribute the total required torque to the front axle motor, left rear wheel motor, and right rear wheel motor according to the distribution coefficients, thereby achieving control of the wheel torque. The distribution coefficients include the front wheel torque coefficient, the left rear wheel torque coefficient, and the right rear wheel torque coefficient. The values ​​of these three coefficients correspond to the pothole depth. The determining module 12 can determine the value of the distribution coefficient to be used based on the pothole depth and the correspondence between the pothole depth and the distribution coefficients. Then, based on the total required torque, motor speed, accelerator pedal opening, wheel track, effective wheel radius, steering wheel angle, brake pedal opening, and distribution coefficients, it determines the pothole yaw torque, pothole front axle torque, pothole left rear wheel torque, and pothole right rear wheel torque.

[0117] Specifically, the total required torque can be determined using the following formula, which also determines the yaw torque, front wheel torque, left rear wheel torque, and right rear wheel torque: P F =(T F ×n F ) / 9550; P R1 =(T R1 ×n R1 ) / 9550; P R2 =(T R2 ×n R2 ) / 9550; P N =P F +P R1 +P R2 T = λT F +(1-λ)T F +T R1 +T R2 ; T F =λ1T;T R1 +T R2 =(1-λ1)T;T R1 / T R2 λ2; where P F For the required power of the front axle motor, T F n is the torque of the front axle motor. F P is the speed of the front axle motor. R1 For the power required by the left rear wheel motor, T R1 n represents the torque of the left rear wheel motor. R1 P represents the rotational speed of the left rear wheel motor. R2 For the power required by the right rear wheel motor, T R2 The torque of the right rear wheel motor, n R2 P represents the rotational speed of the right rear wheel motor. NLet λ1 and λ2 be the total power demand of the vehicle, and D be the torque distribution coefficients. lr R is the wheelbase of the left and right wheels, R is the effective radius of the wheel, T is the total required torque, and M is the yaw torque.

[0118] Please see Figure 2 In some implementations, 07: controlling vehicle movement based on wheel torque includes:

[0119] 071: Control vehicle movement based on wheel torque required for potholes and yaw torque of potholes.

[0120] The above-mentioned vehicle control method can be applied to the vehicle control device 10, and the control module 13 is used to control the vehicle driving according to the required wheel torque and yaw torque of the dent.

[0121] Understandably, when the vehicle is driving over a pothole, the control module 13 directly controls the vehicle's movement based on the required wheel torque and the yaw torque of the pothole, thereby reducing the vibration amplitude of the vehicle.

[0122] Please see Figure 2 In some implementations, when the current time is not equal to the expected time, 053: determine the wheel torque based on driving information, the current time, and the expected time, including:

[0123] 0533: When the current time is not equal to the expected time, determine the total demand torque for the plane based on the driving information. The total demand torque for the plane includes the demand wheel torque for the plane and the yaw torque for the plane. The current driving information includes at least one of the following: motor speed, accelerator pedal opening, wheel track, effective wheel radius, steering wheel angle, and brake pedal opening. The demand wheel torque for the plane includes at least one of the following: the front axle torque for the plane, the left rear wheel torque for the plane, and the right rear wheel torque for the plane. The front axle torque for the plane is the sum of the left rear wheel torque for the plane and the right rear wheel torque for the plane. The left rear wheel torque for the plane is equal to the right rear wheel torque for the plane.

[0124] The above-described vehicle control method can be applied to the vehicle control device 10. The determining module 12 is used to determine the total planar torque demand based on driving information when the current time is not equal to the expected time. The total planar torque demand includes the planar wheel torque demand and the planar yaw torque. The current driving information includes at least one of the following: motor speed, accelerator pedal opening, wheel track, effective wheel radius, steering wheel angle, and brake pedal opening. The planar wheel torque demand includes at least one of the planar front axle torque, the planar left rear wheel torque, and the planar right rear wheel torque. The planar front axle torque is the sum of the planar left rear wheel torque and the planar right rear wheel torque, and the planar left rear wheel torque is equal to the planar right rear wheel torque.

[0125] Specifically, if the current time is not equal to the expected time, it indicates that the vehicle is not driving over a pothole, but there is a pothole in front of the vehicle. When the vehicle is driving on a normal, flat road surface, the required wheel torque for the flat surface includes the front axle torque, the left rear wheel torque, and the right rear wheel torque. The front axle torque is the sum of the left and right rear wheel torques, and the left rear wheel torque is equal to the right rear wheel torque.

[0126] Please see Figure 2 In some implementations, 07: controlling vehicle movement based on wheel torque includes:

[0127] 073: Control vehicle movement based on the wheel torque and yaw torque required for planar operation.

[0128] The above-mentioned vehicle control method can be applied to the vehicle control device 10, and the control module 13 is used to control the vehicle's movement according to the planar demand wheel torque and planar yaw torque.

[0129] Please see Figure 2 , Figure 10 and Figure 11 In some implementations, the vehicle is equipped with a binocular camera, and the control method further includes:

[0130] 02: Use a binocular camera to capture road condition images to obtain road condition information.

[0131] The above-mentioned vehicle control method can be applied to the vehicle control device 10, and the acquisition module 11 is used to capture road condition images using a binocular camera to obtain road condition information.

[0132] Specifically, the binocular camera 40 consists of two independent cameras with a structure similar to human eyes. The binocular camera 40 can simultaneously capture images of the same scene from two different perspectives. The binocular camera 40 is used in fields such as robot vision, intelligent driving, drones, measurement, and security monitoring. In this application, the binocular camera 40 is mounted on the vehicle's control device 10 and configured to capture images in front of the vehicle's control device 10.

[0133] Please see Figure 2 and Figure 11 In some embodiments, the vehicle is equipped with a binocular camera 40, and the control method further includes:

[0134] 01: Calibrate the binocular camera 40 according to the preset camera calibration algorithm.

[0135] The above-mentioned vehicle control method can be applied to the vehicle control device 10. The acquisition module 11 is used to: acquire a preset camera calibration algorithm; and calibrate the binocular camera 40 according to the preset camera calibration algorithm.

[0136] Specifically, the calibration algorithm for the binocular camera 40 can be the Zhang Zhengyou checkerboard calibration method. The Zhang Zhengyou checkerboard calibration method can be used to calibrate the intrinsic and extrinsic parameters of the binocular camera 40.

[0137] In summary, the vehicle control method provided in this application determines whether potholes exist in the acquired road condition image by inputting it into a preset pothole detection model, thereby determining whether potholes exist on the road ahead of the vehicle. If potholes are present in the road condition image, i.e., if potholes exist on the road ahead of the vehicle, the preset pothole detection model determines the location and depth of the potholes. Subsequently, this application adjusts the wheel torque based on the vehicle's driving information, the pothole location, and the pothole depth. This application can detect road conditions in advance and adjust the wheel torque in a timely manner when the vehicle passes over potholes to reduce vehicle bumps and optimize the smoothness of the vehicle's driving process.

[0138] In some implementations, please refer to Figure 12 This application also provides an electronic device 30, which includes a memory and a processor. The memory is configured to store a computer program, and the processor, when executing the computer program, implements the control method in any of the above embodiments.

[0139] For example, when the processor of electronic device 30 executes a computer program stored in memory, the following control method is implemented:

[0140] 05: Determine the wheel torque based on the vehicle's driving information and dent information;

[0141] 07: Control vehicle movement based on wheel torque;

[0142] 03: Among them, pothole information is obtained based on road condition information.

[0143] For example, when the processor of electronic device 30 executes a computer program stored in memory, it implements the following control method:

[0144] 031: Based on the road condition image and the preset pothole detection model, obtain pothole information, including pothole location and pothole depth.

[0145] For example, when the processor of electronic device 30 executes the computer program stored in the memory, it can also implement the control methods in 01, 02, 0311, 0312, 0313, 0314, 0315, 0316, 0317, 0318, 03191, 03192, 03193, 03194, 051, 0511, 053, 0531, 0532, 0533, 071, and 073.

[0146] In some implementations, please refer to Figure 12This application also provides a vehicle 100, including the vehicle control device 10 of any of the above embodiments or the electronic device 30 of any of the above embodiments.

[0147] Please see Figure 13 In some embodiments, this application also provides a computer-readable storage medium 200 storing a computer program 202 that, when executed by a processor, implements the control method in any of the above embodiments.

[0148] For example, when computer program 202 is executed by processor 20, the following control method is implemented:

[0149] 05: Determine the wheel torque based on the vehicle's driving information and dent information;

[0150] 07: Control vehicle movement based on wheel torque;

[0151] 03: Among them, pothole information is obtained based on road condition information.

[0152] For example, when computer program 202 is executed by processor 20, the following control method is implemented:

[0153] 031: Based on the road condition image and the preset pothole detection model, obtain pothole information, including pothole location and pothole depth.

[0154] For example, when computer program 202 is executed by processor 20, it can also implement the control methods in 01, 02, 0311, 0312, 0313, 03131, 03132, 03133, 03134, 03135, 03136, 03137, 03138, 031381, 051, 0511, 053, 0531, 0532, 0533, 071, and 073.

[0155] In the computer-readable storage medium 200 of this application, the presence of potholes in the acquired road condition image is determined by inputting it into a preset pothole detection model, thereby determining whether potholes exist on the road ahead of the vehicle. If potholes are present in the road condition image, i.e., if potholes exist on the road ahead of the vehicle, the preset pothole detection model determines the location and depth of the potholes. Subsequently, this application adjusts the wheel torque based on the vehicle's driving information, the pothole location, and the pothole depth. This application can detect road conditions in advance and adjust the wheel torque in a timely manner when the vehicle passes over potholes to reduce vehicle bumps and optimize the smoothness of the vehicle's ride.

[0156] In the description of this specification, the references to terms such as "some embodiments," "in one example," "exemplarily," etc., indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0157] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the function involved, as will be understood by those skilled in the art to which embodiments of this application pertain.

[0158] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A method for controlling a vehicle, characterized in that, include: Determine the wheel torque based on the dent information and vehicle driving information; The vehicle's movement is controlled based on the wheel torque. The pothole information is obtained based on road condition information.

2. The control method according to claim 1, characterized in that, The road condition information includes road condition images, and the pothole information is obtained based on the road condition information, including: Based on the road condition image and the preset pothole detection model, pothole information is obtained.

3. The control method according to claim 2, characterized in that, The preset pothole detection model includes a backbone network, a neck network, and a head network. The step of obtaining pothole information based on the road condition image and the preset pothole detection model includes: Acquire road condition images and input the road condition images into the backbone network for feature extraction processing to obtain feature-extracted images; The extracted feature image is input into the neck network for feature fusion processing to obtain a fused feature image; The feature-fused image is input into the head network for computation and processing to determine the pit information.

4. The control method according to claim 3, characterized in that, The feature fusion image includes a left-view feature map and a right-view feature map, and the pit information includes the pit location. The step of inputting the feature fusion image into a head network for computational processing to determine the pit information includes: The left-view feature map and the right-view feature map are input into the head network, and the gray value of the right-view feature map is subtracted from the gray value of the left-view feature map to obtain the left disparity map. The location of the pit is determined based on the left parallax diagram.

5. The control method according to claim 3, characterized in that, The feature fusion image includes a left-view feature map and a right-view feature map, and the pit information includes pit depth. The step of inputting the feature fusion image into a head network for computational processing to determine the pit information includes: The left-view feature map and the right-view feature map are input into the head network, and the gray value of the right-view feature map is subtracted from the gray value of the left-view feature map to obtain the left disparity map. The target road surface equation is obtained by fitting a preset road surface equation to the left disparity map; The depth of the pothole is determined based on the target road surface equation.

6. The control method according to claim 3, characterized in that, The feature fusion image includes a left-view feature map and a right-view feature map, and the pit information includes the pit location. The step of inputting the feature fusion image into a head network for computational processing to determine the pit information includes: The left-view feature map and the right-view feature map are input into the head network, and the gray value of the left-view feature map is subtracted from the gray value of the right-view feature map to obtain the right disparity map. The location of the pit is determined based on the right parallax diagram.

7. The control method according to claim 3, characterized in that, The feature fusion image includes a left-view feature map and a right-view feature map, and the pit information includes pit depth. The step of inputting the feature fusion image into a head network for computational processing to determine the pit information includes: The left-view feature map and the right-view feature map are input into the head network, and the gray value of the left-view feature map is subtracted from the gray value of the right-view feature map to obtain the right disparity map. The target road surface equation is obtained by fitting a preset road surface equation to the right disparity map; The depth of the pothole is determined based on the target road surface equation.

8. The control method according to claim 5 or 7, characterized in that, Determining the pothole depth based on the target road surface equation includes: The target road surface equation is substituted into a preset pothole depth formula to determine the pothole depth.

9. The control method according to claim 1, characterized in that, The step of determining the wheel torque based on the driving information and the pothole information includes: The expected time is determined based on the driving information and the pothole information; The wheel torque is determined based on the driving information, the current time, and the expected time. The expected time is later than the current time.

10. The control method according to claim 9, characterized in that, The driving information includes driving speed, the pothole information includes pothole location, and the step of determining the expected time based on the driving information and the pothole information includes: The expected time is determined based on the driving speed and the location of the pothole, whereby the expected time is the arrival time of the vehicle when it reaches the location of the pothole.

11. The control method according to claim 9, characterized in that, Determining the wheel torque based on the driving information, the current time, and the expected time includes: If the current time equals the expected time, the total torque required for the pothole is determined based on the driving information; The total torque required for the dent is allocated according to a preset allocation coefficient to determine the required wheel torque and yaw torque for the dent. The driving information includes at least one of motor speed, accelerator pedal opening, wheel track, effective wheel radius, steering wheel angle, and brake pedal opening. The distribution coefficient includes at least one of front wheel torque coefficient, left rear wheel torque coefficient, and right rear wheel torque coefficient. The required wheel torque for the pothole includes at least one of pothole front axle torque, pothole left rear wheel torque, and pothole right rear wheel torque.

12. The control method according to claim 11, characterized in that, The method of controlling vehicle movement based on the wheel torque includes: The vehicle's movement is controlled based on the required wheel torque and the yaw torque of the pothole.

13. The control method according to claim 9, characterized in that, Determining the wheel torque based on the driving information, the current time, and the expected time includes: If the current time is not equal to the expected time, the total planar torque demand is determined based on the driving information. The total planar torque demand includes the planar wheel torque demand and the planar yaw torque. The current driving information includes at least one of the following: motor speed, accelerator pedal opening, wheel track, effective wheel radius, steering wheel angle, and brake pedal opening. The required wheel torque for the plane includes at least one of the following: plane front axle torque, plane left rear wheel torque, and plane right rear wheel torque. The plane front axle torque is the sum of the plane left rear wheel torque and the plane right rear wheel torque, and the plane left rear wheel torque is equal to the plane right rear wheel torque.

14. The control method according to claim 13, characterized in that, The method of controlling vehicle movement based on the wheel torque includes: The vehicle's movement is controlled based on the required wheel torque and the yaw torque of the plane.

15. The control method according to any one of claims 1-14, characterized in that, The vehicle is equipped with a binocular camera, and the control method further includes: The binocular camera is used to capture road condition images to obtain the road condition information.

16. The control method according to any one of claims 1-14, characterized in that, The vehicle is equipped with a binocular camera, and the control method further includes: The binocular camera is calibrated according to a preset camera calibration algorithm.

17. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory being configured to store a computer program, and the processor, when executing the computer program, implementing the control method according to any one of claims 1-16.

18. A vehicle, characterized in that, The electronic device included in claim 17.

19. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the control method according to any one of claims 1-16.

Images