Vehicle control method and device

Abstract

The embodiment of the invention discloses a vehicle control method and device, and the method comprises the steps: obtaining an environment image of a vehicle, processing the environment image through a terrain recognition model obtained through pre-training, determining a predicted terrain corresponding to the environment image, and enabling the terrain recognition model to be a machine learning model, the predicted terrain is rock, snowfield, sand, city or mud, and the terrain recognition model is trained in advance, so that the terrain recognition model has accurate terrain recognition capability, and the target driving mode of the vehicle can be determined according to the predicted terrain, and the target driving mode is a rock mode, a snowfield mode, a sand mode, an economic mode or a mud mode. Thus, the vehicle is controlled according to the target driving mode, accurate and real-time driving mode switching of the vehicle for the terrain is achieved, and the user experience is improved.

Description

Technical Field

[0001] The present invention relates to the field of automobiles, and in particular to a vehicle control method and device. Background Art

[0002] All-terrain system (ATS) is a control system that enables vehicles to drive stably in different terrains (such as snow, desert, etc.) by switching different driving modes. It is commonly configured in high-end four-wheel drive off-road vehicles. The current mainstream all-terrain switching is mainly switched by manual knobs, and different driving modes can be switched through manual knobs to adapt to different terrains. Some models can also determine the terrain based on the slippage of the four wheels of the vehicle, and then realize automatic switching of driving modes. However, this method of determining terrain has certain limitations in accuracy, so it is impossible to achieve accurate driving mode switching based on terrain. Summary of the invention

[0003] In order to solve the above technical problems, the embodiments of the present application provide a vehicle control method and device, which can perform terrain prediction by processing environmental images through a terrain recognition model, thereby realizing automatic switching of the vehicle's driving mode and improving user experience.

[0004] The present application provides a vehicle control method, the method comprising:

[0005] Acquire an environmental image of the vehicle;

[0006] Processing the environment image using a pre-trained terrain recognition model to determine a predicted terrain corresponding to the environment image, wherein the terrain recognition model is a machine learning model, and the predicted terrain is rock, snow, sand, city, or mud;

[0007] determining a target driving mode of the vehicle according to the predicted terrain, the target driving mode being a rock mode, a snow mode, a sand mode, an economy mode or a mud mode;

[0008] The vehicle is controlled according to the target driving mode.

[0009] Optionally, determining the target driving mode of the vehicle according to the predicted terrain includes:

[0010] A target driving mode of the vehicle is determined based on at least one of power information from a power control unit and driving information from a motion control unit, and the predicted terrain.

[0011] Optionally, controlling the vehicle according to the target driving mode includes:

[0012] The vehicle is controlled according to the target driving mode and a control signal from a vehicle control unit.

[0013] Optionally, the method further includes:

[0014] The initial model is trained using training images so that the initial model is converted into the terrain recognition model, and at least two images in the training images have different disturbance factors, and the disturbance factors include at least one of weather, lighting, and shooting angle.

[0015] Optionally, the method further includes:

[0016] After adding category labels to the environment image, the terrain recognition model is optimized using the environment image with the category labels.

[0017] Optionally, the terrain recognition model includes multiple groups of feature processing layers and multiple fully connected layers connected in sequence, and the feature processing layer includes a convolution layer, an activation function layer, a normalization layer and a maximum pooling layer.

[0018] The embodiment of the present application provides a vehicle control device, characterized in that the device includes:

[0019] An image acquisition unit, used to acquire an environmental image of the vehicle;

[0020] A terrain prediction unit, used to process the environment image using a pre-trained terrain recognition model to determine a predicted terrain corresponding to the environment image, wherein the terrain recognition model is a machine learning model, and the predicted terrain is rock, snow, sand, city or mud;

[0021] a mode determination unit, configured to determine a target driving mode of the vehicle according to the predicted terrain, the target driving mode being a rock mode, a snow mode, a sand mode, an economy mode or a mud mode;

[0022] A control unit is used to control the vehicle according to the target driving mode.

[0023] Optionally, the mode determination unit is specifically used to:

[0024] A target driving mode of the vehicle is determined based on at least one of power information from a power control unit and driving information from a motion control unit, and the predicted terrain.

[0025] Optionally, the control unit is specifically used for:

[0026] The vehicle is controlled according to the target driving mode and a control signal from a vehicle control unit.

[0027] Optionally, the device further comprises:

[0028] A training unit is used to train an initial model using training images so that the initial model is converted into the terrain recognition model, at least two images in the training images have different disturbance factors, and the disturbance factors include at least one of weather, lighting, and shooting angle.

[0029] Optionally, the device further comprises:

[0030] After adding category labels to the environment image, the terrain recognition model is optimized using the environment image with the category labels.

[0031] Optionally, the terrain recognition model includes multiple groups of feature processing layers and multiple fully connected layers connected in sequence, and the feature processing layer includes a convolution layer, an activation function layer, a normalization layer and a maximum pooling layer.

[0032] An embodiment of the present application provides a vehicle control method and device. After acquiring an environmental image of the vehicle, the environmental image can be processed using a pre-trained terrain recognition model to determine a predicted terrain corresponding to the environmental image. The terrain recognition model can be a machine learning model that predicts the terrain to be rock, snow, sand, city, or mud. Since the terrain recognition model has been pre-trained, it has accurate terrain recognition capabilities. In this way, the target driving mode of the vehicle can be determined based on the predicted terrain. The target driving mode is rock mode, snow mode, sand mode, economy mode, or mud mode. The vehicle is then controlled based on the target driving mode, thereby achieving accurate and real-time driving mode switching of the vehicle according to the terrain, thereby improving user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0034] Figure 1 A flow chart of a vehicle control method provided in an embodiment of the present application;

[0035] Figure 2 A schematic diagram of a system architecture provided for an embodiment of the present application;

[0036] Figure 3 A schematic diagram of a training image provided in an embodiment of the present application;

[0037] Figure 4 A schematic diagram of a terrain recognition method provided in an embodiment of the present application;

[0038] Figure 5 A structural block diagram of a vehicle control device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0039] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are described in detail below with reference to the accompanying drawings.

[0040] In the following description, many specific details are set forth to facilitate a full understanding of the present application, but the present application may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present application. Therefore, the present application is not limited to the specific embodiments disclosed below.

[0041] The specific implementation of a vehicle control method and device provided by the embodiments of the present application is described in detail below with reference to the accompanying drawings.

[0042] refer to Figure 1 As shown, a flow chart of a vehicle control method provided in an embodiment of the present application may include the following steps.

[0043] S101, obtaining an environmental image of the vehicle.

[0044] In the embodiment of the present application, an environmental image of the vehicle can be obtained, and the environmental image can include at least one of the environmental image in front of the vehicle and the environmental image around the vehicle. The environmental image is used to reflect the characteristics of the environment in which the vehicle is located, and thus can be used for terrain recognition. The terrain can include rocks, snow, sand, cities, or mud. Different driving modes can be used under different terrains. The vehicle has different parameters under different driving modes, so that it can better adapt to the terrain characteristics.

[0045] The driving mode module that controls the driving mode of the vehicle based on the terrain can be called an all-terrain control module. The all-terrain control module can be a functional module in an intelligent driving system (ADAS, Advance Driving Assistance System), or a functional module outside the intelligent driving system. The environmental image can be acquired by a camera connected to the all-terrain control module. For example, the front camera can be used to acquire the environmental image in front of the vehicle, and other cameras can be used to acquire the environmental image behind or to the side of the vehicle. In this way, environmental images in different directions of the vehicle can be acquired by cameras at different positions. The environmental image can include multiple images, and multiple environmental images can be acquired by periodic camera acquisition, which can include images in the same direction of the vehicle or images in different directions of the vehicle.

[0046] S102, using a pre-trained terrain recognition model to process the environment image, and determine a predicted terrain corresponding to the environment image.

[0047] In an embodiment of the present application, after acquiring an environmental image, the environmental image can be processed using a pre-trained terrain recognition model to determine the predicted terrain corresponding to the environmental image. Since the terrain recognition model has been pre-trained, it has good terrain recognition capabilities, so that terrain recognition can be performed quickly and accurately based on the environmental image, thereby achieving terrain prediction.

[0048] Specifically, the predicted terrain can be rock, snow, sand, city or mud, etc. Different terrains have different pixel distributions, so the corresponding predicted terrain can be determined by recognizing the environmental image. The terrain recognition model can recognize the aforementioned terrains. The terrain recognition is comprehensive and can cover the terrain that may be encountered during off-road driving. It can also recognize urban terrain and traffic. Compared with obtaining images of part of the outdoor terrain through a robot and only recognizing part of the outdoor terrain such as grass and paved roads through a model, it has certain advantages.

[0049] The terrain recognition model can be a machine learning model, for example, a deep convolutional neural network (CNN), such as the Alexnet model. Figure 2 As shown, it is a schematic diagram of a system architecture provided by an embodiment of the present application, wherein the all-terrain control module may include a terrain recognition unit. When the terrain recognition model is a CNN model, the terrain recognition unit may be recorded as a CNN control unit. After the terrain recognition unit obtains the environmental image, it may process the environmental image to determine the predicted terrain corresponding to the environmental image. Among them, the camera is connected to the intelligent driving control unit in the intelligent driving control system, the all-terrain control module may be a functional module independent of the intelligent driving control system, and the camera may be connected to the terrain recognition unit to provide the terrain recognition unit with an environmental image.

[0050] Specifically, the initial model can be trained using training images to transform the initial model into a terrain recognition model. The training images have category labels for indicating the terrain corresponding to the training images. In the process of training the initial model, the training images can be divided into a training set, a validation set, and a test set, wherein the distribution ratios of the training set, the validation set, and the test set can be 80%, 10%, and 10%, respectively. The K-cross method can be used for verification during the training process.

[0051] In specific implementation, the training images can be uniformly processed to keep the size of the training images consistent, thereby ensuring the consistency of model training, for example, the size of the training images is 227×227×3. The training images can be data enhanced so that each training image contains information of the three channels of red, green and blue (RGB), so that the terrain recognition model can be trained with more abundant training images, thereby having higher recognition accuracy.

[0052] In specific implementation, the training images may include images under different disturbance factors, that is, at least two images in the training images have different disturbance factors, and the disturbance factors may include at least one of weather, lighting, and shooting angles, that is, at least two images have different weather, or at least two images have different lighting, or at least two images have different shooting angles. For example, the training images may include aerial images, etc. Among them, different weather may include clear weather, rainy days, foggy days, etc., different lighting may include daytime, night, etc., and different shooting angles may include aerial photography, horizontal angle photography, etc. In this way, on the basis of the image set of the conventional field of view, images under different disturbance factors are added to form training images, so that the terrain recognition model can still stably predict the terrain for different disturbance factors, thereby improving the accuracy of terrain prediction.

[0053] refer to Figure 3 FIG. 1 is a schematic diagram of a training image provided in an embodiment of the present application. The training image can be referred to as Figure 3 A(Aerial image), Figure 3 B (image in clear weather), Figure 3 C (image in rainy weather) and Figure 3 D (images at night), such that at least two images have different weather, at least two images have different lighting, and at least two images have different shooting angles.

[0054] In the embodiment of the present application, the terrain recognition model may include multiple groups of feature processing layers and multiple fully connected layers connected in sequence, the feature processing layer may include a convolution layer, an activation function layer, a normalization layer and a maximum pooling layer, wherein the normalization layer is not shown, and the terrain recognition model may also include an input layer and an output layer, refer to Figure 4 As shown, it is a schematic diagram of a terrain recognition method provided in an embodiment of the present application, wherein an environmental image can be input into the input layer as terrain pattern picture data, the number of groups of feature processing layers can be four groups, the number of layers of fully connected layers can be two layers, and the fully connected layer is used for classification output so that the result of the output layer is the predicted terrain corresponding to the input image, and the predicted terrain is, for example, a city.

[0055] The terrain recognition model may have at least one of an offline self-learning function or a cloud-based deep self-learning function, that is, the terrain recognition model may be optimized based on the acquired images with category labels, so that the terrain recognition model continuously self-learns, expands the learning database, and ensures the recognition accuracy of the terrain recognition model. Specifically, after adding category labels to the environmental images, the terrain recognition model may be optimized using the environmental images with category labels.

[0056] S103: determining a target driving mode of the vehicle according to the predicted terrain.

[0057] In an embodiment of the present application, the target driving mode of the vehicle can be determined according to the predicted terrain. The target driving mode can be a rock mode, a snow mode, a sand mode, an economic mode or a mud mode. There can be a corresponding relationship between the predicted terrain and the target driving mode, and the target driving mode can be determined according to the corresponding relationship. For example, when the predicted terrain is rock, the target driving mode can be a rock mode, when the predicted terrain is snow, the target driving mode can be a snow mode, when the target terrain is sand, the target driving mode can be a sand mode, when the predicted terrain is city, the target driving mode can be an economic mode, and when the predicted terrain is mud, the target driving mode can be a mud mode. When there are multiple environmental images, the target driving mode of the vehicle can be determined according to the multiple predicted terrains corresponding to the multiple environmental images. For example, the target terrain with the largest number among the multiple predicted terrains corresponding to the multiple environmental images can be used as the final terrain, and then the target driving mode of the vehicle is determined according to the final terrain.

[0058] refer to Figure 2 As shown, the all-terrain control module may include a main control unit, which may obtain the predicted terrain from the terrain recognition unit, and then determine the target driving mode of the vehicle. In addition, the all-terrain control module may also include at least one of a motion control unit and a power control unit, the motion control unit is used to collect the driving information of the vehicle, and the power control unit is used to collect the power information of the vehicle, such as the battery state of charge (SOC) of the vehicle, and the driving information is such as the driving speed of the vehicle, the motor torque, etc.

[0059] Specifically, the target driving mode of the vehicle can be determined based on at least one of the driving information from the motion control unit and the power information from the power control unit, as well as the predicted terrain. In this way, the selection of the target driving mode can be limited by the driving information and the power information, so that the determined target driving mode can better meet the actual needs of the vehicle. For example, when the predicted terrain is rocky, but the battery state of charge of the vehicle is low, it means that the battery state of charge is not sufficient to support the activation of the rock mode. At this time, the target driving mode can be determined to be the economic mode. Figure 2As shown, the main control unit can obtain at least one of the driving information and the power information, and determine the target driving mode of the vehicle based on this combined with the predicted terrain.

[0060] S104, controlling the vehicle according to the target driving mode.

[0061] In an embodiment of the present application, the vehicle can be controlled according to the target driving mode. Specifically, the vehicle can be controlled according to the target driving mode and a control signal of a vehicle control unit (VCU). In actual operation, when the target driving mode conflicts with the driving mode determined by the vehicle control unit, the driving mode determined by the vehicle control unit shall prevail.

[0062] refer to Figure 2 As shown, the main control unit in the all-terrain control module can determine the final driving mode according to the control signal of the vehicle control unit and the target driving mode, and then send the final driving mode to each sub-control unit so that each sub-control unit controls the vehicle according to the final driving mode. Alternatively, the main control unit can send the target driving mode to the vehicle control unit so that the vehicle control unit determines the final driving mode according to its own control signal and the target driving mode, and sends the final driving mode to each sub-control unit, or the target driving mode and the control signal of the vehicle control unit can be displayed to the user so that the user can make a selection, and the driving mode selected by the user is used as the final driving mode, and the vehicle control unit or the main control unit sends the final driving mode to the sub-control unit.

[0063] Among them, sub-control units such as intelligent driving control unit, front motor control unit, rear motor control unit, battery management unit, steering control unit, body temperature system, body stability control unit, suspension control unit, instrument switch control unit, etc. can match and adjust the functional state according to the mapping relationship between functional state and terrain mode in different terrain modes. The communication between the main control unit and the self-control unit can be based on the existing communication bus of the vehicle, so that the all-terrain control module can be compatible with the existing communication architecture, that is, adding an all-terrain control module to the existing communication architecture can realize the function of automatically switching driving modes, which is easy to modify.

[0064] An embodiment of the present application provides a vehicle control method. After acquiring an environmental image of the vehicle, the environmental image can be processed using a pre-trained terrain recognition model to determine a predicted terrain corresponding to the environmental image. The terrain recognition model can be a machine learning model that predicts the terrain to be rock, snow, sand, city, or mud. Since the terrain recognition model has been pre-trained, it has accurate terrain recognition capabilities. In this way, the target driving mode of the vehicle can be determined based on the predicted terrain. The target driving mode is rock mode, snow mode, sand mode, economy mode, or mud mode. The vehicle is then controlled based on the target driving mode, thereby achieving accurate and real-time driving mode switching of the vehicle based on the terrain, thereby improving user experience.

[0065] Based on the above vehicle control method, the present application embodiment also provides a vehicle control device, referring to Figure 5 As shown, it is a structural block diagram of a vehicle control device provided in an embodiment of the present application, which may include:

[0066] An image acquisition unit 110 is used to acquire an environmental image of the vehicle;

[0067] A terrain prediction unit 120, configured to process the environment image using a pre-trained terrain recognition model to determine a predicted terrain corresponding to the environment image, wherein the terrain recognition model is a machine learning model, and the predicted terrain is rock, snow, sand, city, or mud;

[0068] a mode determination unit 130, configured to determine a target driving mode of the vehicle according to the predicted terrain, the target driving mode being a rock mode, a snow mode, a sand mode, an economy mode or a mud mode;

[0069] The control unit 140 is configured to control the vehicle according to the target driving mode.

[0070] Optionally, the mode determination unit is specifically used to:

[0071] A target driving mode of the vehicle is determined based on at least one of power information from a power control unit and driving information from a motion control unit, and the predicted terrain.

[0072] Optionally, the control unit is specifically used for:

[0073] The vehicle is controlled according to the target driving mode and a control signal from a vehicle control unit.

[0074] Optionally, the device further comprises:

[0075] A training unit is used to train an initial model using training images so that the initial model is converted into the terrain recognition model, at least two images in the training images have different disturbance factors, and the disturbance factors include at least one of weather, lighting, and shooting angle.

[0076] Optionally, the device further comprises:

[0077] After adding category labels to the environment image, the terrain recognition model is optimized using the environment image with the category labels.

[0078] Optionally, the terrain recognition model includes multiple groups of feature processing layers and multiple fully connected layers connected in sequence, and the feature processing layer includes a convolution layer, an activation function layer, a normalization layer and a maximum pooling layer.

[0079] An embodiment of the present application provides a vehicle control device. After acquiring an environmental image of the vehicle, the environmental image can be processed using a pre-trained terrain recognition model to determine a predicted terrain corresponding to the environmental image. The terrain recognition model can be a machine learning model that predicts the terrain to be rock, snow, sand, city, or mud. Since the terrain recognition model has been pre-trained, it has accurate terrain recognition capabilities. In this way, the target driving mode of the vehicle can be determined based on the predicted terrain. The target driving mode is rock mode, snow mode, sand mode, economy mode, or mud mode. The vehicle is then controlled based on the target driving mode, thereby achieving accurate and real-time driving mode switching of the vehicle according to the terrain, thereby improving user experience.

[0080] Through the description of the above implementation methods, it can be known that those skilled in the art can clearly understand that all or part of the steps in the above-mentioned embodiment method can be implemented by means of software plus a general hardware platform. Based on such an understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a storage medium, such as a read-only memory (ROM) / RAM, a magnetic disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network communication device such as a router) to execute the methods described in each embodiment of the present application or some parts of the embodiments.

[0081] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0082] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0083] The above is only a preferred implementation of the present application. Although the present application has been disclosed as a preferred embodiment, it is not intended to limit the present application. Any technician familiar with the art can use the above disclosed methods and technical contents to make many possible changes and modifications to the technical solution of the present application without departing from the scope of the technical solution of the present application, or modify it into an equivalent embodiment of equivalent changes. Therefore, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present application without departing from the content of the technical solution of the present application still falls within the scope of protection of the technical solution of the present application.

Claims

1. A vehicle control method, characterized in that: The method comprises: Acquire an environmental image of the vehicle; Processing the environment image using a pre-trained terrain recognition model to determine a predicted terrain corresponding to the environment image, wherein the terrain recognition model is a machine learning model, and the predicted terrain is rock, snow, sand, city, or mud; determining a target driving mode of the vehicle according to the predicted terrain, the target driving mode being a rock mode, a snow mode, a sand mode, an economy mode or a mud mode; The vehicle is controlled according to the target driving mode.

2. The method according to claim 1, characterized in that The determining the target driving mode of the vehicle according to the predicted terrain comprises: A target driving mode of the vehicle is determined based on at least one of power information from a power control unit and driving information from a motion control unit, and the predicted terrain.

3. The method according to claim 1, characterized in that The controlling the vehicle according to the target driving mode comprises: The vehicle is controlled according to the target driving mode and a control signal from a vehicle control unit.

4. The method according to claim 1, characterized in that The method further comprises: The initial model is trained using training images so that the initial model is converted into the terrain recognition model, and at least two images in the training images have different disturbance factors, and the disturbance factors include at least one of weather, lighting, and shooting angle.

5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: After adding category labels to the environment image, the terrain recognition model is optimized using the environment image with the category labels.

6. The method according to any one of claims 1 to 4, characterized in that: The terrain recognition model includes multiple groups of feature processing layers and multiple fully connected layers connected in sequence, and the feature processing layer includes a convolution layer, an activation function layer, a normalization layer and a maximum pooling layer.

7. A vehicle control device, characterized in that: The device comprises: An image acquisition unit, used to acquire an environmental image of the vehicle; A terrain prediction unit, used to process the environment image using a pre-trained terrain recognition model to determine a predicted terrain corresponding to the environment image, wherein the terrain recognition model is a machine learning model, and the predicted terrain is rock, snow, sand, city or mud; a mode determination unit, configured to determine a target driving mode of the vehicle according to the predicted terrain, the target driving mode being a rock mode, a snow mode, a sand mode, an economy mode or a mud mode; A control unit is used to control the vehicle according to the target driving mode.

8. The device according to claim 7, characterized in that The mode determination unit is specifically used for: A target driving mode of the vehicle is determined based on at least one of power information from a power control unit and driving information from a motion control unit, and the predicted terrain.

9. The device according to claim 7, characterized in that The control unit is specifically used for: The vehicle is controlled according to the target driving mode and a control signal from a vehicle control unit.

10. The device according to claim 7, characterized in that The device also includes: A training unit is used to train an initial model using training images so that the initial model is converted into the terrain recognition model, at least two images in the training images have different disturbance factors, and the disturbance factors include at least one of weather, lighting, and shooting angle.

Images