Control method of vehicle and apparatus therefor, driving system of vehicle, and vehicle

Abstract

The application discloses a vehicle control method and device, a driving system of a vehicle and the vehicle, and relates to the technical field of vehicle control methods. The vehicle control method comprises the following steps: controlling a virtual driving control device to be formed in the vehicle; receiving actual operation actions input by a user according to the virtual driving control device; and if it is identified that the actual operation actions are the same as at least one preset operation action in a preset operation action set, controlling the vehicle to execute a control instruction corresponding to the at least one preset operation action. A passenger does not need to sit in a main driving position, and the vehicle can be controlled through the virtual driving control device, so that the vehicle is conveniently controlled.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of vehicles, and more particularly to a control method of a vehicle and an apparatus thereof, a driving system of a vehicle and a vehicle. BACKGROUND

[0002] Automatic driving technology is becoming mature and is widely used in related industries and people's social life, especially in the aspects of coach mode, emergency avoidance and automatic parking. With the rapid development of intelligent AI systems, automatic driving is increasingly inclined to digital control. At present, car driving still needs traditional physical operation systems, such as physical steering wheel, physical buttons, etc., which are mainly located at the main driving position, and passengers need to sit at the main driving position to drive the car, which is not convenient for controlling the vehicle. SUMMARY

[0003] The present application is proposed to solve at least one of the above problems. According to a first aspect of the present application, a control method of a vehicle is provided, which comprises: controlling a virtual driving control device formed in the vehicle; receiving an actual operation action input by a user according to the virtual driving control device; and if it is identified that the actual operation action is the same as at least one preset operation action in a preset operation action set, controlling the vehicle to execute a control instruction corresponding to the at least one preset operation action.

[0004] In an embodiment of the present application, if it is identified that the actual operation action is the same as at least one preset operation action in a preset operation action set, the control method further comprises: if it is identified that the actual operation action is the same as at least two preset operation actions in the preset operation action set, controlling the vehicle to execute a control instruction corresponding to part or all of the at least two preset operation actions according to a preset rule.

[0005] In an embodiment of the present application, the control method further comprises: if the control instructions corresponding to the at least two preset operation actions do not conflict with each other, respectively controlling the vehicle to execute a control instruction corresponding to each of the at least two preset operation actions.

[0006] In an embodiment of the present application, the control instruction corresponding to part or all of the at least two preset operation actions is executed by the vehicle according to the preset rules, including: if there is a conflict between the control instructions corresponding to the at least two preset operation actions, the control instruction corresponding to one preset operation action is executed by the vehicle according to a preset conflict resolution rule.

[0007] In an embodiment of the present application, the control instruction corresponding to one preset operation action is executed by the vehicle according to the preset conflict resolution rule, including: the control instruction corresponding to the first identified preset operation action is executed according to the time sequence of the at least two preset operation actions.

[0008] In an embodiment of the present application, the control instruction corresponding to one preset operation action is executed by the vehicle according to the preset conflict resolution rule, including: the control instruction corresponding to the preset operation action with the largest moving range of the actual operation action is executed according to the moving range of the actual operation action corresponding to each preset operation action.

[0009] In an embodiment of the present application, the control method of the vehicle further includes: updating the virtual driving control device according to the control instruction.

[0010] In an embodiment of the present application, the preset operation action includes a preset gesture operation action.

[0011] In an embodiment of the present application, the virtual driving control device includes a virtual steering wheel; the set of preset operation actions includes a preset holding action of the virtual steering wheel, and a preset steering action of moving clockwise or counterclockwise around the virtual steering wheel; wherein the moving direction around the virtual steering wheel represents the steering of the vehicle, and the moving range around the virtual steering wheel represents the steering angle of the vehicle.

[0012] In an embodiment of the present application, if at least two actual operation actions input at the same time are identified to be the same as the preset steering action, the control instruction corresponding to the preset steering action with the largest moving range in all the actual operation actions is input.

[0013] In an embodiment of the present application, the virtual driving control device comprises at least one first type of virtual control button; the preset operation action set comprises a first type of preset button action of clicking the first type of virtual control button and inputting a preset movement action; wherein a movement amplitude of the preset movement action has a correlation with an increase / decrease amplitude of a control instruction corresponding to the first type of virtual control button.

[0014] In an embodiment of the present application, the first type of virtual control button comprises an accelerator button; and the movement amplitude of the preset movement action has a correlation with an increase / decrease amplitude of an accelerator input of the vehicle.

[0015] In an embodiment of the present application, the first type of control button comprises a brake button; and the movement amplitude of the preset movement action has a correlation with an increase / decrease amplitude of a brake input of the vehicle.

[0016] In an embodiment of the present application, the virtual driving control device comprises at least one second type of virtual control button; and the preset operation action set comprises a second type of preset button action of opening or closing the second type of virtual control button.

[0017] In an embodiment of the present application, the second type of virtual control button comprises a button for controlling at least one function of opening or closing a door, a light, a seat, a wiper, a light, a sunroof, a door lock, an air conditioner, a gear, an engine, and navigation of the vehicle.

[0018] In an embodiment of the present application, the control method further comprises: receiving user information of the user; verifying whether the user satisfies a preset driving condition according to the user information of the user; and after the user satisfies the preset driving condition and an instruction of waking up virtual driving is received, controlling the virtual driving control device to be formed in the vehicle.

[0019] In an embodiment of the present application, the user information comprises identity information, and the preset driving condition comprises a driving qualification condition; and verifying whether the user satisfies a preset driving condition according to the user information of the user comprises verifying whether the user has a driving qualification according to the identity information of the user.

[0020] In an embodiment of the present application, the user information comprises current state information, and the preset driving condition comprises a preset state condition; and verifying whether the user satisfies a preset state condition according to the user information of the user comprises verifying whether the user currently satisfies the preset state condition according to the current state information of the user.

[0021] According to the second aspect of the present application, a control device of a vehicle is further provided, the control device comprising: a storage medium and a processor, the storage medium storing a computer program run by the processor, the computer program, when run by the processor, causing the control device of the vehicle to perform any one of the vehicle control methods described above.

[0022] According to the third aspect of the present application, a driving system of a vehicle is further provided, the driving system comprising: a virtual presentation device, a collection device, and any one of the control devices of the vehicle described above, wherein the virtual presentation device is configured to form a virtual driving control device in the vehicle; and the collection device is configured to collect actual operation actions input by a user according to the virtual driving control device.

[0023] In an embodiment of the present application, the virtual presentation device is a projection device configured to form an image of the driving control device in the vehicle.

[0024] In an embodiment of the present application, the projection device is a holographic projection device configured to form a holographic image of the driving control device in the vehicle.

[0025] In an embodiment of the present application, the holographic projection device is a 3D air imaging device.

[0026] In an embodiment of the present application, the collection device is a camera device.

[0027] In an embodiment of the present application, the camera device is a 3D TOF camera.

[0028] In an embodiment of the present application, the driving system further comprises a wake-up module configured to receive an indication of opening or closing the virtual driving input by the user.

[0029] In an embodiment of the present application, the wake-up module is a physical switch or a voice control switch arranged on the vehicle.

[0030] According to the fourth aspect of the present application, a vehicle is further provided, the vehicle comprising: any one of the driving systems of the vehicle described above.

[0031] According to the vehicle control method and device, the driving system of the vehicle and the vehicle provided by the embodiment of the present application, a virtual driving control device is formed in the vehicle; an actual operation action input by a user according to the virtual driving control device is received; if it is identified that the actual operation action is the same as at least one preset operation action in a preset operation action set, the vehicle is controlled to execute a control instruction corresponding to the at least one preset operation action, so as to realize the driving of the vehicle by the passenger. Compared with the prior art, the passenger does not need to sit in the main driving position, and the vehicle can be controlled through the virtual driving control device, so that the vehicle is controlled conveniently. BRIEF DESCRIPTION OF DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort on the basis of these drawings.

[0033] Figure 1 A flowchart of the vehicle control method according to an embodiment of the present application is shown in the figure.

[0034] Figure 2 A schematic diagram of projecting a holographic image according to an embodiment of the present application is shown in the figure.

[0035] Figure 3 A schematic diagram of a camera field of view of a camera device according to another embodiment of the present application is shown in the figure.

[0036] Figure 4 A schematic diagram of a vehicle main and auxiliary driving system according to another embodiment of the present application is shown in the figure.

[0037] Figure 5 A working flowchart of an auxiliary driving system according to an embodiment of the present application is shown in the figure.

[0038] Figure 6 A schematic diagram of a projected holographic image according to an embodiment of the present application is shown in the figure.

[0039] Figure 7 A working flowchart of an auxiliary driving system according to another embodiment of the present application is shown in the figure.

[0040] Figure 8 A working flowchart of an auxiliary driving system according to an embodiment of the present application is shown in the figure.

[0041] Figure 9 A schematic diagram of three kinds of auxiliary driving systems according to an embodiment of the present application is shown in the figure.

[0042] Reference signs:

[0043] 11 - front passenger seat 12 - driver seat 21 - holographic projection device

[0044] 22 - camera 221 - light supplement lamp 31 - steering wheel

[0045] 32 - control button interface 41 - front passenger 42 - driver DETAILED DESCRIPTION

[0046] In order to make the objectives, technical solutions and advantages of the present application more apparent, the following will describe the example embodiments according to the present application in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the example embodiments described herein. Based on the embodiments of the present application described in the present application, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present application.

[0047] In the following description, a large number of specific details are given in order to provide a more thorough understanding of the present application. However, it should be apparent to those skilled in the art that the present application can be practiced without one or more of these details. In other instances, well-known features have not been described in detail in order to avoid obscuring the present application.

[0048] It should be understood that the present application can be implemented in different forms and should not be interpreted as being limited to the embodiments presented herein. On the contrary, these embodiments are provided to make the disclosure thorough and complete, and to fully convey the scope of the present application to those skilled in the art.

[0049] The terms used herein are only for the purpose of describing specific embodiments and not as a limitation of the present application. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising", when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0050] In order to thoroughly understand the present application, detailed structures will be presented in the following description in order to explain the technical solutions presented by the present application. The alternative embodiments of the present application are described in detail as follows, however, in addition to these detailed descriptions, the present application can have other implementation manners.

[0051] Some embodiments of the present application will be described in detail with reference to the drawings. The following embodiments and features of the embodiments described below can be combined with each other in the case of no conflict.

[0052] Firstly, the application scenario of the vehicle control method in the present application needs to be introduced, which is applied to the control process of the vehicle.

[0053] Reference Figure 1 The vehicle control method provided in the embodiments of the present application comprises:

[0054] Step 1: Control to form a virtual driving control device in the vehicle;

[0055] Step 2: Receive the actual operation action input by the user according to the virtual driving control device;

[0056] Step 3: If it is identified that the actual operation action is the same as at least one preset operation action in the preset operation action set, control the vehicle to execute the control instruction corresponding to the at least one preset operation action.

[0057] In the above scheme, by controlling to form a virtual driving control device in the vehicle, receiving the actual operation action input by the user according to the virtual driving control device, and if it is identified that the actual operation action is the same as at least one preset operation action in the preset operation action set, controlling the vehicle to execute the control instruction corresponding to the at least one preset operation action, the driving of the occupant on the vehicle is realized. Compared with the existing mode, the occupant in the present application does not need to sit in the main driving position, and can control the vehicle through the virtual driving control device, thereby facilitating the control of the vehicle. The above steps will be described in detail below with reference to the drawings.

[0058] Firstly, with reference to Figure 1 , a virtual driving control device is formed in the vehicle. The virtual driving control device is a virtual driving control device, which is not an entity structure arranged on the vehicle in the present application, but a virtual driving control device presented by means such as but not limited to projection, virtual imaging, virtual eyes, etc. The virtual driving control device can include various types of virtual control devices such as but not limited to steering wheel, control button, control knob, etc.

[0059] When the virtual driving control device is formed in the vehicle, various methods can be used. For example, a virtual presentation device can be arranged in the vehicle to form a virtual driving control device in the vehicle by controlling the virtual presentation device.

[0060] In setting the virtual presentation device, various ways can be adopted. Exemplarily, the virtual presentation device can be a projection device for forming an image of the driving control device in the vehicle, which is a virtual image that can be observed by the user without the aid of external glasses and the like, thereby facilitating multiple users to observe the virtual driving control device. In a more preferred embodiment, the projection device can be a holographic projection device for forming a holographic image of the driving control device in the vehicle. In a more preferred way, the holographic projection device can be a 3D air imaging device. It should be understood that the virtual presentation device is not limited to the setting way of the projection device, and in addition, can also be other setting ways. For example, it can also be AR glasses, VR glasses and the like, and only the virtual imaging device that can be observed by the user through glasses with related functions is within the protection scope of the embodiments of the present application.

[0061] Exemplarily, with reference to Figure 2 , Figure 3 and Figure 4 , the virtual presentation device can be a holographic projection device 21. In setting the holographic projection device 21, various ways can be adopted. Exemplarily, the holographic projection device 21 can adopt a device with a projection function such as but not limited to a 3D air imaging device. Exemplarily, the holographic projection device 21 can be used to project a holographic image of the main driving system of the main driving position 12 (steering wheel 31 and control buttons, etc.), receive the operation instruction of the co-driver 41 provided by the analysis and recognition module, simulate the entity system, and update the holographic image in real time.

[0062] Exemplarily, with reference to Figure 2 , Figure 3 and Figure 4 , the holographic projection device 21 is used to project a holographic image containing at least the steering wheel 31 and the control buttons in the co-driver position 11. The holographic image is a virtual non-entity 3D image, and when the holographic projection device 21 is turned on, the holographic image containing at least the steering wheel 31 and the control buttons can be projected in the co-driver position 11. The holographic image projected by the projection device can be in front of the co-driver 41, and the projection device can be set in front of the co-driver 41 (near the storage box) or above the head of the co-driver 41. The main driving position 12 has the entity steering wheel 31, brake and throttle controllers, while the co-driver position 11 has the non-entity steering wheel 31, brake and throttle control button interface 32.

[0063] Next, with reference to Figure 1The actual operation action input by the user according to the virtual driving control device is received. That is, the user can input various types of operation actions such as, but not limited to, gestures, head actions, shoulder actions, body actions, and the like according to the virtual control device. When the user inputs the actual operation action according to the virtual control device, the actual operation action is input at a corresponding position according to various functions prompted by the virtual control device or according to the prompt of the virtual control device. This facilitates the collection device to collect the actual operation action input by the user according to the virtual driving control device, and also facilitates subsequent identification of whether the actual operation action is the same as at least one preset operation action in the preset operation action set. That is, after identifying the actual operation action, whether the actual operation action is the same as one preset operation action in the preset operation action set is also identified according to the preset operation action set and the virtual driving control device.

[0064] When the actual operation action input by the user according to the virtual driving control device is received, various methods can be used. For example, a collection device can be provided in the vehicle to collect the actual operation action input by the user according to the virtual driving control device, and then the actual operation action collected by the collection device is received.

[0065] When the collection device is provided, various methods can be used. For example, referring to Figure 2 , Figure 3 and Figure 4 , the collection device can be a camera. The camera collects the actual operation action of the user by photographing the actual operation action of the user, thereby collecting the actual operation action of the user through multiple frames of images. For example, the camera can be a 3D TOF camera to more accurately identify various types of actual operation actions input by the user. Of course, it should be noted that the collection device is not limited to the camera, and other methods can also be used. For example, a sensor can also be used. A sensor wearable device worn on the body of the user can be provided, which can identify the actual operation action of the user as actual operation action information input by the user.

[0066] For example, when the camera 22 is provided, referring to Figure 2 , Figure 3 and Figure 4The camera 22 is configured to acquire gesture data generated by the co-pilot 41 when operating on the holographic image. That is, after the holographic projection device 21 projects the holographic image containing at least the steering wheel 31 and the control buttons on the co-pilot seat 11, the co-pilot 41 who has obtained the driving right can operate on the holographic image. The operation mode can include clicking the control buttons in the holographic image, or holding or turning the steering wheel 31 in the holographic image. Since the operation is on the holographic image, the camera 22 can acquire gesture data generated by the co-pilot 41 when operating on the holographic image. The gesture data can include information such as but not limited to gesture position or gesture posture. That is, the gesture data can obtain the operation object and operation purpose of the co-pilot 41 through the position or posture of the operation gesture, so as to provide the analysis and recognition device to recognize the operation instruction input by the co-pilot 41 through the virtual holographic image. The field of view of the camera 22 can cover the entire co-pilot space of the co-pilot seat 11. The camera 22 can be arranged above the co-pilot 41 (near the vanity mirror) or on the door pillar. For example, referring to Figure 3 A fill light 221 can also be arranged on the camera 22 to provide light filling, thereby improving the camera effect.

[0067] For example, the camera 22 can be a camera such as but not limited to a 3D TOF (Time of Flight) camera. It should be understood that the camera 22 is not limited to the above-mentioned manner, and other manners can also be used.

[0068] For example, the camera 22 can include an image acquisition unit configured to deliver gray-scale and depth images of the co-pilot seat 11 scene to the analysis and recognition device, and also for face recognition (which can be used to start the co-pilot system) and three-dimensional reconstruction (for performing driving), detecting, tracking and positioning the gestures of the co-pilot, such as hand button actions and positions, and steering angles when holding the steering wheel 31, etc.

[0069] Next, referring to Figure 1 If it is recognized that the actual operation action is the same as at least one preset operation action in the preset operation action set, the vehicle is controlled to execute the control instruction corresponding to the same preset operation action. That is, if the actual operation action input by the user according to the virtual driving control device is the same as one of the preset operation actions in the preset operation action set, the vehicle can be controlled to execute the control instruction corresponding to the same preset operation action, thereby realizing the driving of the vehicle by the passenger. Compared with the prior art, the passenger in the present application does not need to sit in the main driver seat, but can control the vehicle through the virtual driving control device, thereby facilitating the control of the vehicle.

[0070] For example, if the actual operation action is identified as being the same as at least one of the preset operation actions in the preset operation action set, the control vehicle to execute the control instruction corresponding to the same at least one of the preset operation actions can include: if the actual operation action is identified as being the same as one of the preset operation actions in the preset operation action set, controlling the vehicle to execute the control instruction corresponding to the same one of the preset operation actions.

[0071] For example, if the actual operation action is identified as being the same as at least one of the preset operation actions in the preset operation action set, the control vehicle to execute the control instruction corresponding to the same at least one of the preset operation actions can include: if the actual operation action is identified as being the same as at least two of the preset operation actions in the preset operation action set, controlling the vehicle to execute the control instruction corresponding to part or all of the at least two of the preset operation actions according to a preset rule. The preset rule is set in advance and can be determined according to factors such as whether the control instructions conflict, time sequence, movement amplitude, etc.

[0072] For example, when the vehicle is controlled to execute the control instruction corresponding to part or all of the at least two of the preset operation actions according to the preset rule, if the control instructions corresponding to the at least two of the preset operation actions do not conflict, the vehicle is controlled to execute the control instruction corresponding to each of the at least two of the preset operation actions, respectively.

[0073] For example, when the vehicle is controlled to execute the control instruction corresponding to part or all of the at least two of the preset operation actions according to the preset rule, if the control instructions corresponding to the at least two of the preset operation actions conflict, the vehicle is controlled to execute the control instruction corresponding to one of the at least two of the preset operation actions according to a preset conflict resolution rule. It should be noted that the determination of whether the control instructions conflict is based on the type of control instruction, the driving state, etc. or even the driving habits of the driver. For example, the control instructions between the accelerator and the brake conflict.

[0074] For example, when the vehicle is controlled to execute the control instruction corresponding to one of the at least two of the preset operation actions according to the preset conflict resolution rule, the time sequence can be used as a conflict resolution mechanism. For example, the control instruction corresponding to the first identified one of the at least two of the preset operation actions can be executed according to the time sequence of the at least two of the preset operation actions.

[0075] For example, when the vehicle is controlled to execute the control instruction corresponding to one of the at least two preset operation actions according to the preset conflict resolution rule, the moving range can be used as the conflict resolution mechanism. For example, the control instruction corresponding to the preset operation action with the largest moving range of the actual operation action among the at least two preset operation actions can be executed according to the moving range of the actual operation action corresponding to each preset operation action.

[0076] For example, the control method of the vehicle can further include: controlling the virtual driving control device to be updated according to the control instruction. That is, after it is identified that the actual operation action input by the user is the same as at least one preset operation action in the preset operation action set and the vehicle is controlled to execute the control instruction corresponding to the at least one preset operation action, the virtual driving control device is further controlled to be updated according to the control instruction, so as to facilitate the user to understand the current control state input for the vehicle.

[0077] In determining each preset operation action in the preset operation action set, various manners can be used. For example, the preset operation action set includes a plurality of preset operation actions. The preset operation action in the preset operation action set can include a preset gesture operation action. Specifically, all preset operation actions in the preset operation action set can be preset gesture operation actions. Of course, in other embodiments, the preset operation action can also be an operation action of other parts of the body. For example, the preset operation action can include a preset head operation action, a preset shoulder operation action, a preset upper body operation action, and even a preset foot operation action, a preset leg operation action, etc. Some preset operation actions in the preset operation action set can be preset gesture operation actions, and some preset operation actions can be preset operation actions of other parts of the body. That is, the preset operation actions in the preset operation action set can be preset operation actions based on the same or different body parts.

[0078] In determining the virtual driving control device, various manners can be used. For example, the virtual driving control device can include a virtual steering wheel. At this time, the preset operation action set can include: a preset holding action of the virtual steering wheel, and a preset steering action of moving clockwise or counterclockwise around the virtual steering wheel; wherein the moving direction around the virtual steering wheel represents the steering of the vehicle, and the moving range around the virtual steering wheel represents the steering angle of the vehicle. It should be explained that the preset holding action and the preset steering action can be preset gesture operation actions, and can also be preset operation actions of other parts of the body such as but not limited to a preset head operation action, a preset shoulder operation action, etc.

[0079] Exemplarily, when the virtual driving control device can include a virtual steering wheel, if it is identified that the at least two actual operation actions input at the same time are all the same as the preset steering action, the control instruction corresponding to the preset steering action with the largest moving range in all the actual operation actions is input.

[0080] In some embodiments, the target steering angle gesture can include a left-turn gesture and a right-turn gesture. The left-turn gesture is a single hand rotating clockwise around the steering wheel 31, and the left-turn gesture is used to control the vehicle to turn left; the right-turn gesture is a single hand rotating counterclockwise around the steering wheel 31, and the right-turn gesture is used to control the vehicle to turn right. The amplitude of the single hand rotating around the steering wheel 31 is positively correlated with the steering angle of the steering wheel 31; and when both hands rotate in the same direction around the steering wheel 31, the larger or smaller value of the double-hand rotating amplitude is used as the steering angle of the steering wheel 31.

[0081] Exemplarily, the virtual driving control device can further include at least one first type of virtual control button. That is, the first type of virtual control button belongs to a type of virtual control button, and the number of the first type of virtual control button can be one, two or more. At this time, the preset operation action set can include a first type of preset button action of clicking the first type of virtual control button and inputting a preset moving action; wherein the moving range of the preset moving action has a correlation with the increase / decrease range of the control instruction corresponding to the first type of virtual control button. That is, the first type of virtual button not only includes the clicking action of clicking the corresponding button, but also includes the action of inputting the control range. It should be explained that the first type of preset button action can be a preset gesture operation action, and can also be a preset operation action of other parts of the body such as but not limited to a preset head operation action, a preset shoulder operation action, etc.

[0082] The type of the first type of virtual control button can be a plurality of types of buttons on the vehicle. As exemplarily introduced below.

[0083] Exemplarily, the first type of virtual control button can include an accelerator button. At this time, the moving range of the preset moving action has a correlation with the increase / decrease range of the input vehicle accelerator.

[0084] Exemplarily, referring to Figure 7 The control button in the control button interface 32 in the holographic image can include an accelerator button. The analysis and identification device can be preset with an accelerator target operation gesture. The analysis and identification device can obtain an operation instruction containing control of increasing or decreasing the vehicle accelerator according to the accelerator target operation gesture and the gesture data obtained by the camera device 22.

[0085] In some embodiments, the target operation gesture of the accelerator can include: an accelerator increasing gesture and an accelerator decreasing gesture; wherein the accelerator increasing gesture is that the accelerator button is clicked and the fingers of the first hand of the front passenger 41 are tightened to the palm, and the accelerator increasing gesture is used to control the accelerator of the vehicle to increase; the accelerator decreasing gesture is that the accelerator button is clicked and the fingers of the first hand are loosened, and the accelerator decreasing gesture is used to control the accelerator of the vehicle to decrease. For example, when the accelerator button is not clicked, the low-speed driving speed can be set by default. In some embodiments, the first hand can be the left hand of the front passenger 41, or the right hand of the front passenger 41. When the first hand is a solid fist, it means that the maximum accelerator is output.

[0086] For example, the first type of control button can also include a brake button. At this time, the movement amplitude of the preset movement action has a correlation with the increase and decrease amplitude of the input vehicle brake.

[0087] For example, with reference to Figure 7 The control button in the control button interface 32 in the holographic image can include a brake button. The analysis and recognition device can be preset with a target operation gesture of the brake. The analysis and recognition device can obtain an operation instruction containing the control of the brake amount of the vehicle according to the target operation gesture of the brake and the gesture data obtained by the camera device 22.

[0088] In some embodiments, the target operation gesture of the brake can include: an increase brake amount gesture and a decrease brake amount gesture. Wherein the increase brake amount gesture is that the brake button is clicked and the fingers of the second hand of the front passenger 41 are tightened to the palm, and the increase brake amount gesture is used to control the brake amount of the vehicle to increase. The decrease brake amount gesture is that the brake button is clicked and the fingers of the second hand are loosened, and the decrease brake amount gesture is used to control the brake amount of the vehicle to decrease. For example, when the brake button is not clicked, the no-brake state driving can be set by default. In some embodiments, the second hand can be the left hand of the front passenger 41, or the right hand of the front passenger 41. When the second hand is a solid fist, it means that the maximum brake amount is output.

[0089] For example, the virtual driving control device can also include at least one second type of virtual control button. That is, the second type of virtual control button belongs to a type of virtual control button, and the number of the second type of virtual control button can be one, two or more. At this time, the preset operation action set can include a second type of preset button action of opening or closing the second type of virtual control button. That is, the second type of virtual button only contains the clicking action of clicking the corresponding button, and does not contain the action of inputting the control amplitude. It needs to be explained that the second type of preset button action can be a preset gesture operation action, but also a preset operation action of other parts of the body such as but not limited to a preset head operation action, a preset shoulder operation action, etc.

[0090] The function of the second type of virtual control button can be implemented in various ways. The following examples introduce several setting ways.

[0091] For example, the second type of virtual control button can include a button for controlling at least one function of opening or closing the door, the light, the seat, the wiper, the light, the sunroof, the door lock, the air conditioner, the gear, the engine, and the navigation of the vehicle.

[0092] For example, referring to FIG. 2, the control button interface 32 in the holographic image can include a button for controlling at least one function of opening or closing the door, the light, the seat, the wiper, the light, the sunroof, the door lock, the air conditioner, the gear, the engine, and the navigation of the vehicle. Figure 7 For example, the control button interface 32 in the holographic image can include a button for controlling at least one function of opening or closing the door, the light, the seat, the wiper, the light, the sunroof, the door lock, the air conditioner, the gear, the engine, and the navigation of the vehicle.

[0093] For example, the click trigger mode of the control button can be the index finger click as the button action recognition instruction. When the index finger touches the projected control button, the operation instruction corresponding to the control button is input, and the corresponding function is executed by the vehicle control system.

[0094] In some embodiments, the control method can further include a judgment logic for determining whether the user meets the preset driving condition before forming the virtual driving control device. For example, the control method can further include the following steps. First, the user information of the user is received. Then, it is verified whether the user meets the preset driving condition according to the user information of the user. Then, after the user meets the preset driving condition and receives the instruction for waking up the virtual driving, the virtual driving control device is controlled to be formed in the vehicle. That is, only after it is verified that the user meets the preset driving condition, the virtual driving control device is controlled to be formed in the vehicle, so as to prevent the user from not meeting the preset driving condition, i.e., immediately opening the virtual driving control device, causing the user to input incorrect actual operation actions for the virtual driving control device, thus miscontrolling the vehicle and causing the vehicle to lose control or other dangers.

[0095] The user information can include various information that can identify the user. For example, the user information can include identity information, such as, but not limited to, an ID number, a user portrait, a user age, a user gender, and the like. The preset driving condition can be determined based on various considerations, such as, but not limited to, whether the user has a driving qualification, whether the user has a preset state condition required for driving, whether the user is drunk, and the like. For example, the preset driving condition can include a driving qualification. In this case, verifying whether the user meets the preset driving condition based on the user information of the user can include verifying whether the user has a driving qualification based on the identity information of the user. This can prevent a user without a driving qualification from triggering the virtual driving control device and driving the vehicle, thereby avoiding driving risks or accidents.

[0096] For example, the user information can include current state information, such as, but not limited to, whether the user is ill, whether the user is drunk, whether the user is sleeping, and the like. In this case, the preset driving condition can include a preset state condition corresponding to the current state information of the user, which can be used to determine whether the user has a driving state based on the current state information of the user. In this case, for example, verifying whether the user meets the preset state condition based on the user information of the user can include verifying whether the user currently meets the preset state condition based on the current state information of the user. This can verify the current state of the user, and prevent the user from triggering the virtual driving control device and driving the vehicle in an unsuitable state, such as, but not limited to, being ill, being drunk, or being asleep, thereby avoiding driving risks or accidents.

[0097] In the various embodiments described above, the virtual driving control device is formed in the vehicle, and the actual operation action input by the user based on the virtual driving control device is received. If it is identified that the actual operation action is the same as at least one preset operation action in a preset operation action set, the vehicle is controlled to execute a control instruction corresponding to the at least one preset operation action, thereby enabling the occupant to drive the vehicle. In comparison with the prior art, the occupant does not need to sit in the driver's seat, but can control the vehicle through the virtual driving control device, thereby facilitating the control of the vehicle.

[0098] Further, the embodiments of the present application also provide a vehicle control device, comprising a storage medium and a processor, the storage medium storing a computer program which is run by the processor, and the computer program, when run by the processor, causes the vehicle control device to perform any of the vehicle control methods described above. The storage medium may, for example, include a storage card of a smart phone, a storage component of a tablet computer, a hard disk of a personal computer, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disc read-only memory (CD-ROM), a USB memory, or any combination of the above storage media. The computer-readable storage medium may be any combination of one or more computer-readable storage media.

[0099] The vehicle control device not only controls the formation of the virtual driving control device in the vehicle and controls the receiving of the actual operation action input by the user according to the virtual driving control device, but also analyzes the recognized actual operation action and each preset operation action in the preset operation action set to determine whether the actual operation action is the same as at least one preset operation action in the preset operation action set. If it is recognized that the actual operation action is the same as at least one preset operation action in the preset operation action set, the control device controls the vehicle to execute the control instruction corresponding to the same at least preset operation action.

[0100] The analysis and recognition device can be provided in the control device, and is configured to determine whether the actual operation action is the same as at least one preset operation action in the preset operation action set. Figure 5 The analysis and recognition device is configured to obtain the operation instruction including the steering angle of the steering wheel 31 and / or the key operation according to the gesture data obtained by the camera 22. That is, the analysis and recognition device can receive the gesture data obtained by the camera 22 and analyze the gesture data to obtain the operation instruction including the steering angle of the steering wheel 31 and / or the key operation. The analysis and recognition device is also configured to transmit the operation instruction to the holographic projection device 21 and the vehicle control system of the vehicle in real time, update the holographic image according to the operation instruction by the holographic projection device 21, and execute the operation instruction by the vehicle control system. After the analysis and recognition device analyzes the operation instruction, the operation instruction is transmitted to the holographic projection device 21 in real time, the holographic projection device 21 updates the display mode of the steering angle of the steering wheel 31 and / or the opening or closing of the control key in the holographic projection according to the latest operation instruction received, realizes the synchronization of the holographic projection and the operation instruction, and enables the co-pilot 41 to see the latest operation control mode. Figure 7The vehicle control system described above can be a control system with control function such as but not limited to EPS (Electric Power Steering), BCM (Body Control Module), DiLink (Central Multimedia Host) and IVI (In vehicle infotainment), which receives operation instruction and executes operation instruction to realize driving the car. Of course, the vehicle control system can also be an intelligent driving control system in the autonomous car, which contains steering wheel 31 control system, central multimedia host, BCM, in-vehicle intelligent voice system and communication system, etc.

[0101] Exemplarily, the analysis and recognition device can be a separate control module among the control devices. The analysis and recognition device can include a storage medium and a processor, the storage medium stores a computer program which is run by the processor, and the computer program enables the processor to calculate the operation instruction containing the steering angle of the steering wheel 31 and / or the key operation according to the gesture data obtained by the camera 22 when the computer program is run by the processor. The storage medium can include, for example, a memory card of a smart phone, a storage component of a tablet computer, a hard disk of a personal computer, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disc read-only memory (CD-ROM), a USB memory, or any combination of the above storage medium. The computer readable storage medium can be any combination of one or more computer readable storage medium. Exemplarily, the analysis and recognition device can be arranged in the vehicle terminal of the vehicle, and can be arranged in the control end of the vehicle terminal such as but not limited to the central multimedia host.

[0102] The analysis and recognition device can be implemented in various ways when obtaining the operation instruction containing the steering angle of the steering wheel 31 and / or the key operation according to the gesture data obtained by the camera 22. Several embodiments are exemplarily introduced as follows.

[0103] Exemplarily, the analysis and recognition device can be preset with a target operation gesture of the steering wheel 31, which can include a target holding gesture of holding the steering wheel 31 and a target steering angle gesture of rotating the steering wheel 31 clockwise or counterclockwise. The analysis and recognition device can obtain the operation instruction containing the steering angle of the steering wheel 31 according to the target operation gesture of the steering wheel 31 and the gesture data obtained by the camera 22. Exemplarily, when the camera 22 detects that the co-pilot 41 extends the target holding gesture on the steering wheel 31 of the holographic image, the gesture data can be input to the analysis and recognition device, and the analysis and recognition device recognizes that the co-pilot 41 has held the steering wheel 31, and the steering condition is established.

[0104] In addition, the application further provides a driving system of a vehicle, which comprises a virtual presentation device, a collection device, and any one of the vehicle control devices. The virtual presentation device is used to form a virtual driving control device in the vehicle. The collection device is used to collect actual operation actions input by a user according to the virtual driving control device.

[0105] In the setting of the virtual presentation device, various methods can be adopted. For example, the virtual presentation device can be a projection device, which is used to form an image of the driving control device in the vehicle. The image is a virtual image, which can be observed by the user without the aid of external glasses, thereby facilitating multiple users to observe the virtual driving control device. In a more preferred embodiment, the projection device can be a holographic projection device, which is used to form a holographic image of the driving control device in the vehicle. In a more preferred method, the holographic projection device can be a 3D air imaging device. It should be understood that the virtual presentation device is not limited to the setting method of the projection device, and other setting methods can also be used. For example, AR glasses, VR glasses, and other virtual imaging devices that can only be observed by the user through glasses with related functions are also within the protection scope of the application.

[0106] In the setting of the collection device, various methods can be adopted. For example, the collection device can be a camera. The camera collects the actual operation actions of the user by taking pictures, thereby collecting the actual operation actions of the user through multiple frames of images. For example, the camera can be a 3D TOF camera, which can more accurately identify various types of actual operation actions input by the user. It should be noted that the setting of the collection device is not limited to the method of using the camera, and other methods can also be used. For example, a sensor can also be used. A sensor wearing device can be set on the user's body, which can identify the actual operation actions of the user as the collection of the actual operation action information input by the user.

[0107] In some embodiments, the driving system can further comprise a wake-up module, which is used to receive an indication of the user inputting to open or close the virtual driving, and control the virtual presentation device and the collection device to work or stop working.

[0108] In setting the wake-up module, various ways can be adopted. Exemplarily, the wake-up module can be a physical switch arranged on the vehicle. In some embodiments, the wake-up device can also be a voice control switch arranged on the vehicle. In other embodiments, the wake-up device can also be a collection device, and the wake-up device is preset with a wake-up operation action and a shutdown operation action, and after the collection device collects the wake-up operation action or the shutdown operation action, the corresponding control of the virtual presentation device and the collection device is performed or stopped.

[0109] Exemplarily, the driving system can be a co-driver system. In application, after obtaining the driving right at the co-driver seat 11, the co-driver system is activated to turn on the holographic projection device 21 to project a holographic image containing at least the steering wheel 31 and the control buttons at the co-driver seat 11. As shown in Figure 6 , the holographic image can be divided into two regions, one region is the steering wheel 31 region, and the other region is the control button interface 32 region. The steering wheel 31 of the holographic image corresponds to the steering function, and the buttons of the control button interface 32 can include control buttons such as but not limited to doors, seats, wipers, lights, sunroofs, door locks, brakes, engine switches, and navigation functions.

[0110] Referring to Figure 5 , after the co-driver system is turned on, the projection device can project a holographic image of the main driver system at the co-driver seat 11, and the camera device 22 can obtain gesture data of the co-driver passenger 41 at the co-driver seat 11 and the co-driver passenger 41 operating on the holographic image. The camera device 22 can specifically detect, identify and track the gesture and position of the co-driver passenger 41 through three-dimensional modeling, recognition positioning and the like to obtain gesture data. The gesture data can be 3D gesture data or 2D gesture data. Then the analysis and recognition device outputs the steering angle of the steering wheel 31 and the operation command of other functions according to the positional relationship between the gesture data and the holographic image to the vehicle control system, which is executed by the vehicle control system to realize the driving of the vehicle by the co-driver passenger 41.

[0111] The vehicle control system can include a steering wheel 31 control system for receiving the operation instructions of the buttons and the steering of the steering wheel 31 provided by the analysis and recognition device to control the driving of the vehicle.

[0112] In a more preferred embodiment, referring to Figure 7 , the co-driver system can further include a wake-up module for waking up the holographic projection device 21, the camera device 22 and the analysis and recognition device. That is, the wake-up module is used to wake up the start of the co-driver passenger 41 operation system. The setting way of the wake-up module can adopt various setting ways. As exemplarily introduced below.

[0113] Exemplarily, referring to Figure 8 andFigure 9 The wake-up module can be a physical switch arranged on the vehicle. The physical switch can be arranged on the driver seat 12, and the driver 42 can click the physical switch to transfer the driving right to the co-driver 41 when it is necessary to transfer the driving right to the co-driver 41. Of course, the physical switch can also be arranged on the co-driver seat 11, and the co-driver 41 can click the physical switch to transfer the driving right to himself when it is necessary to transfer the driving right to the co-driver 41.

[0114] For example, referring to Figure 9 The photographing device can also be used to transmit the identity image information of the co-driver 41 to the vehicle terminal or mobile terminal. At this time, the vehicle terminal or mobile terminal can determine whether the co-driver 41 has the driving ability according to the identity image information. When the photographing device transmits the identity image information of the co-driver 41 to the vehicle terminal, the vehicle terminal determines whether the co-driver 41 has the driving ability. When the photographing device transmits the identity image information of the co-driver 41 to the mobile terminal, the mobile terminal determines whether the co-driver 41 has the driving ability. Correspondingly, at this time, the user can remotely switch the driving right through the mobile terminal. When it is determined that the co-driver 41 has the driving ability according to the identity image information, the vehicle terminal or mobile terminal can control the wake-up module to wake up the holographic projection device 21, the photographing device 22 and the analysis and recognition device. That is, the wake-up module is also used to receive the wake-up instruction of the vehicle terminal or mobile terminal, and wake up the holographic projection device, the photographing device and the analysis and recognition device. The mobile terminal can be a mobile phone, and the user of the mobile phone can remotely control the wake-up module to wake up the holographic projection device 21, the photographing device 22 and the analysis and recognition device through the APP on the mobile phone to determine whether the co-driver 41 has the driving ability.

[0115] For example, referring to Figure 8 The Face ID (facial recognition) information of the co-driver 41, the driving license and the wake-up module of the co-driver system can be registered to the APP of the mobile phone. When the co-driver 41 needs to obtain the holographic co-driver system right, the control method for determining whether the co-driver system is started can be used to remotely start the co-driver system through the APP of the mobile phone to identify the face of the co-driver 41 and compare the data.

[0116] For example, referring to Figure 9, the wake-up module can also be a physical voice control switch set on the vehicle. The main driver 42 or the co-driver 41 can control the wake-up module to wake up the holographic projection device 21, the camera device 22 and the analysis and recognition device by inputting a voice instruction (wake-up word, such as please open the co-driver system) to open the co-driver system to the voice control switch. At this time, the camera device can still transmit the identity image information of the co-driver 41 to the vehicle terminal or the mobile terminal of the vehicle. The vehicle terminal or the mobile terminal of the vehicle can still determine whether the co-driver 41 has the driving ability according to the identity image information. When the camera device transmits the identity image information of the co-driver 41 to the vehicle terminal, the vehicle terminal determines whether the co-driver 41 has the driving ability. When the camera device 22 transmits the identity image information of the co-driver 41 to the mobile terminal, the mobile terminal determines whether the co-driver 41 has the driving ability.

[0117] For example, referring to Figure 9 , the main driver 42 inputs a voice keyword (such as please open the co-driver system) to wake up the co-driver system to start the verification procedure. The camera device 22 collects the face image video stream of the co-driver 41 to the vehicle terminal (which can be a central control host) for face recognition comparison to confirm that the co-driver 41 has the driving ability (driving license). After the comparison is successful, the vehicle terminal can analyze the posture of the co-driver 41 through the camera device 22 to confirm that the co-driver 41 is ready to drive, and then completely start the co-driver system.

[0118] The vehicle control system can also include a domain control system, which can be a central control multimedia host, a BCM vehicle body controller, a steering wheel 31 control system and other related control systems of the vehicle. The central control multimedia host set with a face recognition algorithm responds to the wake-up and face data management, and gives the face recognition result and the start instruction through the algorithm recognition and comparison of the face image sent by the image collection unit. The BCM vehicle body controller is used to receive the operation instruction of the co-driver 41 provided by the analysis and recognition device to control the turn signal, the windshield wiper and the door, etc. The main functions of the cloud platform or the cloud server can include establishing the corresponding connection between the mobile terminal (mobile phone terminal) and the vehicle terminal (vehicle terminal), forwarding the mobile phone APP request, sending the vehicle terminal corresponding feedback message to the mobile phone APP, and saving the necessary information (such as start record). The main functions of the mobile phone APP include establishing the connection with the vehicle terminal through the cloud server, registering the face request and initiating the face data management request (such as automobile TSP, vehicle management platform (face registration information management) and cloud storage platform (face recognition system recognition result data)), and setting the face recognition switch. The vehicle control system can also include an intelligent voice system, and the main function of the intelligent voice system is to recognize and respond to the voice instruction and give a signal to the central control multimedia host.

[0119] In addition, referring to Figure 7A communication system can also be provided in the vehicle, which is used for communication within the vehicle or between the inside and outside of the vehicle. For example, the communication system can include an in-vehicle CAN / CAN FD and an out-of-vehicle 4G / 5G network. The in-vehicle CAN / CAN FD network can be used for the opening and closing of the holographic co-driver system, and the out-of-vehicle 4G / 5G network can be used for face registration, necessary data uploading, and remote control to control the opening and closing of the co-driver system.

[0120] In the various embodiments shown above, the holographic projection device 21 projects a holographic image containing at least the steering wheel 31 and control buttons on the co-driver seat 11; the camera 22 acquires gesture data generated by the co-driver 41 when operating on the holographic image; the analysis and recognition device obtains operation instructions containing the steering angle of the steering wheel 31 and / or button operation according to the gesture data, and transmits the operation instructions to the holographic projection device 21 and the vehicle control system of the vehicle in real time. The holographic projection device 21 updates the holographic image according to the operation instructions, and the vehicle control system executes the operation command to realize the driving of the vehicle by the co-driver 41. Compared with the existing mode, the present application adds a co-driver system to the co-driver seat 11, and the co-driver 41 can input operation instructions containing the steering angle of the steering wheel 31 and / or button operation to the vehicle through non-physical 3D holographic images to realize the driving of the vehicle. This is convenient for the main driver 42 to transfer the driving right to the co-driver 41, and also convenient for the co-driver 41 to drive the vehicle.

[0121] In some embodiments of the present application, a 3D TOF three-dimensional reconstruction co-driver scene is used, including the driver's actions, gestures, and holographic images, to output the operation instructions expressed by the driver to the vehicle control system to realize the driving of the vehicle. Compared with the existing mode, the present application has additional effects in the following scenarios.

[0122] (1) The main driver 42 is not convenient to drive, such as in the scene of difficult to stop on the roadside on expressways, highways, and busy roads, and needs to transfer the driving right temporarily. The co-driver system is started, and the co-driver 41 drives the vehicle;

[0123] (2) The main driver 42 has an accident, such as sudden illness or being hit by a flying object in front, and loses the driving ability. The co-driver 41 starts the driving system at any time to avoid traffic accidents;

[0124] (3) The main driver 42 is a novice and needs to drive with a companion. The co-driver system is started, and the vehicle becomes a training vehicle;

[0125] (4) The main driver 42 uses full automatic driving, and the co-driver 41 can obtain the driving ability when there is a program error to ensure safe driving.

[0126] Meanwhile, the above embodiments show a plurality of starting modes, which can start the co-driver system through a mobile phone APP, voice or physical switch, so as to ensure that the co-driver's authority cannot be used at will, and to give the co-driver the driving function equivalent to the main driver at any time.

[0127] In addition, the embodiments of the present application further provide a vehicle, which can include the driving system of any of the above vehicles. Figures 2-4 The driving system can be used as the main driving system of the vehicle, and the driving system is arranged at the main driving position of the vehicle. The driving system can also be used as the co-driving system of the vehicle, and the driving system is arranged at the co-driving position of the vehicle. Even, the driving system can also be arranged at the rear seat of the vehicle, as a supplement of the main driving system and the co-driving system. The main driving system is used for the main driver of the vehicle body, and the co-driving system is used for the co-driver of the vehicle body. The vehicle can be a traditional manual transmission car, an automatic transmission car, or even an automatic driving car.

[0128] The present application has been described by the above embodiments, but it should be understood that the above embodiments are only for the purpose of example and illustration, and are not intended to limit the present application to the scope of the described embodiments. In addition, those skilled in the art can understand that the present application is not limited to the above embodiments, and more variants and modifications can be made according to the teachings of the present application, which all fall within the scope of the present application. The protection scope of the present application is defined by the attached claims and their equivalent scope.

Claims

1. A method for controlling a vehicle, characterized in that, include: A virtual driving control device is formed within the vehicle, the virtual driving control device including at least one type of virtual control button; Receive actual operation actions input by the user based on the virtual driving control device; If it is detected that the actual operation action is the same as at least one preset operation action in the preset operation action set, the vehicle is controlled to execute the control command corresponding to the same at least one preset operation action; If it is detected that the actual operation action is the same as at least two preset operation actions in the preset operation action set, the vehicle is controlled to execute the control commands corresponding to some or all of the preset operation actions in the at least two preset operation actions according to the preset rules. If there is a conflict between the control commands corresponding to the at least two preset operation actions, then according to the preset conflict resolution rules, the vehicle is controlled to execute the control command corresponding to one of the at least two preset operation actions, wherein the preset conflict resolution rules include the range of movement; The preset operation action set includes: a first type of preset button action that clicks the first type of virtual control button and inputs a preset movement action; wherein, the movement range of the preset movement action is related to the increase or decrease range of the control command corresponding to the first type of virtual control button.

2. The control method as described in claim 1, characterized in that, The control command, which controls the vehicle to execute some or all of the preset operation actions according to preset rules, includes: If the control commands corresponding to the at least two preset operation actions do not conflict, then the vehicle is controlled to execute the control command corresponding to each of the at least two preset operation actions.

3. The control method as described in claim 1, characterized in that, The step of controlling the vehicle to execute the control command corresponding to one of the at least two preset operation actions according to preset conflict resolution rules includes: According to the time sequence of the identified at least two preset operation actions, execute the control command corresponding to the first identified preset operation action among the at least two preset operation actions.

4. The control method as described in claim 1, characterized in that, The step of controlling the vehicle to execute the control command corresponding to one of the at least two preset operation actions according to preset conflict resolution rules includes: According to the movement range of the actual operation action corresponding to each preset operation action, execute the control command corresponding to the preset operation action with the largest actual operation movement range among the at least two preset operation actions.

5. The control method as described in claim 1, characterized in that, Also includes: The virtual driving control device is updated according to the control command.

6. The control method as described in claim 1, characterized in that, The preset operation actions include preset gesture operation actions.

7. The control method as described in claim 1, characterized in that, The virtual driving control device includes a virtual steering wheel; The preset operation action set includes: preset grip action of the virtual steering wheel, and preset steering action of moving clockwise or counterclockwise around the virtual steering wheel; wherein, the direction of movement around the virtual steering wheel represents controlling the steering of the vehicle, and the magnitude of movement around the virtual steering wheel represents controlling the steering angle of the vehicle.

8. The control method as described in claim 7, characterized in that, If it is detected that at least two of the actual operation actions input simultaneously are the same as the preset steering action, then the control command corresponding to the preset steering action with the largest movement range among all the actual operation actions is input.

9. The control method as described in claim 1, characterized in that, The first type of virtual control buttons includes: accelerator button; The movement range of the preset movement action is related to the throttle input of the vehicle.

10. The control method as described in claim 1, characterized in that, The first type of virtual control buttons includes: a brake button; The movement range of the preset movement action is related to the increase or decrease of the braking amount of the vehicle.

11. The control method as described in claim 1, characterized in that, The virtual driving control device includes: at least one second-type virtual control button; The preset operation action set includes: the second type of preset button action for turning the second type of virtual control buttons on or off.

12. The control method as described in claim 11, characterized in that, The second type of virtual control buttons includes buttons that control at least one of the following functions of the vehicle: doors, lights, seats, wipers, lights, sunroof, door locks, air conditioning, gear shift, engine, and navigation.

13. The control method as described in claim 1, characterized in that, Also includes: Receive the user's user information; Verify whether the user meets the preset driving conditions based on the user's user information; After the user meets the preset driving conditions and receives the command to activate virtual driving, the virtual driving control device is formed in the vehicle.

14. The control method as described in claim 13, characterized in that, The user information includes identity information, and the preset driving conditions include driving qualification conditions; The step of verifying whether the user meets the preset driving conditions based on the user's user information includes: verifying whether the user has a driving qualification based on the user's identity information.

15. The control method as described in claim 13, characterized in that, The user information includes current status information, and the preset driving conditions include preset status conditions; The step of verifying whether the user meets the preset state conditions based on the user's user information includes: verifying whether the user currently meets the preset state conditions based on the user's current state information.

16. A vehicle control device, characterized in that, include: A storage medium and a processor, wherein the storage medium stores a computer program executed by the processor, the computer program, when executed by the processor, causes the vehicle control device to perform the vehicle control method as described in any one of claims 1 to 15.

17. A driving system for a vehicle, characterized in that, include: A virtual presentation device for creating a virtual driving control device within the vehicle; The data acquisition device is used to collect the actual operating actions input by the user based on the virtual driving control device; The vehicle control device as described in claim 16.

18. The driving system as claimed in claim 17, characterized in that, The virtual presentation device is a projection device, which is used to generate an image of the driving control device inside the vehicle.

19. The driving system as claimed in claim 18, characterized in that, The projection device is a holographic projection device, which is used to form a holographic image of the driving control device inside the vehicle.

20. The driving system as claimed in claim 19, characterized in that, The holographic projection device is a 3D aerial imaging device.

21. The driving system as claimed in claim 17, characterized in that, The acquisition device is a camera device.

22. The driving system as claimed in claim 21, characterized in that, The camera device is a 3D TOF camera.

23. The driving system as claimed in claim 17, characterized in that, Also includes: The wake-up module is used to receive user input indicating whether to turn virtual driving on or off.

24. The driving system as claimed in claim 23, characterized in that, The wake-up module is a physical switch or a voice control switch installed on the vehicle.

25. A vehicle, characterized in that, include: The driving system of the vehicle as described in any one of claims 17 to 24.

Images