Vehicle driving system, method and vehicle

Abstract

A vehicle driving system, method and vehicle, the vehicle driving system comprising a holographic projection element, an image collector and a controller, wherein the holographic projection element is configured to project a holographic image of a vehicle operating device to a passenger cabin of the vehicle; the image collector is configured to acquire image data of the passenger cabin, the image data comprising the holographic image and a passenger in the passenger cabin; and the controller is configured to identify a control gesture of the passenger to the holographic image according to the image data, and control the vehicle according to the control gesture when the passenger has driving authority. The application can control the vehicle by the passenger in the passenger cabin when the driver in the main driving cabin cannot drive the vehicle, so as to improve driving safety.

Description

Technical Field

[0001] This application relates to the field of vehicle driving technology, and more specifically to a vehicle driving system, method and vehicle. Background Technology

[0002] Current driving systems primarily focus on improving the driving experience and safety for the driver in the driver's seat, without considering the possibility of passengers in the front passenger seat or other cabins driving the vehicle. If the driver in the driver's seat becomes unable to drive, the front passenger and other passengers are also unable to drive, leaving the vehicle in a state of uncontrolled operation, thus compromising driving safety.

[0003] In view of the above-mentioned technical problems, the present invention provides a new vehicle driving system, method and vehicle to at least partially solve the above problems. Summary of the Invention

[0004] This application is made to address at least one of the aforementioned problems. According to one aspect of this application, a vehicle driving system is provided, the vehicle driving system comprising: a holographic projection element for projecting a holographic image of a vehicle control device onto the passenger cabin of the vehicle; an image acquisition unit for acquiring image data of the passenger cabin, the image data including the holographic image and a passenger in the passenger cabin; and a controller for recognizing the passenger's control gestures on the holographic image based on the image data, and controlling the vehicle based on the control gestures when the passenger has driving authority.

[0005] In one embodiment of this application, the step of recognizing the passenger's control gestures on the holographic image based on the image data includes: establishing a three-dimensional model including the holographic image and the passenger's driving posture based on the image data; and determining the passenger's control gestures on the holographic image based on the coordinate position change of the passenger's driving posture relative to the holographic image in the three-dimensional model.

[0006] In one embodiment of this application, the controller is further configured to: identify the passenger based on the image data, and grant the passenger driving authority over the vehicle after the identification is successful.

[0007] In one embodiment of this application, the vehicle control device includes at least one of the following: a steering wheel and vehicle buttons.

[0008] In one embodiment of this application, the controller includes an image processing controller, a steering wheel controller, and a vehicle button controller. The image processing controller is configured to: recognize the passenger's steering wheel control gestures on a holographic image of the steering wheel based on the image data, and generate steering wheel control commands based on the steering wheel control gestures; and / or, recognize the passenger's vehicle button control gestures on a holographic image of the vehicle buttons based on the image data, and generate vehicle button control commands based on the vehicle button control gestures; the steering wheel controller is configured to: control the vehicle based on the steering wheel control commands; and the vehicle button controller is configured to: control the vehicle based on the vehicle button control commands.

[0009] In one embodiment of this application, when the control gesture is to slide one hand along the holographic projection of the steering wheel, the controller controls the rotation direction of the vehicle according to the sliding direction of the one hand, and controls the rotation angle of the vehicle according to the sliding amplitude of the one hand; when the control gesture is to slide both hands along the holographic projection of the steering wheel, the controller controls the rotation direction of the vehicle according to the sliding direction of both hands, and controls the rotation angle of the vehicle according to the maximum or minimum sliding amplitude of the two hands.

[0010] In one embodiment of this application, the vehicle buttons include at least one of the following: door buttons, seat buttons, windshield wiper buttons, headlight buttons, sunroof buttons, door lock buttons, brake buttons, engine buttons, and accelerator buttons.

[0011] In one embodiment of this application, when the control gesture is to click the holographic projection of the accelerator button and touch the holographic projection of the steering wheel with one hand and tighten it, the controller controls the vehicle to accelerate and controls the degree of acceleration of the vehicle according to the degree of tightening of the hand; when the control gesture is to click the holographic projection of the accelerator button and touch the holographic projection of the steering wheel with one hand and release it, the controller controls the vehicle to decelerate and controls the degree of deceleration of the vehicle according to the degree of relaxation of the hand; when the control gesture is to click the holographic projection of the brake button and touch the holographic projection of the steering wheel with one hand and tighten it, the controller controls the vehicle to brake and controls the magnitude of braking force according to the degree of tightening of the hand.

[0012] In one embodiment of this application, the image acquisition device includes two cameras, which are respectively positioned above and below the holographic image projected by the holographic projection element.

[0013] In one embodiment of this application, the vehicle driving system further includes: supplementary lighting for illuminating the passenger cabin.

[0014] In one embodiment of this application, the passenger cabin includes at least one of the following: a front passenger cabin and a rear passenger cabin.

[0015] According to another aspect of this application, a vehicle driving method is provided, the vehicle driving method comprising: projecting a holographic image of a vehicle control device onto the passenger cabin of the vehicle; acquiring image data of the passenger cabin, the image data including the holographic image and a passenger in the passenger cabin; recognizing the passenger's control gestures on the holographic image based on the image data; and controlling the vehicle based on the control gestures when the passenger has driving authority.

[0016] In one embodiment of this application, the step of recognizing the passenger's control gestures on the holographic image based on the image data includes: establishing a three-dimensional model including the holographic image and the passenger's driving posture based on the image data; and determining the passenger's control gestures on the holographic image based on the coordinate position change of the passenger's driving posture relative to the holographic image in the three-dimensional model.

[0017] In one embodiment of this application, the vehicle driving method further includes: identifying the passenger based on the image data, and granting the passenger driving authority over the vehicle after the identification is successful.

[0018] In one embodiment of this application, the vehicle control device includes at least one of the following: a steering wheel and vehicle buttons.

[0019] In one embodiment of this application, controlling the vehicle according to the control gesture includes: when the control gesture is a single hand sliding along the holographic projection of the steering wheel, controlling the rotation direction of the vehicle according to the sliding direction of the single hand, and controlling the rotation angle of the vehicle according to the sliding amplitude of the single hand; when the control gesture is both hands sliding along the holographic projection of the steering wheel, controlling the rotation direction of the vehicle according to the sliding direction of both hands, and controlling the rotation angle of the vehicle according to the maximum or minimum sliding amplitude of the single hand.

[0020] In one embodiment of this application, the vehicle buttons include at least one of the following: door buttons, seat buttons, windshield wiper buttons, headlight buttons, sunroof buttons, door lock buttons, brake buttons, engine buttons, and accelerator buttons.

[0021] In one embodiment of this application, controlling the vehicle according to the control gesture includes: when the control gesture is to click the holographic projection of the accelerator button and contact the holographic projection of the steering wheel with one hand and tighten it, controlling the vehicle to accelerate, and controlling the degree of acceleration of the vehicle according to the degree of tightening of the hand; when the control gesture is to click the holographic projection of the accelerator button and contact the holographic projection of the steering wheel with one hand and relax it, controlling the vehicle to decelerate, and controlling the degree of deceleration of the vehicle according to the degree of relaxation of the hand; when the control gesture is to click the holographic projection of the brake button and contact the holographic projection of the steering wheel with one hand and tighten it, controlling the vehicle to brake, and controlling the magnitude of braking force according to the degree of relaxation of the hand.

[0022] In one embodiment of this application, the passenger cabin includes at least one of the following: a front passenger cabin and a rear passenger cabin.

[0023] According to another aspect of this application, a vehicle is provided, including the vehicle driving system described in any one of the preceding claims.

[0024] According to the vehicle driving system, method, and vehicle of the present application embodiments, by projecting a holographic image of the vehicle control device into the passenger cabin, passengers in the passenger cabin can achieve vehicle control capabilities consistent with those of the driver in the driver's cabin through control gestures on the holographic projection. Thus, in the event that the driver in the driver's cabin is unable to drive the vehicle, passengers in the passenger cabin can control the vehicle, thereby improving driving safety. Attached Figure Description

[0025] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0026] Figure 1 A structural block diagram of a vehicle driving system according to an embodiment of this application is shown.

[0027] Figure 2 A structural block diagram of a vehicle driving system according to another embodiment of this application is shown.

[0028] Figure 3 A flowchart illustrating the operation of a vehicle driving system according to an embodiment of this application is shown.

[0029] Figure 4 A flowchart illustrating a vehicle driving method according to an embodiment of this application is shown. Detailed Implementation

[0030] The following description provides numerous specific details to offer a more thorough understanding of this application. However, it will be apparent to those skilled in the art that this application can be practiced without one or more of these details. In other instances, certain technical features well-known in the art have not been described to avoid confusion with this application.

[0031] It should be understood that this application can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, providing these embodiments will make the disclosure thorough and complete, and will fully convey the scope of this application to those skilled in the art. In the drawings, for clarity, the dimensions and relative dimensions of layers and regions may be exaggerated. The same reference numerals denote the same elements throughout.

[0032] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. When used herein, the singular forms “a,” “an,” and “the” are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the terms “compose” and / or “comprising,” when used in this specification, identify the presence of the stated features, integers, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, operations, elements, components, and / or groups. When used herein, the term “and / or” includes any and all combinations of the associated listed items.

[0033] Current driving systems primarily focus on improving the driving experience and safety for the driver in the driver's seat, without considering the possibility of passengers in the front passenger seat or other cabins driving the vehicle. In actual driving situations, there are often circumstances where the driver in the driver's seat is unable to drive the car, for example:

[0034] 1. In situations where it is difficult to pull over to the side of the road, such as on expressways, highways, and busy roads, the driver in the front seat may be unable to control the car for a short period of time due to making or receiving phone calls, but in this case, it is necessary to find a safe place to park.

[0035] 2. The pilot in the main cockpit experiences an accident during flight (such as sudden illness, being hit by a projectile in front, etc.), resulting in loss of piloting ability;

[0036] 3. The driver in the main cockpit is a novice and unfamiliar with vehicle control;

[0037] 4. The vehicle is in fully automated driving mode, and the automated driving software suddenly malfunctions and the driver in the main cockpit has no time to react.

[0038] In the existing technology, when the driver in the main seat is unable to drive the car, the passengers in the front passenger seat and other seats are also unable to drive the car, resulting in the car being in an uncontrolled state, which affects driving safety.

[0039] To fully understand this application, a detailed structure will be presented in the following description to illustrate the technical solutions proposed in this application. Preferred embodiments of this application are described in detail below; however, in addition to these detailed descriptions, this application may have other implementation methods.

[0040] Example 1

[0041] The following is for reference. Figure 1 This application describes a vehicle driving system according to an embodiment of the present application. For example... Figure 1 As shown, the vehicle driving system 100 includes a holographic projection element 110, an image acquisition unit 120, and a controller 130. The holographic projection element 110 is used to project a holographic image of the vehicle control device into the passenger cabin of the vehicle. The image acquisition unit 120 is used to acquire image data of the passenger cabin, including the holographic image and the passenger in the passenger cabin. The controller 130 is used to recognize the passenger's control gestures on the holographic image based on the image data, and to control the vehicle based on the control gestures when the passenger has driving authority.

[0042] Specifically, in addition to being controlled by the driver in the driver's cabin, the vehicle can also be controlled by passengers in the passenger cabin through the vehicle driving system 100. When passengers in the passenger cabin control the vehicle, a holographic image of the vehicle control device is first projected into the passenger cabin through the holographic projection element 110. The vehicle control device is the same as the one in the driver's cabin; that is, the holographic projection element 110 can replicate the vehicle control device in the driver's cabin into the passenger cabin, allowing passengers in the passenger cabin to have the same vehicle control capabilities as the driver in the driver's cabin. After the holographic projection of the vehicle control device is in the passenger cabin, the image acquisition device 120 can collect image data of the passenger cabin, including the holographic image and the passenger. The image acquisition device 120 is communicatively connected to the controller 130 and can send the collected image data to the controller 130. After receiving the image data, the controller 130 can process and analyze the image data to identify the passenger's control gestures on the holographic image and control the vehicle according to the control gestures when the passenger has driving authority. This allows passengers in the passenger compartment to take control of the vehicle when the driver in the driver's seat is unable to drive, thereby improving driving safety.

[0043] It should be noted that the vehicle control mentioned above includes not only driving behaviors such as driving straight and turning, but also other vehicle control behaviors such as opening and closing doors, adjusting seats, controlling wipers, controlling lights, opening and closing the sunroof, unlocking or locking the vehicle, braking, accelerating, and decelerating the vehicle, without limitation.

[0044] According to the vehicle driving system 100 of this application embodiment, by projecting a holographic image of the vehicle control device into the passenger cabin, passengers in the passenger cabin can achieve vehicle control capabilities consistent with those of the driver in the driver's cabin through control gestures on the holographic projection. Thus, in the event that the driver in the driver's cabin is unable to drive the car, passengers in the passenger cabin can control the vehicle, thereby improving driving safety.

[0045] In one example, multiple identification methods (such as fingerprint recognition, iris recognition, facial recognition, voiceprint recognition, etc.) can be used to determine whether a passenger in the passenger cabin has driving privileges.

[0046] For example, the controller 130 can identify passengers in the passenger cabin based on the image data collected by the image acquisition device 120. If the identification fails, it means that the passenger does not have driving authority for the vehicle; if the identification passes, the passenger is granted driving authority for the vehicle. The identification algorithm used by the controller 130 can be Face ID facial recognition algorithm, convolutional neural network algorithm, eigenface algorithm, Fisher face algorithm, etc., and is not limited thereto.

[0047] In one example, the holographic projection element 110 can be a medium-free 3D aerial imaging module, a holographic projector, etc., and is not limited thereto. For example, such as... Figure 2 As shown, the holographic projection element 110 can preferably be a medium-free 3D air imaging module. The medium-free 3D air imaging module can be installed in front of the passenger in the passenger cabin. It can build a 3D floating image of the vehicle control device in the passenger cabin, so that the passenger in the passenger cabin can see real three-dimensional information without any equipment assistance.

[0048] In one example, the passenger cabin may include one or more of the front passenger cabin and the rear passenger cabin, without limitation. When the passenger cabin is the front passenger cabin, the holographic projection element 110 projects a holographic image of the vehicle control device onto the front passenger cabin, thereby enabling the passenger in the front passenger cabin to have vehicle control capabilities consistent with those of the driver in the driver's cabin. When the passenger cabin is the rear passenger cabin, the holographic projection element 110 projects a holographic image of the vehicle control device onto the rear passenger cabin, thereby enabling the passenger in the rear passenger cabin to have vehicle control capabilities consistent with those of the driver in the driver's cabin. Of course, the passenger cabin may also include both the front passenger cabin and the rear passenger cabin, and the holographic projection element 110 may project a holographic image of the vehicle control device onto either the front passenger cabin or the rear passenger cabin, thereby enabling the passengers in both the front passenger cabin and the rear passenger cabin to have vehicle control capabilities consistent with those of the driver in the driver's cabin.

[0049] It should be noted that a vehicle can have one row of rear seats (for example, sedans typically have only one row of rear seats), or it can have two or more rows of rear seats (for example, large SUVs and commercial vehicles can have two rows of rear seats). When there are two or more rows of rear seats in a vehicle, the holographic projection element 110 can project a holographic image of the vehicle control device onto any one of the rows of rear seats, or it can project a holographic image of the vehicle control device onto multiple rows or even all of the rear seats, depending on the actual driving situation. Moreover, each row of rear seats can include only one passenger compartment, or it can include two or more passenger compartments. When there are two or more passenger compartments in a row of rear seats, the holographic projection element 110 can project a holographic image of the vehicle control device onto any one of the passenger compartments, or it can project a holographic image of the vehicle control device onto multiple or even all of the passenger compartments, again depending on the actual driving situation.

[0050] In one example, the vehicle control device can be a steering wheel, vehicle buttons, or both; there is no limitation in this regard. The steering buttons can include one or more of the following: door buttons, seat buttons, wiper buttons, headlight buttons, sunroof buttons, door lock buttons, brake buttons, engine buttons, and accelerator buttons; there is no limitation in this regard either. When the holographic projection element 110 projects a holographic projection of the steering wheel onto the passenger cabin, passengers in the passenger cabin can use gestures to control the holographic projection of the steering wheel to perform driving actions such as driving straight and turning. When the holographic projection element 110 projects a holographic projection of the vehicle buttons onto the passenger cabin, passengers in the passenger cabin can use gestures to control the holographic projection of the vehicle buttons to perform control actions such as opening and closing doors, adjusting seats, controlling wipers, controlling headlights, opening and closing the sunroof, unlocking or locking the vehicle, braking, accelerating, and decelerating.

[0051] In one example, the image acquisition device 120 can be a camera, such as a binocular camera or a TOF (Time of Flight) camera, without limitation. Taking a binocular camera as an example, a binocular camera can acquire two images of the subject through two cameras at different angles, so that the controller 130 can subsequently calculate the depth information of the object through a series of visual algorithms, thereby realizing 3D modeling and image synthesis.

[0052] In one example, the image acquisition device 120 may include two cameras, positioned above and below the holographic image projected by the holographic projection element 110, respectively. This allows for the acquisition of image data of the passenger and the holographic projection from both vertical and horizontal angles, eliminating the problem of difficulty in recognizing passenger gestures due to mutual obstruction between the holographic projection and the passenger (especially the passenger's hands). Of course, this application does not preclude the image acquisition device 120 from including a greater number of cameras. When including a greater number of cameras, their positions can be configured according to actual conditions and are not limited to being positioned above and below the holographic image.

[0053] In one example, such as Figure 2 As shown, the vehicle driving system 100 may also include supplemental lighting, which can provide supplemental lighting to the passenger cabin to increase the brightness of the passenger cabin (for example, it can illuminate the upper body area of ​​the passenger in a dark environment), so that the image acquisition unit 120 can obtain clearer image data.

[0054] For example, such as Figure 2As shown, the image acquisition device 120 includes two binocular cameras, which are respectively positioned above and below the holographic image projected by the holographic projection element 110. The field of view of the camera above the holographic image can cover the holographic projection and the passenger's upper limbs, while the field of view of the camera below the holographic image can cover the passenger's upper body (including the face and upper limbs). This allows for the acquisition of information about the passenger's hands and the holographic projection from different angles. Either of the binocular cameras can be a short-focal-length wide-angle camera to achieve clear image capture at a shorter distance and capture a wider scene. Additionally, a supplementary light can be positioned between the two individual cameras of any binocular camera.

[0055] In one example, such as Figure 2 As shown, the controller 130 may include an image processing controller and a steering wheel controller. The image processing controller can recognize the passenger's steering wheel control gestures based on the holographic image of the steering wheel, and generate steering wheel control commands based on the steering wheel control gestures. The steering wheel controller can control the vehicle according to the steering wheel control commands, enabling the vehicle to perform driving behaviors such as driving straight and turning. Or, as... Figure 2 As shown, the controller 130 may include an image processing controller and a vehicle button controller. The image processing controller can recognize the passenger's holographic gestures when operating the vehicle buttons based on image data, and generate vehicle button control commands based on these gestures. The vehicle button controller can then control the vehicle according to these commands, enabling actions such as door opening / closing, seat adjustment, wiper control, headlight control, sunroof opening / closing, vehicle unlocking / locking, braking, acceleration, and deceleration. Alternatively, as... Figure 2 As shown, the controller 130, in addition to the image processing controller, may also include a steering wheel controller and a vehicle button controller. The image processing controller can recognize the passenger's steering wheel control gestures based on the holographic image of the steering wheel, and can also recognize the passenger's vehicle button control gestures based on the holographic image of the vehicle buttons. Thus, it can further generate steering wheel control commands based on the steering wheel control gestures and vehicle button control commands based on the vehicle button control gestures. The steering wheel controller can control the vehicle according to the steering wheel control commands, enabling the vehicle to perform driving behaviors such as driving straight and turning. The vehicle button controller can control the vehicle according to the vehicle button control commands, enabling the vehicle to perform control behaviors such as door opening and closing, seat adjustment, wiper control, headlight control, sunroof opening and closing, vehicle unlocking or locking, vehicle braking, vehicle acceleration, and vehicle deceleration.

[0056] In one example, the image processing controller can be the vehicle's existing central multimedia host, ECU (Electronic Control Unit, also known as vehicle computer, on-board computer, etc.), multimedia cockpit controller, or other image processing components added to the vehicle that can analyze and process the image data acquired by the image acquisition unit 120. There is no limitation on this. Taking the multimedia cockpit controller as an example, in addition to image processing, the multimedia cockpit controller can also perform calculations, processing, and judgments on various other vehicle information according to its stored programs to achieve other corresponding functions (such as audio processing, video processing, network processing, etc.).

[0057] In one example, the steering wheel controller can be a steering wheel control element already present in the vehicle, or it can be another element added to the vehicle that can be used to control the movement of the vehicle; there is no limitation on this.

[0058] In one example, the vehicle button controller can be the vehicle's existing Body Control Module (BCM), or it can be any other component added to the vehicle that can be used to control the vehicle body; there is no limitation on which one is used.

[0059] In one example, the controller 130 may employ various image recognition processing algorithms (such as convolutional neural network algorithms, recurrent neural network algorithms, deep reinforcement learning algorithms, etc.) to recognize the image data acquired by the image acquisition unit 120 in order to determine the passenger's control gestures on the holographic image.

[0060] For example, such as Figure 3 As shown, the controller 130 (preferably a central multimedia host) can first obtain distance information from the image data through a binocular ranging algorithm. Then, the controller 130 establishes a three-dimensional model based on the image data. The established three-dimensional model includes holographic images and the passenger's driving posture. Afterward, the controller 130 can identify the coordinate position change of the passenger's driving posture relative to the holographic image in the three-dimensional model according to the corresponding recognition algorithm (such as AI algorithm, which can realize gesture detection, recognition, tracking and positioning, etc.), thereby determining the passenger's control gestures on the holographic image (which may include control gestures on the holographic projection of the steering wheel and control gestures on the holographic projection of the vehicle buttons).

[0061] It should be noted that when the image acquisition device 120 includes two or more cameras, the controller 130 can first stitch together multiple image data collected by the two or more cameras using image stitching and fusion algorithms to obtain image data with a larger field of view. Then, it can perform 3D modeling on the stitched image data and determine the passenger's control gestures on the holographic image based on the coordinate position changes of the passenger's driving posture relative to the holographic image in the 3D model. Alternatively, instead of stitching together the multiple image data collected by the two or more cameras, it can first build a 3D model for the image data collected by each camera separately, and then stitch together the multiple 3D models to obtain a complete 3D model. Then, it can determine the passenger's control gestures on the holographic image based on the coordinate position changes of the passenger's driving posture relative to the holographic image in the complete 3D model.

[0062] In one example, based on the changes in the passenger's driving posture relative to the coordinate position of the holographic image, multiple types of control gestures can be identified, thereby generating multiple types of control commands for controlling the vehicle.

[0063] Taking vehicle steering as an example, when the controller 130 recognizes that the passenger's control gesture is to hold the steering wheel, it means that the steering condition is met. Then, the controller 130 can realize vehicle steering according to the specific control gesture.

[0064] For example, when the recognized control gesture is a passenger's single hand sliding along the holographic projection of the steering wheel, the controller 130 can control the vehicle's rotation direction according to the sliding direction of the passenger's single hand, and control the vehicle's rotation angle according to the sliding amplitude of the passenger's single hand.

[0065] For example, when the control gesture is for the passenger to slide one hand along the holographic projection of the steering wheel clockwise, the controller 130 can control the vehicle to make a right turn, and the angle of the right turn is related to the sliding range of the passenger's hand; when the control gesture is for the passenger to slide one hand along the holographic projection of the steering wheel counterclockwise, the controller 130 can control the vehicle to make a left turn, and the angle of the left turn is related to the sliding range of the passenger's hand.

[0066] For example, when the recognized control gesture is that the passenger slides both hands along the holographic projection of the steering wheel, the controller 130 can control the vehicle's rotation direction according to the sliding direction of the passenger's hands, and control the vehicle's rotation angle according to the maximum or minimum single-hand sliding amplitude of the passenger's hands.

[0067] For example, when the control gesture is for the passenger to slide both hands along the holographic projection of the steering wheel clockwise, the controller 130 can control the vehicle to make a right turn, and the angle of the right turn is related to the maximum or minimum single-hand sliding range of the passenger's hands; when the control gesture is for the passenger to slide both hands along the holographic projection of the steering wheel counterclockwise, the controller 130 can control the vehicle to make a left turn, and the angle of the left turn is related to the maximum or minimum single-hand sliding range of the passenger's hands.

[0068] For other vehicle control actions such as door opening and closing, seat adjustment, wiper control, headlight control, sunroof opening and closing, vehicle unlocking or locking, vehicle braking, vehicle acceleration, and vehicle deceleration, the controller 130 can realize the corresponding vehicle control action when it recognizes that the operation gesture has triggered the corresponding vehicle button.

[0069] For example, when the recognized control gesture is a holographic projection of clicking the door button, the controller 130 can control the door to open or close; when the recognized control gesture is a holographic projection of clicking the seat button, the controller 130 can adjust the seat; when the recognized control gesture is a holographic projection of clicking the wiper button, the controller 130 can control the wipers to move or stop; when the recognized control gesture is a holographic projection of clicking the headlight button, the controller 130 can control the headlights to turn on or off; when the recognized control gesture is a holographic projection of clicking the sunroof button, the controller 130 can control the sunroof to open or close; when the recognized control gesture is a holographic projection of clicking the door lock button, the controller 130 can control the vehicle to unlock or lock; when the recognized control gesture is a holographic projection of rotating the brake button, the controller 130 can control the vehicle to brake, and can control the braking force according to the rotation angle; when the recognized control gesture is a holographic projection of rotating the accelerator button, the controller 130 can control the vehicle to accelerate or decelerate, and can control the degree of acceleration or deceleration according to the rotation angle.

[0070] Of course, this application does not exclude the possibility that the vehicle can be controlled by simultaneously manipulating the holographic projection of the steering wheel and the holographic projection of the vehicle buttons.

[0071] Taking the holographic projection of the accelerator button as an example, when the holographic projection of the accelerator button is not touched, the vehicle can be set to drive at low speed by default. When the recognized control gesture is clicking the holographic projection of the accelerator button and the passenger touches and tightens the holographic projection of the steering wheel with one hand, the controller 130 can control the vehicle to accelerate and control the degree of acceleration based on the degree of tightening of the passenger's hand; when the recognized control gesture is clicking the holographic projection of the accelerator button and the passenger touches and relaxes the holographic projection of the steering wheel with one hand, the controller 130 can control the vehicle to decelerate and control the degree of deceleration based on the degree of relaxation of the passenger's hand. It can be understood that maximum acceleration is achieved when the passenger's hand is tightened into a fist, and maximum deceleration is achieved when the passenger's hand is relaxed into a palm.

[0072] Taking the holographic projection of the brake button as an example, when the recognized control gesture is clicking the holographic projection of the brake button and the passenger touches and tightens the holographic projection of the steering wheel with one hand, the controller 130 can control the vehicle braking and control the braking force according to the degree of tightening of the passenger's hand. It can be understood that maximum braking is achieved when the passenger tightens their hand to a solid fist.

[0073] It should be noted that, in the above process, the reason for recognizing whether the passenger's hand is in contact with the holographic projection of the steering wheel while simultaneously controlling the holographic projection of the vehicle buttons is to determine whether the passenger is in a normal driving state and has the authority to control the vehicle by clicking the holographic projection of the vehicle buttons, thereby further improving the vehicle's driving safety.

[0074] Example 2

[0075] According to another aspect of this application, a method for driving a vehicle is also provided. For example... Figure 4 As shown, the vehicle driving method 400 may include the following steps:

[0076] In step S410, a holographic image of the vehicle control device is projected into the passenger cabin of the vehicle.

[0077] In step S420, image data of the passenger cabin is acquired, including holographic images and passengers in the passenger cabin;

[0078] In step S430, the passenger's control gestures on the holographic image are identified based on the image data, and the vehicle is controlled based on the control gestures when the passenger has driving authority.

[0079] Specifically, in addition to being controlled by the driver in the driver's seat, the vehicle can also be controlled by passengers in the passenger cabin through vehicle driving method 400. When passengers in the passenger cabin control the vehicle, a holographic image of the vehicle control device is first projected into the passenger cabin. This vehicle control device is the same as the one in the driver's seat; that is, the vehicle control device in the driver's seat can be replicated in the passenger cabin, giving passengers the same vehicle control capabilities as the driver in the driver's seat. After the holographic projection of the vehicle control device in the passenger cabin, image data of the passenger cabin can be collected, including the holographic image and the passenger. The image data can then be processed and analyzed to identify the passenger's control gestures on the holographic image, and the vehicle can be controlled according to the gestures when the passenger has driving authority. Therefore, in situations where the driver in the driver's seat is unable to drive the car, passengers in the passenger cabin can control the vehicle, thereby improving driving safety.

[0080] It should be noted that the vehicle control mentioned above includes not only driving behaviors such as driving straight and turning, but also other vehicle control behaviors such as opening and closing doors, adjusting seats, controlling wipers, controlling lights, opening and closing the sunroof, unlocking or locking the vehicle, braking, accelerating, and decelerating the vehicle, without limitation.

[0081] According to the vehicle driving method 400 of this application embodiment, by projecting a holographic image of the vehicle control device into the passenger cabin, passengers in the passenger cabin can achieve vehicle control capabilities consistent with those of the driver in the driver's cabin through control gestures on the holographic projection. Thus, in the event that the driver in the driver's cabin is unable to drive the car, passengers in the passenger cabin can control the vehicle, thereby improving driving safety.

[0082] In one example, multiple identification methods (such as fingerprint recognition, iris recognition, facial recognition, voiceprint recognition, etc.) can be used to determine whether a passenger in the passenger cabin has driving privileges.

[0083] For example, the vehicle driving method 400 can identify passengers in the passenger cabin based on image data. If the identification fails, it means that the passenger does not have driving authority; if the identification passes, the passenger is granted driving authority. The identification algorithm used can be Face ID facial recognition algorithm, convolutional neural network algorithm, eigenface algorithm, Fisher face algorithm, etc., and is not limited thereto.

[0084] In one example, the passenger cabin may include one or more of the front passenger cabin and the rear passenger cabin, without limitation. When the passenger cabin is the front passenger cabin, a holographic image of the vehicle control device is projected onto the front passenger cabin, thereby enabling the passenger in the front passenger cabin to have vehicle control capabilities consistent with those of the driver in the driver's cabin. When the passenger cabin is the rear passenger cabin, a holographic image of the vehicle control device is projected onto the rear passenger cabin, thereby enabling the passengers in the rear passenger cabin to have vehicle control capabilities consistent with those of the driver in the driver's cabin. Of course, the passenger cabin may also include both the front passenger cabin and the rear passenger cabin, thus allowing holographic images of the vehicle control device to be projected onto either the front passenger cabin or the rear passenger cabin, enabling passengers in both the front passenger cabin and the rear passenger cabin to have vehicle control capabilities consistent with those of the driver in the driver's cabin.

[0085] It should be noted that a vehicle can have one row of rear seats (for example, sedans typically have only one row of rear seats), or it can have two or more rows of rear seats (for example, large SUVs and commercial vehicles can have two rows of rear seats). When a vehicle has two or more rows of rear seats, a holographic image of the vehicle control system can be projected onto any one of those rows, or onto multiple rows or even all of them, depending on the actual driving situation. Furthermore, each row of rear seats can include only one passenger compartment, or it can include two or more passenger compartments. Similarly, when a single row of rear seats contains two or more passenger compartments, a holographic image of the vehicle control system can be projected onto any one of those passenger compartments, or onto multiple or even all of them, again depending on the actual driving situation.

[0086] In one example, the vehicle control device can be a steering wheel, vehicle buttons, or both, without limitation. The steering buttons can include one or more of the following: door buttons, seat buttons, wiper buttons, headlight buttons, sunroof buttons, door lock buttons, brake buttons, engine buttons, and accelerator buttons, without limitation. When a holographic projection of the steering wheel is projected onto the passenger cabin, passengers can control the vehicle's movement (straight driving, turning, etc.) through gestures on the holographic projection. When a holographic projection of the vehicle buttons is projected onto the passenger cabin, passengers can control the vehicle's operation (door opening / closing, seat adjustment, wiper control, headlight control, sunroof opening / closing, vehicle unlocking / locking, braking, acceleration, deceleration, etc.) through gestures on the holographic projection.

[0087] In one example, various image recognition processing algorithms (such as convolutional neural network algorithms, recurrent neural network algorithms, deep reinforcement learning algorithms, etc.) can be used to identify the acquired image data in order to determine the passenger's gestures in manipulating the holographic image.

[0088] For example, distance information in the image data can be obtained first through a binocular ranging algorithm, and then a three-dimensional model can be built based on the image data. The three-dimensional model includes a holographic image and the passenger's driving posture. Then, the coordinate position change of the passenger's driving posture relative to the holographic image in the three-dimensional model can be identified and processed according to the corresponding gesture recognition algorithm (such as an AI algorithm that can realize gesture detection, recognition, tracking and positioning). This will determine the passenger's control gestures on the holographic image (which may include control gestures on the holographic projection of the steering wheel and control gestures on the holographic projection of the vehicle buttons).

[0089] It should be noted that when there are two or more images acquired, the two or more image data can be stitched together using image stitching and fusion algorithms to obtain image data with a larger field of view. Then, a 3D model can be created from the stitched image data, and the passenger's control gestures on the holographic image can be determined based on the coordinate changes of the passenger's driving posture relative to the holographic image in the 3D model. Alternatively, instead of stitching the two or more image data, a 3D model can be created for each image data separately, and then the multiple 3D models can be stitched together to obtain a complete 3D model. The passenger's control gestures on the holographic image can then be determined based on the coordinate changes of the passenger's driving posture relative to the holographic image in the complete 3D model.

[0090] In one example, based on the changes in the passenger's driving posture relative to the coordinate position of the holographic image, multiple types of control gestures can be identified, thereby generating multiple types of control commands for controlling the vehicle.

[0091] Taking vehicle steering as an example, when the passenger's control gesture is detected as holding the steering wheel, it means that the steering conditions are met, and then the vehicle can be steered according to the specific control gesture.

[0092] For example, when the recognized control gesture is a passenger's single hand sliding along the holographic projection of the steering wheel, the vehicle's rotation direction can be controlled according to the sliding direction of the passenger's single hand, and the vehicle's rotation angle can be controlled according to the sliding amplitude of the passenger's single hand.

[0093] For example, when the control gesture is for the passenger to slide one hand clockwise along the holographic projection of the steering wheel, the vehicle can be controlled to turn right, and the angle of the right turn is related to the sliding range of the passenger's hand; when the control gesture is for the passenger to slide one hand counterclockwise along the holographic projection of the steering wheel, the vehicle can be controlled to turn left, and the angle of the left turn is related to the sliding range of the passenger's hand.

[0094] For example, when the recognized control gesture is that the passenger slides both hands along the holographic projection of the steering wheel, the vehicle's rotation direction can be controlled according to the sliding direction of the passenger's hands, and the vehicle's rotation angle can be controlled according to the maximum or minimum single-hand sliding amplitude of the passenger's hands.

[0095] For example, when the control gesture is for the passenger to slide both hands along the holographic projection of the steering wheel clockwise, the vehicle can be controlled to turn right, and the angle of the right turn is related to the maximum or minimum single-hand sliding range of the passenger's hands; when the control gesture is for the passenger to slide both hands along the holographic projection of the steering wheel counterclockwise, the vehicle can be controlled to turn left, and the angle of the left turn is related to the maximum or minimum single-hand sliding range of the passenger's hands.

[0096] For other vehicle control actions such as door opening and closing, seat adjustment, wiper control, headlight control, sunroof opening and closing, vehicle unlocking or locking, vehicle braking, vehicle acceleration, and vehicle deceleration, the corresponding vehicle control action can be achieved when the control gesture is recognized as triggering the corresponding vehicle button.

[0097] For example, when the recognized control gesture is clicking the holographic projection of the door button, the door can be opened or closed; when the recognized control gesture is clicking the holographic projection of the seat button, the seat can be adjusted; when the recognized control gesture is clicking the holographic projection of the wiper button, the wipers can be turned on or off; when the recognized control gesture is clicking the holographic projection of the headlight button, the headlights can be turned on or off; when the recognized control gesture is clicking the holographic projection of the sunroof button, the sunroof can be opened or closed; when the recognized control gesture is clicking the holographic projection of the door lock button, the vehicle can be unlocked or locked; when the recognized control gesture is rotating the holographic projection of the brake button, the vehicle can be braked, and the braking force can be controlled according to the rotation angle; when the recognized control gesture is rotating the holographic projection of the accelerator button, the vehicle can be accelerated or decelerated, and the degree of acceleration or deceleration can be controlled according to the rotation angle.

[0098] Of course, this application does not exclude the possibility that the vehicle can be controlled by simultaneously manipulating the holographic projection of the steering wheel and the holographic projection of the vehicle buttons.

[0099] Taking the holographic projection of the accelerator button as an example, when the accelerator button is not touched, the vehicle is set to travel at low speed by default. When the recognized control gesture is clicking the holographic projection of the accelerator button and the passenger touches and tightens the holographic projection of the steering wheel with one hand, the vehicle can be accelerated, and the degree of acceleration can be controlled according to the degree of tightening of the passenger's hand. When the recognized control gesture is clicking the holographic projection of the accelerator button and the passenger touches and relaxes the holographic projection of the steering wheel with one hand, the vehicle can be decelerated, and the degree of deceleration can be controlled according to the degree of relaxation of the passenger's hand. It can be understood that maximum acceleration is achieved when the passenger's hand is tightened into a fist, and maximum deceleration is achieved when the passenger's hand is relaxed into a palm.

[0100] Taking the holographic projection of the brake button as an example, when the recognized control gesture is clicking the holographic projection of the brake button and the passenger touches and tightens the holographic projection of the steering wheel with one hand, the vehicle braking can be controlled, and the braking force can be controlled according to the degree of tightening of the passenger's hand. It can be understood that maximum braking is achieved when the passenger tightens their hand into a solid fist.

[0101] It should be noted that, in the above process, the reason for recognizing whether the passenger's hand is in contact with the holographic projection of the steering wheel while simultaneously controlling the holographic projection of the vehicle buttons is to determine whether the passenger is in a normal driving state and has the authority to control the vehicle by clicking the holographic projection of the vehicle buttons, thereby further improving the vehicle's driving safety.

[0102] Example 3

[0103] According to another aspect of this application, a vehicle is also provided. The vehicle includes a vehicle driving system.

[0104] The vehicle driving system can be implemented as the vehicle driving system 100 mentioned above, which can be referred to in the above introduction and will not be repeated here.

[0105] Based on the above description, the vehicle driving system, method, and vehicle according to the embodiments of this application project a holographic image of the vehicle control device onto the passenger cabin, enabling passengers in the passenger cabin to achieve vehicle control capabilities consistent with those of the driver in the driver's cabin through control gestures on the holographic projection. This allows passengers in the passenger cabin to control the vehicle when the driver in the driver's cabin is unable to drive, thereby improving driving safety.

[0106] Although exemplary embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above exemplary embodiments are merely illustrative and are not intended to limit the scope of this application. Various changes and modifications can be made therein by those skilled in the art without departing from the scope and spirit of this application. All such changes and modifications are intended to be included within the scope of this application as claimed in the appended claims.

[0107] Similarly, it should be understood that, in order to simplify this application and aid in understanding one or more aspects of the application, various features of this application may sometimes be grouped together in a single embodiment, figure, or description thereof in the description of exemplary embodiments of this application. However, this approach should not be construed as reflecting an intention that the claimed application requires more features than are expressly recited in each claim. Rather, as reflected in the corresponding claims, the point of application is that the corresponding technical problem can be solved with fewer features than all of a single disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of this application.

[0108] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of this application and form different embodiments. For example, in the claims, any one of the claimed embodiments can be used in any combination.

[0109] It should be noted that the above embodiments are illustrative of this application and not limiting of it, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

Claims

1. A vehicle driving system, characterized in that, The vehicle driving system includes: A holographic projection element is used to project holographic images of the vehicle control device into the passenger cabin of the vehicle. An image acquisition device is used to acquire image data of the passenger cabin, the image data including the holographic image and the passengers in the passenger cabin; The controller is used to recognize the passenger's gestures in relation to the holographic image based on the image data, and to control the vehicle based on the gestures when the passenger has driving privileges. The vehicle control device includes a steering wheel and vehicle buttons; The vehicle buttons include a brake button and an accelerator button; When the control gesture is to click the holographic projection of the accelerator button and touch the holographic projection of the steering wheel with one hand and tighten it, the controller controls the vehicle to accelerate and controls the degree of acceleration of the vehicle according to the degree of tightening of the hand; when the control gesture is to click the holographic projection of the accelerator button and touch the holographic projection of the steering wheel with one hand and release it, the controller controls the vehicle to decelerate and controls the degree of deceleration of the vehicle according to the degree of relaxation of the hand. The control gesture is that when the controller clicks the holographic projection of the brake button and touches and tightens the holographic projection of the steering wheel with one hand, the controller controls the vehicle to brake and controls the braking force according to the degree of tightening of the hand.

2. The vehicle driving system as described in claim 1, characterized in that, The step of recognizing the passenger's gestures in relation to the holographic image based on the image data includes: A three-dimensional model including the holographic image and the passenger's driving posture is established based on the image data; The passenger's control gestures on the holographic image are determined based on the change in the coordinate position of the passenger's driving posture relative to the holographic image in the three-dimensional model.

3. The vehicle driving system as described in claim 1 or 2, characterized in that, The controller is also used for: The passenger is identified based on the image data, and upon successful identification, the passenger is granted driving privileges to drive the vehicle.

4. The vehicle driving system as described in claim 1, characterized in that, The controller includes an image processing controller, a steering wheel controller, and a vehicle button controller. The image processing controller is configured to: recognize the passenger's steering wheel control gestures on the holographic image of the steering wheel based on the image data, and generate steering wheel control commands based on the steering wheel control gestures; and / or, recognize the passenger's vehicle button control gestures on the holographic image of the vehicle buttons based on the image data, and generate vehicle button control commands based on the vehicle button control gestures. The steering wheel controller is used to: control the vehicle according to the steering wheel control commands; The vehicle button controller is used to control the vehicle according to the vehicle button operation commands.

5. The vehicle driving system as described in claim 1, characterized in that, When the control gesture is to slide one hand along the holographic projection of the steering wheel, the controller controls the rotation direction of the vehicle according to the sliding direction of the hand, and controls the rotation angle of the vehicle according to the sliding range of the hand. The control gesture is that when both hands slide along the holographic projection of the steering wheel, the controller controls the rotation direction of the vehicle according to the sliding direction of the hands, and controls the rotation angle of the vehicle according to the maximum or minimum single-hand sliding amplitude of the hands.

6. The vehicle driving system as described in claim 1, characterized in that, The vehicle buttons also include at least one of the following: door buttons, seat buttons, windshield wiper buttons, headlight buttons, sunroof buttons, door lock buttons, and engine buttons.

7. The vehicle driving system as described in claim 1, characterized in that, The image acquisition device includes two cameras, which are respectively positioned above and below the holographic image projected by the holographic projection element.

8. The vehicle driving system as described in claim 1, characterized in that, The vehicle driving system also includes: supplementary lighting for illuminating the passenger cabin.

9. The vehicle driving system as described in claim 1, characterized in that, The passenger cabin includes at least one of the following: the front passenger cabin and the rear passenger cabin.

10. A method for driving a vehicle, characterized in that, The vehicle driving method includes: A holographic image of the vehicle control device is projected into the passenger cabin of the vehicle. Acquire image data of the passenger cabin, the image data including the holographic image and the passengers in the passenger cabin; The system identifies the passenger's gestures in relation to the holographic image based on the image data, and controls the vehicle based on the gestures when the passenger has driving privileges. The vehicle control device includes a steering wheel and vehicle buttons; The vehicle buttons include a brake button and an accelerator button; Controlling the vehicle according to the control gesture includes: The control gesture is as follows: when the accelerator button is clicked and the steering wheel is touched and tightened with one hand, the vehicle is accelerated, and the degree of acceleration is controlled according to the degree of tightening of the hand; when the accelerator button is clicked and the steering wheel is touched and relaxed with one hand, the vehicle is decelerated, and the degree of deceleration is controlled according to the degree of relaxation of the hand. The control gesture is to control the vehicle braking by clicking the holographic projection of the brake button and touching and tightening the holographic projection of the steering wheel with one hand, and to control the braking force according to the degree of relaxation of the hand.

11. The vehicle driving method as described in claim 10, characterized in that, The step of recognizing the passenger's gestures in relation to the holographic image based on the image data includes: A three-dimensional model including the holographic image and the passenger's driving posture is established based on the image data; The passenger's control gestures on the holographic image are determined based on the change in the coordinate position of the passenger's driving posture relative to the holographic image in the three-dimensional model.

12. The vehicle driving method as described in claim 10 or 11, characterized in that, The vehicle driving method also includes: The passenger is identified based on the image data, and upon successful identification, the passenger is granted driving privileges to drive the vehicle.

13. The vehicle driving method as described in claim 10, characterized in that, Controlling the vehicle according to the control gesture includes: The control gesture is that when one hand slides along the holographic projection of the steering wheel, the direction of the vehicle's rotation is controlled according to the sliding direction of the hand, and the angle of the vehicle's rotation is controlled according to the sliding amplitude of the hand. The control gestures involve sliding both hands along the holographic projection of the steering wheel, controlling the vehicle's rotation direction based on the sliding direction of the hands, and controlling the vehicle's rotation angle based on the maximum or minimum single-hand sliding amplitude of the hands.

14. The vehicle driving method as claimed in claim 10, characterized in that, The vehicle buttons also include at least one of the following: door buttons, seat buttons, windshield wiper buttons, headlight buttons, sunroof buttons, door lock buttons, and engine buttons.

15. The vehicle driving method as described in claim 10, characterized in that, The passenger cabin includes at least one of the following: the front passenger cabin and the rear passenger cabin.

16. A vehicle, characterized in that, Includes the vehicle driving system as described in any one of claims 1 to 9.

Images