Gear shifting system and method based on holographic projection and new energy automobile

Abstract

The invention relates to the technical field of automobile gear shifting, in particular to a gear shifting system and method based on holographic projection and a new energy automobile. The system comprises a holographic projection module installed in an auxiliary instrument desk and used for projecting a pre-generated gear operation holographic image to the position above the auxiliary instrument desk, and the gear operation holographic image is obtained by carrying out holographic projection on a physical gear operation mechanism; the video acquisition module is used for acquiring a driver operation video when a driver performs gear operation on the gear operation holographic image; the holographic projection module adjusts the content of the gear operation holographic image in real time according to the driver operation video; and the gear shifting actuator is used for switching the current gear to a target gear corresponding to the adjusted gear operation holographic image after the holographic projection module stops adjusting the content of the gear operation holographic image. By adopting the method, the occupation of the operation space in the vehicle can be reduced.

Description

Technical Field

[0001] The present application relates to the technical field of automobile gear shifting, and in particular to a gear shifting system and method based on holographic projection and a new energy vehicle. Background Art

[0002] At present, electronic shift systems have gradually replaced traditional mechanical shift systems, and the operation modes of electronic shift systems are various, including lever type, knob type, hand-held type and button type, etc. However, with the increasing number of car functions and on-board equipment, the operating space in the car is constantly occupied, resulting in increasingly tight installation space for physical electronic shift systems; on the other hand, most drivers are accustomed to placing daily items such as mobile phones, power banks and water cups on the auxiliary instrument panel. After installing the physical electronic shift system, the space available for drivers on the auxiliary instrument panel is further reduced. Summary of the invention

[0003] Based on this, a gear shifting system, method and new energy vehicle based on holographic projection are provided to reduce the occupation of the operating space in the vehicle.

[0004] In a first aspect, a gear shifting system based on holographic projection is provided, the system comprising:

[0005] A holographic projection module installed in the auxiliary instrument panel projects a pre-generated gear operation holographic image onto the upper side of the auxiliary instrument panel, wherein the gear operation holographic image is obtained by holographically projecting a physical gear operation mechanism;

[0006] A video acquisition module, which acquires a driver operation video of the driver performing a gear operation on the gear operation holographic image;

[0007] The holographic projection module adjusts the content of the gear operation holographic image in real time according to the driver operation video;

[0008] The gear shift actuator switches the current gear to the target gear corresponding to the adjusted gear operation holographic image after the holographic projection module stops adjusting the content of the gear operation holographic image.

[0009] In combination with the first aspect, in a first implementable embodiment of the first aspect, the system further includes:

[0010] An image processing module, for identifying a hand movement trajectory of the driver based on the driver's operation video;

[0011] The holographic projection module adjusts the content of the gear operation holographic projection in real time according to the hand movement trajectory of the driver.

[0012] In combination with the first possible implementation of the first aspect, in a second possible implementation of the first aspect, the image processing module identifies the driver's hand movement trajectory according to the driver's operation video, including:

[0013] The image processing module divides the driver's operation video into multiple continuous image frames; detects the hand area in each image frame, detects the hand key point in each hand area and calculates the position of the hand key point; and obtains the driver's hand movement trajectory based on the position change of the hand key point in the multiple continuous image frames.

[0014] In combination with the first possible implementation of the first aspect, in a third possible implementation of the first aspect, the holographic projection module adjusts the content of the gear operation holographic projection in real time according to the hand movement trajectory of the driver, including:

[0015] The holographic projection module obtains a pre-stored reference hand movement trajectory corresponding to the user's gear operation on the gear operation holographic image; calculates the similarity between the driver's hand movement trajectory and the reference hand movement trajectory; if the similarity is greater than or equal to a preset similarity, adjusts the content of the gear operation holographic projection in real time according to the driver's hand movement trajectory.

[0016] In combination with the first aspect, in a fourth possible implementation manner of the first aspect, the system further includes:

[0017] an information interaction module, providing a gear operation mode library for the driver to select, wherein the gear operation mode library includes at least one of the following gear operation modes: lever type, knob type or button type, and each of the gear operation modes corresponds to a physical gear operation mechanism; setting the gear operation mode selected by the driver or the preset default gear operation mode as the current gear operation mode;

[0018] The holographic projection module stores the gear operation holographic image generated by holographic projection of the physical gear operating mechanism corresponding to each gear operation mode in advance, and projects the gear operation holographic image corresponding to the current gear operation mode set by the information interaction module onto the top of the auxiliary instrument panel.

[0019] In combination with the fourth implementable embodiment of the first aspect, in a fifth implementable embodiment of the first aspect, the system further includes:

[0020] A voice module, which plays a shift success voice after the shift actuator switches the current gear to the target gear;

[0021] and / or,

[0022] The information interaction module displays a shift success text content after the shift actuator switches the current gear to the target gear.

[0023] In combination with the fifth possible implementation of the first aspect, in the sixth possible implementation of the first aspect, when the driver's hand movement trajectory corresponding to the driver's operation video does not match the pre-stored reference hand movement trajectory, the voice module plays an operation error voice; and / or the information interaction module displays an operation error text content.

[0024] In combination with the first aspect, in a seventh possible implementation manner of the first aspect, the video acquisition module includes:

[0025] A first camera installed above the left A-pillar of the driver's seat collects a first operation video of the driver performing a gear operation on the gear operation holographic image;

[0026] A second camera installed at the upper center of the front windshield or the rearview mirror inside the vehicle collects a second operation video of the driver performing a gear operation on the gear operation holographic image;

[0027] The image fusion unit performs image fusion processing on the first operation video and the second operation video to obtain the driver operation video.

[0028] In a second aspect, a gear shifting method based on holographic projection is provided, the method comprising:

[0029] Projecting a pre-generated gear operation holographic image onto the top of the auxiliary instrument panel, wherein the gear operation holographic image is obtained by holographically projecting a physical gear operation mechanism;

[0030] Collecting a driver operation video of the driver performing a gear operation on the gear operation holographic image;

[0031] Adjust the content of the gear operation holographic image in real time according to the driver operation video;

[0032] When the holographic projection module stops adjusting the content of the gear operation holographic image, the current gear is switched to the target gear corresponding to the adjusted gear operation holographic image.

[0033] In a third aspect, a new energy vehicle is provided, wherein the vehicle comprises a gear shifting system based on holographic projection as described in the second aspect.

[0034] The above-mentioned shifting system, method and new energy vehicle based on holographic projection, the system includes a holographic projection module, a video acquisition module and a shifting actuator installed in the auxiliary instrument panel, wherein: the holographic projection module projects the pre-generated gear operation holographic image to the top of the auxiliary instrument panel, and the video acquisition module collects the driver's operation video of the driver performing the gear operation on the gear operation holographic projection; the holographic projection module adjusts the content of the gear operation holographic image in real time according to the driver's operation video; when the holographic projection module stops adjusting the content of the gear operation holographic image, the shifting actuator switches the current gear to the target gear corresponding to the adjusted gear operation holographic image. It can be seen that the shifting system of the present application projects the physical gear operation mechanism to the top of the auxiliary instrument panel by holographic projection, and the driver realizes the gear switching operation by interacting with the holographic projection image of the physical gear operation mechanism. Compared with the prior art, the present application reduces the occupation of the operating space in the car and provides more usable space for the driver. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 is a structural block diagram of a gear shifting system based on holographic projection in one embodiment;

[0036] Figure 2 is a structural block diagram of a gear shifting system based on holographic projection in one embodiment;

[0037] Figure 3 is a structural block diagram of a gear shifting system based on holographic projection in one embodiment;

[0038] Figure 4 is a structural block diagram of a gear shifting system based on holographic projection in one embodiment;

[0039] Figure 5 is a structural block diagram of an image processing module in one embodiment;

[0040] Figure 6 The figure is a flowchart of gear shifting based on holographic projection in one embodiment. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0042] It should be noted that the illustrations provided in this embodiment only illustrate the basic concept of the present application in a schematic manner, and therefore the drawings only show components related to the present application rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0043] The structures, proportions, sizes, etc. illustrated in the drawings in this specification are only used to match the contents disclosed in the specification so as to facilitate understanding and reading by persons familiar with this technology. They are not used to limit the conditions under which this application can be implemented, and therefore have no substantive technical significance. Any structural modification, change in proportion or adjustment of size, without affecting the effects and purposes that can be achieved by this application, should still fall within the scope of the technical contents disclosed in this application.

[0044] The directions or positional relationships indicated by the terms "upper", "lower", "left", "right", "middle", "longitudinal", "lateral", "horizontal", "inner", "outer", "radial", "circumferential" and the like in this specification are based on the directions or positional relationships shown in the drawings and are only for the convenience of simplifying the description. They do not indicate or imply that the devices or elements referred to must have a specific direction, be constructed and operate in a specific direction, and therefore cannot be understood as limiting the present application. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance.

[0045] As cars have more and more functions and on-board equipment, such as car refrigerators, the operating space in the car is constantly being occupied, resulting in increasingly tight installation space for the physical electronic shift system. In addition, drivers are accustomed to placing daily items such as mobile phones, power banks and water cups on the auxiliary instrument panel. After installing the physical electronic shift system, the space available for the driver on the auxiliary instrument panel is further reduced.

[0046] To this end, the present application provides a shifting system based on holographic projection, which projects the holographic image of the physical gear operating mechanism onto the top of the auxiliary instrument panel, and realizes the gear switching through the interaction between the driver and the holographic image of the physical gear operating mechanism. The present application only needs to install the holographic projection module in the auxiliary instrument panel, and install an image acquisition module in the car to collect the driver's gesture operation video, and the installation position of the image acquisition module can be the left A-pillar of the driver's seat, the upper center of the inner side of the front windshield, and the rearview mirror inside the car, etc., and will not occupy the space commonly used by the driver such as the auxiliary instrument panel. Therefore, compared with the prior art, the present application reduces the occupation of the operating space in the car, ensuring that the driver has more usable space.

[0047] Next, the gear shifting system based on holographic projection of the present application will be introduced in detail through the following embodiments.

[0048] In one embodiment, Figure 1 As shown, a gear shifting system based on holographic projection is provided, comprising:

[0049] A holographic projection module installed in the auxiliary instrument panel projects a pre-generated holographic image of the gear operation onto the upper side of the auxiliary instrument panel;

[0050] A video acquisition module, which acquires a driver operation video of the driver performing a gear operation on the gear operation holographic image;

[0051] The holographic projection module adjusts the content of the gear operation holographic image in real time according to the driver operation video;

[0052] The gear shift actuator switches the current gear to the target gear corresponding to the adjusted gear operation holographic image after the holographic projection module stops adjusting the content of the gear operation holographic image.

[0053] The pre-generated gear operation holographic image is obtained by pre-holographically projecting the physical gear operating mechanism. In order to adapt to the gear shifting operation habits or preferences of different drivers, the present application provides a variety of gear operation holographic images of the physical gear operating mechanism corresponding to different gear operation modes, so that the driver can choose according to his own habits or preferences.

[0054] In some embodiments, Figure 2 As shown, the shifting system of the present application further includes an information interaction module, and exemplarily, the information interaction module is an in-vehicle infotainment system; the information interaction module provides a gear operation mode library for the driver to select, and the gear operation mode library provides a plurality of different gear operation modes, and each of the gear operation modes corresponds to a physical gear operation mechanism;

[0055] Exemplarily, the gear operation mode library includes at least one of the following gear operation modes: lever type, knob type and button type, and the corresponding physical gear operating mechanisms are respectively a lever type gear operating mechanism, a knob type gear operating mechanism and a button type gear operating mechanism; each physical gear operating mechanism includes at least a gear operating member and a gear indication icon, such as the lever type gear operating mechanism includes a gear operating lever and an icon of the gear corresponding to the gear after the gear operating lever is operated in a preset operating mode, the knob type gear operating mechanism includes a knob and an icon of the gear corresponding to the gear after the knob is rotated in a preset rotation mode, and the button type gear operating mechanism includes icons of different gears and a button corresponding to each gear.

[0056] Before using the vehicle for the first time or driving the vehicle on a daily basis, the driver can select one of a variety of different gear operation modes as the current gear operation mode. If the driver does not specify the gear operation mode, the preset default gear operation mode is set as the current gear operation mode; wherein, the default gear operation mode can be the first gear operation mode arranged in the gear operation mode library, or it can be the gear operation mode with the highest usage rate obtained according to big data statistics, or it can be the gear operation mode that the driver uses most frequently from the first time the vehicle was used to the present moment. This application does not limit this.

[0057] Furthermore, the holographic projection module stores a gear operation holographic image generated by holographically projecting the physical gear operating mechanism corresponding to each gear operation mode in advance; the holographic projection module calls the gear operation holographic image corresponding to the current gear operation mode set by the information interaction module and projects it above the auxiliary instrument panel; similar to operating the physical gear operating mechanism, the driver can perform gear operations on the gear operation holographic image, and the video acquisition module will capture the driver's operation video.

[0058] In some embodiments, in order to improve the recognition accuracy of the driver's operation, the driver's operation video is collected by cameras installed at at least two different positions, and at least two different cameras capture the driver's operation video from different angles. Figure 3 As shown, the video acquisition module includes: a first camera, a second camera and an image fusion unit, wherein: the first camera is installed above the left A-pillar of the driver's seat, and collects a first operation video of the driver performing a gear operation on the gear operation holographic image; the second camera is installed above the center of the front windshield or the rearview mirror inside the vehicle, and collects a second operation video of the driver performing a gear operation on the gear operation holographic image; the image fusion unit performs image fusion processing on the first operation video and the second operation video to obtain the driver's operation video.

[0059] Among them, the image fusion unit can decompose the first operation video into multiple continuous first images according to the time sequence, and decompose the second operation video into multiple continuous second images according to the same time sequence, and fuse the first image and the second image at each time point by using pixel-level fusion or feature-level fusion to obtain a fused image at each time point, and merge the fused images at all time points together according to the time sequence to obtain the driver's operation video.

[0060] The first camera is installed above the left A-pillar of the driver's seat, and the second camera is installed above the center of the front windshield or on the rearview mirror inside the vehicle. On the one hand, it can avoid being blocked by objects and facilitate capturing the driver's operations. On the other hand, it can also avoid occupying the space used by the driver in daily life.

[0061] In some embodiments, Figure 4 As shown, the gear shifting system of the present application also includes an image processing module, which identifies the driver's hand movement trajectory based on the driver's operation video, so that the holographic projection module adjusts the content of the gear operation holographic projection in real time according to the driver's hand movement trajectory.

[0062] Furthermore, the image processing module identifies the driver's hand movement trajectory based on the driver's operation video, specifically including: the image processing module divides the driver's operation video into multiple continuous image frames; detects a hand area in each of the image frames, detects a hand key point in each of the hand area and calculates the position of the hand key point; obtains the driver's hand movement trajectory based on the position change of the hand key point in the multiple continuous image frames.

[0063] It should be noted that by collecting driver operation videos through cameras installed in at least two different positions, the image fusion unit performs image fusion processing on the first operation video and the second operation video, so that the image processing module can accurately detect the position changes of the hands and key points of the hands, avoiding misrecognition caused by angles and movement amplitudes.

[0064] Optionally, the image processing module uses computer vision technology and machine learning technology to identify the driver's hand movement trajectory, including: (1) video preprocessing: decomposing the driver's operation video into multiple continuous image frames through a video processing library (such as OpenCV); in order to improve the accuracy of gesture recognition, each image frame can be enhanced, such as denoising, sharpening and adjusting contrast; (2) hand detection: using the clustering properties of skin color in a specific color space, the hand area in the image frame can be preliminarily detected, and the hand area can be further confirmed by combining shape analysis technology (such as edge detection, contour extraction); or by training a convolutional neural network model to identify the hand area in the image frame; (3) hand tracking: calculating the hand through the optical flow method The movement of each pixel or each feature point in the hand area between multiple consecutive image frames can be used to track the driver's hand movement trajectory; or the hand feature points can be matched between multiple consecutive image frames, such as SIFT, SURF or ORB feature points, to track the driver's hand movement trajectory. A recurrent neural network model can also be trained to predict the position change of the hand between different image frames to obtain the driver's hand movement trajectory. (4) Trajectory extraction: Since the hand position between image frames may be noisy or jittery, the position of the detected hand key points needs to be smoothed, such as by linear interpolation and Kalman filtering. The smoothed hand key points are connected in time series to form the driver's hand movement trajectory.

[0065] After the image processing module calculates the driver's hand movement trajectory, the holographic projection module adjusts the content of the gear operation holographic projection in real time according to the driver's hand movement trajectory. This step specifically includes: the holographic projection module obtains a pre-stored reference hand movement trajectory corresponding to the user's gear operation on the gear operation holographic image; calculates the similarity between the driver's hand movement trajectory and the reference hand movement trajectory; if the similarity is greater than or equal to the preset similarity, adjusting the content of the gear operation holographic projection in real time according to the driver's hand movement trajectory.

[0066] It should be noted that the gear operation holographic image corresponding to each gear operation mode in the holographic projection module stores the standard reference hand movement trajectory corresponding to the gear operation, and the reference hand movement trajectory includes the hand movement trajectory of switching from any gear to other gears; in order to improve efficiency, when calculating the similarity, the starting point and moving direction of the driver's hand movement trajectory can be identified, and then the reference hand movement trajectory that is consistent with the starting point and moving direction of the driver's hand movement trajectory will be screened out from all the reference hand movement trajectories, and the similarity between it and the driver's hand movement trajectory is calculated.

[0067] In some embodiments, based on the reference hand movement trajectory and the driver's hand movement trajectory, the Euclidean distance between two trajectory points at each time point is calculated, and then the Euclidean distances at all time points are averaged or summed to obtain the similarity between the reference hand movement trajectory and the driver's hand movement trajectory; or, the similarity is obtained by calculating the Fréchet distance between the reference hand movement trajectory and the driver's hand movement trajectory.

[0068] Since the video acquisition module continuously and in real time captures the driver's operation video, the image processing module also continuously and in real time calculates the driver's hand movement trajectory. Therefore, before calculating a complete driver's hand movement trajectory that can realize gear switching, the calculated driver's hand movement trajectory has not reached the length of its corresponding reference hand movement trajectory. At this time, it is only necessary to calculate the similarity of the repeated part of the driver's hand movement trajectory and the reference hand movement trajectory. When the similarity of the repeated part is greater than or equal to the preset similarity, it is considered that the driver's hand movement trajectory is consistent with the reference hand movement trajectory, and the content of the gear operation holographic projection is adjusted in real time according to the driver's hand movement trajectory.

[0069] Among them, adjusting the content of the gear operation holographic projection in real time according to the driver's hand movement trajectory refers to moving the gear operating part in the gear operation holographic image according to the driver's hand movement trajectory; exemplarily, for a lever-type gear operation holographic image, the projected holographic image moves the gear lever according to the driver's hand movement trajectory; for a knob-type gear operation holographic image, the projected holographic image rotates the knob according to the driver's hand movement trajectory; for a button-type gear operation holographic image, the projected holographic image presses the corresponding button according to the driver's hand movement trajectory.

[0070] When the gear operation holographic image stops adjusting, the target gear that the gear operation holographic image points to is determined. For example, for a lever-type gear operation holographic image, the gear that the gear operation lever points to is the target gear, for a knob-type gear operation holographic image, the gear that the knob points to is the target gear, and for a button-type gear operation holographic image, the gear that the pressed button points to is the target gear. In some embodiments, the gear operation holographic image stops adjusting when the driver stops the hand action for more than a preset time, or when the driver puts both hands on the steering wheel or opens the door.

[0071] In order to improve the driver's adaptability in switching from the physical gear operating mechanism to the virtual gear operation holographic image, and reduce the possibility of gear shifting failure due to the driver's improper actions, the present application provides a corresponding gear operation tutorial for the gear operation holographic image corresponding to each gear operation mode. The gear operation tutorial can be a teaching video for performing gear operation on the gear operation holographic image, or it can be real-time teaching. For example, the real-time teaching can be generating a guide icon on the gear operation holographic image and playing a guide voice to guide the driver to follow the guide icon and the guide voice to learn the gear switching operation.

[0072] In some embodiments, when the driver uses the current gear operation mode for the first time, the gear operation learning mode is automatically entered to learn through the gear operation tutorial; if this is not the first time that the driver uses the current gear operation mode, the gear operation learning mode may not be entered, but a jump option to enter the gear operation learning mode is also provided on the interface of the information interaction module, and the driver can click on the jump option to enter the gear operation learning mode at any time.

[0073] In some embodiments, after the driver completes a gear operation tutorial, a similar inquiry message such as "Do you want to learn again?" is initiated. When the driver replies with an affirmative response such as "yes", the gear operation learning mode is entered again, and the gear operation learning mode is exited until the driver replies with a negative response such as "no".

[0074] In some embodiments, after the driver completes the gear operation learning through the gear operation tutorial, a preset gear operation test is automatically initiated. This test is only used to test whether the driver can correctly operate the gear operation holographic image according to the gear specified in the test. In this test, the signal receiving end of the shift actuator is temporarily offline and the gear switching is not performed. When the driver passes the gear switching test, the communication of the signal receiving end of the shift actuator is restored, and the driver can subsequently interact with the gear operation holographic image to achieve gear switching.

[0075] In some embodiments, Figure 5 As shown, the shifting system of the present application also includes a voice module. After the shifting actuator switches the current gear to the target gear, the voice module plays a shift success voice and / or the information interaction module displays a shift success text content, and the shift success text content is displayed on the interface of the information interaction module. Exemplarily, the shift success voice and shift success text content are respectively "You have successfully shifted, the current gear is XX" and other voice and text content that prompt the user that the shift is successful and the current gear position.

[0076] In some embodiments, when the hand movement trajectory of the driver corresponding to the driver's operation video does not match the pre-stored reference hand movement trajectory, that is, the similarity between the two is lower than the preset similarity, the voice module plays the operation error voice and / or the information interaction module displays the operation error text content, and the operation error text content is displayed on the interface of the information interaction module. Exemplarily, the operation error voice and the operation error text content are respectively voice and text content such as "Your operation is wrong, please restart the operation" to prompt the user that the gear shift failed.

[0077] The above-mentioned gear shifting system based on holographic projection projects the holographic image of the physical gear operating mechanism onto the top of the auxiliary instrument panel through the holographic projection module, collects the driver's operation video on the holographic image of the physical gear operating mechanism, controls the holographic image of the physical gear operating mechanism to adjust in real time following the driver's operation, and switches the current gear to the target gear pointed to by the holographic image of the adjusted physical gear operating mechanism. Compared with the prior art, the gear shifting system of the present application does not occupy the space on the surface of the auxiliary instrument panel, providing more usable space for the driver.

[0078] Each module in the above-mentioned holographic projection-based shifting system can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each of the above modules.

[0079] In one embodiment, Figure 6 As shown, a shifting method based on holographic projection is provided. Taking the method applied to the shifting system based on holographic projection described in the above embodiment as an example, the method includes the following steps:

[0080] Step 201, projecting a pre-generated gear operation holographic image onto the top of the auxiliary instrument panel, wherein the gear operation holographic image is obtained by holographically projecting a physical gear operation mechanism;

[0081] Step 202, collecting a driver operation video of the driver performing a gear operation on the gear operation holographic image;

[0082] Step 203, adjusting the content of the gear operation holographic image in real time according to the driver operation video;

[0083] Step 204: after the holographic projection module stops adjusting the content of the gear operation holographic image, the current gear is switched to the target gear corresponding to the adjusted gear operation holographic image.

[0084] In some embodiments, adjusting the content of the gear operation holographic image in real time according to the driver's operation video includes: identifying the driver's hand movement trajectory according to the driver's operation video; and adjusting the content of the gear operation holographic projection in real time according to the driver's hand movement trajectory.

[0085] In some embodiments, identifying the driver's hand movement trajectory based on the driver's operation video includes: dividing the driver's operation video into multiple consecutive image frames; detecting a hand area in each of the image frames, detecting a hand key point in each of the hand areas and calculating the position of the hand key point; and obtaining the driver's hand movement trajectory based on the position change of the hand key point in the multiple consecutive image frames.

[0086] In some embodiments, adjusting the content of the gear operation holographic projection in real time according to the driver's hand movement trajectory includes: obtaining a pre-stored reference hand movement trajectory corresponding to the user's gear operation on the gear operation holographic image; calculating the similarity between the driver's hand movement trajectory and the reference hand movement trajectory; if the similarity is greater than or equal to a preset similarity, adjusting the content of the gear operation holographic projection in real time according to the driver's hand movement trajectory.

[0087] In some embodiments, a gear operation mode library is provided for the driver to select, and the gear operation mode library includes at least one of the following gear operation modes: lever type, knob type or button type, and each gear operation mode corresponds to a physical gear operation mechanism; the gear operation mode selected by the driver or the preset default gear operation mode is set as the current gear operation mode; and a gear operation holographic image generated by holographic projection of the physical gear operating mechanism corresponding to each gear operation mode is stored, and the gear operation holographic image corresponding to the set current gear operation mode is projected above the sub-instrument console.

[0088] In some implementations, after the gear shift actuator switches the current gear to the target gear, a gear shift success voice message is played and / or a gear shift success text message is displayed.

[0089] In some embodiments, when the driver's hand movement trajectory corresponding to the driver's operation video does not match the pre-stored reference hand movement trajectory, an operation error voice is played and / or the information interaction module displays an operation error text content.

[0090] In some embodiments, collecting a driver operation video of the driver performing a gear operation on the gear operation holographic image includes: collecting a first operation video of the driver performing a gear operation on the gear operation holographic image through a first camera installed above the left A-pillar of the driver's seat; collecting a second operation video of the driver performing a gear operation on the gear operation holographic image through a second camera installed above the center of the front windshield or the rearview mirror inside the vehicle; performing image fusion processing on the first operation video and the second operation video to obtain the driver operation video.

[0091] For the specific limitations on the gear shifting method based on holographic projection, please refer to the limitations on the gear shifting system based on holographic projection above, which will not be repeated here.

[0092] It should be understood that although Figure 6 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 6 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0093] In one embodiment, a new energy vehicle is provided, wherein the vehicle comprises a gear shifting system based on holographic projection as described in the above embodiment.

[0094] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0095] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0096] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A gear shifting system based on holographic projection, characterized in that: The system comprises: A holographic projection module installed in the auxiliary instrument panel projects a pre-generated gear operation holographic image onto the upper side of the auxiliary instrument panel, wherein the gear operation holographic image is obtained by holographically projecting a physical gear operation mechanism; A video acquisition module, which acquires a driver operation video of the driver performing a gear operation on the gear operation holographic image; The holographic projection module adjusts the content of the gear operation holographic image in real time according to the driver operation video; The gear shift actuator switches the current gear to the target gear corresponding to the adjusted gear operation holographic image after the holographic projection module stops adjusting the content of the gear operation holographic image.

2. The gear shifting system based on holographic projection according to claim 1, characterized in that: The system further comprises: An image processing module, for identifying a hand movement trajectory of the driver based on the driver's operation video; The holographic projection module adjusts the content of the gear operation holographic projection in real time according to the hand movement trajectory of the driver.

3. The gear shifting system based on holographic projection according to claim 2 is characterized in that: The image processing module identifies the driver's hand movement trajectory according to the driver's operation video, including: The image processing module divides the driver's operation video into multiple continuous image frames; detects the hand area in each image frame, detects the hand key point in each hand area and calculates the position of the hand key point; and obtains the driver's hand movement trajectory based on the position change of the hand key point in the multiple continuous image frames.

4. The gear shifting system based on holographic projection according to claim 2, characterized in that: The holographic projection module adjusts the content of the gear operation holographic projection in real time according to the hand movement trajectory of the driver, including: The holographic projection module obtains a pre-stored reference hand movement trajectory corresponding to the user's gear operation on the gear operation holographic image; calculates the similarity between the driver's hand movement trajectory and the reference hand movement trajectory; if the similarity is greater than or equal to a preset similarity, adjusts the content of the gear operation holographic projection in real time according to the driver's hand movement trajectory.

5. The gear shifting system based on holographic projection according to claim 1, characterized in that: The system further comprises: an information interaction module, providing a gear operation mode library for the driver to select, wherein the gear operation mode library includes at least one of the following gear operation modes: lever type, knob type or button type, and each of the gear operation modes corresponds to a physical gear operation mechanism; setting the gear operation mode selected by the driver or the preset default gear operation mode as the current gear operation mode; The holographic projection module stores the gear operation holographic image generated by holographic projection of the physical gear operating mechanism corresponding to each gear operation mode in advance, and projects the gear operation holographic image corresponding to the current gear operation mode set by the information interaction module onto the top of the auxiliary instrument panel.

6. The gear shifting system based on holographic projection according to claim 5, characterized in that: The system further comprises: A voice module, which plays a shift success voice after the shift actuator switches the current gear to the target gear; and / or, The information interaction module displays a shift success text content after the shift actuator switches the current gear to the target gear.

7. The gear shifting system based on holographic projection according to claim 6, characterized in that: When the driver's hand movement trajectory corresponding to the driver's operation video does not match the pre-stored reference hand movement trajectory, the voice module plays an operation error voice and / or the information interaction module displays an operation error text content.

8. The gear shifting system based on holographic projection according to claim 1, characterized in that: The video acquisition module comprises: A first camera installed above the left A-pillar of the driver's seat collects a first operation video of the driver performing a gear operation on the gear operation holographic image; A second camera installed at the upper center of the front windshield or the rearview mirror inside the vehicle collects a second operation video of the driver performing a gear operation on the gear operation holographic image; The image fusion unit performs image fusion processing on the first operation video and the second operation video to obtain the driver operation video.

9. A gear shifting method based on holographic projection, characterized in that: The method comprises: Projecting a pre-generated gear operation holographic image onto the top of the auxiliary instrument panel, wherein the gear operation holographic image is obtained by holographically projecting a physical gear operation mechanism; Collecting a driver operation video of the driver performing a gear operation on the gear operation holographic image; Adjust the content of the gear operation holographic image in real time according to the driver operation video; When the holographic projection module stops adjusting the content of the gear operation holographic image, the current gear is switched to the target gear corresponding to the adjusted gear operation holographic image.

10. A new energy vehicle, characterized in that: The automobile comprises a holographic projection-based gear shifting system as claimed in any one of claims 1 to 8.

Images