Method and system for assisting gesture control in vehicle

Abstract

A method and system for assisting in gesture control in a vehicle is provided. The method includes: identifying an action made by a user to activate generation of a gesture interaction area; and generating the gesture interaction area in a predetermined three-dimensional space based on the action made by the user to activate the generation of the gesture interaction area.

Description

Technical Field

[0001] The present invention relates to gesture control, and more particularly, to a method and system for assisting gesture control in a vehicle. Background Art

[0002] When a user interacts with a vehicle during driving, it poses a challenge to safe driving by using a touch device. For this reason, various in-vehicle applications have been developed to enable the user to control the functions of the vehicle in a contactless manner such as by voice, gesture, etc., thereby enhancing driving safety. However, in practical applications, there is no clear gesture recognition area in the vehicle, making it impossible for the user to accurately interact with the vehicle using gestures. For example, when the user does not want to interact with the vehicle using gestures, the vehicle recognizes the user's gestures; when the user wants to interact with the vehicle using gestures, the vehicle does not respond. Therefore, a technology is needed to solve the problem of low gesture recognition efficiency in the vehicle. Summary of the Invention

[0003] The Summary of the Invention is provided to introduce in a simplified form some concepts that will be further described in the following Detailed Description. The Summary of the Invention is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to help determine the scope of the claimed subject matter.

[0004] According to an embodiment of the present invention, there is provided a method for assisting gesture control in a vehicle, including: recognizing a generated action made by a user to activate a gesture interaction area; and generating the gesture interaction area in a predetermined three-dimensional space based on the generated action made by the user to activate the gesture interaction area.

[0005] According to another embodiment of the present invention, there is provided a system for assisting gesture control in a vehicle, including: a gesture interaction area activation module configured to recognize a generated action made by a user to activate a gesture interaction area; and a gesture interaction area generation module configured to generate the gesture interaction area in a predetermined three-dimensional space based on the generated action made by the user to activate the gesture interaction area.

[0006] By reading the following detailed description and referring to the associated drawings, these and other features and advantages will become apparent. It should be understood that the foregoing general description and the following detailed description are illustrative and not restrictive of the claimed aspects. Brief Description of the Drawings

[0007] To understand the manner in which the above-described features of the present invention are used in detail, the above briefly summarized content can be described more specifically with reference to the various embodiments, some aspects of which are shown in the accompanying drawings. It should be noted, however, that the drawings only show some typical aspects of the present invention and should not be considered as limiting its scope, as the description may allow other equally effective aspects.

[0008] Figure 1 FIG. 1 shows an architecture diagram of a system 100 for assisting gesture control in a vehicle according to an embodiment of the present invention;

[0009] Figure 2 FIG. 2 shows a schematic diagram of the installation positions of a gesture interaction area activation module 101 and a gesture interaction area generation module 102 in a vehicle according to an embodiment of the present invention;

[0010] Figure 3 FIG. 3 shows Figure 2 a side view of a camera, an ultrasonic speaker array, and a touchless gesture interaction area generated by the ultrasonic speaker array;

[0011] Figure 4 FIG. 4 shows a flowchart of a method 400 for assisting gesture control in a vehicle according to an embodiment of the present invention;

[0012] Figure 5 FIG. 5 shows a further detailed description of the method 400 in combination with Figure 1 the system 100 according to an embodiment of the present invention;

[0013] Figure 6 FIG. 6 shows a block diagram of an exemplary computing device according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0014] The present invention will be described in detail below with reference to the accompanying drawings, and the features of the present invention will be further revealed in the following detailed description.

[0015] The following detailed description refers to the accompanying drawings showing exemplary embodiments of the present invention. However, the scope of the present invention is not limited to these embodiments, but is defined by the appended claims. Therefore, embodiments other than those shown in the drawings, such as modified versions of the shown embodiments, are still encompassed by the present invention.

[0016] References to "one embodiment", "an embodiment", "an example embodiment", etc. in this specification mean that the embodiment may include a particular feature, structure, or characteristic, but every embodiment may not necessarily include the particular feature, structure, or characteristic. Moreover, these phrases do not necessarily refer to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it should be understood that the particular feature, structure, or characteristic can be implemented in combination with other embodiments within the knowledge of those skilled in the relevant art, whether or not explicitly described.

[0017] For ease of explanation, only the embodiments of applying the technical solution of the present invention to "vehicles" are described in detail herein. However, those skilled in the art can fully understand that the technical solution of the present invention can be applied to any means of transportation such as trains, subways, ships, etc. Unless otherwise specified, the term "A or B" used in this specification refers to "A and B" and "A or B", rather than meaning that A and B are exclusive.

[0018] Term introduction:

[0019] IVI (In-Vehicle Infotainment): It is an in-vehicle integrated information processing system that uses a dedicated in-vehicle central processor and is based on the vehicle body bus system and Internet services. IVI can implement a series of applications including 3D navigation, real-time traffic conditions, IPTV, assisted driving, fault detection, vehicle information, body control, mobile office, wireless communication, online-based entertainment functions, etc., greatly improving the vehicle's electronic, networked, and intelligent levels.

[0020] Generally speaking, the present invention provides a solution for assisting gesture control in a vehicle. In this solution, when the user's hand is recognized by a camera or when the user's voice command is recognized by a voice recognition module, an ultrasonic speaker array is triggered to provide a touchless area in the air of a predetermined three-dimensional space where the user's gesture can be detected. The user can interact with the vehicle in real time by making various gestures in this touchless area.

[0021] Figure 1 The architecture diagram of a system 100 for assisting gesture control in a vehicle according to an embodiment of the present invention is shown.

[0022] The system 100 includes a gesture interaction area activation module 101, a gesture interaction area generation module 102, and an IVI module 103. Those skilled in the art can fully understand that the above module division is only for the purpose of clear explanation. The functions of one or more of the above modules can be combined into a single module or split into more modules. Moreover, one or more of the above modules can be implemented in the form of software, hardware, or a combination thereof. In addition, the data transmission method between each module can adopt a method known in the art, which is not within the discussion scope of the present invention.

[0023] According to an embodiment of the present invention, the gesture interaction area activation module 101 can be configured to recognize an action made by the user to activate the generation of the gesture interaction area.

[0024] For example, the gesture interaction area activation module 101 can be embodied as a camera (e.g., an infrared camera set, i.e., including one or more infrared cameras), which can be configured to recognize the presence of the user's hand within its viewfinder range. For example, the user can activate the generation of the gesture interaction area by placing the hand within the viewfinder range of the camera.

[0025] Again, for example, the gesture interaction area activation module 101 can be embodied as a voice recognition device, which can be configured to capture and recognize the user's voice command. For example, the user can activate the generation of the gesture interaction area by speaking a voice command (e.g., "generate a gesture interaction area").

[0026] According to an embodiment of the present invention, the gesture interaction area generation module 102 can be configured to generate a gesture interaction area in a predetermined three-dimensional space in response to an action made by the user to activate the gesture interaction area.

[0027] Specifically, the gesture interaction area generation module 102 can be embodied as an ultrasonic speaker array, which can generate an ultrasonic area. The user can make various gestures in this ultrasonic area to interact with the vehicle. In practice, the ultrasonic speaker array emits self-correlated ultrasonic signals to generate a touchless area in a predetermined three-dimensional space. As the user makes gestures in this touchless area, these ultrasonic signals will emit reflected signals after contacting the gestures made by the user, and the reflected signals are collected by the microphones of the ultrasonic speaker array. After receiving the reflected signals, the speed and distance features of the signals are extracted, and finally the interaction instruction expressed by the gesture is obtained.

[0028] According to an embodiment of the present invention, the IVI module 103 may be configured to receive a message indicating that an action detected for activating the generation of the gesture interaction area transmitted by the gesture interaction area activation module 101, and send a gesture interaction area generation message to the gesture interaction area generation module 102 based on the received message. However, in another embodiment, the gesture interaction area activation module 101 and the gesture interaction area generation module 102 may directly interact with each other.

[0029] Figure 2 The schematic diagram shows the installation positions of the gesture interaction area activation module 101 and the gesture interaction area generation module 102 in a vehicle according to an embodiment of the present invention. In this schematic diagram, the gesture interaction area activation module 101 is embodied as a camera.

[0030] See Figure 2 , the gesture interaction area activation module 101 (e.g., a camera) and the gesture interaction area generation module 102 (e.g., an ultrasonic speaker array) may be installed under the armrest 104 between seats to facilitate the user to activate the generation of the gesture interaction area by placing the hand within the camera's view range. However, those skilled in the art can clearly understand that this installation position is only illustrative. For example, the gesture interaction area activation module 101 may be integrated with an existing in-vehicle camera without additional installation.

[0031] Figure 3 The figure shows Figure 2 a side view of the camera 101, the ultrasonic speaker array 102, and the touchless gesture interaction area 301 generated by the ultrasonic speaker array 102.

[0032] As can be seen from Figure 3 , the touchless gesture interaction area 301 generated by the ultrasonic speaker array 102 is located in a three-dimensional space at a certain height above the armrest, so that the user can easily and comfortably place the hand into this area and make gestures in this area 301 in a manner approximately horizontal to the armrest.

[0033] Although, in Figure 3 , the contactless gesture interaction area is shown in the form of a touchless pad, however, those skilled in the art can fully understand that Figure 3 the size, shape, position, etc. of the contactless gesture interaction area shown can vary according to the actual needs or usage habits of the user.

[0034] Thus, it can be seen that through the solution of the present invention, the user can activate the generation of the gesture interaction area in a specific way. Therefore, by making gestures in the generated gesture interaction area, there will be no bad user experience that irrelevant gestures are misrecognized or interactive gestures cannot be recognized in time.

[0035] Figure 4 The flowchart of a method 400 for assisting gesture control in a vehicle according to an embodiment of the present invention is shown.

[0036] At 405, an action made by the user to activate the generation of the gesture interaction area is recognized. According to an embodiment of the present invention, the action may be that the user places the hand within the viewfinder range of the camera. According to another embodiment of the present invention, the action may be that the user issues a voice command.

[0037] At 410, based on the action made by the user to activate the generation of the gesture interaction area, a gesture interaction area is generated in a predetermined three-dimensional space. According to an embodiment of the present invention, the gesture interaction area is a contactless ultrasonic gesture interaction area generated by an ultrasonic speaker array.

[0038] Figure 5 The combination according to an embodiment of the present invention is shown Figure 1 of the system 100 for a further detailed description of the method 400.

[0039] At 505, the gesture interaction area activation module 101 recognizes an action made by the user to activate the generation of the gesture interaction area. According to an embodiment of the present invention, when the gesture interaction area activation module 101 is a camera, the action may be that the user places the hand within the viewfinder range of the camera. According to another embodiment of the present invention, when the gesture interaction area activation module 101 is a voice recognition device, the action may be a voice command issued by the user.

[0040] At 510, in response to the recognition, the gesture interaction area activation module 101 sends a message (e.g., Hand_Detect_On message) indicating that the action made to activate the generation of the gesture interaction area is detected to the IVI module 103.

[0041] At 515, in response to receiving the message, the IVI module 103 sends a gesture interaction area generation message (e.g., Touchless_Pad_On message) to the gesture interaction area generation module 102.

[0042] At 520, in response to receiving the gesture interaction area generation message, the gesture interaction area generation module 102 generates a gesture interaction area. According to an embodiment of the present invention, the gesture interaction area generation module 102 generates a contactless gesture interaction area in the form of ultrasonic waves in a predetermined three-dimensional space.

[0043] At 525, the IVI module 103 recognizes the gesture made by the user in the generated gesture interaction area so that the user can interact with the vehicle.

[0044] Figure 6 FIG. Figure 6 shows a block diagram of an exemplary computing device according to an embodiment of the present invention, which is an example of a hardware device applicable to various aspects of the present invention.

[0045] Referring Figure 6 , a computing device 600 will now be described, which is an example of a hardware device applicable to various aspects of the present invention. The computing device 600 can be any machine configured to perform processing and / or computing, and can be, but is not limited to, a workstation, a server, a desktop computer, a laptop computer, a tablet computer, a personal digital assistant, a smartphone, an in-vehicle computer, or any combination thereof. The foregoing various methods / devices / servers / client devices can be implemented in whole or at least in part by the computing device 600 or a similar device or system.

[0046] The computing device 600 may include components that can be connected or communicate via one or more interfaces and a bus 602. For example, the computing device 600 may include a bus 602, one or more processors 604, one or more input devices 606, and one or more output devices 608. The one or more processors 604 can be any type of processor and may include, but are not limited to, one or more general-purpose processors and / or one or more dedicated processors (e.g., specialized processing chips). The input device 606 can be any type of device capable of inputting information into the computing device and may include, but are not limited to, a mouse, a keyboard, a touch screen, a microphone, and / or a remote controller. The output device 608 can be any type of device capable of presenting information and may include, but are not limited to, a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The computing device 600 may also include a non-transitory storage device 610 or be connected to the non-transitory storage device. The non-transitory storage device can be any storage device that is non-transitory and capable of implementing data storage, and the non-transitory storage device may include, but are not limited to, a disk drive, an optical storage device, a solid-state memory, a floppy disk, a flexible disk, a hard disk, a magnetic tape, or any other magnetic medium, an optical disk, or any other optical medium, a ROM (read-only memory), a RAM (random access memory), a cache memory, and / or any storage chip or cartridge, and / or any other medium from which a computer can read data, instructions, and / or code. The non-transitory storage device 610 can be separated from the interface. The non-transitory storage device 610 may have data / instructions / code for implementing the above methods and steps. The computing device 600 may also include a communication device 612. The communication device 612 can be any type of device or system capable of enabling communication with internal devices and / or communication with a network and may include, but are not limited to, a modem, a network card, an infrared communication device, a wireless communication device, and / or a chipset, such as a Bluetooth device, an IEEE 1302.11 device, a WiFi device, a WiMax device, a cellular communication device, and / or similar devices.

[0047] When the computing device 600 is used as an in-vehicle device, it can also be connected to external devices (e.g., a GPS receiver, sensors for sensing different environmental data (such as an acceleration sensor, a wheel speed sensor, a gyroscope, etc.)). In this way, the computing device 600 can receive, for example, positioning data and sensor data indicating the form condition of the vehicle. When the computing device 600 is used as an in-vehicle device, it can also be connected to other devices for controlling the driving and operation of the vehicle (e.g., an engine system, a windshield wiper, an anti-lock braking system, etc.).

[0048] In addition, the non-transitory storage device 610 may have map information and software components, so that the processor 604 can implement route guidance processing. In addition, the output device 606 may include a display for displaying maps, displaying positioning markers of the vehicle, and displaying images indicating the driving status of the vehicle. The output device 606 may also include a speaker or a headphone jack for audio guidance.

[0049] The bus 602 may include, but is not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus. In particular, for in-vehicle devices, the bus 602 may also include a Controller Area Network (CAN) bus or other architectures designed for automotive applications.

[0050] The computing device 400 may also include a working memory 614, which can be any type of working memory capable of storing instructions and / or data beneficial to the operation of the processor 604 and may include, but is not limited to, random access memory and / or read-only storage devices.

[0051] The software components may be located in the working memory 614, and these software components include, but are not limited to, an operating system 616, one or more application programs 618, drivers, and / or other data and code. The instructions for implementing the above methods and steps may be included in the one or more application programs 618, and the modules / units / components of the various devices / servers / client devices described above may be implemented by the processor 604 reading and executing the instructions of the one or more application programs 618.

[0052] It should also be recognized that changes can be made according to specific requirements. For example, custom hardware may also be used, and / or specific components may be implemented in hardware, software, firmware, middleware, microcode, hardware description language, or any combination thereof. In addition, connections with other computing devices, such as network input / output devices, etc., may be adopted. For example, the disclosed methods and parts or all of the devices may be implemented by programming hardware (such as programmable logic circuits including Field Programmable Gate Arrays (FPGAs) and / or Programmable Logic Arrays (PLAs)) with assembly language or hardware programming languages (such as VERILOG, VHDL, C++) using the logic and algorithms according to the present invention.

[0053] Although aspects of the present invention have been described so far with reference to the accompanying drawings, the above methods, systems, and devices are merely examples, and the scope of the present invention is not limited to these aspects, but is defined only by the appended claims and their equivalents. Various components may be omitted or may also be replaced by equivalent components. Additionally, the steps may be implemented in an order different from the order described in the present invention. Furthermore, the various components may be combined in various ways. It is also important that, as technology develops, many of the components described may be replaced by equivalent components that emerge later.

Claims

1. A method for assisting gesture control in a vehicle, comprising: identifying a generated action made by a user to activate a gesture interaction area; and generating the gesture interaction area in a predetermined three-dimensional space based on the generated action made by the user to activate the gesture interaction area.

2. The method according to claim 1, characterized in that, The action is that the user places a hand within the viewing range of a camera.

3. The method according to claim 1, characterized in that The action is that the user issues a voice command.

4. The method according to claim 1, characterized in that The gesture interaction area is a contactless ultrasonic gesture interaction area generated by an ultrasonic speaker array.

5. The method of claim 1, further comprising: Identifying a gesture made by the user within the gesture interaction area to interact with the vehicle.

6. A system for assisting gesture control in a vehicle, comprising: a gesture interaction area activation module configured to identify a generated action made by a user to activate a gesture interaction area; and a gesture interaction area generation module configured to generate the gesture interaction area in a predetermined three-dimensional space based on the generated action made by the user to activate the gesture interaction area.

7. The system according to claim 6, wherein The gesture interaction area activation module is a camera, and the action is that the user places a hand within the viewing range of the camera.

8. The system according to claim 6, wherein, The gesture interaction area activation module is a voice recognition device, and the action is that the user issues a voice command.

9. The system according to claim 6, wherein The gesture interaction area generation module is an ultrasonic speaker array, and the gesture interaction area is a contactless ultrasonic gesture interaction area generated by the ultrasonic speaker array.

10. The system according to claim 6, further comprising: An in-vehicle infotainment system configured to identify a gesture made by the user within the gesture interaction area to interact with the vehicle.

Images