Depth-based gesture control method and system for it
The depth value-based gesture operation method in vehicles uses stereoscopic images to recognize and execute commands, improving driving safety by allowing complex hand movements for device control without diverting the driver's attention.
Patent Information
- Application Number
- DE102017125578
- Authority / Receiving Office
- DE · DE
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2017-11-02
- Publication Date
- 2024-11-28
- Estimated Expiration
- 2037-11-02
AI Technical Summary
The increasing number of electronic devices in vehicles requires drivers to operate multiple devices simultaneously, diverting their attention and compromising driving safety, whether through direct operation or use of integrated touch screens.
A depth value-based gesture operation method using stereoscopic images to detect and execute commands, incorporating a stereo image camera module and processing unit to recognize and execute gestures, allowing precise three-dimensional hand movements for device control.
Enhances driving safety by enabling precise gesture recognition, reducing the need for drivers to look away from the road and allowing for more complex hand movements, thereby minimizing distractions.
Smart Images

Figure 00000000_0000_ABST
Abstract
Description
FIELD OF THE INVENTION
[0001] The present invention generally relates to a gesture operation method, and more particularly to a depth-based gesture operation method in which operation can be performed by recognizing operation gestures in the depth values of a stereoscopic image captured by a mobile vehicle. BACKGROUND OF THE INVENTION
[0002] Cars are the most popular mobile vehicles for modern families. Car manufacturers introduce a variety of new models, and consumers also gradually phase out old cars. After replacing old cars with new ones, consumers can also purchase electronic subsystems for mobile vehicles, such as GPS systems, tire pressure monitoring systems, car multimedia devices, dashboard cameras, and windshield displays, opening up a huge consumer market. Manufacturers of electronic devices for mobile vehicles also develop various electronic devices applicable to mobile vehicles, generating huge profits every year.
[0003] Accordingly, today's mobile vehicle generally includes one or more electronic devices. Users can turn the installed electronic devices on or off according to the current usage scenario. For example, when driving the mobile vehicle to a known destination, the driver can turn off GPS navigation, whereas when traveling to an unknown location, the driver can turn on GPS navigation and play music. Unfortunately, as the number of electronic devices in a mobile vehicle increases, the driver must operate multiple electronic devices simultaneously. In addition, the driver must also operate basic electronic devices such as wipers, lights, and windows. This may consequently result in the driver being too busy to drive safely.
[0004] To solve these problems, some manufacturers further integrate the devices into a mobile vehicle. The driver can operate the devices integrated into the system using a touchscreen in the car. This allows most devices in the mobile vehicle to be operated by the driver. However, the screen of such an integrated system is usually located to one side of the driver. Whenever the driver needs to operate the devices, they must temporarily avert their gaze. Consequently, this method can still pose problems with driving safety.
[0005] Patent application publication US 2015 / 0 177 842 A1 discloses a gesture control method having the features of the preamble of claim 1 and a gesture control system designed to carry out the method having the features of the preamble of claim 8. Patent application publication US 2009 / 0 324 008 A1 describes a method, a device and a computer program for carrying out gesture analyses. SUMMARY
[0006] One object of the present invention is to provide a depth-based gesture control method. After acquiring stereoscopic images, an algorithm is executed to detect whether command gestures are contained in the stereoscopic images. Subsequently, the actions corresponding to the command gesture can be executed.
[0007] To achieve the above object and achieve efficiency, the present invention provides a depth-based gesture control method for use in a mobile vehicle, which gesture control method has the features of claim 1. Further embodiments of the gesture control method are the subject of the claims dependent on claim 1.The depth-based gesture control method applicable to mobile vehicles includes the steps of acquiring a first stereoscopic image, detecting whether a plurality of depth values of the first stereoscopic image include a trigger gesture by executing a first algorithm, acquiring a second stereoscopic image, detecting whether a plurality of depth values of the second stereoscopic image include a command gesture by executing a second algorithm, and executing an action corresponding to the command gesture by a processing unit. A first camera unit captures a first image, and a second camera unit captures a second image. A processing unit generates a first stereoscopic image depending on the first image, the second image, and the included angle.A processing unit judges whether the number of the plurality of depth values of the first stereoscopic image is less than a threshold value. If so, a structured light projection module continuously projects a plurality of light planes. A structured light camera unit receives a plurality of light pattern signals and obtains a third stereoscopic image through calculations in accordance with the plurality of light pattern signals. The processing unit combines the first stereoscopic image and the third stereoscopic image to form a combined first stereoscopic image. According to one embodiment of the present invention, a clustering algorithm is applied to detect the position of the trigger gesture in the mobile vehicle. The processing unit judges whether executing the action corresponding to the command gesture is permissible according to the position of the trigger gesture in the mobile vehicle.If so, the processing unit executes the action corresponding to the command gesture.
[0008] According to one embodiment of the present invention, a first camera unit captures a third image, and a second camera unit captures a fourth image. A processing unit generates a second stereoscopic image depending on the third image, the fourth image, and the included angle.
[0009] According to one embodiment of the present invention, a structured light projection module continuously projects a plurality of light planes. A structured light camera unit receives a plurality of light pattern signals and performs calculations to obtain a first stereoscopic image or a second stereoscopic image according to the light pattern signals.
[0010] According to one embodiment of the present invention, a first camera unit captures a third image, and a second camera unit captures a fourth image. A processing unit generates a second stereoscopic image based on the third image, the fourth image, and the included angle. A processing unit judges whether the number of the plurality of depth values of the second stereoscopic image is less than a threshold value. If so, a structured light projection module continuously projects a plurality of light planes. A structured light camera unit receives a plurality of light pattern signals and obtains a fourth stereoscopic image through calculations in accordance with the plurality of light pattern signals. The processing unit combines the second stereoscopic image and the fourth stereoscopic image to form a combined second stereoscopic image.
[0011] According to one embodiment of the present invention, a structured light projection module continuously projects a plurality of light planes. A structured light camera unit receives a plurality of light pattern signals and obtains a first stereoscopic image through calculations in accordance with the plurality of light pattern signals. A processing unit judges whether the number of the plurality of depth values of the first stereoscopic image is less than a threshold value. If so, a first camera unit acquires a first image and a second camera unit acquires a second image. The processing unit generates a third stereoscopic image depending on the first image, the second image, and the included angle. Furthermore, the processing unit combines the first stereoscopic image and the third stereoscopic image to form a combined first stereoscopic image.
[0012] According to one embodiment of the present invention, a structured light projection module continuously projects a plurality of light planes. A structured light camera unit receives a plurality of light pattern signals and obtains a second stereoscopic image through calculations in accordance with the plurality of light pattern signals. A processing unit judges whether the number of the plurality of depth values of the second stereoscopic image is less than a threshold value. If so, a first camera unit acquires a third image and a second camera unit acquires a fourth image. The processing unit generates a fourth stereoscopic image depending on the third image, the fourth image, and the included angle. Furthermore, the processing unit combines the second stereoscopic image and the fourth stereoscopic image to form a combined second stereoscopic image.
[0013] Furthermore, the present invention provides a depth-based gesture control system having the features of claim 8. Further embodiments of the gesture control system are subject to the claims dependent on claim 8. The system comprises a stereoscopic camera module and a processing unit. The stereoscopic camera module captures a first stereoscopic image and a second stereoscopic image. The processing unit executes a first algorithm to detect whether a plurality of depth values of the first stereoscopic image include a trigger gesture and executes a second algorithm to detect whether a plurality of depth values of the second stereoscopic image include a command gesture. The processing unit then executes an operation corresponding to the command gesture. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 shows a flowchart of the depth-based gesture operation method according to the first embodiment of the present invention, Fig. 2A shows a schematic system diagram of the depth-based gesture control method according to the first embodiment of the present invention, Fig. 2B shows a schematic system diagram of the depth-based gesture control method according to the first embodiment of the present invention, illustrating that the stereo camera module is arranged in a mobile vehicle, Fig. 3 shows a schematic diagram of the stereo camera module in the depth-based gesture operation method according to the first embodiment of the present invention, Fig. 4 shows a schematic diagram of the stereo camera module in the depth-based gesture operation method according to a second embodiment of the present invention, Fig. 5 shows a schematic representation of the depth-based gesture control method according to the second embodiment of the present invention for illustrating the projection of a light plane onto an object, Fig. Fig. 6 is a schematic diagram of the two-dimensional dot matrix included in the light plane in the depth-based gesture operation method according to the second embodiment of the present invention, Fig. 7 shows a flowchart of the depth-based gesture operation method according to a third embodiment of the present invention, Fig. Fig. 8 shows a flowchart of step S1 in the depth-based gesture operation method according to a fourth embodiment of the present invention, Fig. 9 shows a schematic system diagram of the depth-based gesture operation method according to the fourth embodiment of the present invention, and Fig. 10 shows a flowchart of step S5 in the depth value-based gesture operation method according to a fifth embodiment of the present invention. DETAILED DESCRIPTION
[0014] In order to make the structure and features as well as the performance of the present invention more understandable and apparent, a detailed description of the present invention is provided below in conjunction with embodiments and the accompanying drawings.
[0015] According to the prior art, driving safety is compromised when a driver operates devices installed in a mobile vehicle because they frequently have to temporarily shift their line of sight or there are too many devices to operate. Therefore, the present invention provides a gesture operation method.
[0016] The following describes the flow of the depth-based gesture control method according to the first embodiment of the present invention. Fig. 1, which shows a flowchart of the depth-based gesture control method according to the first embodiment of the present invention. As shown in the drawing, the depth-based gesture control method according to the present embodiment includes the steps: Step S1: Acquiring a first stereoscopic image, Step S3: Detect whether a trigger gesture is included, Step S5: Acquiring a second stereoscopic image, Step S7: Detect whether a command gesture is included, and Step S9: Perform the operation corresponding to the command gesture.
[0017] The following describes the system required to implement the depth-based gesture control method according to the present invention. It is based on Fig. 2A, which shows a schematic system diagram of the depth-based gesture control method according to the first embodiment of the present invention. As shown in the figure, the system for the depth-based gesture control method according to the present invention includes a gesture control system 1, which includes a stereo camera module 10 and a processing unit 30.
[0018] The aforementioned gesture control system 1 is installed in a mobile vehicle, so that the stereo camera module 10 photographs the interior of the mobile vehicle. In other words, the stereo camera module 10 is positioned at a location suitable for photographing the driver's seat and / or the passenger's seat. The mobile vehicle can be a car, a truck, a bus, an electric car, or another vehicle.
[0019] As in Fig. 2B, the stereo imaging camera module 10 according to a preferred embodiment of the present invention is arranged in the mobile vehicle such that the stereo imaging camera module is capable of photographing the interior of the mobile vehicle.
[0020] As in Fig. 3, the stereo image camera module 10 includes a first camera unit 101 and a second camera unit 103. An included angle 105 is formed between the first camera unit 101 and the second camera unit 103.
[0021] The aforementioned processing unit 30 is an electronic component capable of performing arithmetic, logical and imaging operations.
[0022] The following describes the flow of the depth-based gesture control method according to the first embodiment of the present invention. Reference is made to the Fig. 1 to 3. During the operation of the electronic device arranged in the mobile vehicle, steps S1 to S9 are executed.
[0023] In step S1 of capturing a first stereoscopic image, the first camera unit 101 captures a first image, and the second camera unit 103 captures a second image. The processing unit 30 performs an operation to obtain a first stereoscopic image based on the first image, the second image, and the included angle 105.
[0024] In step S3 for detecting whether a trigger gesture is included, the processing unit 30 executes a first algorithm to detect whether a trigger gesture is included in a plurality of depth values of the first stereoscopic image. If so, step S5 is performed; otherwise, the process returns to step S1. The first algorithm is a gesture recognition algorithm. Specifically, the processing unit 30 uses the gesture recognition algorithm to detect, depending on the depth values (the Z values) of a plurality of pixels in the first stereoscopic image, whether a trigger gesture is included in the first stereoscopic image.
[0025] In step S5 of capturing a second stereoscopic image, the first camera unit 101 captures a third image, and the second camera unit 103 captures a fourth image. The processing unit 30 performs an operation to obtain a second stereoscopic image depending on the third image, the fourth image, and the included angle 105.
[0026] In step S7 for detecting whether a command gesture is included, processing unit 30 executes a second algorithm to detect whether a command gesture is included in a plurality of depth values of the second stereoscopic image. If so, step S9 is performed; otherwise, the process returns to step S5. The second algorithm is a gesture recognition algorithm. Specifically, processing unit 30 uses the gesture recognition algorithm to detect, depending on the depth values (the Z values) of a plurality of pixels in the second stereoscopic image, whether a command gesture is included in the second stereoscopic image.
[0027] In step S9 for executing the action corresponding to the command gesture, after recognizing the command gesture, the processing unit 30 executes the action corresponding to the command gesture. For example, if the command gesture is to turn on the GPS, the processing unit 30 turns on the GPS. If the command gesture is to turn on an audio device, the processing unit 30 turns on the audio device.
[0028] Then, the depth-based gesture operation method according to the first embodiment of the present invention is completed. Compared with gesture recognition methods using plane images, the present invention, which performs gesture recognition using stereoscopic images, can detect the outlines of hands more precisely and thus realize more accurate recognition performance.
[0029] Additionally, when performing gesture recognition using plane images, the user's hands are limited to planar movements, namely, up, down, left, and right movements. In contrast, when performing gesture recognition using the depth values of stereoscopic images, three-dimensional movements of the user's hands can also be recognized, namely, forward, backward, up, down, left, and right movements. This allows the user to record more gestures in the gesture control system 1, thereby enabling the user to operate one or more devices arranged in the mobile vehicle.
[0030] Furthermore, according to the present invention, the trigger gesture is first detected before recognizing a command gesture. The user must first perform the trigger gesture, and the gesture control system 1 can recognize the trigger gesture. This can prevent the gesture control system 1 from executing incorrect actions when the user unintentionally moves their hands back and forth.
[0031] The following describes the depth-based gesture control method according to the second embodiment of the present invention. Fig. 4, which shows the system required for the depth-based gesture operation method according to the second embodiment of the present invention. The difference between the present embodiment and the first embodiment is that the gesture operation system 1 according to the present embodiment includes a structured light projection module 107 and a structured light camera unit 109. The rest is the same as in the first embodiment. Therefore, the details will not be described again.
[0032] The structured light projection module 107 includes a laser unit 1071 and a lens group 1073. As shown in Fig. As shown in Figure 5, the structured light source projects controllable light spots, light stripes, or light planes onto the surface of an object 2. Subsequently, a sensor, such as a camera, captures the reflected image. After geometric operations, the stereoscopic image of the object is obtained. According to one embodiment of the present invention, an invisible laser is used as the light source due to its excellent coherence, slow attenuation, long measuring distance, and high accuracy. Furthermore, the laser is less affected by other light sources and is therefore preferred over normal light. The laser provided by the laser unit 1071 is scattered after passing through the lens group 1073 and forms a light plane 1075 in space.The lens group 1073 employed in the present invention may include a pattern lens comprising patterned microstructures, thus allowing the light plane formed by the penetrating laser to have pattern characteristics, such as the two-dimensional light spot matrix shown in FIG. Fig. 6 is shown.
[0033] According to the present embodiment, in steps S1 and S5, the structured light projection module 107 continuously projects a plurality of light planes 1075 onto the surface of one or more objects 2 in the mobile vehicle. Subsequently, the structured light camera unit 10 captures a plurality of light pattern signals reflected from the surface of the object 2 after projecting the light planes 1075. The processing unit 30 performs calculations based on the received plurality of light pattern signals to obtain the first stereoscopic image and the second stereoscopic image.
[0034] According to another embodiment of the present invention, the projection distance of the laser can be further increased or reduced by adjusting the power of the laser unit 1071. For example, the power of the laser unit 1071 can be increased in an open space and reduced in a confined space.
[0035] In the depth-based gesture operation method according to the second embodiment of the present invention, since the structured light projection module 107 can spontaneously project structured light onto a surrounding object 2, the method can be applied in a dark or dim environment.
[0036] The following describes the depth-based gesture control method according to the third embodiment of the present invention. Fig. 7. The difference between the present embodiment and the first embodiment is that the method according to the present embodiment further includes steps S4 and S8. The rest is the same as in the first embodiment. Therefore, the details will not be described again.
[0037] In step S4 for identifying the position of the trigger gesture in the mobile vehicle, the processing unit 30 executes a clustering algorithm to identify the position of the trigger gesture in the mobile vehicle. In other words, the processing unit 30 executes the clustering algorithm to identify the position of the user making the trigger gesture in the mobile vehicle. For example, the user making the trigger gesture is in the driver's seat, the passenger seat, or the rear seat.
[0038] According to one embodiment of the present invention, after identifying the position of the user performing the trigger gesture in the mobile vehicle, the user identification can be further captured. For example, the user in the driver's seat is the driver, the user in the front passenger seat is the front passenger or the passenger, or the user in the back seat is the passenger.
[0039] According to one embodiment of the present invention, the clustering algorithm is the K-means clustering algorithm or another algorithm that can group objects.
[0040] In step S8 for determining whether the execution of the command gesture is permitted, the processing unit 30 determines whether the execution of the action corresponding to the command gesture is permitted depending on the position of the trigger gesture in the mobile vehicle. In other words, the processing unit 30 will determine whether the action corresponding to the command gesture should be performed depending on the position of the user executing the trigger gesture.
[0041] Thereafter, the depth-based gesture control method according to the third embodiment of the present invention is completed. By employing the present embodiment, different authorization levels can be granted according to different positions. For example, the driver may have authorization to operate the windshield wiper, the car lamp, and the GPS, the front passenger may have authorization to operate the GPS and the multimedia system, and the passengers may have authorization to operate the multimedia system. This can reduce the risk of users inadvertently activating devices, such as the windshield wiper, which may disturb the driver. Consequently, the safety of the present invention is improved.
[0042] Next, the depth-based gesture operation method according to the fourth embodiment of the present invention will be described. Fig. 8 and Fig. 9, which shows a flowchart of step S1, which further includes steps S101 to S105, and a schematic system diagram of the depth-based gesture control method according to the fourth embodiment of the present invention. The difference between the present and the first embodiment is that, according to the present embodiment, steps S101 to S105 are also provided after step S1. The gesture control system 1 comprises a first camera unit 101, a second camera unit 103, a structured light projection module 107, and a structured light camera unit 109.
[0043] In step S1 of capturing a first stereoscopic image, the first camera unit 101 captures a first image, and the second camera unit 103 captures a second image. The processing unit 30 performs an operation to obtain a first stereoscopic image depending on the first image, the second image, and the included angle 105.
[0044] In step S101, the processing unit 30 determines whether the depth value of the first stereoscopic image is less than a threshold value. If so, step S103 is executed; otherwise, the processing unit proceeds to step S3. In other words, the processing unit 30 searches the pixels of the first stereoscopic image and collects statistics on the number of pixels containing depth values, thereby determining the number of depth values. The processing unit 30 then determines whether the number of depth values is less than a threshold value.
[0045] In step S103 of acquiring a third stereoscopic image, the structured light projection module 107 continuously projects a plurality of light planes 1075 onto the surface of one or more objects 2 in the mobile vehicle. Subsequently, the structured light camera unit 109 acquires a plurality of light pattern signals reflected from the surface of the object 2 after projecting the light planes 1075. The processing unit 30 performs calculations based on the received plurality of light pattern signals to obtain the third stereoscopic image.
[0046] In step S105 for fusing the first stereoscopic image and the third stereoscopic image, the processing unit 30 performs the fusion according to an image fusion algorithm to obtain a fused first stereoscopic image.
[0047] Subsequently, in step S3, the processing unit judges according to a first algorithm whether the fused first stereoscopic image contains the trigger gesture.
[0048] According to the present embodiment, the first stereoscopic image can also be acquired during the execution of step S1 using the structured light projection module 107 and the structured light camera unit 109. In this case, in step S103, the processing unit 30 performs an operation to obtain the third stereoscopic image depending on the first image acquired by the first camera unit 101, the second image acquired by the second camera unit 103, and the included angle 105.
[0049] Since the first and second images captured by the first and second camera units 101, 103 in a dimly lit location may appear partially or completely dark, the calculated first stereoscopic image may not include a depth value in some pixels. Furthermore, the light pattern signals in the first stereoscopic image captured by the structured light projection module 107 and the structured light camera unit 109 in a bright environment will be incomplete due to light interference. Under this condition, the obtained first stereoscopic image may not include a depth value in some pixels. If the pixels that do not include a depth value are excessive in the first stereoscopic image, it is difficult for the first algorithm to detect whether the first stereoscopic image includes the trigger gesture.According to the present embodiment, once the number of depth values in the first stereoscopic image is judged to be less than a threshold value, the third stereoscopic image is additionally captured and fused with the first stereoscopic image. The fused first stereoscopic image thus generated will contain a sufficient number of depth values to thereby facilitate the first algorithm's detection of whether the fused first stereoscopic image includes the trigger gesture.
[0050] According to one embodiment of the present invention, the threshold is 50% of the total number of pixels of the first stereoscopic image. In other words, the threshold is 500 if the first stereoscopic image contains 1000 pixels.
[0051] Next, the depth-based gesture operation method according to the fifth embodiment of the present invention will be described. Fig. 10, which shows a flowchart of step S5, which further includes steps S501 to S505 in the depth-based gesture control method according to the fifth embodiment of the present invention. The difference between the present and the first embodiment is that, according to the present embodiment, steps S501 to S505 are also provided after step S5. The system required according to the present embodiment corresponds to that according to the fourth embodiment. Fig. 9. The gesture control system 1 comprises a first camera unit 101, a second camera unit 103, a structured light projection module 107, and a structured light camera unit 109.
[0052] In step S5 of capturing a second stereoscopic image, the first camera unit 101 captures a third image, and the second camera unit 103 captures a fourth image. The processing unit 30 performs an operation to obtain a second stereoscopic image depending on the third image, the fourth image, and the included angle 105.
[0053] In step S101, the processing unit 30 determines whether the depth value of the second stereoscopic image is less than a threshold value. If so, step S503 is executed; otherwise, the processing unit proceeds to step S7. In other words, the processing unit 30 searches the pixels of the second stereoscopic image and collects statistics on the number of pixels containing depth values, thereby determining the number of depth values. The processing unit 30 then determines whether the number of depth values is less than a threshold value.
[0054] In step S53 of acquiring a fourth stereoscopic image, the structured light projection module 107 continuously projects a plurality of light planes 1075 onto the surface of one or more objects 2 in the mobile vehicle. Subsequently, the structured light camera unit 109 acquires a plurality of light pattern signals reflected from the surface of the object 2 after the light planes 1075 are projected. The processing unit 30 performs calculations based on the received plurality of light pattern signals to obtain the fourth stereoscopic image.
[0055] In step S505 for fusing the second stereoscopic image and the fourth stereoscopic image, the processing unit 30 performs the fusion according to an image fusion algorithm to obtain a fused second stereoscopic image.
[0056] Thereafter, in step S7, the processing unit judges according to a second algorithm whether the fused second stereoscopic image contains the trigger gesture.
[0057] According to the present embodiment, the second stereoscopic image can also be acquired during the execution of step S5 using the structured light projection module 107 and the structured light camera unit 109. In this case, in step S503, the processing unit 30 performs an operation to obtain the fourth stereoscopic image depending on the third image acquired by the first camera unit 101, the fourth image acquired by the second camera unit 103, and the included angle 105.
[0058] According to one embodiment of the present invention, the threshold is 50% of the total number of pixels of the first stereoscopic image. In other words, the threshold is 500 if the second stereoscopic image contains 1000 pixels.
[0059] According to one embodiment of the present invention, the first stereoscopic image may consist of continuous images representing the continuous gesture of the trigger gesture. In step S3, the processing unit 30 executes the first algorithm and the hidden Markov model (HMM) to determine whether the first stereoscopic image contains the trigger gesture.
[0060] According to one embodiment of the present invention, the first stereoscopic image may consist of continuous images representing the continuous gesture of the trigger gesture. In step S3, the processing unit 30 executes the first algorithm and the hidden Markov model (HMM) to determine whether the first stereoscopic image contains the trigger gesture.
[0061] According to an embodiment of the present invention, the plurality of light pattern signals are formed by the reflection of the plurality of light planes from one or more objects.
[0062] Accordingly, the present invention complies with the statutory requirements due to its novelty, inventive step, and applicability. However, the foregoing description relates only to exemplary embodiments of the present invention and is not intended to limit the scope and spirit of the present invention. All equivalent changes and modifications that can be made in accordance with the form, structure, features, or spirit defined in the claims of the present invention are intended to be covered by the appended claims of the present invention.
Claims
[1] A depth-based gesture control method for use in a mobile vehicle, wherein the mobile vehicle includes a stereo camera module (10), wherein the stereo camera module (10) includes a first camera unit (101), a second camera unit (103), a structured light projection module (107) and a structured light camera unit (109), and wherein an included angle (105) is formed between the first camera unit (101) and the second camera unit (103), the gesture control method comprising the steps of: Capturing a first stereoscopic image, Detecting whether a plurality of depth values of the first stereoscopic image include a trigger gesture by executing a first algorithm, Capturing a second stereoscopic image, Detecting whether a plurality of depth values of the second stereoscopic image include a command gesture by executing a second algorithm, and Carrying out a work process corresponding to the command gesture by a processing unit (30), capturing a first image by the first camera unit (101), capturing a second image by the second camera unit (103), characterized by generating the first stereoscopic image by the processing unit (30) as a function of the first image, the second image and the included angle (105), Judging by the processing unit (30) whether the number of the plurality of depth values of the first stereoscopic image is smaller than a threshold value, continuously projecting a plurality of light planes by the structured light projection module (107) when the number of the plurality of depth values of the first stereoscopic image is less than the threshold value, Receiving a plurality of light pattern signals by the structured light camera unit (109), Calculating based on the plurality of light pattern signals to obtain a third stereoscopic image, and Combining the first stereoscopic image and the third stereoscopic image by the processing unit (30) to form a fused first stereoscopic image. [2] A depth-based gesture control method according to claim 1, further comprising the steps of: Identifying a position of the trigger gesture in the mobile vehicle using a clustering algorithm, Judging by the processing unit (30) whether the execution of the process corresponding to the command gesture is permissible depending on the position of the trigger gesture in the mobile vehicle, and Carrying out the operation corresponding to the command gesture by the processing unit (30), if this is permissible. [3] A depth-based gesture control method according to claim 1, further comprising the steps of: capturing a third image by the first camera unit (101), capturing a fourth image by the second camera unit (103), and Generating the second stereoscopic image by the processing unit (30) as a function of the third image, the fourth image and the included angle (105). [4] A depth-based gesture control method according to claim 1, further comprising the steps of: continuously projecting a plurality of light planes by the structured light projection module (107), Receiving a plurality of light pattern signals by the structured light camera unit (109), and Calculating based on the plurality of light pattern signals to obtain the first stereoscopic image or the second stereoscopic image. [5] A depth-based gesture control method according to claim 1, further comprising the steps of: capturing a third image by the first camera unit (101), capturing a fourth image by the second camera unit (103), generating the second stereoscopic image by the processing unit (30) as a function of the third image, the fourth image and the included angle (105), Judging by the processing unit (30) whether the number of the plurality of depth values of the second stereoscopic image is less than a threshold value, continuously projecting a plurality of light planes by the structured light projection module (107) when the number of the plurality of depth values of the second stereoscopic image is less than the threshold value, Receiving a plurality of light pattern signals by the structured light camera unit (109), Calculating based on the plurality of light pattern signals to obtain a fourth stereoscopic image, and Combining the second stereoscopic image and the fourth stereoscopic image by the processing unit (30) to form a fused second stereoscopic image. [6] A depth-based gesture control method according to claim 1, further comprising the steps of: continuously projecting a plurality of light planes by the structured light projection module (107), Receiving a plurality of light pattern signals by the structured light camera unit (109), Calculation based on the plurality of light pattern signals to obtain the first stereoscopic image, Judging by the processing unit (30) whether the number of the plurality of depth values of the first stereoscopic image is smaller than a threshold value, capturing a first image by the first camera unit (101) when the number of the plurality of depth values of the first stereoscopic image is smaller than the threshold value, capturing a second image by the second camera unit (103), generating a third stereoscopic image by the processing unit (30) as a function of the first image, the second image and the included angle (105), and Combining the first stereoscopic image and the third stereoscopic image by the processing unit (30) to form a fused first stereoscopic image. [7] A depth-based gesture control method according to claim 1, further comprising the steps of: Continuously projecting a plurality of light planes by the structured light projection module (107), Receiving a plurality of light pattern signals by the structured light camera unit (109), Calculation based on the plurality of light pattern signals to obtain the second stereoscopic image, Judging by the processing unit (30) whether the number of the plurality of depth values of the second stereoscopic image is smaller than a threshold value, capturing a third image by the first camera unit (101) when the number of the plurality of depth values of the second stereoscopic image is smaller than the threshold value, capturing a fourth image by the second camera unit (103), generating a fourth stereoscopic image by the processing unit (30) as a function of the third image, the fourth image and the included angle (105), and Combining the second stereoscopic image and the fourth stereoscopic image by the processing unit (30) to form a fused second stereoscopic image. [8] Depth-based gesture control system (1) for a mobile vehicle, comprising: a stereo image camera module (10) which captures a first stereoscopic image and a second stereoscopic image, in which the stereo image camera module (10) includes a first camera unit (101), a second camera unit (103), a structured light projection module (107) and a structured light camera unit (109), wherein an included angle (105) is formed between the first camera unit (101) and the second camera unit (103), and a processing unit (30) which detects, by executing a first algorithm, whether a plurality of depth values of the first stereoscopic image include a trigger gesture, and which detects, by executing a second algorithm, whether a plurality of depth values of the second stereoscopic image include a command gesture, wherein the processing unit (30) can execute an operation corresponding to the command gesture, characterized by , that the processing unit (30) determines a first stereoscopic image as a function of a first image captured by the first camera unit (101), a second image captured by the second camera unit (103), and the included angle (105), wherein the processing unit (30) assesses whether the number of the plurality of depth values of the first stereoscopic image is less than a threshold value, wherein the structured light projection module (107) continuously projects a plurality of light planes if the number of the plurality of depth values of the first stereoscopic image is less than the threshold value, wherein the structured light camera unit (109) receives a plurality of light pattern signals, wherein the processing unit (30) determines a third stereoscopic image by calculations based on the plurality of light pattern signals,and wherein the processing unit (30) combines the first stereoscopic image and the third stereoscopic image to form a fused first stereoscopic image., [9] A depth-based gesture control system (1) according to claim 8, wherein the processing unit (30) identifies a position of the trigger gesture in the mobile vehicle by executing a clustering algorithm, and judges, depending on the position of the trigger gesture in the mobile vehicle, whether the execution of the operation corresponding to the command gesture is permitted, and if it is permitted, the processing unit (30) can execute the operation corresponding to the command gesture. [10] Depth-based gesture control system (1) according to claim 8, wherein the processing unit (30) determines a second stereoscopic image depending on a third image captured by the first camera unit (101), a fourth image captured by the second camera unit (103) and the included angle (105). [11] The depth-based gesture control system (1) according to claim 8, wherein the structured light projection module (107) continuously projects a plurality of light planes, the structured light camera unit (109) receives a plurality of light pattern signals, and the processing unit (30) determines the first stereoscopic image or the second stereoscopic image by calculations based on the plurality of light pattern signals. [12] Depth-based gesture control system (1) according to claim 8, wherein the processing unit (30) determines a second stereoscopic image depending on a third image captured by the first camera unit (101), a fourth image captured by the second camera unit (103), and the included angle (105), wherein the processing unit (30) judges whether the number of the plurality of depth values of the second stereoscopic image is less than a threshold value, wherein the structured light projection module (107) continuously projects a plurality of light planes when the number of the plurality of depth values of the second stereoscopic image is less than the threshold value, wherein the structured light camera unit (109) receives a plurality of light pattern signals, wherein the processing unit (30) determines a fourth stereoscopic image by calculations based on the plurality of light pattern signals,and wherein the processing unit (30) combines the second stereoscopic image and the fourth stereoscopic image to form a fused second stereoscopic image., [13] Depth-based gesture control system (1) according to claim 8, wherein the structured light projection module (107) continuously projects a plurality of light planes, wherein the structured light camera unit (109) receives a plurality of light pattern signals, wherein the processing unit (30) determines the first stereoscopic image by calculations based on the plurality of light pattern signals, wherein the processing unit (30) judges whether the number of the plurality of depth values of the first stereoscopic image is less than a threshold value, wherein the processing unit (30) determines a third stereoscopic image depending on a first image captured by the first camera unit (101), a second image captured by the second camera unit (103), and the included angle (105), if the number of the plurality of depth values of the first stereoscopic image is less than the threshold value,and wherein the processing unit (30) combines the first stereoscopic image and the third stereoscopic image to form a fused first stereoscopic image., [14] Depth-based gesture control system (1) according to claim 8, wherein the structured light projection module (107) continuously projects a plurality of light planes, wherein the structured light camera unit (109) receives a plurality of light pattern signals, wherein the processing unit (30) determines the second stereoscopic image by calculations based on the plurality of light pattern signals, wherein the processing unit (30) judges whether the number of the plurality of depth values of the second stereoscopic image is less than a threshold value, wherein the processing unit (30) determines a fourth stereoscopic image depending on a third image acquired by the first camera unit (101), a fourth image acquired by the second camera unit (103) and the included angle (105) if the number of the plurality of depth values of the second stereoscopic image is less than the threshold value,and wherein the processing unit (30) combines the second stereoscopic image and the fourth stereoscopic image to form a fused second stereoscopic image.,
Citation Information
Patent Citations
Method, appartaus and computer program product for providing gesture analysis
US20090324008A1
3D Gesture Based User Authorization and Device Control Methods
US20150177842A1
