VR equipment and method for realizing gesture recognition based on flash laser radar

Abstract

The invention discloses VR equipment and method for realizing gesture recognition based on a flash laser radar. The VR equipment comprises a glove, the reflectivity of the glove on the surfaces of fingers, the palm center and the palm back is larger than that of the surfaces of other positions, and a flash laser radar emits laser beams to the glove and receives the laser beams reflected by the glove; and the head-mounted display is used for identifying the gesture presented by the glove according to the signal intensity in the laser beam reflected by the glove. The glove is not provided with an electronic component, does not need to be powered, cannot generate the problems of power consumption, heat dissipation, safety and the like, is low in manufacturing cost, production difficulty and failure rate, is not prone to damage, can reduce the difficulty of gesture capture and judgment of a head-mounted display, improves the gesture recognition precision, and improves the user experience. And related algorithms on the head-mounted display are simple and easy to realize.

Description

Technical Field

[0001] Embodiments of the present application relate to the field of virtual reality technology, and in particular, to a VR device and method for gesture recognition based on a flash lidar. Background Art

[0002] With the development of virtual reality (VR) technology and augmented reality (AR) technology, the research on VR device and AR device controllers has become a current research hotspot. Currently, the controllers of VR devices and AR devices mainly include gamepads, gloves, and finger rings, etc. VR devices and AR devices can achieve interaction with head-mounted displays through the controllers.

[0003] In the related art, the current VR devices and AR devices usually have the following problems in gesture recognition interaction:

[0004] 1. Gloves and finger rings usually have electronic components, such as LED light sources, etc., which need to be powered, and problems such as power consumption, heat dissipation, and safety will occur.

[0005] 2. Such gloves and finger rings with electronic components usually have a high production cost, great production difficulty, and a high failure rate, and are easily damaged.

[0006] 3. It is difficult for the head-mounted display to capture and judge gestures, and the accuracy is low, which poses high requirements on the sensors and related algorithms on the head-mounted display. Summary of the Invention

[0007] A VR device and method for gesture recognition based on a flash lidar provided by embodiments of the present application can solve or partially solve the above deficiencies or other deficiencies in the prior art.

[0008] According to a first aspect of the present application, a VR device for gesture recognition based on a flash lidar is provided, including: a glove, the reflectivity of the surface of the glove at the finger, palm, and back of the hand positions is greater than that of the remaining positions, and the flash lidar emits a laser beam to the glove and receives the laser beam reflected back by the glove; and a head-mounted display, which recognizes the gesture presented by the glove according to the signal intensity in the laser beam reflected back by the glove.

[0009] In an embodiment of the present application, the reflectivity of the surface of the glove at the finger, palm, and back of the hand positions is greater than 80%.

[0010] In one embodiment of the present application, the reflectivity of the glove on the surfaces of the 9 interphalangeal joints of the fingers, the palm, and the back of the hand is greater than 80%, and the areas of the palm and the back of the hand with a reflectivity greater than 80% are larger than the 9 interphalangeal joints of the fingers.

[0011] In one embodiment of the present application, the reflectivity of the glove on both the side facing the palm and the side facing the back of the hand of the 9 interphalangeal joints is greater than 80%.

[0012] In one embodiment of the present application, the reflectivity of the glove on the palm and the side of the 9 interphalangeal joints facing the palm is different from the reflectivity of the glove on the back of the hand and the side of the 9 interphalangeal joints facing the back of the hand.

[0013] In one embodiment of the present application, the surface of the glove at the finger, palm, and back of the hand positions has a film layer of a material with a reflectivity greater than 80%.

[0014] In one embodiment of the present application, the flash lidar is located in the head-mounted display.

[0015] In one embodiment of the present application, the flash lidar is located at a fixed position in the environment.

[0016] In one embodiment of the present application, the flash lidar obtains the reflection dot matrix of the glove based on the signal intensity in the laser beam reflected back by the glove; the head-mounted display matches the reflection dot matrix with the stored gesture dot matrix pattern and identifies the gesture presented by the glove according to the matching result.

[0017] In one embodiment of the present application, the flash lidar obtains the reflection dot matrix of the glove based on the signal intensity in the laser beam reflected back by the glove; the head-mounted display concatenates the reflection dot matrix into a hand bone network and identifies the gesture presented by the glove according to the hand bone network.

[0018] In one embodiment of the present application, the head-mounted display concatenates the reflection dot matrix into the hand bone network according to the area, number, position, distance between reflection points, and distance between the reflection points and the flash lidar in the reflection dot matrix.

[0019] In one embodiment of the present application, the head-mounted display also performs corresponding operations according to the instructions corresponding to the gesture presented by the glove.

[0020] According to a second aspect of the present application, a method for gesture recognition based on a flash lidar is provided, which is applied to a VR device. The method includes: emitting a laser beam to a glove through the flash lidar and receiving the laser beam reflected back by the glove, wherein the reflectivity of the surfaces of the fingers, palm, and back of the hand of the glove is greater than that of the remaining positions; and recognizing the gesture presented by the glove by the head-mounted display according to the signal intensity in the laser beam reflected back by the glove.

[0021] The VR device for gesture recognition based on a flash lidar provided according to the embodiments of the present application makes the reflectivity of the surfaces of the fingers, palm, and back of the hand of the glove greater than that of the remaining positions, scans the glove using the flash lidar, and can recognize the gesture presented by the glove according to the signal intensity of the reflected signals generated at different reflectivity positions on the glove surface. At this time, the glove is a passive component, without electronic components, does not require power supply, and will not cause problems such as power consumption, heat dissipation, and safety. Moreover, compared with gloves with electronic components, such gloves have lower manufacturing costs, lower production difficulties, lower failure rates, are not easily damaged, can reduce the difficulty of gesture capture and judgment by the head-mounted display, improve the accuracy of gesture recognition, and the relevant algorithms on the head-mounted display are simple and easy to implement.

[0022] The content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become easily understandable through the following description. Brief Description of the Drawings

[0023] By reading the detailed description of the non-limiting embodiments with reference to the following drawings, other features, objectives, and advantages of the present application will become more obvious. The drawings are used to better understand the solution and do not constitute a limitation to the present application. Among them:

[0024] Figure 1 is a schematic diagram of a VR device for gesture recognition based on a flash lidar according to an embodiment of the present application;

[0025] Figure 2 is a schematic diagram of the reflection points on a glove according to an embodiment of the present application connected into a hand bone network;

[0026] Figure 3A and Figure 3B is a schematic diagram of a gesture presented by a glove according to an embodiment of the present application;

[0027] Figure 4 is a schematic diagram of a gesture presented by a glove according to another embodiment of the present application;

[0028] Figure 5 Schematic diagram of a gesture presented by a glove according to another embodiment of the present application;

[0029] Figure 6 Schematic diagram of a gesture presented by a glove according to still another embodiment of the present application;

[0030] Figure 7 Schematic flowchart of a method for gesture recognition based on a flash lidar according to an embodiment of the present application. Detailed implementation manners

[0031] The following makes an explanation of exemplary embodiments of the present application with reference to the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0032] In the accompanying drawings, for ease of explanation, the thickness, dimensions, and shapes of the components have been slightly adjusted. The drawings are only examples and are not drawn to an exact scale. As used herein, terms such as "substantially", "about", and similar terms are used as terms indicating approximation, rather than terms indicating degree, and are intended to illustrate the inherent deviations in measured or calculated values that would be recognized by those of ordinary skill in the art.

[0033] It should also be understood that expressions such as "including", "having", "comprising", "containing", and / or "including" are open-ended rather than closed-ended expressions in this specification, which means the presence of the stated features, elements, and / or components, but does not exclude the presence of one or more other features, elements, components, and / or their combinations. In addition, when an expression such as "at least one of..." appears after a list of listed features, it modifies the entire list of features, rather than just an individual element in the list. In addition, when describing the embodiments of the present application, the use of "may" means "one or more embodiments of the present application". And the term "exemplary" is intended to refer to an example or illustration.

[0034] Unless otherwise defined, all terms used herein (including engineering terms and scientific and technical terms) have the same meaning as commonly understood by those of ordinary skill in the art to which the present application pertains. It should also be understood that, unless clearly stated in the present application, words defined in a common dictionary should be interpreted as having a meaning consistent with their meaning in the context of the related art, and should not be interpreted in an idealized or overly formal sense.

[0035] It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments may be combined with each other. Additionally, unless explicitly defined or inconsistent with the context, the specific steps included in the methods described in the present application do not have to be limited to the recorded order, but may be executed in any order or in parallel.

[0036] In addition, those skilled in the art can understand that the quantities shown in the drawings of the present application and in the following text, such as the quantity of light films, etc., are only shown for the convenience of illustration, and the specific quantity can be set according to actual needs without departing from the teachings of the disclosed content of the present application.

[0037] With the development of virtual reality technology and augmented reality technology, the research on the controllers of VR devices and AR devices has become a current research hotspot. Currently, the controllers of VR devices and AR devices mainly include hand controllers, gloves, and finger rings, etc. VR devices and AR devices can achieve interaction with the head-mounted display through the controller. Gesture recognition is a natural interaction method and also the ultimate goal of the development of the controllers of VR devices and AR devices. Among them, gloves and finger rings take into account the naturalness and accuracy of gesture recognition interaction and are currently the commonly used controllers for gesture recognition interaction of VR devices and AR devices.

[0038] In the related art, the current VR devices and AR devices usually have the following problems when performing gesture recognition interaction:

[0039] 1. Gloves and finger rings usually have electronic components, such as LED light sources, etc., which need to be powered, and problems such as power consumption, heat dissipation, and safety will occur.

[0040] 2. Such gloves and finger rings with electronic components usually have a relatively high production cost, a large production difficulty, and a relatively high failure rate, and are easily damaged.

[0041] 3. It is difficult for the head-mounted display to capture and judge gestures, and the accuracy is relatively low, which poses high requirements on the sensors and related algorithms on the head-mounted display.

[0042] To solve the above problems, an embodiment of the present application provides a VR device 1000 that realizes gesture recognition based on a flash lidar.

[0043] Figure 1 A schematic diagram of a VR device 1000 that realizes gesture recognition based on a flash lidar according to an embodiment of the present application is shown. As Figure 1As shown in the figure, the VR device 1000 for gesture recognition based on a flash lidar may include: a glove 100 and a head-mounted display 200. Among them, the reflectivity of the glove 100 on the surfaces of the fingers 110, 120, 130, 140, 150, the palm center 160 and the back of the hand is greater than that of the remaining positions. The flash lidar emits a laser beam to the glove 100 and receives the laser beam reflected back by the glove 100. The head-mounted display 200 recognizes the gesture presented by the glove 100 according to the signal intensity in the laser beam reflected back by the glove 100.

[0044] Due to the difference in the reflectivity of the surface of the glove 100, where reflectivity is a physical quantity used to describe the ratio of the energy reflected by an object to the incident energy. When the flash lidar scans the glove 100 by emitting a laser beam, the intensity of the reflected signals generated by the laser beam at different reflectivity positions on the surface of the glove 100 is different. The intensity of the reflected signal is large at the positions with high reflectivity. For example, the intensity of the reflected signals generated at the fingers 110, 120, 130, 140, 150, the palm center 160 and the back of the hand of the glove 100 is large, and the intensity of the reflected signal is small at the positions with low reflectivity. For example, the intensity of the reflected signals generated at the remaining positions of the glove 100 other than the fingers 110, 120, 130, 140, 150, the palm center 160 and the back of the hand is small. Therefore, the head-mounted display 200 can identify the positions on the surface of the glove 100 with high reflectivity according to the reflected signals with high intensity in the laser beam reflected back by the glove 100, and thus identify the gesture shape presented by the glove 100 according to the positions on the surface of the glove 100 with high reflectivity. For example, the gestures presented by the glove 100 may include an open palm, a clenched fist, fingers together, etc.

[0045] For the VR device 1000 for gesture recognition based on a flash lidar according to the embodiments of the present application, by making the reflectivity of the surfaces of the fingers 110, 120, 130, 140, 150, the palm center 160 and the back of the hand of the glove 100 greater than that of the remaining positions, using the flash lidar to scan the glove 100, and according to the different intensities of the reflected signals generated at different reflectivity positions on the surface of the glove 100, the gesture presented by the glove 100 can be recognized according to the signal intensity in the laser beam reflected back by the glove 100. At this time, the glove is a passive component, without any electronic components, does not require power supply, and will not cause problems such as power consumption, heat dissipation and safety. Moreover, compared with gloves provided with electronic components, such gloves have lower manufacturing costs, lower production difficulties, lower failure rates, are not easily damaged, and at the same time can reduce the difficulty of gesture capture and judgment by the head-mounted display, improve the accuracy of gesture recognition, and the relevant algorithms on the head-mounted display are simple and easy to implement.

[0046] It should be noted that the embodiments of the present application do not limit the materials, manufacturing methods, etc. used for the glove 100. For example, the material of the glove 100 can be leather, rubber, knitted fabric, etc., and the manufacturing method of the glove 100 can be sewing, knitting, dipping, etc. The embodiments of the present application do not limit the implementation method for making the reflectivity of the surface of the glove 100 greater at the positions of the fingers 110, 120, 130, 140, 150, the palm 160 and the back of the hand than at the remaining positions. For example, materials with a reflectivity greater than that of the remaining positions can be provided at the positions of the fingers 110, 120, 131, 140, 150, the palm 160 and the back of the hand on the surface of the glove 100 by methods such as coating, printing, weaving, inlaying, etc. Compared with the existing solutions for sensors to sense gestures, adding materials with a small rigidity and a high reflectivity to the glove makes the glove comfortable to wear.

[0047] It should be noted that the embodiments of the present application do not limit the form of the head-mounted display 200. For example, the head-mounted display 200 can be a virtual reality helmet or an augmented reality glasses, etc.

[0048] It should be noted that the embodiments of the present application do not limit the position where the flash lidar is set. For example, the flash lidar can be set at a fixed position in the environment in an outside-in positioning mode, such as setting the flash lidar at a fixed position indoors to better observe the position change information of the gesture. It can also be set on the head-mounted display 200 in an inside-out positioning mode. The embodiments of the present application do not limit the number of flash lidars set. For example, one flash lidar can be set in the center of the head-mounted display 200, or one flash lidar can be set in the center and on both sides of the head-mounted display 200 respectively to expand the field of view angle of the flash lidar. It is worth noting that when the inside-out mode is adopted and the user wears the head-mounted display, since the fingers are all within about 0.5 m of the flash lidar, the flash lidar can more easily distinguish the differences in the spatial positions of the fingers.

[0049] It should be noted that the embodiments of the present application do not limit the form of the gesture presented by the glove 100. For example, the gesture presented by the glove 100 can be formed by one hand or two hands.

[0050] In some alternative embodiments of the present application, the reflectivity of the glove 100 on the surfaces of the fingers 110, 120, 131, 140, 150, the palm 160, and the back of the hand can be greater than 80%. Optionally, the reflectivity of the glove 100 on the surfaces of the remaining positions other than the fingers 110, 120, 131, 140, 150, the palm 160, and the back of the hand can be less than 20%, so as to highlight the positions of the fingers 110, 120, 131, 140, 150, the palm 160, and the back of the hand of the glove 100 from the remaining positions of the glove 100 due to the large difference in reflectivity between the two. Optionally, for the positions of the fingers 110, 120, 131, 140, 150, the palm 160, and the back of the hand of the glove 100, materials with a reflectivity greater than 80%, such as metals, ceramics, resins, etc., can be used. In an alternative example, a film layer of a material with a reflectivity greater than 80% can be provided on the surfaces of the fingers 110, 120, 131, 140, 150, the palm 160, and the back of the hand of the glove 100. Weaving an optical film layer with a reflectivity greater than 80% into the corresponding positions of the glove, such as using a soft metal film layer, can improve the wearing comfort of the glove. The embodiments of the present application do not limit the reflectivity values of the positions of the fingers 110, 120, 131, 140, 150, the palm 160, and the back of the hand of the glove 100, nor the types of materials used to achieve such reflectivity values.

[0051] In some alternative embodiments of the present application, the reflectivity of the glove 100 on the surfaces of the entire fingers 110, 120, 131, 140, 150, the palm 160, and the back of the hand can be greater than 80%. For example, reflective strips with a reflectivity greater than 80% can be provided along the entire finger extension direction at the finger positions on the surface of the glove 100, and large-area reflective points with a reflectivity greater than 80% can be provided at the palm and back-of-hand positions on the surface of the glove 100, so as to reflect the state of the fingers through the shape of the reflective strips. In some other alternative embodiments of the present application, the reflectivity of the glove 100 on the surfaces of the 9 interphalangeal joints of the fingers, the palm, and the back of the hand can be greater than 80%, and the area of the palm and the back of the hand with a reflectivity greater than 80% is larger than the area of the 9 interphalangeal joints of the fingers with a reflectivity greater than 80%. For example, reflective points with a reflectivity greater than 80% can be provided at the positions of the 9 interphalangeal joints on the surface of the glove 100, and large-area reflective points with a reflectivity greater than 80% can be provided at the palm and back-of-hand positions on the surface of the glove 100, so as to reflect the state of the fingers through the distribution of the reflective points. The embodiments of the present application do not limit the shape of the area with a reflectivity greater than 80% provided on the surface of the glove 100 for indicating the positions of the fingers.

[0052] In some alternative embodiments of the present application, the reflectivity of the 100 gloves on one side or the other side of the nine interphalangeal joints facing the palm or the back of the hand can be made greater than 80%. For example, reflective points with a reflectivity greater than 80% can be set only at the positions on the surface of the glove 100 where the nine interphalangeal joints face the palm, and large-area reflective points with a reflectivity greater than 80% can be set at the positions of the palm and the back of the hand on the surface of the glove 100; or reflective points with a reflectivity greater than 80% can be set only at the positions on the surface of the glove 100 where the nine interphalangeal joints face the back of the hand, and large-area reflective points with a reflectivity greater than 80% can be set at the positions of the palm and the back of the hand on the surface of the glove 100. In some other alternative embodiments of the present application, the reflectivity of the glove 100 on both the side of the nine interphalangeal joints facing the palm and the side facing the back of the hand can be made greater than 80%. For example, reflective points with a reflectivity greater than 80% can be set at the positions on both the side of the nine interphalangeal joints facing the palm and the side facing the back of the hand on the surface of the glove 100, and large-area reflective points with a reflectivity greater than 80% can be set at the positions of the palm and the back of the hand on the surface of the glove 100. The embodiments of the present application do not limit the setting method of the reflective points with a reflectivity greater than 80% provided on the surface of the glove 100 for indicating the positions of the fingers.

[0053] Optionally, when the reflectivity of the glove 100 on both the side of the nine interphalangeal joints facing the palm and the side facing the back of the hand is made greater than 80%, the reflectivity of the glove 100 on the palm and the side of the nine interphalangeal joints facing the palm can be made different from the reflectivity on the back of the hand and the side of the nine interphalangeal joints facing the back of the hand, that is, the front and back sides of the palm of the glove 100 have different reflectivities. For example, reflective points with a reflectivity greater than 90% can be set at the positions on the surface of the glove 100 where the nine interphalangeal joints face the palm, large-area reflective points with a reflectivity greater than 90% can be set at the position of the palm on the surface of the glove 100, reflective points with a reflectivity greater than 80% and less than 90% can be set at the positions on the surface of the glove 100 where the nine interphalangeal joints face the back of the hand, and large-area reflective points with a reflectivity greater than 80% and less than 90% can be set at the position of the back of the hand on the surface of the glove 100.

[0054] In some alternative embodiments of the present application, the flash lidar can obtain the reflection dot matrix of the glove 100 based on the signal intensity in the laser beam reflected back by the glove. The head-mounted display 200 can match the reflection dot matrix obtained by the flash lidar with the stored gesture dot matrix patterns, and identify the gesture presented by the glove 100 according to the matching result. Among them, if there is a gesture dot matrix pattern in the stored gesture dot matrix patterns that is consistent with the reflection dot matrix obtained by the flash lidar, the gesture corresponding to the gesture dot matrix pattern is identified as the gesture presented by the glove 100. If there is no gesture dot matrix pattern in the stored gesture dot matrix patterns that is consistent with the reflection dot matrix obtained by the flash lidar, a prompt message can be fed back indicating that the gesture presented by the glove 100 cannot be recognized. The stored gesture dot matrix patterns can be set in the head-mounted display 200 or in the cloud server, and the embodiments of the present application do not limit this. Gesture recognition through preset gesture dot matrix diagrams can reduce the amount of data processing performed by the head-mounted display 200 and improve the efficiency of gesture recognition.

[0055] In some other alternative embodiments of the present application, as Figure 2 shown, the flash lidar can obtain the reflection dot matrix of the glove 100 based on the signal intensity in the laser beam reflected back by the glove. The head-mounted display 200 can concatenate the reflection dot matrix obtained by the flash lidar into a hand bone network, and identify the gesture presented by the glove 100 according to the hand bone network. Optionally, the head-mounted display 200 can concatenate the reflection dot matrix into a hand bone network according to information such as the area, quantity, position of the reflection points in the reflection dot matrix, the distance between the reflection points, and the distance between the reflection points and the flash lidar. Among them, the areas presented by the reflection points corresponding to the palm and the back of the hand are significantly larger than the areas presented by the reflection points corresponding to the dorsal interphalangeal joints. Gesture recognition by concatenating the hand bone network can meet the needs of gesture recognition in different scenarios, improve the accuracy of gesture recognition, can recognize a variety of gestures, and enhance the simulation and perception of the human body.

[0056] In some other alternative embodiments of the present application, the flash lidar can obtain the reflection dot matrix of the glove 100 based on the signal intensity in the laser beam reflected back by the glove. The head-mounted display 200 can match the reflection dot matrix obtained by the flash lidar with the stored gesture dot matrix patterns. If there is a gesture dot matrix pattern in the stored gesture dot matrix patterns that is consistent with the reflection dot matrix obtained by the flash lidar, the gesture corresponding to the gesture dot matrix pattern is recognized as the gesture presented by the glove 100. If there is no gesture dot matrix pattern in the stored gesture dot matrix patterns that is consistent with the reflection dot matrix obtained by the flash lidar, the reflection dot matrix obtained by the flash lidar can be concatenated into a hand bone network, and the gesture presented by the glove 100 can be recognized based on the hand bone network.

[0057] In some alternative embodiments of the present application, different gestures can correspond to different operation instructions. For example Figure 3A and Figure 3B the gesture in, that is, the OK gesture, can correspond to an operation instruction indicating affirmation or agreement; Figure 4 the gesture in, that is, the gesture with the thumb and little finger extended and the middle three fingers curled towards the palm, can correspond to an operation instruction indicating increase or continuation; Figure 5 the gesture in, that is, the gesture of praise, can correspond to an operation instruction indicating return; Figure 6 the gesture in, that is, the gesture with the fingers extended and the palm exposed, can correspond to an operation instruction indicating pause. After recognizing the gesture presented by the glove 100, the head-mounted display 200 can execute corresponding operations according to the instruction corresponding to the gesture presented by the glove 100.

[0058] In some alternative embodiments of the present application, since the flash lidar can obtain the gesture presented by the glove 100 through one scan of the glove 100, continuous recognition of the change of the gesture can be performed to obtain the gesture corresponding to a series of actions, realizing the recognition of the gesture action.

[0059] The VR device 1000 for gesture recognition based on flash lidar provided by the embodiments of the present application will be described below with reference to the accompanying drawings and specific embodiments.

[0060] In some alternative embodiments of the present application, reflection points with a reflectivity greater than 80% are provided at the positions of the 9 interphalangeal joints, the palm and the back of the hand of the glove 100, and the areas of the reflection points on the palm and the back of the hand are larger than the areas of the reflection points of the 9 interphalangeal joints. As Figure 3A shown, when the glove 100 presents Figure 3A the gesture in, according to Figure 3A it can be known that 10 reflection points on the palm, the back of the hand and the interphalangeal joints of the glove 100 will be presented, that isFigure 3A The circled position can, based on the time when the laser beam is emitted and reflected back, determine the distance between the reflection point on the glove 100 and the flash lidar. According to the size of the area of the reflection point on the glove 100, it is possible to determine the position of the palm and the back of the hand. If both the reflection point on the palm of the glove 100 and the reflection point on the back of the hand are obtained, it can be shown that the side of the hand is facing the flash lidar. By judging the positions of other reflection points on the glove 100, the reflection points of the palm and the back of the hand can be distinguished. Based on the outermost reflection point on the glove 100 and its position relative to the palm, the position of the thumb can be determined, and thus the reflection point of the index finger can be judged. According to the positions of the reflection points of other fingers on the glove 100, the distances between the reflection points, the distances between the reflection points and the flash lidar, and the number of the reflection points, it is possible to judge the gesture presented by the glove 100. As Figure 3B shown, reflection points can also be provided on both the side of the 9 interphalangeal joints of the glove 100 facing the back of the hand and the side facing the palm. Since the number of reflection points is increased and the width of the finger is fixed, gesture recognition can be better performed thereby.

[0061] In some other alternative embodiments of the present application, reflection points with a reflectivity greater than 80% are provided at the positions of the 9 interphalangeal joints, the palm and the back of the hand of the glove 100, and the areas of the reflection points on the palm and the back of the hand are larger than the areas of the reflection points of the 9 interphalangeal joints. As Figure 4 shown, when the glove 100 presents Figure 4 the gesture in Figure 4 it can be known that 10 reflection points on the palm, the back of the hand and the interphalangeal joints of the glove 100 will be presented, that is Figure 4 the circled position. According to the number of reflection points on the glove 100, if the reflection point located at the palm is blocked, it can be judged that the palm is facing the flash lidar. According to the fact that there are two reflection points and one reflection point at both ends of the glove 100, the positions of the thumb and the little finger can be judged. According to the fact that the reflection point in the middle on the glove 100 is farther from the reflection point at the palm than the reflection point of the little finger at the farthest, it can be judged that the middle finger is folded. According to the fact that the reflection point at the palm of the glove 100 is blocked, it can be judged that the middle finger is curled towards the palm, and thus the gesture presented by the glove 100 can be recognized.

[0062] In still some other alternative embodiments of the present application, reflection points with a reflectivity greater than 80% are provided at the positions of the 9 interphalangeal joints, the palm and the back of the hand of the glove 100, and the areas of the reflection points on the palm and the back of the hand are larger than the areas of the reflection points of the 9 interphalangeal joints. As Figure 5 shown, when the glove 100 presents Figure 5 the gesture in Figure 5It can be known that 10 reflection points on the palm, the back of the hand and the interphalangeal joints of the glove 100 will be presented, that is Figure 5 The positions circled in Figure 5 . It can be judged that neither the palm nor the back of the hand is within the scanning range of the flash lidar according to the fact that there are no large-area reflection points on the glove 100. The back of the hand cannot be blocked, and it can only be facing away from the flash lidar. Moreover, according to the occlusion of the palm and the positional relationship between the reflection points on the glove 100, the gesture presented by the glove 100 can be recognized.

[0063] In still another alternative example of the present application, reflection points with a reflectivity greater than 80% are provided at the positions of the 9 interphalangeal joints, the palm and the back of the hand of the glove 100, and the areas of the reflection points on the palm and the back of the hand are larger than the areas of the reflection points of the 9 interphalangeal joints. As Figure 6 shown, when the glove 100 presents Figure 6 the gesture in Figure 6 , according to Figure 6 It can be known that 10 reflection points on the palm, the back of the hand and the interphalangeal joints of the glove 100 will be presented, that is Figure 6 The positions circled in Figure 6 . The gesture presented by the glove 100 can be recognized according to the reflection points on the palm of the glove 100 and the reflection points on the side of the interphalangeal joints facing the palm.

[0064] The embodiment of the present application also provides a method 2000 for gesture recognition based on a flash lidar, Figure 7 which shows a schematic flowchart of the method 2000 for gesture recognition based on a flash lidar according to the embodiment of the present application. The method 2000 for gesture recognition based on a flash lidar according to the embodiment of the present application can be applied to the VR device 1000 in any of the above embodiments. As Figure 7 shown, the method 2000 for gesture recognition based on a flash lidar may include the following steps:

[0065] S300, emitting a laser beam to the glove through the flash lidar and receiving the laser beam reflected back by the glove, wherein the reflectivity of the surface of the glove at the positions of the fingers, the palm and the back of the hand is greater than that of the remaining positions; and

[0066] S400, recognizing the gesture presented by the glove through the head-mounted display according to the signal intensity in the laser beam reflected back by the glove.

[0067] It should be understood that the steps shown in the method 2000 are not exclusive, and other steps may be executed before, after or between any of the shown steps. In addition, some of the shown steps may be executed simultaneously, or may be executed in an order different from Figure 1 the order shown.

[0068] Optionally, the reflectivity of the glove on the surfaces of the finger, palm, and back of the hand positions is greater than 80%.

[0069] Optionally, the reflectivity of the glove on the surfaces of the 9 interphalangeal joints of the finger, palm, and back of the hand positions is greater than 80%, and the area of the palm and the back of the hand with a reflectivity greater than 80% is larger than that of the 9 interphalangeal joints of the finger.

[0070] Optionally, the reflectivity of both sides of the 9 interphalangeal joints of the glove facing the palm and facing the back of the hand is greater than 80%.

[0071] Optionally, the reflectivity of the glove on the palm and the side of the 9 interphalangeal joints facing the palm is different from the reflectivity on the back of the hand and the side of the 9 interphalangeal joints facing the back of the hand.

[0072] Optionally, the surface of the glove at the finger, palm, and back of the hand positions has a film layer of a material with a reflectivity greater than 80%.

[0073] Optionally, the flash lidar is located in the head-mounted display.

[0074] Optionally, the flash lidar is located at a fixed position in the environment.

[0075] Optionally, step S300 includes:

[0076] Obtaining the reflection dot matrix of the glove based on the signal intensity in the laser beam reflected back by the glove through the flash lidar;

[0077] Step S400 includes:

[0078] Matching the reflection dot matrix with the stored gesture dot matrix pattern through the head-mounted display, and identifying the gesture presented by the glove according to the matching result.

[0079] Optionally, step S300 includes:

[0080] Obtaining the reflection dot matrix of the glove based on the signal intensity in the laser beam reflected back by the glove through the flash lidar;

[0081] Step S400 includes:

[0082] Connecting the reflection dot matrix in series into a hand bone network through the head-mounted display, and identifying the gesture presented by the glove according to the hand bone network.

[0083] Optionally, the reflective dot matrix is connected in series as a hand bone network through the head-mounted display, and the gesture presented by the glove is recognized according to the hand bone network, including:

[0084] The head-mounted display connects the reflective dot matrix in series as the hand bone network according to the area, quantity, position, distance between the reflective dots, and distance between the reflective dots and the flash lidar.

[0085] Optionally, the method 2000 for gesture recognition based on flash lidar further includes:

[0086] The head-mounted display performs corresponding operations according to the instructions corresponding to the gesture presented by the glove.

[0087] The above specific embodiments do not limit the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principle of the present disclosure shall be included within the protection scope of the present disclosure.

Claims

1. A VR device for gesture recognition based on a flash lidar, characterized in that, it includes: a glove, the reflectivity of the surface of the glove at the finger, palm and back of the hand positions is greater than that of the remaining positions, and the flash lidar emits a laser beam to the glove and receives the laser beam reflected back by the glove; and a head-mounted display, which recognizes the gesture presented by the glove according to the signal intensity in the laser beam reflected back by the glove.

2. The VR device according to claim 1, characterized in that, the reflectivity of the surface of the glove at the finger, palm and back of the hand positions is greater than 80%.

3. The VR device according to claim 2, characterized in that, the reflectivity of the surface of the glove at the 9 interphalangeal joints of the finger, palm and back of the hand positions is greater than 80%, and the area of the palm and the back of the hand with a reflectivity greater than 80% is larger than that of the 9 interphalangeal joints of the finger.

4. The VR device according to claim 3, characterized in that, the reflectivity of both sides of the 9 interphalangeal joints of the glove facing the palm and the back of the hand is greater than 80%.

5. The VR device according to claim 4, characterized in that, the reflectivity of the glove at the palm and the side of the 9 interphalangeal joints facing the palm is different from the reflectivity of the glove at the back of the hand and the side of the 9 interphalangeal joints facing the back of the hand.

6. The VR device according to claim 1, characterized in that, the flash lidar is located on the head-mounted display.

7. The VR device according to claim 1, characterized in that, the flash lidar is located at a fixed position in the environment.

8. The VR device according to claim 6 or 7, characterized in that, the flash lidar obtains the reflection dot matrix of the glove according to the signal intensity in the laser beam reflected back by the glove; the head-mounted display matches the reflection dot matrix with the stored gesture dot matrix pattern, and recognizes the gesture presented by the glove according to the matching result.

9. The VR device according to claim 6 or 7, characterized in that, the flash lidar obtains the reflection dot matrix of the glove according to the signal intensity in the laser beam reflected back by the glove; the head-mounted display connects the reflection dot matrix in series into a hand bone network, and recognizes the gesture presented by the glove according to the hand bone network.

10. A method for gesture recognition based on a flash lidar, characterized in that, applied to a VR device, the method includes: emitting a laser beam to a glove through a flash lidar and receiving the laser beam reflected back by the glove, wherein the reflectivity of the surface of the glove at the finger, palm and back of the hand positions is greater than that of the remaining positions; and recognizing the gesture presented by the glove by a head-mounted display according to the signal intensity in the laser beam reflected back by the glove.

Images