Gesture detection method and device, intelligent glasses, system and storage medium

Abstract

The invention relates to the technical field of data processing, and provides a gesture detection method and device, intelligent glasses, a system and a storage medium, and the method comprises the steps: obtaining a target image obtained by photographing a hand of a user through the intelligent glasses; receiving a detection signal transmitted by a signal transmitting device of a user; determining the position of the signal transmitting equipment relative to the intelligent glasses according to the receiving time of the detection signal; determining the position of a hand region in the target image according to the position of the signal transmitting equipment relative to the intelligent glasses; and performing gesture detection on the target image according to the position of the hand region in the target image so as to reduce the omission ratio and the false detection ratio of the hand of the user in the target image, improve the positioning accuracy of the hand of the user in the target image and improve the gesture detection effect of the target image.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a gesture detection method, device, smart glasses, system and storage medium. Background Art

[0002] At present, smart glasses can achieve human-computer interaction through gesture detection. Generally, gesture detection can be achieved through deep learning methods. However, when using deep learning methods to achieve gesture detection, it is easy to fail to detect the user's hand, the missed detection rate and false detection rate of the user's hand are high, and the positioning of the user's hand is inaccurate, resulting in poor gesture detection effect, affecting the user's human-computer interaction experience with smart glasses. Summary of the invention

[0003] The main purpose of the present application is to provide a gesture detection method, device, smart glasses, system and storage medium, aiming to reduce the missed detection rate and false detection rate of the user's hands in the target image, improve the positioning accuracy of the user's hands in the target image, and enhance the gesture detection effect of the target image.

[0004] In a first aspect, the present application provides a gesture detection method for smart glasses, the gesture detection method comprising the following steps:

[0005] Acquire a target image obtained by photographing a user's hand through the smart glasses;

[0006] Receiving a detection signal transmitted by a signal transmitting device of the user;

[0007] Determining the position of the signal transmitting device relative to the smart glasses according to the reception time of the detection signal;

[0008] Determining a position of a hand region in the target image according to a position of the signal transmitting device relative to the smart glasses;

[0009] Performing gesture detection on the target image according to the position of the hand area in the target image.

[0010] In a second aspect, the present application further provides a gesture detection device, the gesture detection device comprising:

[0011] An acquisition module is used to acquire a target image obtained by photographing a user's hand by the smart glasses;

[0012] A receiving module, used to receive a detection signal transmitted by a signal transmitting device of the user;

[0013] A first determination module, configured to determine a position of the signal transmitting device relative to the smart glasses according to a reception time of the detection signal;

[0014] A second determination module, configured to determine a position of a hand region in the target image according to a position of the signal transmitting device relative to the smart glasses;

[0015] The detection module is used to perform gesture detection on the target image according to the position of the hand area in the target image.

[0016] In a third aspect, the present application further provides a pair of smart glasses, the smart glasses comprising a memory and a processor;

[0017] The memory is used to store computer programs;

[0018] The processor is used to execute the computer program and implement the gesture detection method as described above when executing the computer program.

[0019] In a fourth aspect, the present application further provides a gesture detection system, the gesture detection system comprising:

[0020] A signal transmitting device, wherein the signal transmitting device is used to transmit a detection signal;

[0021] The aforementioned smart glasses.

[0022] In a fifth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the gesture detection method as described above are implemented.

[0023] The present application provides a gesture detection method, device, smart glasses, system and storage medium. The gesture detection method includes: obtaining a target image obtained by photographing a user's hand with smart glasses; receiving a detection signal emitted by a signal transmitting device; determining the position of the signal transmitting device relative to the smart glasses according to the reception time of the detection signal; determining the position of the hand area in the target image according to the position of the signal transmitting device relative to the smart glasses; and performing gesture detection on the target image according to the position of the hand area in the target image. Based on the determination of the position of the signal transmitting device relative to the smart glasses, for example, the position of the hand area in the target image can be determined by the position of the signal transmitting device relative to the smart glasses, combined with the position of the signal transmitting device relative to the user's hand and the corresponding shooting range of the smart glasses, which is conducive to improving the accuracy of positioning the user's hand in the target image. In addition, based on the improvement of the accuracy of positioning the user's hand in the target image, it is conducive to improving the accuracy of determining the user's hand in the target image, so as to reduce the missed detection rate and false detection rate of the user's hand in the target image. Based on the reduction of the missed detection rate and false detection rate of the user's hand in the target image, the user's gesture in the target image can be detected, which is conducive to improving the gesture detection effect of the target image. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0025] Figure 1 is a flowchart of a gesture detection method provided in an embodiment of the present application;

[0026] Figure 2 is a schematic block diagram of a gesture detection device provided in an embodiment of the present application;

[0027] Figure 3 is a schematic block diagram of the structure of a pair of smart glasses provided in an embodiment of the present application;

[0028] Figure 4 It is a schematic block diagram of a gesture detection system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0030] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0031] The embodiments of the present application provide a gesture detection method, device, smart glasses, system and storage medium. The gesture detection method can be applied to smart glasses.

[0032] Exemplarily, smart glasses may include VR glasses, AR glasses, etc., which are not limited here.

[0033] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0034] See also Figure 1 , Figure 1It is a flow chart of a gesture detection method provided in an embodiment of the present application. It should be noted that the gesture detection method provided in an embodiment of the present application can be used in smart glasses.

[0035] like Figure 1 As shown, the gesture detection method includes steps S101 to S105.

[0036] S101, obtaining a target image obtained by photographing a user's hand with smart glasses.

[0037] Exemplarily, when using smart glasses, the user needs to wear the smart glasses in front of the user's eyes. For example, the smart glasses can be mounted on the user's head to wear the smart glasses in front of the user's glasses. In some embodiments, the smart glasses are provided with a camera, for example, and the camera can be used to photograph the user's hand to obtain a target image, but this is not limited to this and is not limited here.

[0038] For example, when a user wears smart glasses, such as when the user wears VR glasses or AR glasses, the user has an interactive demand for the smart glasses. In some embodiments, the user can interact with the smart glasses by making gestures. Based on this, the smart glasses have a detection demand for the user's gestures. For example, the smart glasses can capture the user's hand within the shooting range corresponding to the smart glasses to obtain a target image, so as to subsequently detect the user's gesture in the target image.

[0039] In some embodiments, in order to avoid an increase in power consumption of the smart glasses due to the smart glasses being in a shooting state for a long time and to improve the battery life of the smart glasses, the process of the smart glasses shooting the user's hands can be controlled.

[0040] Exemplarily, when a first preset operation of the user on the signal transmitting device is detected, the smart glasses are controlled to photograph the user's hands to obtain a target image; when a second preset operation of the user on the signal transmitting device is detected, the smart glasses are prohibited from photographing images.

[0041] For example, the user can perform a first preset operation on the signal transmitting device when his hand is within the shooting range of the smart glasses. The user can perform a second preset operation on the signal transmitting device when his hand is not within the shooting range of the smart glasses. When the smart glasses detect the first preset operation of the user on the signal transmitting device, they can control the smart glasses to shoot the user's hand according to the detected first preset operation to obtain the target image. Among them, the first preset operation can be a touch operation, a key operation, etc. of the signal transmitting device by the user, which is not limited here. Of course, it is not limited to this. The smart glasses can also control the smart glasses to shoot images by detecting the user's operation on the smart glasses. For example, when the user's third preset operation on the smart glasses is detected, the smart glasses are controlled to shoot the user's hand to obtain the target image. Among them, the third preset operation can be a touch operation, an eye movement operation, a tapping operation, etc. of the smart glasses by the user, which is not limited here.

[0042] For example, when the smart glasses detect the second preset operation of the user on the signal transmitting device, the smart glasses can be prohibited from capturing images based on the detected second preset operation. The second preset operation can be a touch operation, a button operation, etc. of the signal transmitting device by the user, which is not limited here. Of course, it is not limited to this. The smart glasses can also prohibit the smart glasses from capturing images by detecting the user's operation on the smart glasses. For example, when the user's fourth preset operation on the smart glasses is detected, the smart glasses are prohibited from capturing images. The fourth preset operation can be a touch operation, an eye movement operation, a tapping operation, etc. of the smart glasses by the user, which is not limited here.

[0043] Taking the example that both the first preset operation and the second preset operation are the user's key operation on the signal transmitting device, when the user's key operation on the signal transmitting device is detected, the smart glasses are controlled to shoot the user's hand to obtain the target image; when the user's key operation on the signal transmitting device is detected again, the smart glasses are prohibited from shooting images. Of course, it is not limited to this. For example, based on the current state of the shooting function of the smart glasses being on or off, combined with the detected user's key operation on the signal transmitting device, the smart glasses can be controlled to shoot the user's hand to obtain the target image, or the smart glasses can be prohibited from shooting images. There is no limitation here.

[0044] In an exemplary embodiment, the smart glasses can control or prohibit the smart glasses from capturing images by, for example, detecting whether the user's hand is within the corresponding capturing range of the smart glasses. For example, when it is detected that the user's hand is within the corresponding capturing range of the smart glasses, the smart glasses are controlled to capture the user's hand to obtain the target image; when it is detected that the user's hand is not within the corresponding capturing range of the smart glasses, the smart glasses are prohibited from capturing images.

[0045] However, the number of users may include one or more. When there are multiple users, the users may include users wearing the smart glasses and users not wearing the smart glasses; wherein the user wearing the smart glasses may be referred to as the first user, and the user not wearing the smart glasses may be referred to as the second user. For example, in a case where the smart glasses control or prohibit the smart glasses from capturing images by detecting whether there is a user's hand within the shooting range of the smart glasses, for example, only the hand of the second user may appear within the shooting range of the smart glasses. In this case, it is easy for the smart glasses to control the smart glasses to capture the image of the hand of the second user upon detecting that the hand of the second user is within the shooting range of the smart glasses. In this case, it is easy for the smart glasses to mistakenly perform gesture detection based on the hand image of the second user, thereby controlling the smart glasses, resulting in a poor user experience of the smart glasses for the first user.

[0046] Based on this, by detecting the user's operation on the signal transmitting device or smart glasses, controlling or prohibiting the smart glasses from capturing images, the smart glasses can be controlled to capture images based on the actual needs of the first user to ensure that the image captured by the smart glasses includes the hand area of ​​the first user, which is beneficial to improving the accuracy of controlling or prohibiting the smart glasses from capturing images. By ensuring that the image captured by the smart glasses includes the hand area of ​​the first user, when the hand area of ​​the first user in the image is subsequently determined based on the image captured by the smart glasses, the situation in which the hand area of ​​the first user in the image cannot be determined because the image only includes the hand area of ​​the second user can be avoided, and the situation in which the hand area of ​​the second user in the image is mistakenly determined as the hand area of ​​the first user because the image only includes the hand area of ​​the second user can be reduced, which is beneficial to subsequently improving the accuracy of gesture detection for the first user and improving the first user's experience of using the smart glasses.

[0047] S102: Receive a detection signal transmitted by a user's signal transmitting device.

[0048] Exemplarily, a communication connection is established between the smart glasses and the signal transmitting device, and the smart glasses and the signal transmitting device can communicate with each other, for example, the smart glasses can receive the detection signal transmitted by the signal transmitting device. In some embodiments, when detecting the user's gesture according to the target image, it is necessary to determine the position of the user's hand in the target image. Based on this, the position of the user's hand in the target image can be determined with the help of the user's signal transmitting device. Among them, the signal transmitting device can be a finger ring, a finger clip, a bracelet, etc.

[0049] For example, taking the signal transmitting device mounted on the user's hand as an example, the smart glasses can obtain the position of the signal transmitting device relative to the user's hand. For example, the user's signal transmitting device can transmit a detection signal to the smart glasses. When the smart glasses receive the detection signal transmitted by the user's signal transmitting device, they can subsequently determine the position of the signal transmitting device relative to the smart glasses based on the received detection signal. In the case of determining the position of the signal transmitting device relative to the smart glasses, since the signal transmitting device is mounted on the user's hand, the position of the user's hand relative to the smart glasses can be determined based on the position of the signal transmitting device relative to the smart glasses, so as to subsequently determine the position of the user's hand in the target image, thereby detecting the user's gesture. Of course, it is not limited to this. The signal transmitting device can also be mounted on a part other than the user's hand, such as the user's shoulder, the user's neck, the user's foot, etc. For example, the smart glasses can pre-acquire the position of the signal transmitting device relative to the user's hand when the user's hand is in the shooting range corresponding to the smart glasses, so as to determine the position of the user's hand relative to the smart glasses in combination with the position of the signal transmitting device relative to the smart glasses. This is not limited here.

[0050] In some embodiments, taking the case where there are multiple users, and the multiple users include the first user and the second user as an example, if the acquired target image includes the hands of the first user and the second user, based on the fact that the first user is a user wearing or using smart glasses, it is necessary to identify the hands of the first user from the target image. Based on this, the signal transmitting device can be mounted on the hand of the first user, and the detection signal transmitted by the signal transmitting device of the first user is received, so as to determine the position of the signal transmitting device relative to the smart glasses according to the detection signal transmitted by the signal transmitting device of the first user received, and determine the position of the hand of the first user relative to the smart glasses based on the position of the signal transmitting device relative to the smart glasses, so as to determine the position of the hand of the first user in the target image, and detect the gesture of the first user in the target image. For example, when the hand of the second user is equipped with a signal transmitting device, since there is no communication connection between the smart glasses and the signal transmitting device of the second user, the smart glasses cannot receive the detection signal transmitted by the signal transmitting device of the second user. When the hand of the second user is not equipped with a signal transmitting device, the second user cannot transmit the detection signal through the signal transmitting device, and the smart glasses cannot receive the detection signal transmitted by the signal transmitting device of the second user. Based on this, when the position of the first user's hand in the target image is subsequently determined, the interference of the second user's hand area in the target image with the first user's hand area can be eliminated by receiving the detection signal transmitted by the first user's signal transmitting device. Of course, this is not limited to this and is not limited here.

[0051] In this way, by receiving the detection signal emitted by the user's signal transmitting device, when subsequently determining the position of the user's hand in the target image, the interference of other users' hands in the target image on the user's hand can be reduced, the missed detection rate and false detection rate of the user's hand in the target image can be reduced, and the positioning accuracy of the user's hand in the target image can be improved, which is beneficial to the subsequent improvement of the gesture detection effect on the target image.

[0052] In some embodiments, in order to avoid the increase in power consumption of the smart glasses due to the smart glasses being in a gesture detection state for a long time and to improve the battery life of the smart glasses, the gesture detection function of the smart glasses can be turned on or off by controlling the signal transmitting device to transmit a detection signal or stop transmitting the detection signal.

[0053] Exemplarily, the signal transmitting device transmits a detection signal when detecting a first preset operation of the signal transmitting device by the user, and stops transmitting the detection signal when detecting a second preset operation of the signal transmitting device by the user. The first preset operation or the second preset operation may be a touch operation, a key operation, etc. of the signal transmitting device by the user, which is not limited here.

[0054] In some embodiments, when a first preset operation of a user on a signal transmitting device is detected, a detection signal transmitted by the user's signal transmitting device is received.

[0055] For example, taking the first preset operation as the user's key operation on the signal transmitting device as an example, when a key operation of the user on the signal transmitting device is detected, the signal transmitting device transmits a detection signal, and the smart glasses can receive the detection signal emitted by the user's signal transmitting device for subsequent determination of the position of the signal transmitting device relative to the smart glasses. Of course, it is not limited to this and is not limited here. When the smart glasses receive the detection signal emitted by the user's signal transmitting device, the gesture detection function of the smart glasses can be turned on to perform gesture detection on the target image obtained by the smart glasses taking a picture of the user's hand.

[0056] In some embodiments, when a second preset operation of the user on the signal transmitting device is detected, the detection signal transmitted by the user's signal transmitting device is stopped.

[0057] For example, taking the second preset operation as the user's key operation on the signal transmitting device as an example, when the user's key operation on the signal transmitting device is detected again, the signal transmitting device stops transmitting the detection signal, and the smart glasses stop receiving the detection signal transmitted by the user's signal transmitting device. Of course, it is not limited to this and is not limited here. In the case where the smart glasses stop receiving the detection signal transmitted by the user's signal transmitting device, the gesture detection function of the smart glasses can be turned off to stop gesture detection of the target image obtained by the smart glasses photographing the user's hand.

[0058] In this way, by detecting the user's first preset operation or the second preset operation on the signal transmitting device, the smart glasses are controlled to receive or stop receiving the detection signal transmitted by the user's signal transmitting device, which is beneficial to improving the convenience of controlling the smart glasses to perform gesture detection or stop performing gesture detection, and is beneficial to reducing the power consumption of the smart glasses and improving the battery life of the smart glasses.

[0059] S103: Determine the position of the signal transmitting device relative to the smart glasses according to the reception time of the detection signal.

[0060] In some embodiments, since the position of the signal transmitting device is different from the position of the smart glasses, for example, the position of the user's head is different from the position of the user's hand, after the signal transmitting device transmits a detection signal to the smart device, the smart glasses need to wait for a certain time before they can receive the detection signal transmitted by the signal transmitting device. Based on this, the position of the signal transmitting device relative to the smart glasses can be determined according to the reception time of the detection signal. For example, the smart glasses can obtain the transmission time of the detection signal transmitted by the signal transmitting device to the smart glasses and the reception time of the detection signal received by the smart glasses to determine the position of the signal transmitting device relative to the smart glasses. For example, the distance of the signal transmitting device relative to the smart glasses is determined according to the transmission time of the detection signal transmitted by the signal transmitting device to the smart glasses and the reception time of the detection signal received by the smart glasses. According to the distance of the signal transmitting device relative to the smart glasses, the position of the signal transmitting device relative to the smart glasses is determined within the shooting range corresponding to the smart glasses. Of course, it is not limited to this and is not limited here.

[0061] In an exemplary embodiment, when the target image includes the hand areas of multiple users, such as the hand area of ​​the first user and the hand area of ​​the second user, the signal transmitting device of the first user can transmit a detection signal to the smart glasses, and the smart glasses can determine the distance of the signal transmitting device relative to the smart glasses based on the transmission time of the detection signal transmitted by the signal transmitting device to the smart glasses and the reception time of the detection signal received by the smart glasses. Based on the distance of the signal transmitting device relative to the smart glasses, combined with the shooting range corresponding to the smart glasses, the position of the signal transmitting device relative to the smart glasses is determined for subsequent determination of the position of the hand area of ​​the first user in the target image. For example, based on the position of the signal transmitting device relative to the smart glasses and combined with the position of the signal transmitting device relative to the hand of the first user, the position of the hand of the first user in the target image is determined, thereby eliminating the interference of the hand area of ​​the second user in the target image with the hand area of ​​the first user. Of course, it is not limited to this, and no limitation is made here.

[0062] In this way, by determining the position of the signal transmitting device relative to the smart glasses based on the reception time of the detection signal, the position of the hand area in the target image can be subsequently determined. Moreover, when the target image includes the hand areas of multiple users, by determining the position of the signal transmitting device relative to the smart glasses, it is beneficial to subsequently reduce the interference of other users' hands in the target image on the user's hands, reduce the missed detection rate and false detection rate of the user's hands in the target image, improve the positioning accuracy of the user's hands in the target image, and enhance the gesture detection effect of the target image.

[0063] Exemplarily, the smart glasses are provided with a plurality of signal receivers, and the positions of the plurality of signal receivers on the smart glasses are different. For example, the plurality of signal receivers may be 3 signal receivers, 5 signal receivers, 6 signal receivers, etc., which are not limited here.

[0064] In some embodiments, the distance of the signal transmitting device relative to each signal receiver is determined based on the reception time of the detection signal received by each signal receiver; the position of the signal transmitting device relative to the smart glasses is determined based on the distance of the signal transmitting device relative to each signal receiver among the multiple signal receivers.

[0065] For example, based on the different positions of multiple signal receivers on the smart glasses, the corresponding reception time of the detection signal received by each signal receiver can be the same or different. According to the reception time of the detection signal received by each signal receiver, for example, the distance of the signal transmitting device relative to each signal receiver can be determined in combination with the transmission time of the detection signal. Of course, it is not limited to this and is not limited here. According to the distance of the signal transmitting device relative to each signal receiver in the multiple signal receivers, for example, the position of the signal transmitting device relative to the smart glasses can be determined based on a multilateral measurement method.

[0066] In an exemplary embodiment, taking the multiple signal receivers as three signal receivers as an example, the multilateral measurement method may include a trilateral measurement method. For example, the distance of the signal transmitting device relative to each signal receiver is determined according to the time when the three signal receivers receive the detection signal. For example, according to the time when the A signal receiver receives the detection signal, the distance of the signal transmitting device a relative to the A signal receiver is determined, according to the time when the B signal receiver receives the detection signal, the distance of the signal transmitting device b relative to the B signal receiver is determined, and according to the time when the C signal receiver receives the detection signal, the distance of the signal transmitting device c relative to the C signal receiver is determined. Of course, it is not limited to this, and no limitation is made here.

[0067] In this way, based on the setting of multiple signal receivers, it is helpful to improve the positioning accuracy of the signal transmitting device relative to the smart glasses. Based on the determination of the position of the signal transmitting device relative to the smart glasses, it can be used for subsequent determination of the position of the hand area in the target image, which is helpful to reduce the missed detection rate and false detection rate of the user's hands in the target image, improve the positioning accuracy of the user's hands in the target image, and enhance the gesture detection effect of the target image.

[0068] Exemplarily, the signal transmitting device is provided with a first signal transmitter and a second signal transmitter, and the transmission speeds of the detection signals transmitted by the first signal transmitter and the second signal transmitter are different. For example, the detection signal transmitted by the first signal transmitter may be a Bluetooth signal, and the detection signal transmitted by the second signal transmitter may be an ultrasonic signal, and the transmission speed of the Bluetooth signal is different from the transmission speed of the ultrasonic signal, but this is not limited to this and is not limited here.

[0069] In some embodiments, the distance of the signal transmitting device relative to the signal receiver is determined based on the time when the signal receiver receives the detection signal of the first signal transmitter and the time when the signal receiver receives the detection signal of the second signal transmitter.

[0070] For example, since the transmission speeds of the detection signals emitted by the first signal transmitter and the second signal transmitter are different, the reception time of the detection signal of the first signal transmitter received by the signal receiver is correspondingly different from the reception time of the detection signal of the second signal transmitter received by the signal receiver. Based on this, the time difference can be determined according to the reception time of the detection signal of the first signal transmitter received by the same signal receiver and the reception time of the detection signal of the second signal transmitter, and the distance of the signal transmitting device relative to the signal receiver can be determined according to the time difference.

[0071] For example, the distance between the signal transmitting device and the signal receiver can be determined based on the time difference based on a preset distance function. For example, the preset distance function can be expressed as:

[0072]

[0073] Among them, D is used to indicate the distance between the signal transmitting device and the signal receiver, t is used to indicate the time difference, v1 is used to indicate the transmission speed of the detection signal transmitted by the first signal transmitter, and v2 is used to indicate the transmission speed of the detection signal transmitted by the second signal transmitter.

[0074] Based on this, the determined time difference, the transmission speed of the detection signal emitted by the first signal transmitter, and the transmission speed of the detection signal emitted by the second signal transmitter can be substituted into the preset distance function to obtain the distance of the signal transmitting device relative to the signal receiver. Based on the determination of the distance of the signal transmitting device relative to the signal receiver, the position of the signal transmitting device relative to the smart glasses can be determined in combination with the shooting range corresponding to the smart glasses. For example, the position range of the signal transmitting device relative to the signal receiver is determined based on the distance of the signal transmitting device relative to the signal receiver, and the position of the signal transmitting device relative to the smart glasses is determined based on the overlap between the position range of the signal transmitting device relative to the signal receiver and the shooting range corresponding to the smart glasses. Of course, it is not limited to this and is not limited here.

[0075] In some embodiments, the multi-lateral measurement method may include a trilateral measurement method. Take the case where the multiple signal receivers include a first signal receiver, a second signal receiver, and a third signal receiver, the signal transmitting device is provided with a first signal transmitter and a second signal transmitter, the transmission speeds of the detection signals transmitted by the first signal transmitter and the second signal transmitter are different, and the signal transmitting device is carried on the user's hand. For example, the time difference between the first signal transmitted by the first signal transmitter and the second signal transmitted by the second signal transmitter can be received according to the three received signals. For example, if the time difference between the first signal and the second signal received by the first signal receiver is t1, then the distance of the signal transmitting device relative to the first signal receiver can be determined to be D1, the time difference between the first signal and the second signal received by the second signal receiver is t2, then the distance of the signal transmitting device relative to the second signal receiver can be determined to be D2, and the time difference between the first signal and the second signal received by the third signal receiver is t3, then the distance of the signal transmitting device relative to the third signal receiver can be determined to be D3. For example, the position range of the signal transmitting device relative to the smart glasses can be determined based on the distance D1 of the signal transmitting device relative to the first signal receiver, the distance D2 of the signal transmitting device relative to the second signal receiver, and the distance D3 of the signal transmitting device relative to the third signal receiver, and the position of the signal transmitting device relative to the smart glasses can be determined in combination with the shooting range corresponding to the smart glasses. Within the shooting range corresponding to the smart glasses, if there is a preset position with a distance D1 relative to the first signal receiver of the smart glasses, the distance D2 of the preset position relative to the second signal receiver of the smart glasses, and the distance D3 of the preset position relative to the third signal receiver of the smart glasses, the position of the signal transmitting device relative to the smart glasses can be determined based on the preset position. Of course, it is not limited to this, and no limitation is made here.

[0076] For example, the distance of the signal transmitting device relative to the signal receiver can be determined based on the correspondence between the preset time difference and the distance of the signal transmitting device relative to the signal receiver. For example, when the time difference is determined, the distance of the signal transmitting device relative to the signal receiver corresponding to the time difference can be found from the correspondence between the preset time difference and the distance of the signal transmitting device relative to the signal receiver, thereby determining the distance of the signal transmitting device relative to the signal receiver. Of course, it is not limited to this and is not limited here.

[0077] In this way, by determining the distance between the signal transmitting device and the signal receiver, based on the signal receiver being set on the smart glasses, the position of the signal transmitting device relative to the smart glasses can be subsequently determined, thereby helping to improve the convenience of determining the position of the signal transmitting device relative to the smart glasses.

[0078] S104: Determine the position of the hand area in the target image according to the position of the signal transmitting device relative to the smart glasses.

[0079] For example, a spatial relative coordinate system Z can be established with the smart glasses as the reference system. For example, the spatial relative coordinate system Z is established with the center point of the smart glasses as the coordinate origin, but this is certainly not limited to this and is not limited here. In some embodiments, the position of the signal transmitting device in the spatial relative coordinate system Z can be determined based on the position of the signal transmitting device relative to the smart glasses. For example, the position coordinates of the signal transmitting device in the spatial relative coordinate system Z are (U, V, W). The target image obtained by photographing the user's hand based on the smart glasses can be a two-dimensional image, then the position coordinates (U, V, W) of the signal transmitting device in the spatial relative coordinate system Z need to be converted to obtain the position of the hand area in the target image, for example, the position of the hand area in the target image is determined to be (u, v). Of course, this is not limited to this and is not limited here.

[0080] In this way, by determining the position of the hand area in the target image according to the position of the signal transmitting device relative to the smart glasses, it is beneficial to subsequently detect the user's gesture in the target image and improve the gesture detection effect of the target image. In addition, when the target image includes the hands of multiple users, the interference of other users' hands on the positioning accuracy of the user's hands can be reduced, the missed detection rate and false detection rate of the user's hands in the target image can be reduced, and the positioning accuracy of the user's hands in the target image can be improved.

[0081] In some implementations, the position of the signal transmitting device relative to the smart glasses is converted according to a preset position conversion function to obtain the position of the hand area in the target image.

[0082] For example, the preset position conversion function can convert the position of the signal transmitting device relative to the smart glasses into the position of the hand area in the target image, for example, converting the position coordinates (U, V, W) of the signal transmitting device in the spatial relative coordinate system Z into the position of the hand area in the target image as (u, v). Of course, it is not limited to this and is not limited here.

[0083] In an exemplary embodiment, a camera is provided on the smart glasses. For example, the position of the camera on the smart glasses can be calibrated by a calibration algorithm, and based on the calibrated position of the camera on the smart glasses, a preset position conversion function required for converting the position of the signal transmitting device relative to the smart glasses into the position of the hand area in the target image is determined, which is of course not limited to this and is not limited here.

[0084] In this way, by converting the position of the signal transmitting device relative to the smart glasses according to a preset position conversion function, the position of the hand area in the target image is obtained, which is conducive to improving the convenience of determining the position of the hand area in the target image.

[0085] S105: Perform gesture detection on the target image according to the position of the hand area in the target image.

[0086] For example, the position of the hand area in the target image is equivalent to the position of the user's hand in the target image. According to the position of the hand area in the target image, gesture detection is performed on the target image, and the gesture of the user in the target image can be detected. Based on this, when the target image includes the hands of multiple users, the interference of the hands of other users on the positioning accuracy of the user's hands can be eliminated, and the gesture of the user in the target image can be detected.

[0087] In this way, gesture detection is performed on the target image through the position of the hand area in the target image, which is beneficial to improving the accuracy of locating the hand in the target image, improving the accuracy of gesture detection of the user in the target image, and improving the gesture detection effect of the target image.

[0088] In some implementations, based on the position of the hand region in the target image, a hand contour corresponding to the hand region is identified; and based on the hand contour corresponding to the hand region, a gesture corresponding to the hand region is determined.

[0089] For example, when the position of the hand area in the target image is determined, the position of the hand area in the target image can be expanded to identify the hand contour corresponding to the hand area, that is, to identify the hand contour of the user in the target image. Based on the determination of the hand contour corresponding to the hand area, it can be compared with the preset gesture contour to determine the gesture corresponding to the hand area. Of course, it is not limited to this and is not limited here.

[0090] In this way, determining the gesture corresponding to the hand area based on the hand contour corresponding to the hand area is conducive to improving the convenience of detecting the gesture of the user in the target image, thereby helping to improve the accuracy of gesture detection for the total user of the target image and improving the gesture detection effect for the target image.

[0091] In some implementations, the smart glasses can be controlled to execute a preset function corresponding to a gesture based on the gesture corresponding to the hand area.

[0092] For example, when the target image includes the hands of multiple users, the position of the hand area of ​​the user wearing or using smart glasses in the target image can be determined from the hands of the multiple users, thereby identifying the gesture of the user wearing or using smart glasses in the target image. Based on this, according to the gesture interaction between the user wearing or using smart glasses and the smart glasses, the smart glasses can be controlled to execute the preset function corresponding to the gesture interaction, which is conducive to improving the user experience of the smart glasses.

[0093] The gesture detection method provided in the above embodiment obtains a target image obtained by photographing the user's hands with smart glasses; receives a detection signal transmitted by the user's signal transmitting device; determines the position of the signal transmitting device relative to the smart glasses based on the reception time of the detection signal; determines the position of the hand area in the target image based on the position of the signal transmitting device relative to the smart glasses; performs gesture detection on the target image based on the position of the hand area in the target image to reduce the missed detection rate and false detection rate of the user's hands in the target image, improve the positioning accuracy of the user's hands in the target image, and enhance the gesture detection effect of the target image.

[0094] See also Figure 2 , Figure 2 is a schematic block diagram of a gesture detection device provided in an embodiment of the present application. The gesture detection device can be configured in a server or an electronic device to execute the aforementioned gesture detection method.

[0095] like Figure 2 As shown, the gesture detection device includes: an acquisition module 110 , a receiving module 120 , a first determination module 130 , a second determination module 140 and a detection module 150 .

[0096] The acquisition module 110 is used to acquire a target image obtained by photographing a user's hand through smart glasses, where the smart glasses are mounted on the user's head.

[0097] The receiving module 120 is used to receive the detection signal transmitted by the signal transmitting device of the user, and the signal transmitting device is carried on the hand of the user.

[0098] The first determination module 130 is used to determine the position of the signal transmitting device relative to the smart glasses according to the reception time of the detection signal.

[0099] The second determination module 140 is used to determine the position of the hand area in the target image according to the position of the signal transmitting device relative to the smart glasses.

[0100] The detection module 150 is used to perform gesture detection on the target image according to the position of the hand area in the target image.

[0101] Exemplarily, the smart glasses are provided with a plurality of signal receivers, and the positions of the plurality of signal receivers on the smart glasses are different. The first determination module 130 includes a first determination submodule and a second determination submodule.

[0102] A first determination submodule, configured to determine the distance of the signal transmitting device relative to each of the signal receivers according to the reception time of the detection signal received by each of the signal receivers;

[0103] The second determination submodule is used to determine the position of the signal transmitting device relative to the smart glasses according to the distance of the signal transmitting device relative to each of the multiple signal receivers.

[0104] Exemplarily, the signal transmitting device is provided with a first signal transmitter and a second signal transmitter, and the detection signals transmitted by the first signal transmitter and the second signal transmitter have different transmission speeds. The first determination submodule includes a distance determination submodule.

[0105] The distance determination submodule is used to determine the distance of the signal transmitting device relative to the signal receiver based on the reception time of the detection signal of the first signal transmitter received by the signal receiver and the reception time of the detection signal of the second signal transmitter received by the signal receiver.

[0106] Exemplarily, the signal transmitting device transmits the detection signal when detecting a first preset operation of the signal transmitting device by the user, and stops transmitting the detection signal when detecting a second preset operation of the signal transmitting device by the user. The receiving module 120 includes a first receiving submodule.

[0107] The first receiving submodule is used to receive a detection signal transmitted by the signal transmitting device of the user when a first preset operation of the user on the signal transmitting device is detected.

[0108] Exemplarily, the gesture detection device further includes a second receiving submodule.

[0109] The second receiving submodule is used to stop receiving the detection signal transmitted by the signal transmitting device of the user when a second preset operation of the user on the signal transmitting device is detected.

[0110] Exemplarily, the second determination module 140 includes a conversion submodule.

[0111] The conversion submodule is used to convert the position of the signal transmitting device relative to the smart glasses according to a preset position conversion function to obtain the position of the hand area in the target image.

[0112] Exemplarily, the detection module 150 includes a recognition submodule and a gesture determination submodule.

[0113] The recognition submodule is used to recognize the hand contour corresponding to the hand area according to the position of the hand area in the target image.

[0114] The gesture determination submodule is used to determine the gesture corresponding to the hand area according to the hand contour corresponding to the hand area.

[0115] It should be noted that technicians in the relevant field can clearly understand that for the convenience and simplicity of description, the specific working processes of the above-described devices and modules and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0116] The method of the present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0117] Exemplarily, the above method and apparatus may be implemented in the form of a computer program, which may be run on smart glasses. Exemplarily, smart glasses may include VR glasses, AR glasses, etc., which are not limited here.

[0118] See also Figure 3 , Figure 3It is a schematic block diagram of the structure of a pair of smart glasses provided in an embodiment of the present application.

[0119] like Figure 3 As shown, the smart glasses include a memory and a processor. The memory and the processor may be connected via a system bus, and the memory may include a storage medium and an internal memory.

[0120] The storage medium may store an operating system and a computer program. When the computer program is executed, the processor may execute any gesture detection method.

[0121] The processor is used to provide computing and control capabilities to support the operation of the entire smart glasses.

[0122] The internal memory provides an environment for the operation of the computer program in the storage medium. When the computer program is executed by the processor, the processor can execute any gesture detection method.

[0123] Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the smart glasses to which the scheme of the present application is applied. The specific smart glasses may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0124] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0125] In one embodiment, the processor is used to execute a computer program and implement the following steps when executing the computer program:

[0126] Acquire a target image obtained by photographing a user's hand through the smart glasses;

[0127] Receiving a detection signal transmitted by a signal transmitting device of the user;

[0128] Determining the position of the signal transmitting device relative to the smart glasses according to the reception time of the detection signal;

[0129] Determining a position of a hand region in the target image according to a position of the signal transmitting device relative to the smart glasses;

[0130] Performing gesture detection on the target image according to the position of the hand area in the target image.

[0131] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of gesture detection described above can refer to the corresponding process in the aforementioned gesture detection method embodiment, and will not be repeated here.

[0132] See also Figure 4 , Figure 4 It is a schematic block diagram of a gesture detection system provided in an embodiment of the present application.

[0133] like Figure 4 As shown, the gesture detection system includes a signal transmitting device and the aforementioned smart glasses.

[0134] Exemplarily, the signal transmitting device is used to transmit a detection signal.

[0135] Exemplarily, smart glasses can be used to implement the steps of the gesture detection method of the embodiment of the present application.

[0136] The specific principles and implementation methods of the gesture detection system provided in the embodiments of the present application are similar to the gesture detection method of the aforementioned embodiments, and will not be described in detail here.

[0137] The embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. The method implemented when the computer program is executed by a processor can refer to the various embodiments of the gesture detection method of the present application.

[0138] The computer-readable storage medium may be an internal storage unit of the smart glasses described in the foregoing embodiments, such as a hard disk or memory of the smart glasses. The computer-readable storage medium may also be an external storage device of the smart glasses, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc., equipped on the smart glasses.

[0139] It should be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0140] It should also be understood that the term "and / or" used in the specification of this application and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations. It should be noted that, in this article, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "including a..." does not exclude the presence of other identical elements in the process, method, article or system including the element.

[0141] The serial numbers of the embodiments of the present application are for description only and do not represent the advantages and disadvantages of the embodiments. The above description is only a specific implementation mode of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A gesture detection method, characterized in that: For smart glasses, the gesture detection method includes: Acquire a target image obtained by photographing a user's hand through the smart glasses; Receiving a detection signal transmitted by a signal transmitting device of the user; Determining the position of the signal transmitting device relative to the smart glasses according to the reception time of the detection signal; Determining a position of a hand region in the target image according to a position of the signal transmitting device relative to the smart glasses; Performing gesture detection on the target image according to the position of the hand area in the target image.

2. The gesture detection method according to claim 1, characterized in that: The smart glasses are provided with a plurality of signal receivers, and the positions of the plurality of signal receivers on the smart glasses are different; The determining, according to the reception time of the detection signal, the position of the signal transmitting device relative to the smart glasses comprises: According to the reception time of the detection signal received by each of the signal receivers, correspondingly determining the distance of the signal transmitting device relative to each of the signal receivers; The position of the signal transmitting device relative to the smart glasses is determined according to the distance of the signal transmitting device relative to each of the multiple signal receivers.

3. The gesture detection method according to claim 2, characterized in that: The signal transmitting device is provided with a first signal transmitter and a second signal transmitter, and the transmission speeds of the detection signals transmitted by the first signal transmitter and the second signal transmitter are different; The determining the distance of the signal transmitting device relative to each of the signal receivers according to the reception time of the detection signal received by each of the signal receivers comprises: The distance of the signal transmitting device relative to the signal receiver is determined according to the reception time of the detection signal of the first signal transmitter received by the signal receiver and the reception time of the detection signal of the second signal transmitter received by the signal receiver.

4. The gesture detection method according to claim 1, characterized in that: The signal transmitting device transmits the detection signal when detecting a first preset operation of the signal transmitting device by the user, and stops transmitting the detection signal when detecting a second preset operation of the signal transmitting device by the user; The receiving the detection signal transmitted by the signal transmitting device of the user includes: When a first preset operation of a user on a signal transmitting device is detected, receiving a detection signal transmitted by the signal transmitting device of the user; The gesture detection method further includes: When a second preset operation of the user on the signal transmitting device is detected, the detection signal transmitted by the signal transmitting device of the user is stopped.

5. The gesture detection method according to any one of claims 1 to 4, characterized in that: The determining the position of the hand area in the target image according to the position of the signal transmitting device relative to the smart glasses includes: According to a preset position conversion function, the position of the signal transmitting device relative to the smart glasses is converted to obtain the position of the hand area in the target image.

6. The gesture detection method according to any one of claims 1 to 4, characterized in that: The performing gesture detection on the target image according to the position of the hand area in the target image includes: According to the position of the hand area in the target image, identifying the hand contour corresponding to the hand area; A gesture corresponding to the hand area is determined according to a hand contour corresponding to the hand area.

7. A gesture detection device, characterized in that: The gesture detection device comprises: An acquisition module is used to acquire a target image obtained by photographing a user's hand by the smart glasses; A receiving module, used to receive a detection signal transmitted by a signal transmitting device of the user; A first determination module, configured to determine a position of the signal transmitting device relative to the smart glasses according to a reception time of the detection signal; A second determination module, configured to determine a position of a hand region in the target image according to a position of the signal transmitting device relative to the smart glasses; The detection module is used to perform gesture detection on the target image according to the position of the hand area in the target image.

8. A smart glasses, characterized in that: The smart glasses include a memory and a processor; The memory is used to store computer programs; The processor is configured to execute the computer program and implement the gesture detection method according to any one of claims 1 to 6 when executing the computer program.

9. A gesture detection system, characterized in that: The gesture detection system comprises: A signal transmitting device, wherein the signal transmitting device is used to transmit a detection signal; Smart glasses as claimed in claim 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the gesture detection method according to any one of claims 1 to 6 are implemented.

Images