Visual tracking wake-up method, device, computer equipment and storage medium

Abstract

The embodiments of this specification provide a wake-up method, device, computer device and storage medium for visual tracking. The method is applied to a smart head-mounted device, on which an ultrasonic transmitting unit and an ultrasonic receiving unit are installed. The method includes: using the ultrasonic transmitting unit to send an ultrasonic detection signal to the surroundings of the smart head-mounted device; using the ultrasonic receiving unit to receive an ultrasonic feedback signal; wherein the ultrasonic feedback signal is an ultrasonic signal reflected by the target object by the ultrasonic detection signal; judging whether the user has made a gesture within a preset detection range according to the ultrasonic feedback signal; in the case where the user has made a gesture, waking up the camera of the smart head-mounted device to use the camera to visually track the user's gesture. The technical solution provided in the embodiments of this specification can reduce the power consumption of the smart head-mounted device for gesture recognition to a certain extent.

Description

Technical Field

[0001] The embodiments of this specification relate to the field of smart head-mounted devices, and specifically to a visual tracking wake-up method, device, computer equipment and storage medium. Background Art

[0002] Gesture capture technology in VR glasses usually detects hand movements and postures through multiple sensors, allowing users to interact in virtual environments through natural gestures. Currently, VR glasses mainly use built-in hand tracking cameras, which use computer vision technology to capture the movement of users' hands. However, when users use VR glasses, these cameras must be in wake-up mode, otherwise there is no guarantee when the user will make gestures. On the one hand, this consumes a higher power consumption of VR glasses, and on the other hand, when there is no need to recognize user gestures, it also affects the user's privacy to a certain extent. Summary of the invention

[0003] In view of this, multiple embodiments of this specification are dedicated to providing a visual tracking wake-up method, apparatus, computer device, storage medium and computer program product to reduce the power consumption of gesture recognition of smart head-mounted devices to a certain extent.

[0004] The implementation mode of this specification proposes a visual tracking wake-up method, which is applied to an intelligent head-mounted device, on which an ultrasonic transmitting unit and an ultrasonic receiving unit are installed. The method includes: using the ultrasonic transmitting unit to send an ultrasonic detection signal to the surroundings of the intelligent head-mounted device; using the ultrasonic receiving unit to receive an ultrasonic feedback signal; wherein the ultrasonic feedback signal is an ultrasonic signal reflected by the target object after the ultrasonic detection signal is reflected; judging whether the user has made a gesture within a preset detection range according to the ultrasonic feedback signal; and if the user has made a gesture, waking up the camera of the intelligent head-mounted device to use the camera to visually track the user's gesture.

[0005] The implementation mode of this specification proposes a wake-up device for visual tracking, and the method is applied to an intelligent head-mounted device, on which an ultrasonic transmitting unit and an ultrasonic receiving unit are installed. The wake-up device for visual tracking includes: an ultrasonic detection module, which is used to use the ultrasonic transmitting unit to send an ultrasonic detection signal to the surroundings of the intelligent head-mounted device; a signal receiving module, which is used to use the ultrasonic receiving unit to receive an ultrasonic feedback signal; wherein the ultrasonic feedback signal is an ultrasonic signal reflected by the target object after the ultrasonic detection signal is reflected by the target object; a gesture judgment module, which is used to judge whether the user has made a gesture within a preset detection range according to the ultrasonic feedback signal; and a wake-up module, which is used to wake up the camera of the intelligent head-mounted device when the user has made a gesture, so as to use the camera to visually track the user's gesture.

[0006] An embodiment of the present specification provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the method described in the above embodiment when executing the computer program.

[0007] The embodiments of this specification provide a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the method described in the above embodiments is implemented.

[0008] The embodiments of this specification provide a computer program product, including computer instructions, and the computer instructions are used to enable a computer to execute the method described in the above embodiments.

[0009] The implementation method of this specification sends ultrasonic signals outward, then receives ultrasonic feedback signals reflected from these ultrasonic signals, and determines whether the user has made a gesture within a preset detection range based on these ultrasonic feedback signals. If the user has made a gesture within the preset detection range, the camera of the smart head-mounted device is awakened to use the camera to recognize the user's gesture, thereby eliminating the need to turn on the camera when starting the smart head-mounted device, thereby reducing the power consumption of the smart head-mounted device for gesture recognition to a certain extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 Shown is a flow chart of a visual tracking wake-up method provided in one embodiment.

[0011] Figure 2 Shown is a schematic diagram of a visual tracking wake-up device provided in one embodiment.

[0012] Figure 3 Shown is a schematic diagram of a computer device provided by an embodiment. DETAILED DESCRIPTION

[0013] In order to enable those skilled in the art to better understand the solution of this specification, the technical solution in the implementation of this specification will be clearly and completely described below in conjunction with the drawings in the implementation of this specification. Obviously, the described implementation is only a part of the implementation of this specification, not all of the implementations. Based on the implementation in this specification, all other implementations obtained by ordinary technicians in this field without creative work are within the scope of protection of this specification.

[0014] Gesture capture technology in VR glasses usually detects hand movements and postures through multiple sensors, allowing users to interact in virtual environments through natural gestures. Currently, VR glasses mainly use built-in hand tracking cameras, which use computer vision technology to capture the movement of users' hands. However, when users use VR glasses, these cameras must be in wake-up mode, otherwise there is no guarantee when the user will make gestures. On the one hand, VR glasses consume higher power consumption, and on the other hand, when there is no need to recognize user gestures, it also affects the user's privacy to a certain extent. If a physical button is added to the VR glasses, the user needs to turn on the button to perform gesture recognition, which is a poor user experience, and there is a possibility that the button will be forgotten to be turned off after it is turned on.

[0015] Therefore, the speaker of the smart head-mounted device itself can be used as an ultrasonic transmitting unit to transmit ultrasonic signals, and its own microphone can be used as an ultrasonic receiving unit to receive ultrasonic signals reflected by objects in the space. Then, based on the reflected ultrasonic signals, it is determined whether hand activity can be detected. This allows the camera of the smart head-mounted device to be woken up only when gesture recognition is required, thereby reducing the power consumption of the smart head-mounted device for gesture recognition to a certain extent.

[0016] See also Figure 1 The embodiment of this specification provides a visual tracking wake-up method, which is applied to a smart head-mounted device, on which an ultrasonic transmitting unit and an ultrasonic receiving unit are installed. The method may include the following steps.

[0017] Step S110: using the ultrasonic transmitting unit to send an ultrasonic detection signal to the surroundings of the smart head mounted device.

[0018] In this embodiment, the smart head-mounted device includes smart glasses, virtual reality headsets (VR headsets), augmented reality helmets (AR headsets), etc. Among them, smart glasses usually have functions such as display information, voice assistants, cameras, etc., and can project information directly onto lenses to provide augmented reality (AR) experience. Virtual reality headsets provide an immersive virtual environment experience through head-mounted devices. VR headsets are usually connected to computers or game consoles, and use sensors to track head movements to provide a 360-degree visual experience. Augmented reality technology combines virtual information with the real world through the screen or lenses of the device.

[0019] In this embodiment, the ultrasonic transmitter unit (Ultrasonic Transmitter Unit) is a device for transmitting ultrasonic signals, which is usually composed of an ultrasonic sensor, a driving circuit, a signal source and a power amplifier. The working principle of the ultrasonic transmitter unit is based on the piezoelectric effect or the magnetostrictive effect, as follows: Piezoelectric effect: When an external voltage acts on a piezoelectric material, the material will undergo mechanical deformation, generate vibrations and emit ultrasonic waves; conversely, when ultrasonic waves pass through piezoelectric materials, the material will generate electrical signals (inverse piezoelectric effect). Signal generation and amplification: Ultrasonic signals are usually high-frequency AC signals (e.g., tens of kHz to several MHz), which need to be generated by a signal source, amplified by a power amplifier, and finally converted into mechanical vibrations (ultrasonic waves) by a transducer. Ultrasonic emission: The ultrasonic transducer converts the electrical signal into an ultrasonic signal and transmits it into the medium. These ultrasonic waves can propagate to the target object for distance measurement.

[0020] In this embodiment, the ultrasonic transmitting unit may include piezoelectric ceramics, piezoelectric films, magnetostrictive materials, ultrasonic transducers, and excitation circuits. These components convert electrical energy into ultrasonic waves through different physical effects (such as piezoelectric effect, magnetostrictive effect, etc.). Among them, the speaker can drive the vibration membrane (or voice coil) of the speaker to vibrate through an electrical signal, thereby generating pressure waves (sound waves) in the air. Therefore, the speaker carried by the smart head-mounted device itself can be used as an ultrasonic transmitting unit.

[0021] In this embodiment, the ultrasonic detection signal may be an ultrasonic signal sent outward by the ultrasonic transmitting unit. Since ultrasonic waves have weak penetration, when they encounter obstacles, the ultrasonic signal may be reflected.

[0022] Step S120: using the ultrasonic receiving unit to receive an ultrasonic feedback signal; wherein the ultrasonic feedback signal is an ultrasonic signal reflected by the ultrasonic detection signal from the target object.

[0023] In this embodiment, the ultrasonic receiving unit is a device for receiving ultrasonic signals and converting them into processable electrical signals. It is usually used in conjunction with an ultrasonic transmitting unit. The working principle of the ultrasonic receiving unit is opposite to that of the ultrasonic transmitting unit. It uses the piezoelectric effect or the electroacoustic effect to convert the received ultrasonic signal (i.e., sound wave) into an electrical signal. The most common way to receive ultrasonic waves is to use piezoelectric materials. When subjected to ultrasonic vibration, piezoelectric materials deform and generate voltage signals. The amplitude of this voltage signal is proportional to the intensity and frequency of the incident ultrasonic wave. The ultrasonic receiving unit usually includes the following key components: receiving sensor (probe): This is the core component for receiving ultrasonic signals. Commonly used sensor materials include piezoelectric ceramics (such as PZT), piezoelectric films or MEMS (micro-electromechanical system) sensors. These sensors can convert the received sound wave energy into electrical signals. Gain control circuit: Ultrasonic signals usually attenuate during propagation, and the received signal strength may be weak. Therefore, the receiving unit usually includes a gain control circuit for amplifying the signal for subsequent processing. Filtering circuit: Because the frequency of ultrasonic signals is usually high, the receiving circuit may need filtering to remove noise or unnecessary frequency components to make the signal clearer. Signal processing unit: After receiving the ultrasonic signal, it is usually sent to the signal processing unit (such as a digital signal processor or microcontroller). This unit is responsible for analyzing and processing the ultrasonic signal to extract the required information, such as distance measurement.

[0024] In this embodiment, the ultrasonic feedback signal is an ultrasonic signal emitted by the ultrasonic transmitting unit and received by the ultrasonic receiving unit after being reflected by the surface of the target object. The target object may be clothing, a chair, a desktop, a wall, etc. Since the surface materials of each target object are different, they have different reflectivities to the ultrasonic wave. Therefore, the signal strength of the ultrasonic feedback signal may also be different at the same distance.

[0025] Step S130: determining whether the user has made a gesture within a preset detection range according to the ultrasonic feedback signal.

[0026] In this embodiment, the sound has a Doppler effect. The Doppler effect of ultrasound refers to the phenomenon that the frequency of the received ultrasound wave changes when there is relative motion between the ultrasound source and the receiver. When the ultrasound wave source and the receiver move relative to each other, the received frequency is different from the frequency when the wave source is emitted. Specifically: when the ultrasound source approaches the receiver, the received frequency becomes higher (the frequency increases), and when the ultrasound source moves away from the receiver, the received frequency becomes lower (the frequency decreases). Therefore, it is possible to determine whether the target object is moving based on the Doppler effect of ultrasound.

[0027] In this embodiment, the preset detection range can be a partial area in front of the body, which can be set by the manufacturer before leaving the factory, or it can be set by the user according to his or her own height and body shape after purchase. Of course, the camera of the smart head-mounted device can also scan the human body, and then input it into the trained distance setting model, and then set it according to the result of the distance setting model. Then, it can be calculated whether the distance between the target object and the ultrasonic receiving unit is within the predetermined spatial range based on the time node when the moving target object is detected and the time node when the ultrasonic transmitting unit sends the frame of ultrasonic detection signal.

[0028] Step S140: When the user makes a gesture, wake up the camera of the smart head mounted device to use the camera to visually track the user's gesture.

[0029] In this embodiment, when the user makes a hand movement within a predetermined range, it means that the user wants to use the smart head-mounted device to recognize gestures, and the camera of the smart head-mounted device can be immediately awakened. If the operation of mutual finger tapping is recognized, the APP can be opened by selecting an option; if the action of light touch is recognized, a typing operation can be performed to interact with a specific element; if a pinching operation is recognized, additional options such as zooming in or out can be displayed; if pinching and dragging is recognized, the window can be moved or slid, etc.

[0030] In this embodiment, the ultrasonic transmitting unit, ultrasonic receiving unit and camera of the smart head-mounted device are used in conjunction, so that the camera does not need to remain awake for a long time, and the camera is awakened when it is necessary to capture the user's gestures. The Doppler effect of ultrasound is used to determine whether there is a hand movement, and the time of sending and receiving ultrasonic signals can be used to calculate whether the hand movement is within the detection range of the smart head-mounted device, and then the camera is awakened, so that the power consumption of the smart device can be reduced to a certain extent without affecting the user experience, and the camera is awakened only when it is needed, which also protects the user's privacy to a certain extent.

[0031] In some embodiments, determining whether the user has made a gesture within a preset detection range based on the ultrasonic feedback signal may include: acquiring a first ultrasonic feedback signal and a second ultrasonic feedback signal; wherein the first ultrasonic feedback signal and the second ultrasonic feedback signal are ultrasonic feedback signals obtained by reflection from the surface of an object for ultrasonic detection signals on different frames; determining whether the signal strength of direct signals in the first ultrasonic feedback signal and the second ultrasonic feedback signal has changed; in the event that the signal strength of the direct signals in the first ultrasonic feedback signal and the second ultrasonic feedback signal has changed, determining whether the user has made a gesture within the preset detection range based on a time node when the signal strength in the second ultrasonic feedback signal changes and a time node when the ultrasonic detection signal corresponding to the second ultrasonic feedback signal is emitted.

[0032] In this embodiment, two different groups of ultrasonic feedback signals (a first ultrasonic feedback signal and a second ultrasonic feedback signal) are first obtained. The two groups of ultrasonic feedback signals are ultrasonic detection signals at different frame rates reflected from the surface of the target object. Then, it is determined whether the signal strength of the ultrasonic direct signal in the two groups of ultrasonic feedback signals has changed. The signal frequency of the ultrasonic direct signal is larger than that of the ultrasonic signal after multiple reflections. If there is no change, it means that the user's hand movement is not detected. If there is a change, it means that the movement of the target object is detected. At this time, it is determined whether the change is within the preset detection range based on the time when the change occurs and the emission time of the second ultrasonic feedback signal. If it is within the preset detection range, it can continue to determine whether the gesture action that has changed is within the preset detection range. If it is within the preset detection range, the camera is woken up.

[0033] In some embodiments, determining whether the signal strength of the direct signal in the first ultrasonic feedback signal and the second ultrasonic feedback signal has changed may include: determining the time nodes at which different ultrasonic reflection signals are received based on the first ultrasonic feedback signal; recording the intensity difference between the signal strength of the second ultrasonic feedback signal and the signal strength of the first ultrasonic feedback signal at each time node; and determining that the signal strength of the direct signal in the first ultrasonic feedback signal and the second ultrasonic feedback signal has changed when the intensity difference is greater than or equal to a preset intensity.

[0034] In this embodiment, if no action occurs within the spatial range, the ultrasonic channel changes very little, and the spatial range can be considered to be a stable environment. Therefore, the relative position relationship between the reflection channel and the direct channel remains almost unchanged. In the case where there is an active target object within the spatial range, the stable model of the signal will be broken. Therefore, when the user does not make a gesture, the signal strength between the first ultrasonic feedback signal and the second ultrasonic feedback signal remains almost unchanged; and when the user makes a gesture, there will be a large difference between the first ultrasonic feedback signal and the second ultrasonic feedback signal. Therefore, it is possible to determine whether there is a gesture action within the preset detection range based on whether the signal strength of two different sets of ultrasonic feedback signals changes.

[0035] In this embodiment, the moment corresponding to the first direct reflection signal in the first ultrasonic feedback signal is recorded as t1, and it is determined whether the signal strength in the second ultrasonic feedback signal at time t1 and the signal strength of the first ultrasonic feedback signal at time t1 are greater than the preset strength. If they are greater than the preset strength, it means that the corresponding ultrasonic reflection signal at this moment has undergone a large change, and it can be considered that there is gesture activity at this point.

[0036] In this embodiment, the multiple groups of second ultrasonic feedback signals are multiple frames of ultrasonic signals emitted by the ultrasonic transmitting unit that are continuously collected. Since the signal change at a single moment may be caused by various reasons such as data error or wind noise, the ultrasonic observation data is unstable. Therefore, the variance comparison analysis of the ultrasonic observation values ​​at the same moment can be performed on the continuous ultrasonic signal frames to see whether it is stable within a certain range. If it is stable within the range of the preset variance, it means that at this moment, the ultrasonic reflection signal in the car is still within a stable range of change.

[0037] In this embodiment, first, multiple groups of ultrasonic observation values ​​can be stored in a fixed-size memory pool. As the memory pool is updated and overflowed, the variance and mean of the ultrasonic observation values ​​in the memory pool are continuously calculated. Normally, when there is gesture activity, the ultrasonic observation value will change dramatically, and the variance value of the memory pool will also increase accordingly; when there is no gesture action, the ultrasonic observation value will not change dramatically, and even if there is a large environmental noise, the variance will remain relatively stable. Therefore, it is possible to determine whether there is human activity at a time node based on the variance of the ultrasonic observation value in a memory pool.

[0038] In this embodiment, in the process of continuously updating the variance, due to the influence of environmental noise, human body detection based on the change of the observed value will have errors, and it is difficult to define the threshold of human presence and absence. Therefore, this embodiment uses the Kalman filtering method to filter the variance to eliminate the influence of environmental noise on the detection results. Kalman filtering is an algorithm that uses the linear system state equation to optimally estimate the system state through the system input and output ultrasonic observation values. Its function is filtering, and its purpose is to reduce the influence of noise and interference on data measurement.

[0039] The Kalman equation adopted in this embodiment is as follows:

[0040] (1) Prediction part:

[0041] The covariance of the system estimate at time K = the system covariance at time k-1 + the process noise covariance, that is:

[0042]

[0043] (2) Kalman gain equation:

[0044] Kalman gain = system estimation covariance at time k / (system estimation covariance at time k + observation noise covariance), that is:

[0045]

[0046] (3) Update the optimal value solution:

[0047] The optimal value of the state at time k = the predicted value of the state + Kalman gain * (measured value - predicted value of the state), that is:

[0048]

[0049] Specifically, in the process of updating the variance, it is necessary to retain the minimum value Var of the variance over a period of time min During this period of time, a certain multiple of the minimum variance value Var during this period of time min *K var Set as the threshold for Kalman filtering.

[0050] Among them, K var The setting range is 10-20 times. Variance indicates the discreteness of data. The variance data in the unmanned state is relatively stable, mainly concentrated near a value, and the variance is small. The noise in the environment only makes the data fluctuate slightly larger. This part of the fluctuation can be included by 10-20 times the minimum variance. varIf the target ultrasonic observation difference value is set within the above range, the filtered target ultrasonic observation difference value can be obtained. Therefore, when the target ultrasonic observation difference value is greater than or equal to the preset variance, the time node corresponding to the target ultrasonic observation difference value can be determined as the second moment.

[0051] In this embodiment, the minimum absolute difference mean of the ultrasonic observation values ​​is calculated, and the product of the minimum absolute difference mean and the second preset multiple is used as the preset variance Dev min *K dev ; Then, compare the ultrasonic observation difference value with the preset variance. If the ultrasonic observation difference value is greater than or equal to the preset variance, it is determined that human activity is detected in the vehicle; otherwise, continue to determine whether there is human activity at the next time node.

[0052] In this embodiment, the second preset multiple is 20-30 times. The absolute deviation mean indicates the degree of dispersion of data around the mean. By continuously updating the minimum absolute deviation mean, the minimum absolute deviation mean is relatively small in a quiet environment; the minimum absolute deviation mean is larger in a noisy environment. In addition, the smaller the second preset multiple is set, the larger the range of sensing whether there are human activities in the vehicle space. By adjusting the second preset multiple, if multiple ultrasonic receiving units are arranged in the vehicle, the units can be non-interfering with each other.

[0053] In some embodiments, judging whether the user has made a gesture within a preset detection range based on the time node when the signal strength in the second ultrasonic feedback signal changes and the emission time node of the ultrasonic detection signal corresponding to the second ultrasonic feedback signal can include: recording a first sampling number at the moment when the second ultrasonic feedback signal is emitted; and, recording a second sampling number at the moment when the signal strength in the second ultrasonic feedback signal changes, multiplying the result obtained by subtracting the second sampling number from the first sampling number by the sampling frequency to obtain the signal transmission time; and determining whether the user has made a gesture within the preset detection range based on the signal transmission time and the propagation speed of the ultrasonic wave.

[0054] In this embodiment, the first sampling number is the number of ultrasonic feedback signals collected by the ultrasonic receiving unit at the time of emission of the ultrasonic detection signal corresponding to the second ultrasonic feedback signal. The second sampling number is the number of ultrasonic feedback signals collected by the ultrasonic receiving unit at the time when the signal strength of the ultrasonic feedback signal changes. Since the time interval of each acquisition is the same in the process of collecting ultrasonic reflection signals. Specifically, for example, the ultrasonic receiving unit collects 48,000 ultrasonic signals per second, the first sampling number is 48, and the second sampling number is 72. Then the result obtained by subtracting the second sampling number from the first sampling number is 24, and then multiplying it by the sampling frequency, that is, 24×60 / 48000=0.0015s. That is, the signal transmission time is 0.0015s. Among them, the signal has gone back and forth from emission to reflection. At this time, the distance between the gesture action and the smart head-mounted device is 340×0.0015 / 2=25.5cm, where 340 is the propagation speed of ultrasonic waves in the air. Then, it is determined whether the distance of 25.5 cm from the smart head-mounted device is within the preset detection range. If it is within the preset detection range, the camera of the smart head-mounted device is awakened; if it is not within the preset detection range, the ultrasonic transmitting unit and the ultrasonic receiving unit are used to continue to monitor whether the user has made gestures within the preset detection range.

[0055] In some embodiments, the method may further include: when it is detected that the smart head-mounted device is in a powered-on state and the smart head-mounted device is worn on the head, starting the ultrasonic transmitting unit and the ultrasonic receiving unit.

[0056] In this embodiment, when the intelligent head-mounted device is detected to be in the on state and the sensor senses that the intelligent head-mounted device is worn on the head, the ultrasonic transmitting unit and the ultrasonic receiving unit are started at the same time, so that it is not easy to cause misjudgment. On the other hand, when the ultrasonic receiving unit and the ultrasonic transmitting unit are started at the same time, although the ultrasonic receiving unit may not receive the ultrasonic feedback signal at the beginning, the continuity of the received ultrasonic signal and the accuracy of calculating the transmission time of the ultrasonic wave in the air can be guaranteed.

[0057] In some embodiments, the method may further include: controlling the smart head mounted device to be in a sleep state when no user gesture action is detected within a preset time range and a preset detection range.

[0058] In this embodiment, if a user gesture is detected within a preset detection range, the camera of the smart head-mounted device is awakened to capture the user's gesture. At this time, if the camera does not capture the user's gesture for a certain period of time or within a preset time range, the camera of the smart head-mounted device can be controlled to be in a dormant state, thereby reducing the power consumption of the smart head-mounted device to a certain extent. The preset time range can be 30 seconds, 1 minute, 2 minutes, 5 minutes, etc., and can be set as needed.

[0059] See also Figure 2 In some embodiments, a visual tracking wake-up device can be provided. The visual tracking wake-up device is applied to an intelligent head-mounted device. An ultrasonic transmitting unit and an ultrasonic receiving unit are installed on the intelligent head-mounted device. The visual tracking wake-up device may include: an ultrasonic detection module, a signal receiving module, a gesture judgment module, and a wake-up module.

[0060] The ultrasonic detection module is used to use the ultrasonic transmitting unit to send an ultrasonic detection signal to the surroundings of the smart head-mounted device.

[0061] A signal receiving module is used to receive an ultrasonic feedback signal using the ultrasonic receiving unit; wherein the ultrasonic feedback signal is an ultrasonic signal reflected by the ultrasonic detection signal through a target object.

[0062] The gesture judgment module is used to judge whether the user has made a gesture action within a preset detection range according to the ultrasonic feedback signal.

[0063] The wake-up module is used to wake up the camera of the smart head-mounted device when the user makes a gesture, so as to use the camera to visually track the user's gesture.

[0064] Regarding the specific functions and effects achieved by the visual tracking wake-up device, please refer to other embodiments of this specification for comparison and explanation, and will not be repeated here. The various modules in the visual tracking wake-up device can be implemented in whole or in part by software, hardware, and a combination thereof. The modules can be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0065] See also Figure 3 In some implementations, a computer device may be provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the method steps in the implementation are implemented.

[0066] In some embodiments, a computer-readable storage medium may be provided, on which a computer program is stored. When the computer program is executed by a processor, the method steps in the embodiments are implemented.

[0067] In some implementations, a computer program product may be provided, including computer instructions, wherein the computer instructions are used to enable a computer to execute the method described in the above implementations.

[0068] Those of ordinary skill in the art can understand that all or part of the processes in the implementation method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the implementation methods of the methods described. Among them, any reference to memory, storage, database or other media used in the various implementations provided in this specification may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0069] It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0070] The descriptions of the various embodiments of this specification are made in a progressive manner. Different embodiments focus on describing the parts that are different from other embodiments. After reading this specification, a person skilled in the art can know that the various embodiments in this specification, as well as the various technical features disclosed in the embodiments, can be combined in more ways. To make the description concise, not all possible combinations of the various technical features in the embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0071] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0072] The multiple embodiments in this specification emphasize the parts that are different from other embodiments, and the various embodiments can be explained in comparison with each other. Any combination of the multiple embodiments in this specification by technicians in the relevant field based on general technical knowledge is included in the disclosure scope of this specification.

[0073] The above description is only an implementation method of this case and is not intended to limit the scope of protection of the claims of this case. For those skilled in the art, this case may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this case should be included in the scope of the claims of this case.

Claims

1. A visual tracking wake-up method, characterized in that: The method is applied to a smart head-mounted device, on which an ultrasonic transmitting unit and an ultrasonic receiving unit are installed, and the method includes: Using the ultrasonic transmitting unit to send an ultrasonic detection signal to the surroundings of the smart head mounted device; Utilizing the ultrasonic receiving unit to receive an ultrasonic feedback signal; wherein the ultrasonic feedback signal is an ultrasonic signal reflected by the target object from the ultrasonic detection signal; Determining whether the user has made a gesture within a preset detection range according to the ultrasonic feedback signal; When the user makes a gesture, the camera of the smart head mounted device is awakened to use the camera to visually track the user's gesture.

2. The method according to claim 1, characterized in that Determining whether the user has made a gesture within a preset detection range according to the ultrasonic feedback signal includes: Acquire a first ultrasonic feedback signal and a second ultrasonic feedback signal; wherein the first ultrasonic feedback signal and the second ultrasonic feedback signal are ultrasonic feedback signals obtained by reflecting the ultrasonic detection signals on different frames from the surface of the object; determining whether the signal strength of the direct signal in the first ultrasonic feedback signal and the second ultrasonic feedback signal changes; When the signal strength of the direct signal in the first ultrasonic feedback signal and the second ultrasonic feedback signal changes, it is determined whether the user has made a gesture within a preset detection range based on the time node when the signal strength in the second ultrasonic feedback signal changes and the emission time node of the ultrasonic detection signal corresponding to the second ultrasonic feedback signal.

3. The method according to claim 2, characterized in that Determining whether the signal strength of the direct signal in the first ultrasonic feedback signal and the second ultrasonic feedback signal changes includes: determining, based on the first ultrasonic feedback signal, time nodes at which different ultrasonic reflection signals are received; Recording the intensity difference between the signal intensity of the second ultrasonic feedback signal and the signal intensity of the first ultrasonic feedback signal at each time node; When the intensity difference is greater than or equal to the preset intensity, it is determined that the signal intensity of the direct signal in the first ultrasonic feedback signal and the second ultrasonic feedback signal has changed.

4. The method according to claim 2, characterized in that: Judging whether the user has made a gesture action within a preset detection range according to a time node at which the signal strength in the second ultrasonic feedback signal changes and a time node at which an ultrasonic detection signal corresponding to the second ultrasonic feedback signal is emitted, includes: Recording the first sampling number at the time when the second ultrasonic feedback signal is emitted; and, recording the second sampling number at the time when the signal strength in the second ultrasonic feedback signal changes; The result obtained by subtracting the second sampling number from the first sampling number is multiplied by the sampling frequency to obtain the signal transmission time; Based on the signal transmission time and the propagation speed of the ultrasonic wave, it is determined whether the gesture action made by the user is within a preset detection range.

5. The method according to claim 1, characterized in that: The method further comprises: When it is detected that the smart head-mounted device is in a powered-on state and the smart head-mounted device is worn on the head, the ultrasonic transmitting unit and the ultrasonic receiving unit are started.

6. The method according to claim 1, characterized in that The method further comprises: When no user gesture action is detected within a preset time range and a preset detection range, the smart head-mounted device is controlled to be in a sleep state.

7. A visual tracking wake-up device, characterized in that: The visual tracking wake-up device is applied to a smart head-mounted device, on which an ultrasonic transmitting unit and an ultrasonic receiving unit are installed. The visual tracking wake-up device includes: An ultrasonic detection module, used to send an ultrasonic detection signal to the surroundings of the smart head mounted device using the ultrasonic transmitting unit; A signal receiving module, used to receive an ultrasonic feedback signal using the ultrasonic receiving unit; wherein the ultrasonic feedback signal is an ultrasonic signal reflected by the ultrasonic detection signal through a target object; A gesture judgment module, used to judge whether the user has made a gesture action within a preset detection range according to the ultrasonic feedback signal; The wake-up module is used to wake up the camera of the smart head-mounted device when the user makes a gesture, so as to use the camera to visually track the user's gesture.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

10. A computer program product, characterized in that The method comprises computer instructions for causing a computer to execute the method according to any one of claims 1 to 6.

Images