Method and apparatus for triggering gesture recognition function

Abstract

The present disclosure relates to a method and device for triggering gesture recognition function, applied to an electronic device equipped with a depth sensor, the method comprising: generating a depth map according to depth data collected by the depth sensor; in the case that it is determined based on the depth map that there is a target object meeting a triggering condition, triggering the electronic device to start gesture recognition function; wherein the triggering condition comprises: a preset region in the field of view range of the depth sensor and a duration in the region reaching a preset duration.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of gesture recognition, and particularly relates to a method and device for triggering gesture recognition. BACKGROUND

[0002] Gesture recognition refers to a process of recognizing movements of various parts of a human body by using an electronic device, and is generally directed to movements of a face or a hand of a human.

[0003] In the related art, there are two ways to implement gesture recognition: one is based on computer vision technology, which captures an image of an object by using a camera installed on an electronic device, and recognizes features in the image to implement gesture recognition; the other is based on depth data, which collects depth data of an object by using a depth sensor installed on an electronic device, and recognizes the depth data to implement gesture recognition.

[0004] However, when implementing gesture recognition by using the above two ways, the electronic device will continuously recognize the collected image or depth data, which increases the cost of gesture recognition. SUMMARY

[0005] The present disclosure provides a method and device for triggering gesture recognition to solve the problems in the related art.

[0006] According to a first aspect of an embodiment of the present disclosure, a method for triggering gesture recognition is provided, which is applied to an electronic device equipped with a depth sensor, and the method comprises:

[0007] generating a depth map according to depth data collected by the depth sensor;

[0008] in a case where it is determined based on the depth map that there is a target object meeting a triggering condition, triggering the electronic device to start a gesture recognition function; wherein the triggering condition comprises: a preset region in a field of view range of the depth sensor and a duration of being located in the region reaching a preset duration.

[0009] According to a second aspect of an embodiment of the present disclosure, a device for triggering gesture recognition is provided, which is applied to an electronic device equipped with a depth sensor, and the device comprises:

[0010] a generating unit configured to generate a depth map according to depth data collected by the depth sensor;

[0011] a triggering unit configured to, in a case where it is determined based on the depth map that there is a target object meeting a triggering condition, trigger the electronic device to start a gesture recognition function; wherein the triggering condition comprises: a preset region in a field of view range of the depth sensor and a duration of being located in the region reaching a preset duration.

[0012] According to a third aspect of the embodiments of the present disclosure, an electronic device is provided, comprising:

[0013] a processor;

[0014] a memory for storing processor-executable instructions;

[0015] The processor is configured to implement the method according to the first aspect.

[0016] According to a fourth aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, which stores a computer program, and the program is executed by a processor to implement the steps in the method according to the first aspect.

[0017] The technical solutions provided by the embodiments of the present disclosure can include the following beneficial effects:

[0018] As can be seen from the above embodiments, the present disclosure sets a triggering method for gesture recognition, generates a depth map according to the depth data collected by the depth sensor installed on the electronic device, and triggers the electronic device to start the gesture recognition function in the case that it is determined based on the depth map that there is a target object meeting the triggering condition, so that the gesture recognition function can only be triggered under certain conditions, thereby avoiding the continuous starting of the gesture recognition function by the electronic device and reducing the cost of gesture recognition; on the other hand, by setting the triggering condition as a preset region within the field of view of the depth sensor and the continuous time length in the region reaching a preset time length, the triggering of the gesture recognition function will not be too difficult to cause failure to trigger, nor too easy to cause random triggering.

[0019] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and other drawings can be obtained by those skilled in the art without creative labor.

[0021] Figure 1 is a schematic architecture diagram of a gesture recognition function triggering according to an embodiment of the present disclosure.

[0022] Figure 2 is a schematic flowchart of a gesture recognition function triggering method according to an embodiment of the present disclosure.

[0023] Figure 3ais a schematic diagram of object preset region movement according to an embodiment of the present disclosure.

[0024] Figure 3b is a schematic diagram of a depth map according to an embodiment of the present disclosure.

[0025] Figure 3c is a schematic diagram of another depth map according to an embodiment of the present disclosure.

[0026] Figure 4a is a schematic diagram of connected regions in a depth map according to an embodiment of the present disclosure.

[0027] Figure 4b is a schematic diagram of a binary map according to an embodiment of the present disclosure.

[0028] Figure 4c is a schematic diagram of calculating a centroid coordinate according to an embodiment of the present disclosure.

[0029] Figure 5 is a schematic diagram of mean filtering according to an embodiment of the present disclosure.

[0030] Figure 6 is a schematic block diagram of an apparatus for triggering of gesture recognition functionality according to an embodiment of the present disclosure.

[0031] Figure 7 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0032] The technical solutions in the embodiments of the present disclosure will be described clearly and completely below with reference to the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present disclosure.

[0033] The terms used in the embodiments of the present disclosure are merely for the purpose of describing particular embodiments, and are not intended to limit the embodiments of the present disclosure. The singular forms "a", "an" and "the" used in the embodiments of the present disclosure and the appended claims are intended to include plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0034] It should be understood that, although the terms first, second, third, etc. can be used herein to describe various information, the information should not be limited to these terms. These terms are only used to differentiate one piece of information from another piece of information of the same type. For example, without departing from the scope of embodiments of the present disclosure, first information can also be referred to as second information, and similarly, second information can also be referred to as first information. Depending on the context, the word "if" as used herein can be interpreted as "when" or "upon" or "in response to determining".

[0035] For the purpose of brevity and clarity, the terms "greater than" or "less than", "higher than" or "lower than" are used herein to characterize the size relationship. However, it can be understood by those skilled in the art that the term "greater than" also covers the meaning of "greater than or equal to", and the term "less than" also covers the meaning of "less than or equal to"; the term "higher than" covers the meaning of "higher than or equal to", and the term "lower than" covers the meaning of "lower than or equal to".

[0036] Gesture recognition refers to a process of recognizing the movement of each part of the human body by using an electronic device, generally aiming at the movement of the face or the hand of a person.

[0037] In the related art, there are two ways to implement gesture recognition: one is based on computer vision technology, which captures the image of an object through the camera equipped on the electronic device, and recognizes the features in the image to implement gesture recognition; the other is based on depth data, which collects the depth data of an object through the depth sensor equipped on the electronic device, and recognizes the depth data to implement gesture recognition. However, when implementing gesture recognition by using the above two ways, the electronic device will continuously recognize the collected image or depth data, which increases the cost of gesture recognition.

[0038] To solve the problems in the related art, the present disclosure proposes a method for triggering gesture recognition function.

[0039] Figure 1 is a schematic architecture diagram of triggering gesture recognition function according to an embodiment of the present disclosure. As shown in Figure 1 The electronic device 10 is equipped with a depth sensor 30, which can collect the depth data of an object 20 within the field of view. The electronic device 10 is configured with processing logic for implementing the technical solutions of the present disclosure, which can generate a depth map based on the collected depth data according to a generation function, and trigger to start gesture recognition function based on the depth map in the case that the object 20 meets the triggering condition. The specific implementation process of the processing logic will be described in detail below in combination with Figures 2-5 .

[0040] Figure 2This is a schematic flowchart illustrating a method for triggering a gesture recognition function according to an embodiment of this disclosure. The method for triggering the gesture recognition function shown in this embodiment can be executed by an electronic device equipped with a depth sensor, including but not limited to communication devices such as mobile phones, tablets, wearable devices, and IoT devices. Figure 2 As shown, the method for triggering the gesture recognition function may include the following steps:

[0041] In step S202, a depth map is generated based on the depth data collected by the depth sensor.

[0042] Depth sensors can refer to sensors used to collect depth data, such as Time-of-Flight (ToF) sensors, structured light sensors, and binocular sensors; this disclosure does not limit the scope of the term. Taking a ToF sensor as an example, it can emit infrared light, laser light, or other light rays into its field of view. When the light rays are reflected back from the object to the sensor, the distance between the sensor and the object being measured can be calculated based on the time difference between the emission and reflection of the light rays.

[0043] A depth map is an image generated from depth data, where the pixel value of each pixel represents the distance between a point within the field of view and the depth sensor. If an object exists within the field of view of the depth sensor, the pixel values ​​on the depth map can reflect the distances between various points of that object and the depth sensor.

[0044] In step S204, if a target object that meets the triggering conditions is determined based on the depth map, the electronic device is triggered to start the gesture recognition function; wherein, the triggering conditions include: being in a preset area within the field of view of the depth sensor and being in that area for a preset duration.

[0045] The field of view of a depth sensor refers to the maximum range of depth data that the depth sensor can acquire. A preset area refers to a region within a pre-defined field of view, such as a cubic area 15-20 cm away from the depth sensor.

[0046] The trigger condition includes: a preset region in the field of view range of the depth sensor and a duration of the target object staying in the region reaching a preset duration. This requires the target object to stay in the preset region, and the duration needs to reach the preset duration. In the case of a preset duration of 5 seconds, if an object stays in the preset region for 2 seconds, leaves the preset region halfway, and then returns to the preset region and continues to stay for 3 seconds, the object does not stay in the preset region continuously, and the duration is only 3 seconds, so the object cannot be regarded as the target object, and the trigger condition is not met; if the object continues to stay in the preset region for 2 seconds, the duration of the object in the preset region will reach 5 seconds, and the object can be regarded as the target object, and the trigger condition is met.

[0047] The electronic device can calculate the duration of the target object in the preset region according to the number of frames of the depth map generated by the depth sensor. Assuming that the depth sensor can generate 10 frames of images per second, then every 10 frames of images, the duration is added by 1 second. If the target object is in the preset region for 50 consecutive frames of images, it means that the duration of the target object in the preset region reaches 5 seconds.

[0048] Based on the depth map, the position of the object in the field of view range can be determined. In the case that the object is in the preset region in the field of view range of the depth sensor and the duration of the object staying in the region reaches the preset duration, the object meets the trigger condition, and the electronic device can start the gesture recognition function for gesture recognition. The gesture recognition function can be implemented based on computer vision technology or based on depth data, and the present disclosure does not limit this.

[0049] In this embodiment, on the one hand, a trigger method is set for gesture recognition, a depth map is generated according to the depth data collected by the depth sensor installed on the electronic device, and in the case that a target object meeting the trigger condition is determined based on the depth map, the electronic device is triggered to start the gesture recognition function, so that the gesture recognition function can only be triggered under certain conditions, thereby avoiding the electronic device from continuously starting the gesture recognition function and reducing the cost of gesture recognition; on the other hand, by setting the trigger condition as a preset region in the field of view range of the depth sensor and a duration of the target object staying in the region reaching a preset duration, the triggering of the gesture function is not too difficult or too easy.

[0050] In addition to the trigger condition mentioned above, the present disclosure also proposes an additional trigger condition to improve the accuracy of trigger determination.

[0051] In an embodiment, the method further includes: determining a moving distance of the target object in the preset duration according to the motion information of the target object; and wherein the trigger condition further includes: the moving distance of the target object in the preset duration does not exceed a preset threshold.

[0052] Since the preset area is a defined region, the target object can move a certain distance even if it is within that region. The electronic device can use the pixel values ​​on the generated depth map as motion information for the target object, thereby determining the distance the target object moves within a preset time period.

[0053] like Figure 3a As shown, sensor 301 is a depth sensor, and region 302 is a preset region. Within a preset time period, object 304 moves 3 centimeters from its original position 303 towards sensor 301. That is, object 304, initially 13 centimeters away from sensor 301, moves to a position 10 centimeters away from sensor 301 after the preset time period. The corresponding changes on the depth map are as follows. Figure 3b As shown, depth map 305 is a depth map generated by sensor 301 based on the collected depth data. The pixel value of pixel 306 represents the distance between object 304 and sensor 301. Within a preset time period, the pixel value of pixel 306 changes from 13 to 10, so the moving distance of the target object within the preset time period can be calculated to be 3 centimeters. If the preset threshold is less than 3 centimeters, object 304 does not meet the triggering condition; if the preset threshold is greater than or equal to 3 centimeters, object 304 meets the triggering condition.

[0054] Of course, the movement of the target object within the preset area can also be repetitive. Figure 3a For example, object 304 can first translate 3 cm from position 303 towards sensor 301 within a preset time period, and then translate back to position 303. At this time, the corresponding changes on the depth map are as follows: Figure 3c As shown, the pixel value of pixel 306 changes from 13 to 10 and then back to 13, from which the translation distance of object 304 is calculated to be 6 centimeters.

[0055] In this embodiment, by further limiting the movement distance of the target object based on the original triggering conditions, the accidental activation of the gesture recognition function is avoided when the object moves a small distance against the user's will, thereby improving the accuracy of the triggering determination of the gesture recognition function.

[0056] In one embodiment, the method further includes: determining the number of frames lost by the depth sensor within a preset time period based on the depth data collected by the depth sensor; wherein the triggering condition further includes: the number of frames lost by the depth sensor within the preset time period does not exceed a preset number.

[0057] Frame loss can refer to frame missing, i.e. the electronic device does not receive the depth data frame sent by the depth sensor, and the frame loss number is the number of frame loss of the depth sensor within a preset time length. In the case that the frame loss number within the preset time length is too large, it indicates that the depth sensor is malfunctioning or the data transmission link is malfunctioning, at this time, if the determination of the triggering of the gesture recognition function is continued, it can lead to determination error. Therefore, the frame loss number needs to be added to the triggering condition.

[0058] The preset number can be set in time units, such as 2 frames per second. In the case that the preset time length is 10 seconds, if the frame loss number within the preset time length is 10 frames, then the frame loss number per second is 1 frame, which is less than the preset 2 frames per second, which meets the triggering condition; if the frame loss number within the preset time length is 30 frames, then the frame loss number per second is 3 frames, which exceeds the preset 2 frames per second, which does not meet the triggering condition.

[0059] In this embodiment, by adding the frame loss number to the triggering condition, the influence of the depth sensor malfunction and the data transmission malfunction on the triggering determination is excluded, thereby improving the accuracy of the triggering determination of the gesture recognition function.

[0060] In an embodiment, the method further comprises: determining the position of the centroid of the target object; and determining that the target object meets the triggering condition in the case that the centroid meets the triggering condition.

[0061] The centroid refers to the center of mass, which refers to a hypothetical point on a material system where the mass is considered to be concentrated. Since the volume of the target object cannot be determined, using the centroid position of the target object instead of the position of the target object can improve the accuracy of the triggering condition determination.

[0062] Further, the determination of the position of the centroid of the target object comprises: taking the connected region with the largest area in the depth map as the target connected region corresponding to the target object; converting the depth map into a binary map and determining the centroid of the binary map as the centroid of the target object; wherein the pixel value of the target connected region in the binary map is a non-0 uniform value, and the pixel value of the remaining region is 0; establishing a coordinate system based on the binary map, and taking the coordinates of the centroid in the coordinate system as the position of the centroid of the target object.

[0063] In the depth map, the image region composed of pixel points with non-0 pixel value and adjacent position is called a connected region. Generally, the connected region with the largest area in the depth map corresponds to the target object, so this connected region can be taken as the target connected region. As shown in FIG. 1, in this depth map, the palm region composed of multiple adjacent pixels with non-0 pixel value is the connected region with the largest area, so this region is determined as the largest connected region. Figure 4a ​

[0064] In a binary image, each pixel has only two values: 0 or a single non-zero value (such as 1 or 2). When the pixel value corresponding to the target object is a single value, the centroid of the target object better reflects its position. For example... Figure 4b As shown, it can be Figure 4a The pixel value corresponding to the middle palm area is set to 1, and the pixel value of the corresponding pixels in the other areas is set to 0 to obtain a binary image. Figure 4b A coordinate system is established based on the binary image shown, as follows: Figure 4c As shown, a coordinate system is established with the lower left corner of the binary graph as the origin.

[0065] The specific formula for calculating the centroid coordinates can be:

[0066]

[0067] Where n is the length of the binary image on the x-axis (number of pixels), and m is the length of the binary image on the y-axis (number of pixels).

[0068] like Figure 4c As shown, the binary graph has a length of 8 on both the x-axis and y-axis. Therefore, the formula for calculating the centroid coordinates is:

[0069]

[0070] according to Figure 4c We can calculate that cx is (3+5+7+8+8+5+2) / 8 = 4.75, cy is (3+5+7+8+8+5+2) / 8 = 19.5, and the centroid coordinates are (4.75, 4.75).

[0071] In this embodiment, by converting the depth map into a binary map and establishing a coordinate system based on the binary map, the electronic device can regard the calculated centroid coordinates as the position of the target object, thereby determining whether the target object meets the triggering conditions.

[0072] In one embodiment, before converting the depth map into a binary map, the method further includes: determining a minimum acquisition distance based on the depth map, the minimum acquisition distance being the minimum pixel value recorded on the depth map; and performing a filtering process on the depth map, the filtering process including: filtering out depth data corresponding to pixels whose acquisition distances are not within a preset distance range; wherein the lower boundary of the preset distance range is the minimum acquisition distance and the upper boundary is the sum of the minimum acquisition distance and the estimated thickness for the target object.

[0073] Generally, electronic devices can set an estimated thickness for the target object. For example, if the target object is a hand, the estimated thickness could be 5 centimeters. Since the target object is within a preset distance range, filtering out depth data corresponding to pixels whose acquisition distance is outside this range can eliminate depth data that does not correspond to the target object, thereby improving the accuracy of trigger detection.

[0074] In this embodiment, by setting an estimated thickness for the target object and filtering out the depth data corresponding to pixels whose acquisition distance is not within the preset distance range, depth data that does not correspond to the target object can be filtered out, thereby improving the accuracy of triggering judgment.

[0075] In one embodiment, the method further includes: performing mean filtering on the filtered depth map; and re-filtering the mean-filtered depth map.

[0076] Mean filtering is a typical linear filtering algorithm that replaces the current pixel's value with the average of the values ​​of its n×n surrounding pixels. By iterating through each pixel in the image, this method can perform mean filtering on the entire image. Mean filtering can remove noise from an image, making it clearer.

[0077] like Figure 5 As shown, mean filtering can be applied to the pixel values ​​of the pixels corresponding to the black box. Let's assume we take the mean of the pixel values ​​of the surrounding 5×5 pixels (i.e., the pixels corresponding to the gray box). The calculation method is as follows:

[0078] [(197+25+106+156+159)+(149+40+107+5+71)+(163+198+226+223+156)+(222+37+68+193+157)+(42+72+250+41+75)] / 25=126

[0079] After calculating the new value, this new value can be used as the pixel value after mean filtering for the current pixel. For Figure 5 In the depth map shown, the mean pixel value of each pixel within its surrounding 5x5 region is calculated and used as the new value for the current pixel, thus obtaining the mean filtering result of the current image. (For edge pixels, only the mean pixel value of the surrounding neighborhood points within the image can be taken.)

[0080] In this embodiment, by applying mean filtering to the filtered depth map, the electronic device can re-process the mean-filtered depth map, thereby further improving the accuracy of the trigger determination of the gesture recognition function.

[0081] Corresponding to the foregoing embodiment of the method for triggering the gesture recognition function, the disclosure also provides an embodiment of a device for triggering the gesture recognition function.

[0082] Figure 6 is a schematic block diagram of a device for triggering a gesture recognition function according to an embodiment of the disclosure. As shown in the figure, the device is applied to an electronic device equipped with a depth sensor, and the device comprises: Figure 6

[0083] a generating unit 602 configured to generate a depth map according to depth data collected by the depth sensor;

[0084] a triggering unit 604 configured to trigger the electronic device to start the gesture recognition function in a case where it is determined based on the depth map that there is a target object meeting a triggering condition; wherein the triggering condition comprises: a preset region within a field of view range of the depth sensor and a duration of being located in the region reaching a preset duration.

[0085] Optionally, the method further comprises:

[0086] a first determining unit 606 configured to determine a moving distance of the target object within the preset duration according to motion information of the target object;

[0087] wherein the triggering condition further comprises: the moving distance of the target object within the preset duration not exceeding a preset threshold.

[0088] Optionally, the method further comprises:

[0089] a second determining unit 608 configured to determine a frame loss number of the depth sensor within the preset duration according to the depth data collected by the depth sensor;

[0090] wherein the triggering condition further comprises: the frame loss number of the depth sensor within the preset duration not exceeding a preset number.

[0091] Optionally, the method further comprises:

[0092] a third determining unit 610 configured to determine a position of a center of mass of the target object;

[0093] a judging unit 612 configured to judge that the target object meets the triggering condition in a case where the center of mass meets the triggering condition.

[0094] Optionally, the third determining unit 610 is specifically configured to:

[0095] take a connected region with a largest area in the depth map as a target connected region corresponding to the target object; ​

[0096] convert the depth map into a binary map, and determine a centroid of the binary map as the centroid of the target object; wherein a pixel value of the target connected region in the binary map is a uniform value other than 0, and a pixel value of a remaining region is 0;

[0097] establish a coordinate system based on the binary map, and take a coordinate of the centroid in the coordinate system as a position of the centroid of the target object.

[0098] Optionally, before converting the depth map into a binary map, the method further comprises:

[0099] a fourth determining unit 614, configured to determine a minimum acquisition distance according to the depth map, the minimum acquisition distance being a minimum pixel value recorded on the depth map;

[0100] a first filtering unit 616, configured to perform filtering processing on the depth map, the filtering processing comprising: filtering out depth data corresponding to a pixel point whose acquisition distance is not within a preset distance range; wherein a lower boundary of the preset distance range is the minimum acquisition distance and an upper boundary of the preset distance range is a sum of the minimum acquisition distance and an estimated thickness of the target object.

[0101] Optionally, the method further comprises:

[0102] a filtering unit 618, configured to perform mean filtering on the filtered depth map;

[0103] a second filtering unit 620, configured to perform the filtering processing again on the mean filtered depth map.

[0104] As to the apparatus in the above-mentioned embodiments, the specific manners in which various modules perform operations have been described in details in the embodiments of the method, and will not be described in details here.

[0105] For the apparatus embodiments, since they basically correspond to the method embodiments, the relevant parts refer to the part of the description of the method embodiments. The apparatus embodiments described above are merely illustrative, wherein the modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, i.e., they can be located in one place or distributed on multiple network modules. According to actual needs, some or all of the modules can be selected to achieve the purposes of the present disclosure. Those skilled in the art can understand and implement without creative labor.

[0106] Embodiments of the present disclosure also propose an electronic device, comprising:

[0107] a processor;

[0108] a memory for storing processor-executable instructions;

[0109] The processor is configured to implement the method according to any one of the above embodiments.

[0110] The processor is configured to implement the method according to any one of the above embodiments.

[0111] With reference to Figure 7 The apparatus 700 can include one or more of the following components: a processing component 702, a memory 704, a power supply component 706, a multimedia component 708, an audio component 710, an input / output (I / O) interface 712, a sensor component 714 and a communication component 716.

[0112] The processing component 702 usually controls overall operations of the apparatus 700, such as operations associated with displaying, making phone calls, data communications, camera operations and recording operations. The processing component 702 can include one or more processors 720 to execute instructions to complete all or part of steps of the methods described above. In addition, the processing component 702 can include one or more modules to facilitate interaction between the processing component 702 and other components. For example, the processing component 702 can include a multimedia module to facilitate the interaction between the multimedia component 708 and the processing component 702.

[0113] The memory 704 is configured to store various types of data to support operations of the apparatus 700. Examples of these data include instructions for any application or method operating on the apparatus 700, contact data, phonebook data, messages, pictures, videos and so on. The memory 704 can be implemented by any type of volatile or non-volatile storage devices or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0114] The power supply component 706 supplies electrical power for various components of the apparatus 700. The power supply component 706 can include a power supply management system, one or more power supplies, and other components associated with generating, managing and distributing electrical power for the apparatus 700.

[0115] The multimedia component 708 includes a screen providing an output interface between the device 700 and a user. In some embodiments, the screen can include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors for sensing a touch, a slide and a gesture on the touch panel. The touch sensor can not only sense a boundary of a touching or a sliding action, but also detect duration and pressure related to the touching or sliding action. In some embodiments, the multimedia component 708 includes a front camera and / or a rear camera. When the device 700 is in an operation mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each of the front and rear camera can be a fixed optical lens system or have a focal length and optical zooming capability.

[0116] The audio component 710 is configured to output and / or input an audio signal. For example, the audio component 710 includes a microphone (MIC) configured to receive an external audio signal when the device 700 is in an operation mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 704 or transmitted via the communication component 716. In some embodiments, the audio component 710 also includes a speaker for outputting an audio signal.

[0117] The I / O interface 712 provides an interface between the processing component 702 and peripheral interface modules, which can be a keypad, a click wheel, buttons, and the like. The buttons can include, but are not limited to, a home button, a volume button, a start button, and a lock button.

[0118] The sensor component 714 includes one or more sensors for providing status assessments of various aspects of the device 700. For example, the sensor component 714 can detect an open / closed position of the device 700, relative positioning of components, such as a display and a keypad of the device 700, a change in position of the device 700 or a component of the device 700, the presence or absence of user contact with the device 700, the orientation or acceleration / deceleration of the device 700, and a temperature change of the device 700, among a plethora of other examples. The sensor component 714 can include a proximity sensor configured to detect presence of a nearby object without any physical touch. The sensor component 714 can also include a light sensor (e.g., a CMOS or CCD image sensor) configured to work in conjunction with the camera component 706. In some embodiments, the sensor component 714 can further include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0119] The communication component 716 is configured to facilitate wired or wireless communication between the device 700 and other devices. The device 700 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, 4G LTE, 5G NR, or a combination thereof. In an exemplary embodiment, the communication component 716 receives broadcast signals or broadcast-related information from external broadcast management systems via a broadcast channel. In an exemplary embodiment, the communication component 716 further includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) techniques, infrared data association (IrDA) techniques, ultra-wideband (UWB) techniques, Bluetooth (BT) techniques, and other techniques.

[0120] In an exemplary embodiment, the device 700 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors, or other electronic elements, for performing the methods described in any one of the above embodiments.

[0121] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions, such as the memory 704 including instructions, is also provided, which can be executed by the processor 720 of the device 700 to complete the above methods. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disc, and an optical data storage device, etc.

[0122] Other embodiments of the present disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the features disclosed herein. It is intended that the present disclosure cover any and all variations of the present disclosure including those variations that can be incorporated into the above detailed description and making use of the general principles of the present disclosure. It is intended that the present disclosure include all such as fall within the scope of the appended claims and their equivalents. The specification and examples given herein are to be considered exemplary only, with the true scope and spirit of the disclosure being indicated by the following claims.

[0123] It is to be understood that the present disclosure is not limited to the precise construction described above and shown in the attached drawings, and that various modifications and changes can be effected therein by those skilled in the art without departing from the scope of the application. The scope of the present disclosure is to be limited only by the appended claims.

Claims

1. A method for triggering a gesture recognition function, characterized in that, The method, applied to an electronic device equipped with a depth sensor, includes: A depth map is generated based on the depth data collected by the depth sensor; Determining the location of the centroid of a target object includes: taking the connected region with the largest area in the depth map as the target connected region corresponding to the target object; converting the depth map into a binary map and determining the centroid of the binary map as the centroid of the target object; wherein, the pixel values ​​corresponding to the target connected region in the binary map are uniform non-zero values, and the pixel values ​​of the remaining regions are 0; establishing a coordinate system based on the binary map, and taking the coordinates of the centroid in the coordinate system as the location of the centroid of the target object; If the centroid satisfies the triggering condition, the target object is determined to meet the triggering condition. If a target object that meets the triggering conditions is determined based on the depth map, the electronic device is triggered to start the gesture recognition function; wherein, the triggering conditions include: being in a preset area within the field of view of the depth sensor and being in that area for a preset duration.

2. The method according to claim 1, characterized in that, The method further includes: Based on the motion information of the target object, determine the distance the target object moves within the preset time period; The triggering condition further includes: the target object's movement distance does not exceed a preset threshold within the preset time period.

3. The method according to claim 1, characterized in that, The method further includes: Based on the depth data collected by the depth sensor, determine the number of frames lost by the depth sensor within the preset time period; The triggering condition further includes: the number of frames lost by the depth sensor within the preset time period does not exceed a preset number.

4. The method according to claim 1, characterized in that, Before converting the depth map into a binary map, the method further includes: The minimum acquisition distance is determined based on the depth map, where the minimum acquisition distance is the minimum pixel value recorded on the depth map; The depth map is filtered, and the filtering process includes: filtering out the depth data corresponding to pixels whose acquisition distance is not within a preset distance range; wherein, the lower boundary of the preset distance range is the minimum acquisition distance and the upper boundary is the sum of the minimum acquisition distance and the estimated thickness for the target object.

5. The method according to claim 4, characterized in that, The method further includes: The filtered depth map is then subjected to mean filtering; The depth map after mean filtering is then subjected to the same filtering process again.

6. A triggering device for gesture recognition function, characterized in that, An electronic device equipped with a depth sensor, the device comprising: Generation unit: Generates a depth map based on the depth data collected by the depth sensor; The third determining unit determines the position of the centroid of the target object, including: taking the connected region with the largest area in the depth map as the target connected region corresponding to the target object; converting the depth map into a binary map and determining the centroid of the binary map as the centroid of the target object; wherein, the pixel values ​​corresponding to the target connected region in the binary map are uniform non-zero values, and the pixel values ​​of the other regions are 0; establishing a coordinate system based on the binary map, and taking the coordinates of the centroid in the coordinate system as the position of the centroid of the target object; The determination unit determines that the target object meets the triggering condition if the centroid meets the triggering condition. Triggering unit: When a target object that meets the triggering conditions is determined to exist based on the depth map, the electronic device is triggered to start the gesture recognition function; wherein, the triggering conditions include: being in a preset area within the field of view of the depth sensor and being in that area for a duration that reaches a preset duration.

7. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to implement the method of any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method described in any one of claims 1 to 5.

Images