A human body intelligent sensing device and gesture recognition method thereof

Abstract

The present invention proposes a human intelligent sensing device and a gesture recognition method thereof, which belongs to the technical field of gesture recognition of intelligent devices. First, the transmission power of the infrared signal transmitter in the device is adjusted so that the waveform curves on both sides received by the infrared receiver at the same time have a small difference; then, the influence of the infrared signal in the environment and the influence caused by human clicking on the screen are filtered out to determine whether the gesture is a valid gesture; finally, according to the order in which the peak values ​​of the waveforms on both sides arrive, it is determined whether the gesture is a left swipe or a right swipe. The present invention adjusts the current of the infrared transmitter to avoid the problem that the waveforms on both sides have a large difference and the gesture cannot be recognized normally due to the different distances between the left and right transmitters and the receiver; and filters out the interference data of the environment and the human operation of the screen according to different judgment methods to avoid gesture misjudgment, causing the device controlled by the device to be frequently triggered, affecting the control accuracy of the device, and reducing the life of the device.

Description

Technical Field

[0001] The invention relates to a human body intelligent sensing device and a gesture recognition method thereof, belonging to the technical field of intelligent device gesture recognition. Background Art

[0002] Gesture recognition of the human body intelligent sensing device is achieved by setting multiple infrared transmitters and an infrared receiver on the device. The receiver is set in the middle of each transmitter, and determines whether the gesture is a left swipe or a right swipe according to the order of arrival of the waveform peaks on both sides received by the receiver. With the further development of intelligent control, human body intelligent sensing devices begin to have an operable screen. Due to the limitations of the manufacturing process of the operable screen, the receiver and the transmitter can only be set on both sides of the screen. For a one-receiver-multiple-transmitter device, the distance between the infrared transmitter and the infrared receiver on the left and right sides cannot be equal. For example, for a one-receiver-two-transmitter device, one transmitter is set on one side of the screen, and the other transmitter and receiver are set on the other side of the screen. In this case, the receiver and one transmitter are very close, and the receiver and the other transmitter are very far away. In this way, the peak waveforms of the two transmitters received by the receiver are quite different, which may lead to misjudgment of gestures.

[0003] At the same time, when receiving data, the receiver will receive signals from the external environment. If these external environmental interference signals are relatively large, it will also cause misjudgment of gesture recognition, that is, the external environmental interference signals will be recognized as gesture signals, which will mistakenly trigger the device controlled by the device, affecting the control accuracy and even the service life of the device; in addition, for devices with operable screens, when people operate the screen, for example, sliding or clicking the screen, because the hand is very close to the receiver, these screen operations will cause great interference to the waveform received by the receiver, which will cause misjudgment of gesture recognition, that is, the signals of human operation of the screen will be recognized as gesture signals, causing the device controlled by the device to be frequently mistakenly triggered, affecting the normal use of the device and its control accuracy, and even affecting the service life of the device, while also affecting the normal operation of the user and the user experience. Summary of the invention

[0004] The purpose of the present invention is to provide a human body intelligent sensing device and a gesture recognition method thereof, so as to solve the problem of gesture misjudgment caused by external environment interference and human operation of the screen.

[0005] The present invention provides a gesture recognition method for a human body intelligent sensing device, the method comprising the following steps:

[0006] 1) Adjust the infrared transmitter current so that the peak difference of the waveforms on both sides received by the infrared receiver is within the set range;

[0007] 2) judging whether there is environmental interference based on the magnitude of the peak value of the waveforms on both sides and the entry threshold and / or the magnitude of the time taken by the waveform from the entry threshold to the peak value and the set time threshold, and filtering out the waveforms generated by the environmental interference; wherein the entry threshold refers to the minimum value of the waveform generated when a person performs gesture control;

[0008] 3) For the waveforms after filtering out environmental interference, determine whether there is interference from human-operated screens based on the difference in width from the entry threshold to the peak value of the waveforms on both sides and / or whether the waveform on one side is greater than the entry threshold when the waveform on the other side reaches the peak value, and filter out the waveforms generated by human-operated screens;

[0009] 4) For the waveform after filtering out interference, the gesture direction is determined by the order in which the peak values ​​of the waveform curves on both sides are reached; when the left waveform curve reaches the peak value before the right waveform curve, it is determined to be a right swipe gesture; when the right waveform curve reaches the peak value before the left waveform curve, it is determined to be a left swipe gesture.

[0010] The present invention firstly adjusts the transmission power of the infrared transmitter by adjusting the current of the infrared transmitter, so as to avoid the problem that the waveforms on the left and right sides are greatly different due to the different distances between the transmitters and the receiver, and the gesture cannot be recognized normally; then, according to different judgment methods, the interference data of the environmental interference and the human operation screen are filtered out to improve the accuracy of gesture recognition, so as to avoid gesture misjudgment, resulting in the frequent false triggering of the equipment controlled by the human intelligent sensing device (such as smart home equipment), affecting the control accuracy of the equipment, and reducing the life of the equipment.

[0011] Furthermore, before environmental interference filtering is performed, the infrared receiver sampling data is preprocessed, and the preprocessing process includes: when the sampling value on either side received by the receiver reaches the entry threshold, the sampling sequence counting is started, and when all the sampling values ​​are less than or equal to the exit threshold, the environmental interference filtering is entered; wherein the exit threshold is equal to the entry threshold.

[0012] By using the above method, the data that is obviously not a gesture is filtered out from the collected data, and there is no need to judge it, thereby avoiding the problem that the subsequent calculation amount will be large due to the excessive amount of real-time collected data.

[0013] Furthermore, the step 2) implements environmental interference filtering by determining whether the waveform peak is less than or equal to a set multiple of the entry threshold. When the waveform peak is less than or equal to a set multiple of the entry threshold, the waveform is considered to be environmental interference and is filtered out. The set multiple is greater than 1.2 times.

[0014] Since the peak value of a normal gesture will definitely be much larger than the entry threshold, and the waveform smaller than the entry threshold is definitely not a normal gesture waveform, some data that does not meet the peak value of the normal gesture waveform can be filtered out through the above method, further reducing the amount of subsequent calculations, while avoiding misjudgment and frequent triggering of the device, affecting the control accuracy of the device, and even reducing the life of the device.

[0015] Furthermore, in step 2), when the time taken by the waveform from the entry threshold to the peak value is not within the set time threshold, the waveform is considered to be environmental interference and is filtered out.

[0016] Furthermore, the time threshold is set to [6ms, 300ms].

[0017] Since normal gesture operations are neither too fast nor too slow, the above method can filter out some non-gesture interference data based on this feature, while avoiding the problem of gesture misjudgment and frequent device triggering, which affects the control accuracy of the device and reduces the life of the device.

[0018] Furthermore, in the step 3), when the difference in width from the entry threshold to the peak value of the waveforms on both sides is greater than or equal to half of the maximum width, the waveform is considered to be interference caused by human operation of the screen and is filtered out; the maximum width refers to the longest time from the entry threshold to the peak value of the waveforms on both sides.

[0019] In actual operation, an effective gesture spans the entire operation screen, and the width of the waveforms on both sides from the entry threshold to the peak value should not differ too much. The above method is used to filter out the interference data generated when the screen is manually operated to avoid misjudgment of gestures, which causes the device to be triggered, affects the normal use of the device and its control accuracy, and also affects the normal operation of the screen by people, thereby affecting the user experience.

[0020] Furthermore, the entry threshold is calibrated by a test curve obtained by a normal operating gesture of a human body at a set distance from the device.

[0021] A large number of test curves are obtained through the experiment described in the above manner to obtain a more accurate entry threshold, thereby avoiding the problem of setting the threshold too low and causing a large amount of subsequent calculations.

[0022] Furthermore, in step 2), when using a set time threshold to judge environmental interference, the set time threshold is converted into a set number of interval points, and the set number of interval points and the number of interval points between the entry threshold and the peak of the waveform are used to judge whether there is environmental interference. When the number of interval points between the entry threshold and the peak of the waveform is not within the set interval point range, the waveform is considered to be environmental interference and is filtered out; wherein the set number of interval points is determined based on the sampling frequency of the infrared receiver and the set time threshold.

[0023] By converting the set time threshold into the set interval points, the environmental interference waveform can also be filtered out to avoid misjudgment of gestures.

[0024] The present invention also proposes a human intelligent sensing device, which includes an infrared signal transmitter, an infrared signal receiver and a signal processor, wherein the signal processor is connected to the infrared signal receiver, at least one infrared signal transmitter is installed on one side of the device, and another infrared signal transmitter and infrared signal receiver are installed on the other side of the device, and the signal processor adopts the gesture recognition method of the human intelligent sensing device as described above.

[0025] The human body intelligent sensing device proposed in the present invention filters out data of abnormal operation gestures in the collected signals through a signal processor, and judges whether the gesture is swiping left or right, so as to solve the problem of gesture misjudgment caused by external environmental interference and human operation of the screen. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a specific flow chart of gesture recognition of the human body intelligent sensing device of the present invention;

[0027] Figure 2 is a waveform curve diagram of a non-gesture signal in an environment in an embodiment of the present invention;

[0028] Figure 3 is a waveform curve diagram when the screen is manually operated in an embodiment of the present invention;

[0029] Figure 4 is a waveform curve diagram of a normal right swipe gesture in an embodiment of the present invention;

[0030] Figure 5 It is a waveform curve diagram of a normal left swipe gesture in an embodiment of the present invention. DETAILED DESCRIPTION

[0031] The specific implementation of the present invention will be further described below in conjunction with the accompanying drawings.

[0032] The present invention is mainly directed to a human intelligent sensing device with an operable screen. Such an operating screen is generally made of glass. In the existing sensing device without an operable screen, the installed infrared signal receiver is generally located at the center of multiple transmitters, that is, the distance between each infrared signal transmitter and the receiver is equal. However, for a human intelligent sensing device with an operable glass screen, due to the limitation of the screen on the installation structure, the infrared signal receiver cannot be installed in the middle of the screen, resulting in different distances between different infrared signal transmitters and the infrared signal receiver, which makes the generated waveform curves differ greatly and cannot accurately identify gestures. For example, in this embodiment, an infrared signal transmitter is installed on the right side of the device, and another infrared signal transmitter and infrared signal receiver are installed on the left side of the device, so that one transmitter is closer to the receiver and the other transmitter is farther from the receiver, which causes the waveform curves on the left and right sides received by the receiver to have a large difference, affecting the accurate judgment of the gesture direction. In addition, the infrared signals in the external environment and the signals generated by the person when operating the screen will affect the waveform received by the receiver, and will cause certain interference to the gesture judgment process. As other embodiments, in addition to one receiving and two transmitting, the present invention is also applicable to a human sensing device with one receiving and multiple transmitting.

[0033] Method Embodiment

[0034] The present invention proposes a gesture recognition method for a human body intelligent sensing device, the specific process is as follows Figure 1 First, adjust the transmission power of the infrared signal transmitter in the device so that the difference between the waveform curves on both sides received by the infrared receiver is small; then filter out the influence of the infrared signal in the environment and the influence caused by human clicking on the screen, and determine that this gesture is a valid gesture; finally, determine whether this gesture is a left swipe or a right swipe based on the order in which the peak values ​​of the waveforms on both sides arrive.

[0035] Step 1. Adjust the transmitter current

[0036] Since the distances between the infrared signal transmitter and the receiver are different, the generated waveform curves are quite different, and the direction of the gesture cannot be accurately identified. In this embodiment, due to the limitation of the device structure, one of the infrared signal transmitters is on the right side of the device, and the other infrared signal transmitter and the infrared signal receiver are on the left side of the device. Due to the different distances, when the transmission power (current magnitude) of the two infrared signal transmitters is the same, the overall intensity value of the curve closer to the infrared signal receiver (i.e., the left side) is much greater than the intensity value of the curve farther away (i.e., the right side), resulting in a large difference in the waveforms of the two transmitters. Gesture recognition is generally determined by the order in which the peak values ​​of the waveforms on both sides arrive. If the peak values ​​on both sides are very different, it may cause misjudgment of gestures.

[0037] To this end, the current of the infrared signal transmitter is adjusted to adjust the infrared signal emission intensity, reduce the waveform difference generated by the reflection data of different infrared signal transmitters sampled by the receiver, so that the period, wave width, and peak value of different waveforms are within the set threshold, and further improve the subsequent detection rate. Among them, the adjustment of the current size is determined based on the pre-simulation experiment. For example, the maximum difference between the peak values ​​of the two waveforms can be no more than 20%, or the maximum difference between the widths of the two waveforms can be no more than 20%.

[0038] Step 2. Preprocess the collected infrared signal

[0039] Since the infrared receiver collects signals in real time, the amount of data collected is relatively large. If all the collected data are judged, the subsequent calculation amount will also be very large. The time for gesture operation of the human body intelligent sensing device accounts for a relatively small proportion of the use time of the human body intelligent sensing device. Most of the collected data may not be gesture operation data. Therefore, it is necessary to screen the collected data and filter out the data that cannot be gesture operation.

[0040] The present invention is provided with an entry threshold and an exit threshold. When the sampling value of the signal transmitted by any infrared signal transmitter and received by the infrared signal receiver after reflection reaches the entry threshold, the sampling sequence counting starts. When all the sampling values ​​drop below the exit threshold, subsequent processing is performed. That is, when the sampling value does not reach the entry threshold, it cannot be a gesture operation, and subsequent processing is not performed on it, and it is directly discarded. Among them, the entry threshold is calibrated according to experiments, specifically, it is determined according to a large number of curves generated by multiple screen operations when a person is at a certain distance from the screen, wherein the calibration experiment is generally performed when the human body is 30 cm away from the screen; at the same time, the exit threshold is equal to the entry threshold.

[0041] Step 3. Filter out environmental interference

[0042] Taking into account the influence of infrared signals in the environment or signals generated by people passing in front of the screen on gesture judgment, the screened data needs to be filtered to remove environmental influences. Under normal circumstances, if it is a valid gesture, the peak value of the waveform curve received by the receiver is much larger than the entry threshold. If the peak value of the screened waveform is less than or equal to the set multiple of the entry threshold, it is considered that this is not a gesture, but interference from the environmental signal, where the minimum set multiple can be 1.2. In this embodiment, the set multiple can be set to 1.5. For example, the entry threshold is set to 100 (light intensity value), that is, when the waveform peak reaches at least 150, it may be a gesture, otherwise it is considered that this is not a gesture. Figure 2 The peak values ​​of the left and right waveforms are around 125, so it is determined that this is not a gesture, but may be caused by someone passing by.

[0043] At the same time, considering that the speed of the gesture operation process will not be too slow or too fast, the time taken by the waveform received by the corresponding receiver to reach the peak from the entry threshold will not be too short or too long, so a set time threshold is set, and whether this is a gesture is determined by judging whether the time taken by the generated waveform from reaching the entry threshold to reaching the peak meets the set time threshold. In this embodiment, the time threshold is set to [12ms, 240ms]. If the time from the entry threshold to the peak is greater than 12ms and less than 240ms, it is considered that this is a gesture; if the time from the entry threshold to the peak is less than 12ms, the sliding speed is too fast, or the time from the entry threshold to the peak is greater than 240ms, the sliding speed is too slow. These two situations do not follow the normal gesture sliding rules, and it is determined that these two situations are not gestures.

[0044] Since the infrared signal receiver collects data at a certain sampling frequency, the number of interval points from the waveform reaching the entry threshold to the peak value when the gesture is satisfied can be determined according to the set time threshold. For example, when the sampling frequency is 2ms, according to the set time threshold [12ms, 240ms], the number of interval points at this time is [6,120]. If the sampling frequency is 3ms, the number of interval points is [4,80]. Similarly, when the number of interval points from the waveform reaching the entry threshold to the peak value meets the set number of interval points at the corresponding sampling frequency, it is determined that this is a gesture. The set time threshold is also obtained through experimental measurement.

[0045] As another implementation, the time threshold may be set to [6ms, 300ms]. By setting the entry threshold and time threshold as above, it is possible to effectively determine whether a single waveform meets the characteristics of a gesture waveform, and filter out data that is obviously not a gesture waveform characteristic.

[0046] Step 4. Filter out human interference

[0047] When people operate the screen, their hands may slide up, down, left, right, or tap the screen, which will affect the waveform received by the receiver. Therefore, it is necessary to filter out the interference caused by these human operations on the screen.

[0048] Under normal circumstances, the operation of a single valid gesture spans the entire operation screen. The widths (time taken) of the waveforms received by the receiver on both sides from the entry threshold to the peak should not differ significantly. Therefore, by comparing the widths (time taken) of the waveforms on both sides from the entry threshold to the peak, it is set that the difference in the widths of the waveforms on both sides from the entry threshold to the peak should not exceed half of the maximum width, so as to filter out artificial interference. For example, if the width of the generated left waveform from the entry threshold to the peak is T1, and the width of the generated right waveform from the entry threshold to the peak is T2, when T2 < T1 / 2 or T1 < T2 / 2, it is considered that this gesture is artificial interference. For example, if the width of the left waveform from the entry threshold to the peak is 90 ms and the width of the right waveform from the entry threshold to the peak is 40 ms, then it is considered that there is artificial interference. It is very likely that when a person's hand operates on the screen, for example, when switching the interface, the hand slides from left to right, but the distance of this slide is not large and does not span the entire screen.

[0049] At the same time, during the process of artificially clicking on the screen, there may be a situation where only one side is clicked, which will result in a large difference in the intensity values of the waveforms received by the receiver on both sides. During the process of a single valid gesture operation, the hand will pass through or infinitely approach the transmitters on both the left and right sides. Under normal circumstances, the difference in the peak values of the curves on both sides is not particularly large. Therefore, it is set that when one waveform reaches the peak, the other waveform cannot be lower than the entry threshold, so as to filter out the interference caused by artificially clicking on the screen. For example, if the left waveform reaches the peak at 120 ms, and the intensity value of the right waveform at this time is less than the set entry threshold, it is determined that this gesture is artificial interference. As Figure 3 shown, where the left waveform curve reaches the peak, while the intensity value of the right waveform curve is lower than the set entry threshold. At this time, it may be that during the process of artificially clicking on the screen, the left side of the screen has been clicked all the time, making the person's hand closer to the infrared signal transmitter on the left, thus resulting in this situation.

[0050] As other implementation methods, step 3 and step 4 can be carried out simultaneously, or artificial interference can be filtered out first and then environmental interference can be filtered out.

[0051] Step 5. Gesture judgment

[0052] For the waveform curves on both sides after data preprocessing in step 2 and interference filtering in steps 3 and 4, determine whether the gesture is a left swipe or a right swipe. Determine whether it is a left swipe or a right swipe according to the order in which the peaks of the waveforms received by the receiver on both sides arrive. If the waveform curve on the left side reaches the peak before the waveform curve on the right side, it can be understood that the hand first approaches the transmitter on the left and then the transmitter on the right, then it is considered that the current gesture is a right swipe; conversely, if the waveform curve on the right side reaches the peak before the waveform curve on the left side, it can also be understood that the hand first approaches the transmitter on the right and then the transmitter on the left, then it is considered that the current gesture is a left swipe. As Figure 4 As shown, from Figure 4 It can be seen that the waveform curves on both sides meet the set conditions in the above steps 3 and 4. The left curve reaches the peak before the right curve. It is considered that the hand first approaches the transmitter on the left and then the transmitter on the right, and the gesture is judged to be a right swipe. Figure 5 As shown, from Figure 5 It can be seen that the waveform curves on both sides meet the set conditions in the above steps 3 and 4. The right curve reaches the peak before the left curve. It is considered that the hand approaches the right transmitter first and then the left transmitter, and the gesture is determined to be a left swipe.

[0053] Device Embodiment

[0054] The present invention also proposes a human body intelligent sensing device, which includes an infrared signal transmitter, an infrared signal receiver and a signal processor, wherein the signal processor is connected to the receiver. In this embodiment, an infrared signal transmitter is installed on the right side of the device, and another infrared signal transmitter and an infrared signal receiver are installed on the left side of the device; as other implementations, there may be multiple infrared transmitters, such as three or four, and in this case, the currents of the multiple transmitters may be adjusted so that the maximum difference in the peak value of the waveform of each receiver does not exceed 20%.

[0055] Among them, the infrared signal transmitter is used to transmit infrared signals, the infrared signal receiver is used to receive infrared signals from the external environment, and an internal integrated irLED chip is used to receive external infrared signals, including infrared signal interference signals and human body reflection signals, and the collected signals are transmitted to the signal processor; the signal processor is used to process the collected infrared signals and determine whether gesture recognition is triggered. When gesture recognition is triggered, the above-mentioned gesture recognition method is used to identify the gesture direction of the signal collected by the receiver. The specific implementation method of the method has been described in detail in the method example and will not be repeated here.

Claims

1. A gesture recognition method for a human body intelligent sensing device, characterized in that: The method comprises the following steps: 1) Adjust the infrared transmitter current so that the peak difference of the waveforms on both sides received by the infrared receiver is within the set range; 2) judging whether there is environmental interference based on the magnitude of the peak value of the waveforms on both sides and the entry threshold and / or the magnitude of the time taken by the waveform from the entry threshold to the peak value and the set time threshold, and filtering out the waveforms generated by the environmental interference; wherein the entry threshold refers to the minimum value of the waveform generated when a person performs gesture control; 3) For the waveforms after filtering out environmental interference, determine whether there is interference from human-operated screens based on the difference in width from the entry threshold to the peak value of the waveforms on both sides and / or whether the waveform on one side is greater than the entry threshold when the waveform on the other side reaches the peak value, and filter out the waveforms generated by human-operated screens; 4) For the waveform after filtering out interference, the gesture direction is determined by the order in which the peak values ​​of the waveform curves on both sides are reached; when the left waveform curve reaches the peak value before the right waveform curve, it is determined to be a right swipe gesture; when the right waveform curve reaches the peak value before the left waveform curve, it is determined to be a left swipe gesture.

2. The gesture recognition method of the human body intelligent sensing device according to claim 1, characterized in that: Before environmental interference filtering, the infrared receiver sampling data is preprocessed. The preprocessing process includes: when the sampling value on either side received by the receiver reaches the entry threshold, the sampling sequence counting starts, and when all sampling values ​​are less than or equal to the exit threshold, the environmental interference filtering begins; wherein the exit threshold is equal to the entry threshold.

3. The gesture recognition method of the human body intelligent sensing device according to claim 1, characterized in that: The step 2) filters out environmental interference by checking whether the waveform peak is less than or equal to a set multiple of the entry threshold. When the waveform peak is less than or equal to a set multiple of the entry threshold, the waveform is considered to be environmental interference and is filtered out. The set multiple is greater than 1.2 times.

4. The gesture recognition method of the human body intelligent sensing device according to claim 1, characterized in that: In the step 2), when the time taken by the waveform from the entry threshold to the peak value is not within the set time threshold, the waveform is considered to be environmental interference and is filtered out.

5. The gesture recognition method of the human body intelligent sensing device according to claim 4, characterized in that: Set the time threshold to [6ms, 300ms].

6. The gesture recognition method of the human body intelligent sensing device according to claim 1, characterized in that: In the step 3), when the difference in width from the entry threshold to the peak value of the waveforms on both sides is greater than or equal to half of the maximum width, the waveform is considered to be interference caused by human operation of the screen and is filtered out; the maximum width refers to the longest time from the entry threshold to the peak value of the waveforms on both sides.

7. The gesture recognition method of the human body intelligent sensing device according to claim 1 or 2, characterized in that: The entry threshold is calibrated by a test curve obtained by a normal operation gesture of a human body at a set distance from the human intelligent sensing device.

8. The gesture recognition method of the human body intelligent sensing device according to claim 1 or 4, characterized in that: In the step 2), when the set time threshold is used to judge environmental interference, the set time threshold is converted into a set number of interval points, and the set number of interval points and the number of interval points between the entry threshold and the peak value of the waveform are used to judge whether there is environmental interference; when the number of interval points between the entry threshold and the peak value of the waveform is not within the set interval point range, the waveform is considered to be environmental interference and is filtered out; wherein the set number of interval points is determined based on the sampling frequency of the infrared receiver and the set time threshold.

9. A human body intelligent sensing device, comprising an infrared signal transmitter, an infrared signal receiver and a signal processor, wherein the signal processor is connected to the infrared signal receiver, characterized in that: At least one infrared signal transmitter is installed on one side of the device, another infrared signal transmitter and an infrared signal receiver are installed on the other side of the device, and the signal processor adopts the gesture recognition method of the human intelligent sensing device as described in any one of claims 1-8.

Images