Control method, control device, human-computer interaction device and storage medium

Abstract

The present disclosure relates to a control method, a control device, a human-computer interaction device and a storage medium. The control method comprises: obtaining reflection signals received by each receiving source in a plurality of receiving sources of a radar module, wherein the reflection signals are radar signals emitted by a transmitting source of the radar module and reflected by a target to the plurality of receiving sources, and the plurality of receiving sources are arranged at intervals in a first direction and a second direction; determining a target distance, a target speed and a first angle according to the reflection signals received by each receiving source in the plurality of receiving sources, wherein the target is perpendicular to a line connecting any two receiving sources in the same direction, the perpendicular line has a first reference angle with one of the any two receiving sources and a second reference angle with the other receiving source, and the first angle is a difference between the first reference angle and the second reference angle; determining a gesture action of the target according to a plurality of groups of the target distance, the target speed and the first angle; and controlling the human-computer interaction device according to the gesture action to improve the user experience.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of human-computer interaction devices, and in particular to a control method, a readable storage medium, a control device, and a human-computer interaction device. BACKGROUND

[0002] In related technologies, with the rapid popularization of intelligent display terminals and XR technology (including: AR Augment Reality, VR virtual reality, and MR Mixed Reality), the intelligent display terminal and the XR device both need to use a special interaction device (a remote controller, a handle, etc.) to complete the control function. In the use process, the interaction device is easy to be lost or run out of power, and in the XR game process, the interaction device is easy to fall and be damaged, which reduces the user experience. SUMMARY

[0003] To overcome the problems in related technologies, the present disclosure provides a control method of a human-computer interaction device, a control device, a human-computer interaction device, and a storage medium.

[0004] According to a first aspect of an embodiment of the present disclosure, a control method of a human-computer interaction device is provided, including:

[0005] obtaining a reflection signal received by each receiving source in a plurality of receiving sources of a radar module, wherein the reflection signal is a radar signal emitted by a target to the plurality of receiving sources after being reflected by the target, and the plurality of receiving sources are arranged at intervals in a first direction and a second direction;

[0006] determining a target distance, a target speed, and a first angle according to the reflection signal received by each receiving source in the plurality of receiving sources, wherein the target is perpendicular to a line connecting any two receiving sources in the same direction, the perpendicular line has a first reference angle with one of the two receiving sources and a second reference angle with the other receiving source, and the first angle is a difference between the first reference angle and the second reference angle;

[0007] determining a gesture action of the target according to a plurality of groups of the target distance, the target speed, and the first angle;

[0008] controlling the human-computer interaction device according to the gesture action.

[0009] In some embodiments, the determining the gesture action of the target according to the plurality of groups of the target distance, the target speed, and the first angle includes:

[0010] obtaining a plurality of continuous target distances from the plurality of groups of the target distance, and obtaining a change of the plurality of continuous target distances.

[0011] When the consecutive at least two target distances are continuously increasing or decreasing, it is determined that the target is sending a close gesture or a far gesture.

[0012] Further, when the changes of the consecutive N target distances are continuously increasing or continuously decreasing, N≥5, it is determined that the target is sending a close gesture or a far gesture.

[0013] In some embodiments, the determination of the gesture action of the target according to the multiple groups of target distances, target speeds and the first angles comprises:

[0014] A plurality of consecutive target speeds are obtained from the multiple groups of target speeds, and a vector direction change of the plurality of consecutive target speeds is obtained.

[0015] When the vector directions of the plurality of target speeds change at the Mth target speed, it is determined that the target is sending a waving gesture.

[0016] Further, when the vector directions of the plurality of target speeds change at the Mth target speed, and the vector directions of the consecutive X target speeds before the Mth target speed and the consecutive X target speeds after the Mth target speed are opposite, X≥5, it is determined that the target is sending a waving gesture.

[0017] Optionally, a plurality of consecutive first angles are obtained from the multiple groups of first angles.

[0018] When the plurality of first angles of any two receiving sources in a first direction change in sign after the Yth first angle, a first difference value between the Y-1th first angle and the Y+1th first angle is calculated.

[0019] When the plurality of first angles of any two receiving sources in a second direction change in sign after the Zth first angle, a second difference value between the Z-1th first angle and the Z+1th first angle is calculated, wherein the sign change refers to a change from a positive value to a negative value or a change from a negative value to a positive value, and the first direction faces the target.

[0020] The absolute value of the first difference value is compared with the absolute value of the second difference value.

[0021] If the absolute value of the first difference value is greater than the absolute value of the second difference value, the waving gesture is a left-right waving gesture.

[0022] If the absolute value of the second difference value is greater than the absolute value of the first difference value, the waving gesture is an up-down waving gesture.

[0023] Further, if the Y-1th first angle is positive and the Y+1th first angle is negative, it is swinging from right to left, if the Y-1th first angle is negative and the Y+1th first angle is positive, it is swinging from left to right;

[0024] If the Z-1th first angle is positive and the Z+1th first angle is negative, it is swinging from up to down, if the Z-1th first angle is negative and the Z+1th first angle is positive, it is swinging from down to up.

[0025] In some embodiments, determining the target distance, the target speed and the first angle according to the reflection signal received by each receiving source in the plurality of receiving sources comprises:

[0026] Determining a range-Doppler map according to the reflection signal received by each receiving source in the plurality of receiving sources;

[0027] Determining an energy maximum value of the reflection signal received by each receiving source according to the range-Doppler map;

[0028] Determining a target position according to at least two energy maximum values exceeding a set threshold and having the same coordinate position;

[0029] Determining the target distance, the target speed and the first angle according to the target position and the phase difference of each receiving source.

[0030] According to a second aspect of the embodiments of the present disclosure, a control device of a human-computer interaction device is provided, and the control device comprises:

[0031] An acquisition module, configured to acquire a reflection signal received by each receiving source in a plurality of receiving sources of a radar module, wherein the reflection signal is a radar signal reflected by a target to the plurality of receiving sources, the radar signal being emitted by an emitting source of the radar module, and the plurality of receiving sources are arranged at intervals in a first direction and a second direction;

[0032] An identification module, configured to determine a target distance, a target speed and a first angle according to the reflection signal received by each receiving source in the plurality of receiving sources, and determine a gesture action of a target according to a plurality of groups of the target distance, the target speed and the first angle, wherein the target is towards a perpendicular line of a connecting line of any two receiving sources in a same direction, an angle of the perpendicular line with one of the any two receiving sources is a first reference angle, and an angle of the perpendicular line with another receiving source is a second reference angle, and the first angle is a difference value of the first reference angle and the second reference angle;

[0033] A control module, configured to control the human-computer interaction device according to the gesture action.

[0034] Further, the acquisition module is specifically configured to: acquire continuous target distances from the multiple groups of target distances.

[0035] The identification module is specifically configured to: obtain changes of the continuous target distances, and determine that the target sends a close gesture action or a far-away gesture action when at least two continuous target distances continuously increase or decrease.

[0036] Further, the identification module is further configured to: determine that the target sends a close gesture action or a far-away gesture action when changes of N continuous target distances are continuous increment or continuous decrement, N≥5.

[0037] In some embodiments, the acquisition module is specifically configured to: acquire continuous target speeds from the multiple groups of target speeds.

[0038] The identification module is specifically configured to: obtain vector direction changes of the continuous target speeds, and determine that the target sends an arm swinging gesture action when vector directions of the multiple target speeds change at an Mth position.

[0039] Further, the identification module is further configured to: determine that the target sends an arm swinging gesture action when vector directions of the multiple target speeds change at an Mth position, and vector directions of continuous X positions before the Mth position and continuous X positions after the Mth position are opposite, X≥5.

[0040] Further, the acquisition module is further configured to: acquire continuous first angles from the multiple groups of first angles.

[0041] The identification module is further configured to: calculate a first difference value of a Y-1th position and a Y+1th position when multiple first angles of any two receiving sources in a first direction change in sign after a Yth position;

[0042] calculate a second difference value of a Z-1th position and a Z+1th position when multiple first angles of any two receiving sources in a second direction change in sign after a Zth position, wherein the sign change refers to a change from a positive value to a negative value or a change from a negative value to a positive value, and the first direction faces the target.

[0043] Compare an absolute value of the first difference value with an absolute value of the second difference value.

[0044] If the absolute value of the first difference value is greater than the absolute value of the second difference value, the arm swinging gesture action is left-right arm swinging.

[0045] If the absolute value of the second difference value is greater than the absolute value of the first difference value, the arm swinging gesture action is up-down arm swinging.

[0046] Further, the identification module is further configured to: if the Y-1th is positive and the Y+1th is negative, the arm swinging is from right to left, if the Y-1th is negative and the Y+1th is positive, the arm swinging is from left to right.

[0047] if the Z-1th is positive and the Z+1th is negative, the arm swinging is from up to down, if the Z-1th is negative and the Z+1th is positive, the arm swinging is from down to up.

[0048] In some embodiments, the identification module is further configured to: determine a range-Doppler map according to the reflected signals received by each of the plurality of receiving sources;

[0049] determine an energy maximum value of the reflected signal received by each of the receiving sources according to the range-Doppler map;

[0050] determine a target position according to at least two of the energy maximum values which exceed a set threshold and have the same coordinate position;

[0051] determine the target distance, the target speed and the first angle according to the target position and the phase difference of each of the receiving sources.

[0052] According to a third aspect of embodiments of the present disclosure, a human-computer interaction device is provided, comprising:

[0053] a processor;

[0054] a memory for storing executable instructions of the processor;

[0055] The processor is configured to perform the control method as described in the above embodiments.

[0056] Further, the human-computer interaction device is configured as any one of a display device and an XR device.

[0057] According to a fourth aspect of embodiments of the present disclosure, a non-transitory computer readable storage medium is provided, when instructions in the storage medium are executed by a processor of a human-computer interaction device, the human-computer interaction device is enabled to perform the control method as described in the above embodiments.

[0058] The technical solutions provided by the embodiments of the present disclosure can include the following beneficial effects:

[0059] According to the control method of the human-computer interaction device, the radar signal is emitted by the emission source of the radar module, the target distance, the target speed and the first angle are determined based on the reflected signals received by the multiple receiving sources, the gesture action is determined according to the target distance, the target speed and the first angle, and the human-computer interaction device interacts with the user based on the gesture action. On the one hand, the human-computer interaction can be directly realized by the gesture action, without the need for remote controllers, handles and other components, so that the use experience can be improved and the use pain points (i.e., not found, insufficient power and the like) caused by the limitations of the device can be reduced. On the other hand, the calculation steps and processing steps of the gesture action recognition are less, and the response speed of the human-computer interaction device is faster.

[0060] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0061] The accompanying drawings, which are incorporated into and form part of the specification, illustrate an embodiment consistent with the present application and, together with the specification, serve to explain the principles of the application.

[0062] Figure 1 is a flowchart of a control method according to an exemplary embodiment.

[0063] Figure 2 is a flowchart of the determination of the approaching gesture action and the moving away gesture action of a control method according to an exemplary embodiment.

[0064] Figure 3 is a flowchart of the determination of the waving gesture action of a control method according to an exemplary embodiment.

[0065] Figure 4 is a flowchart of the determination of the waving gesture action of a control method according to an exemplary embodiment.

[0066] Figure 5 and Figure 6 is a flowchart of the determination of the waving gesture action of a control method according to an exemplary embodiment.

[0067] Figures 7-9 is a process diagram of the first angle change according to an exemplary embodiment.

[0068] Figure 10 is a block diagram of a radar module (general structure of a control device) according to an exemplary embodiment.

[0069] Figure 11 is a block diagram of a control device according to an exemplary embodiment.

[0070] Figure 12is a block diagram of a human-computer interaction device according to an exemplary embodiment. DETAILED DESCRIPTION

[0071] The exemplary embodiments will be described in detail herein with reference to the attached drawings. In the following description, the same numbers are used to indicate the same or similar elements, unless otherwise represented. The embodiments described in the following exemplary embodiments do not represent all the implementations consistent with the present disclosure. Instead, they only represent examples of devices and methods consistent with some aspects of the present disclosure, as detailed in the appended claims.

[0072] Figure 1 is a flow chart of a control method according to an exemplary embodiment, as Figure 1 shown, a control method of a human-computer interaction device 1000 according to an embodiment of the present disclosure.

[0073] The control method comprises:

[0074] Step 10: obtaining the reflection signals received by each receiving source 120 in the plurality of receiving sources 120 of the radar module 100, wherein the reflection signals are radar signals emitted by the transmitting source 110 of the radar module 100 reflected by the target to the plurality of receiving sources 120, and the plurality of receiving sources 120 are arranged at intervals in the first direction and the second direction.

[0075] Step 20: determining the target distance, target speed and first angle according to the reflection signals received by each receiving source 120 in the plurality of receiving sources 120, wherein the target is perpendicular to the line connecting any two receiving sources 120 in the same direction, the angle between the perpendicular line and one of the any two receiving sources 120 is the first reference angle, and the angle between the perpendicular line and the other receiving source 120 is the second reference angle, and the first angle is the difference between the first reference angle and the second reference angle.

[0076] Step 30: determining the gesture action of the target according to the plurality of sets of target distance, target speed and first angle.

[0077] Step 40: controlling the human-computer interaction device 1000 according to the gesture action.

[0078] Wherein, the receiving source 120 can be multiple, the transmitting source 110 is one, the plurality of receiving sources can all receive the reflection signals, the receiving sources 120 are arranged at intervals in the first direction and the second direction, the first direction is orthogonal to the second direction, the plurality of receiving sources 120 have a certain phase difference, the target speed, the target distance and the first angle can be obtained, the target speed is the swing speed of the target (for example: the user's arm), and the target distance is the distance between the target and the human-computer interaction device 1000.

[0079] Specifically, the radar module 100 can be configured as a millimeter wave radar, and the frequency modulation signal range, frequency modulation time, sampling frequency, transmission mode, number of chirps per frame, and number of sampling points per chirp of the millimeter wave are set first, and then the radar signal is emitted by the transmission source 110, the reflected signal is received by the receiving source 120, the target distance, target speed and first angle are calculated based on the reflected signal, and then the gesture action can be determined according to multiple sets of target distance, target speed and first angle within a preset time.

[0080] The preset time can be 3s, 5s, etc. Multiple sets of target distance, target speed and first angle are obtained within the preset time, each set corresponding to a target distance, a target speed and an angle. The multiple sets of data are continuous on the same time line, and then a gesture can be determined according to the front and rear single data or multiple single data on the time node, and multiple gestures can be determined based on the mixed comparison of multiple data on the time node, so as to realize accurate and rapid recognition of gesture actions.

[0081] It should be pointed out that the human-computer interaction device 1000 can be configured as a television, a display device such as a smart screen, a game device with body interaction, etc. It can also be a product used with a display device or a game device. In the daily use process, the display device or the game device realizes the control function through a remote controller or a handle, etc. In the use process, the remote controller or the handle is easy to be lost or run out of power, and the human-computer interaction part such as the remote controller or the handle is easy to be damaged by falling, and the use experience is poor. The existing gesture recognition based on computer vision has many calculation steps and processing steps, and the response speed is slow.

[0082] According to the control method of the human-computer interaction device 1000 of the embodiment of the present disclosure, the radar signal is emitted by the transmission source 110 of the radar module 100, and the target distance, target speed and first angle are determined based on the reflected signals received by multiple receiving sources, and then the gesture action is determined according to the target distance, target speed and first angle. The human-computer interaction device 1000 interacts with the user based on the gesture action. On the one hand, the human-computer interaction can be realized directly through the gesture action, without the need for components such as a remote controller or a handle, which can improve the use experience and reduce the use pain points (i.e. lost or run out of power) caused by device limitations. On the other hand, the calculation steps and processing steps of gesture action recognition are less, and the response speed of the human-computer interaction device 1000 is faster.

[0083] As shown in Figure 2 Step 20 includes sub-step 21 and sub-step 22, wherein sub-step 21 is used to identify whether the gesture action is moving towards or away from the human-computer interaction device 1000, and sub-step 22 can identify whether the gesture action is an arm swinging action.

[0084] Specifically, determining the target's hand gestures based on multiple sets of target distance, target speed, and first angle includes:

[0085] Sub-step 211: Obtain multiple consecutive target distances from multiple sets of target distances, and obtain the changes in multiple consecutive target distances;

[0086] Sub-step 212: When the distance between at least two consecutive targets increases or decreases continuously, it is determined that the target has made a gesture of approaching or moving away.

[0087] In other words, the movement of the target toward or away from the human-computer interaction device 1000 can be determined based on the changes in data at previous and subsequent time points on the same timeline, using the single data point of target distance.

[0088] For example: Within 3 seconds, a total of 50 sets of target distance, target speed and first angle data are collected, corresponding to 50 consecutive target distances. Among them, the 25th target distance is 3m, the 26th target distance is 5m, and the 27th target distance is 5.5m. It can be determined that the target is moving away from the human-computer interaction device by 1000. If the target distance gradually decreases, it can be determined that it is moving towards the human-computer interaction device by 1000.

[0089] Of course, the statement that the target distance gradually increases corresponds to the target moving away from the human-computer interaction device 1000, and the target distance gradually decreases corresponds to the target moving towards the human-computer interaction device 1000 is an example of a usage scenario. The specific target distance gradually increasing can represent movement towards or away from the human-computer interaction device 1000 based on the reference standard. The reference standard for the target distance is not unique. The reference standard can be the wall on which the human-computer interaction device 1000 is set, the relative wall, the human-computer interaction device 1000 itself, etc.

[0090] Understandably, if the distance between two consecutive targets increases or decreases, the accuracy of judging whether the target has made a gesture of approaching or moving away is low. In order to reduce data noise in the judgment process, we can determine whether the target has made a gesture of approaching or moving away when the distance between N consecutive targets changes continuously or continuously, and N≥5. This will improve the accuracy and reliability of obtaining gestures of approaching and moving away.

[0091] exist Figure 3 In the illustrated embodiment, the target's hand gesture is determined based on multiple sets of target distance, target speed, and a first angle, including:

[0092] Step 221: Obtain multiple consecutive target velocities from multiple sets of target velocities, and obtain the vector direction changes of the multiple consecutive target velocities;

[0093] Step 222: When the vector direction of the plurality of target speeds changes at the Mth target speed, it is determined that the target sends an arm swinging gesture action.

[0094] Specifically, the speed is a vector value, and the direction of the speed can be defined by positive and negative. During the arm swinging process of the user, the swinging of the arm corresponds to a change in the centripetal speed. The change in the speed corresponds to a change in the vector direction of the target speed. The vector data is quantified, that is, the positive and negative values of the data are changed. Each target speed is a value calculated based on the phase difference of the plurality of receiving sources, which can be positive or negative.

[0095] Further, the control method of the present disclosure determines whether the user has an arm swinging action based on the change in the vector direction of the target speed, which is specifically represented by the positive and negative changes in the value. The motion speed close to the radar module 100 can be correspondingly positioned as a vector negative, and the motion speed away from the radar module 100 can be correspondingly positioned as a vector positive. Different reference systems can also be established to define different directions as vector positive and vector negative.

[0096] For example, 50 sets of target distance, target speed, and first angle data are collected within 3s, corresponding to 50 continuous target speeds. The 15th target speed is 1 m / s, and the 16th target speed is -0.5 m / s. That is, the vector direction of the target speed changes at the 16th target speed value, which can be determined as the user sending an arm swinging action.

[0097] It can be understood that the recognition accuracy of the arm swinging action is low based on only one target speed with a changed vector direction. When the vector directions of the plurality of target speeds change at the Mth target speed, and the vector directions of the continuous X target speeds before the Mth target speed and the continuous X target speeds after the Mth target speed are opposite, X≥5, it is determined that the target sends an arm swinging gesture action.

[0098] As described above, the 15th target speed is 1 m / s, and the 11th target speed to the 15th target speed are all >0, while the corresponding 16th target speed to the 20th target speed are all <0. At this time, it can be determined that the user sends an arm swinging action, and the gesture action recognition accuracy is higher, and the data noise is filtered to reduce the gesture action recognition error.

[0099] According to the example embodiment of the present disclosure, when the gesture action is recognized as an arm swinging, the direction and radian of the arm swinging action can all represent different control instructions.

[0100] Correspondingly, as shown in Figure 4 , Figure 5 and Figure 6 , a plurality of continuous first angles can be obtained from a plurality of first angles.

[0101] When the sign of multiple first angles of any two receiving sources 120 in the first direction changes after the Yth first angle, calculate the first difference between the (Y-1)th first angle and the (Y+1)th first angle.

[0102] When the sign of multiple first angles of any two receivers 120 in the second direction changes after the Zth first angle, the second difference between the Z-1th first angle and the Z+1th first angle is calculated, where the sign change refers to changing from a positive value to a negative value or from a negative value to a positive value, and the first direction is facing the target;

[0103] Compare the absolute value of the first difference with the absolute value of the second difference;

[0104] If the absolute value of the first difference is greater than the absolute value of the second difference, then the arm-swinging gesture is a left or right arm swing.

[0105] If the absolute value of the second difference is greater than the absolute value of the first difference, then the arm-waving gesture is an up-and-down arm swing.

[0106] Specifically, the target's movement during the arm swing is a dynamic process relative to multiple receiving sources (120°). Figure 7 Appendix Figure 8 Appendix Figure 9 The process of changing the first angle between two receiving sources 120 in the first direction or two receiving sources 120 in the second direction is shown respectively.

[0107] Understandably, during the arm swing, the relative position of the target with respect to the multiple first receiving sources 120 located in the first direction gradually changes in the first direction and / or the relative position of the target with respect to the multiple first receiving sources 120 located in the first direction gradually changes in the second direction. The first angle is the difference between two reference angles, which will also change accordingly. The change process is from positive to negative or from negative to positive, which corresponds to the first direction facing the target, while the second direction is orthogonal to the first direction. That is, the first direction can be used to obtain the angle change when swinging the arm left and right, and the second direction can be used to obtain the angle change when swinging the arm up and down, so as to initially realize the recognition of the arm swing gesture.

[0108] It should be noted that, based on the difference between up-and-down arm swing gestures and left-and-right arm swing gestures, and because the receiving source 120 is configured with multiple sources spaced apart in the first and second directions, the first difference will not be equal to the second difference when the user performs an arm swing gesture.

[0109] It is understandable that the up and down swing arms and the left and right swing arms also have corresponding starting and ending positions, namely, from top to bottom or from bottom to top, from left to right or from right to left.

[0110] Therefore, it can be further determined that if the (Y-1)th first angle is positive and the (Y+1)th first angle is negative, then the arm swing is from right to left; if the (Y-1)th first angle is negative and the (Y+1)th first angle is positive, then the arm swing is from left to right.

[0111] If the (Z-1)th first angle is positive and the (Z+1)th first angle is negative, then the arm swings downwards; if the (Z-1)th first angle is negative and the (Z+1)th first angle is positive, then the arm swings upwards.

[0112] For example, such as Figure 8 As shown, the sign of the first angle changes at this position; at the previous first angle ( Figure 7 The first angle is negative, and the sign of the second first angle is negative. Figure 9 The first angle sign is positive. By changing the signs of the two first angles, the starting and ending directions of the arm swing can be determined, thus completing the full recognition of the arm swing gesture and improving the recognition accuracy.

[0113] In some embodiments, determining the target distance, target velocity, and first angle based on the reflected signals received by each of the plurality of receiving sources 120 includes:

[0114] The range Doppler image is determined based on the reflected signals received by each of the multiple receiving sources 120.

[0115] The maximum energy of the reflected signal received by each receiving source is determined based on the distance Doppler plot;

[0116] The target location is determined based on at least two maximum energy values ​​that exceed a set threshold and are located at the same coordinate position.

[0117] The target distance, target velocity, and first angle are determined based on the target position and the phase difference between each receiving source.

[0118] In other words, in some embodiments, the target distance, target velocity, and first angle can be obtained through the RD spectrum, which can further improve the acquisition efficiency and acquisition accuracy.

[0119] Figure 11 This is a control device 200 for a human-computer interaction device 1000 according to a second aspect of an embodiment of the present disclosure, as illustrated by an example. The control device 200 includes: an acquisition module 210, an identification module 220, and a control module 230.

[0120] The acquisition module 210 is configured to acquire reflected signals received by each of the plurality of receiving sources 120 of the radar module 100, wherein the reflected signals are radar signals emitted by the emitting source 110 of the radar module 100 and reflected by a target to the plurality of receiving sources 120, and the plurality of receiving sources 120 are arranged at intervals in the first direction and the second direction; the identification module 220 is configured to determine target distance, target speed and first angle according to the reflected signals received by each of the plurality of receiving sources 120, and determine a gesture action of the target according to a plurality of groups of target distance, target speed and first angle, wherein the target is perpendicular to a line connecting any two receiving sources 120 in the same direction, the perpendicular line is the first reference angle with one of the any two receiving sources 120, and the second reference angle with the other receiving source 120, and the first angle is the difference between the first reference angle and the second reference angle; and the control module 230 is configured to control the human-computer interaction device 1000 according to the gesture action.

[0121] The control device 200 of the human-computer interaction device 1000 according to the embodiments of the present disclosure can emit radar signals through the emitting source 110 of the radar module 100, determine target distance, target speed and first angle (as shown in Figure 10 The control device 200 of the human-computer interaction device 1000 according to the embodiments of the present disclosure can emit radar signals through the emitting source 110 of the radar module 100, determine target distance, target speed and first angle (as shown in

[0122] According to a third aspect of the embodiments of the present disclosure, a human-computer interaction device 1000 is provided, including a processor 400 and a memory 300 configured to store executable instructions of the processor 400, wherein the processor 400 is configured to execute the control method in the above embodiments.

[0123] The human-computer interaction device 1000 according to the embodiments of the present disclosure can implement the above control method, so that the interaction experience of the human-computer interaction device 1000 is better.

[0124] Figure 12 is a block diagram of a human-computer interaction device 1000 according to an exemplary embodiment. For example, the human-computer interaction device 1000 can be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, a display panel, a television, etc.

[0125] Referring to Figure 12The human interface device 1000 can include the following components: a processing component 1007, a memory 300, a power supply component 1002, a multimedia component 1001, a

[0126] The processing component 1007 usually controls overall operations of the human interface device 1000, such as operations associated with displaying, making phone calls, data

[0127] The memory 300 is configured to store various types of data to support operations of the device 1000. Examples of these data include instructions for any applications or methods operating on the human interface device 1000, contact data, phonebook data, messages, pictures, videos, and so on. The memory 300 can be implemented by any type of volatile or nonvolatile storage devices or a combination thereof, such as static random access memory 300 (SRAM), electrically erasable programmable read-only memory 300 (EEPROM), erasable programmable read-only memory 300 (EPROM), programmable read-only memory 300 (PROM), read-only memory 300 (ROM), magnetic storage, flash memory 300, magnetic or optical disks.

[0128] The power supply component 1002 supplies electrical power for the various components of the human interface device 1000. The power supply component 1002 can include a power supply management system, one or more power supplies, and other components associated with generating, managing and distributing electrical power for the human interface device 1000.

[0129] The multimedia component 1001 includes a screen providing an output interface between the human-machine interactive device 1000 and a user. In some embodiments, the screen can include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes the touch panel, the screen can be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense a touch, a slide and a gesture on the touch panel. The touch sensor can not only sense a boundary of a touching or a sliding action, but also detect duration and pressure related to the touching or sliding action. In some embodiments, the multimedia component 1001 includes a front camera and / or a back camera. When the human-machine interactive device 1000 is in an operation mode, such as a shooting mode or a video mode, the front camera and / or the back camera can receive external multimedia data. Each of the front and back cameras can be a fixed optical lens system or have a focal length and optical zoom capability.

[0130] The audio component 1003 is configured to output and / or input audio signals. For example, the audio component 1003 includes a microphone (MIC) that is configured to receive external audio signals when the human-machine interactive device 1000 is in an operation mode, such as a call mode, a recording mode and a voice recognition mode. The received audio signals can be further stored in the memory 300 or transmitted via the communication component 1005. In some embodiments, the audio component 1003 further includes a speaker for outputting audio signals.

[0131] The I / O interface 1006 provides an interface between the processing component 1007 and peripheral interface modules, which can be a keyboard, a click wheel, a button, etc. The buttons can include, but are not limited to, a home button, a volume button, a start button and a lock button.

[0132] The sensor component 1004 includes one or more sensors to provide status assessments for various aspects of the human-machine interface device 1000. For example, the sensor component 1004 can detect an on / off status of the device 1000, relative positioning of components, such as a display and keypad of the human-machine interface device 1000, a change in position of the human-machine interface device 1000 or a component of the human-machine interface device 1000, presence or absence of user contact with the human-machine interface device 1000, orientation or acceleration / deceleration of the human-machine interface device 1000, and temperature changes of the human-machine interface device 1000. The sensor component 1004 can include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor component 1004 can also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor component 1004 can also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0133] The communication component 1005 is configured to facilitate wired or wireless communication between the human-machine interface device 1000 and other devices. The human-machine interface device 1000 can access a wireless network based on a communication standard, such as WiFi, 2G, or 3G, or a combination thereof. In an example embodiment, the communication component 1005 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an example embodiment, the communication component 1005 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0134] In an example embodiment, the human-machine interface device 1000 can be implemented using one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors 400, other electronic units, or a combination thereof, to perform the above-described methods.

[0135] In an example embodiment, a non-transitory computer-readable storage medium including instructions, such as a memory 300 including instructions, is also provided, which can be executed by the processor 400 of the human-machine interface device 1000 to perform the above-described methods. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory 300 (RAM), a CD-ROM, a magnetic tape, a floppy disc, and an optical data storage device, and the like.

[0136] The human-computer interaction device 1000 is configured as any one of a display device and an XR device.

[0137] According to a fourth aspect of the embodiments of the present disclosure, a non-transitory computer-readable storage medium is provided, which, when instructions in the storage medium are executed by the processor 400 of the human-computer interaction device 1000, enables the human-computer interaction device 1000 to perform the control method as in the above embodiments.

[0138] Exemplarily, the control method of the human-computer interaction device 1000 includes:

[0139] Step 10: acquiring reflected signals received by each receiving source 120 of the plurality of receiving sources 120 of the radar module 100, wherein the reflected signals are radar signals emitted by the transmitting source 110 of the radar module 100 reflected by a target to the plurality of receiving sources 120, and the plurality of receiving sources 120 are arranged at intervals in the first direction and the second direction.

[0140] Step 20: determining the target distance, the target speed and the first angle according to the reflected signals received by each receiving source 120 of the plurality of receiving sources 120, wherein the target is perpendicular to the line connecting any two receiving sources 120 in the same direction, the perpendicular line has a first reference angle with one of the any two receiving sources 120 and a second reference angle with the other receiving source 120, and the first angle is the difference between the first reference angle and the second reference angle;

[0141] Step 30: determining the gesture action of the target according to the plurality of groups of target distance, target speed and first angle;

[0142] Step 40: controlling the human-computer interaction device 1000 according to the gesture action.

[0143] As to the apparatus in the above embodiments, the specific manners in which each module performs operations have been described in detail in the embodiments about the method, and thus will not be described in detail here.

[0144] Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. The specification and examples given are exemplary only and the true scope and spirit of the application is indicated by the following claims. The application is not limited by the embodiments that have been described and shown.

[0145] It should be understood that the application is not limited to the precise construction that has been described and shown in the accompanying drawings, and that various modifications and changes can be made by those skilled in the art without departing from the scope of the application. The scope of the application is indicated by the appended claims, rather than by the description and examples presented above.

Claims

1. A control method of a human-machine interaction device, characterized by, The method comprises: acquiring reflection signals received by each of a plurality of receiving sources of a radar module, wherein the radar module comprises one transmitting source and a plurality of receiving sources, the reflection signals are radar signals transmitted by the transmitting source of the radar module and reflected by a target to the plurality of receiving sources, and the plurality of receiving sources are arranged at intervals in a first direction and a second direction; determining a target distance, a target speed and a first angle according to the reflection signals received by each of the plurality of receiving sources, wherein the target is perpendicular to a line connecting any two receiving sources in the same direction, the perpendicular line has a first reference angle with one of the two receiving sources and a second reference angle with the other receiving source, and the first angle is a difference between the first reference angle and the second reference angle; determining a gesture action of the target according to a plurality of sets of the target distance, the target speed and the first angle, wherein when the gesture action is a swinging arm action, a sign of the first angle changes with a change in a relative position of the target in a direction to the plurality of receiving sources located in the direction, a sign change of the first angle in different directions is related to a direction of the swinging arm action, when the first angle changes in sign in the first direction, the swinging arm action is left-right swinging, and when the first angle changes in sign in the second direction, the swinging arm action is up-down swinging, and the first direction faces the target; controlling the human-computer interaction device according to the gesture action.

2. The control method of the human-machine interaction device according to claim 1, characterized in that, The method for determining the gesture action of the target according to the plurality of sets of the target distance, the target speed and the first angle comprises: acquiring a plurality of continuous target distances from the plurality of sets of the target distance, and obtaining a change in the plurality of continuous target distances; when at least two continuous target distances continuously increase or decrease, it is determined that the target sends a gesture action of approaching or moving away.

3. The control method of the human-machine interaction device according to claim 2, characterized in that, when a change in N continuous target distances is continuously increasing or continuously decreasing, N≥5, it is determined that the target sends a gesture action of approaching or moving away.

4. The control method of the human-machine interaction device according to claim 1, characterized in that, The method for determining the gesture action of the target according to the plurality of sets of the target distance, the target speed and the first angle comprises: acquiring a plurality of continuous target speeds from the plurality of sets of the target speed, and obtaining a vector direction change of the plurality of continuous target speeds; when a vector direction of the plurality of target speeds changes at an Mth target speed, it is determined that the target sends a swinging arm gesture action.

5. The control method of the human-machine interaction device according to claim 4, characterized in that, when a vector direction of the plurality of target speeds changes at an Mth target speed, and a vector direction of X continuous target speeds before the Mth target speed and a vector direction of X continuous target speeds after the Mth target speed are opposite, X≥5, it is determined that the target sends a swinging arm gesture action.

6. The control method of the human-computer interaction device according to claim 4 or 5, wherein: a plurality of continuous first angles are acquired from the plurality of sets of the first angle. calculating a first difference between the Y-1th first angle and the Y+1th first angle when the first angles of any two of the receiving sources in the first direction change sign after the Yth first angle, wherein the sign change refers to changing from positive to negative or from negative to positive, the first direction facing the target; calculating a second difference between the Z-1th first angle and the Z+1th first angle when the first angles of any two of the receiving sources in the second direction change sign after the Zth first angle, wherein the sign change refers to changing from positive to negative or from negative to positive, the first direction facing the target; comparing the absolute value of the first difference with the absolute value of the second difference; if the absolute value of the first difference is greater than the absolute value of the second difference, the arm swinging gesture is left-right arm swinging; if the absolute value of the second difference is greater than the absolute value of the first difference, the arm swinging gesture is up-down arm swinging.

7. The control method of the human-computer interaction device according to claim 6, wherein, if the Y-1th first angle is positive and the Y+1th first angle is negative, it is right-to-left arm swinging, and if the Y-1th first angle is negative and the Y+1th first angle is positive, it is left-to-right arm swinging; if the Z-1th first angle is positive and the Z+1th first angle is negative, it is up-to-down arm swinging, and if the Z-1th first angle is negative and the Z+1th first angle is positive, it is down-to-up arm swinging.

8. The control method of the human-machine interaction device according to claim 1, characterized in that, determining the target distance, the target speed and the first angle according to the reflected signals received by each of the receiving sources in the plurality of receiving sources, comprising: determining a range-Doppler map according to the reflected signals received by each of the receiving sources in the plurality of receiving sources; determining energy maximum values of the reflected signals received by each of the receiving sources according to the range-Doppler map; determining the target position according to at least two of the energy maximum values exceeding a set threshold and having the same coordinate position; determining the target distance, the target speed and the first angle according to the target position and the phase difference of each of the receiving sources.

9. A control device of a human-machine interface device, characterized by comprising: The control device comprises: an acquisition module, configured to acquire reflected signals received by each of the receiving sources in a plurality of receiving sources of a radar module, wherein the radar module comprises one transmitting source and a plurality of receiving sources, the reflected signals are radar signals transmitted by a target to the plurality of receiving sources after being reflected by the target, and the plurality of receiving sources are arranged at intervals in a first direction and a second direction; An identification module is configured to determine a target distance, a target speed and a first angle according to a reflection signal received by each of a plurality of receiving sources, and determine a gesture action of the target according to a plurality of groups of the target distance, the target speed and the first angle, wherein the first angle is an included angle between a line connecting any two receiving sources in the same direction and the target; when the gesture action is a waving action, a sign of the first angle changes with a change in a relative position of the target in a direction and a plurality of receiving sources located in the direction; a sign change of the first angle in different directions is related to a direction of the waving action; when the first angle changes in a sign in a first direction, the waving action is a left-right waving, and when the first angle changes in a sign in a second direction, the waving action is an up-down waving, and the first direction faces the target. A control module is configured to control the human-computer interaction device according to the gesture action.

10. A human-machine interaction device, characterized in that, The human-computer interaction device comprises: a processor; a memory for storing executable instructions of the processor; wherein the processor is configured to implement the control method according to any one of claims 1-8.

11. The human-machine interaction device according to claim 10, characterized in that, The human-computer interaction device is configured as any one of a display device and an XR device.

12. A non-transitory computer-readable storage medium, comprising: When the instructions in the storage medium are executed by the processor of the human-computer interaction device, the human-computer interaction device is enabled to implement the control method according to any one of claims 1-8.

Images