Interaction control method and device, equipment and storage medium

Abstract

Embodiments of the invention provide an interaction control method and apparatus, a device and a storage medium. The method comprises the steps of obtaining a plurality of user gesture images triggered by a user for a target interface element in a current application; determining whether the gesture operation of the user is a hand turning operation based on the plurality of user gesture images; and if the gesture operation of the user is the hand turning operation, displaying the target interface element according to a set interaction response logic corresponding to the hand turning operation. By the adoption of the method and device, a new type of air gesture operation, namely the hand turning operation, is defined, and the interaction modes in the man-machine interaction process are increased. And displaying the target interface element based on an interaction response logic corresponding to the hand turning operation set for the target interface element. By adopting the mode, the current display mode can be switched to another display mode to display the target interface element, so that the display efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of information processing technology, and in particular to an interactive control method, device, equipment and storage medium. Background Art

[0002] With the development of science and technology, more and more intelligent electronic devices have entered people's lives, such as smartphones, smart TVs, smart speakers, wearable devices, self-moving robots, etc. Users can interact with these electronic devices, and different types of electronic devices can interact in different ways. For example, users can perform touch operations on the screen of a smartphone and interact with the smartphone through touch operations. For another example, users can input voice commands to the smart speaker and interact with the smart speaker through voice commands. Or for another example, users can input control commands to smart TVs, wearable devices, etc. through remote controls or air gestures.

[0003] With the popularity of smart TVs, wearable devices, etc., the requirements for interacting with these electronic devices are getting higher and higher. At present, in terms of air gesture operation, the flexibility of using air gesture operation to interactively control electronic devices provided in the relevant technology is relatively poor. Summary of the invention

[0004] Embodiments of the present invention provide an interactive control method, apparatus, device and storage medium for improving the flexibility of interacting with an electronic device through air gesture operations.

[0005] In a first aspect, an embodiment of the present invention provides an interactive control method, the method comprising:

[0006] Obtain multiple user gesture images triggered by the user on the target interface element in the current application;

[0007] Based on the multiple user gesture images, determining whether the user's gesture operation is a hand flip operation;

[0008] If the user's gesture operation is the flipping operation, the target interface element is displayed according to the interaction response logic set corresponding to the flipping operation.

[0009] Optionally, the target interface element includes a first type interface element and a plurality of second type interface elements located within the first type interface element;

[0010] Before displaying the target interface element according to the interaction response logic set corresponding to the flip operation, the method further includes:

[0011] A hand-flipping interaction event is registered for the target interface element in the operating system, wherein the hand-flipping interaction event includes a hand-flipping operation behavior and a corresponding interaction response logic.

[0012] Optionally, the first type interface element includes a list, and the plurality of second type interface elements include a plurality of information thumbnails located in the list;

[0013] The displaying the target interface element according to the set interactive response logic corresponding to the flip operation includes:

[0014] Display the information detail interfaces corresponding to the multiple information thumbnails respectively.

[0015] Optionally, the displaying of information detail interfaces corresponding to the plurality of information thumbnails respectively includes:

[0016] Determining a selected target information thumbnail from among the plurality of information thumbnails;

[0017] The information detail interfaces corresponding to the multiple information thumbnails are displayed respectively, and the information detail interface corresponding to the target information thumbnail is displayed on top.

[0018] Optionally, the first type of interface element includes a list or a pop-up window, and the plurality of second type of interface elements include a plurality of options;

[0019] The displaying the target interface element according to the set interactive response logic corresponding to the flip operation includes:

[0020] Switch from a currently displayed first option to a second option of the plurality of options.

[0021] Optionally, determining whether the user's gesture operation is a hand flip operation based on the multiple user gesture images includes:

[0022] For any user gesture image among the multiple user gesture images, detecting a hand type corresponding to the user's hand in the any user gesture image, wherein the hand type includes a back of a hand or a palm;

[0023] If there is a user gesture image in which the corresponding hand type is switched among the multiple user gesture images, it is determined that the user's gesture operation is the hand flip operation.

[0024] Optionally, if there is a user gesture image in which the corresponding hand type is switched among the multiple user gesture images, determining that the user gesture operation is the hand flip operation includes:

[0025] For any user gesture image among the multiple user gesture images, determining a detection frame corresponding to the user's hand in the any user gesture image;

[0026] Get the width of the detection frame;

[0027] If there is a user gesture image in which the corresponding hand type is switched among the multiple user gesture images, and the width of the detection frame corresponding to each of the multiple user gesture images meets the preset conditions according to the change trend of the image acquisition time sequence, then it is determined that the user's gesture operation is the flip hand operation.

[0028] In a second aspect, an embodiment of the present invention provides an interactive control device, including:

[0029] An acquisition module is used to acquire multiple user gesture images triggered by the user for the target interface element in the current application;

[0030] A determination module, used to determine whether the user's gesture operation is a hand flip operation based on the multiple user gesture images;

[0031] A response module is used to display the target interface element according to the interaction response logic set corresponding to the flipping operation if the user's gesture operation is the flipping operation.

[0032] Optionally, the target interface element includes a first type interface element and a plurality of second type interface elements located within the first type interface element;

[0033] The device further comprises a registration module, wherein the registration module is configured to:

[0034] A hand-flipping interaction event is registered for the target interface element in the operating system, wherein the hand-flipping interaction event includes a hand-flipping operation behavior and a corresponding interaction response logic.

[0035] Optionally, the first type interface element includes a list, and the plurality of second type interface elements include a plurality of information thumbnails located in the list;

[0036] The response module is used to display the information detail interfaces corresponding to the multiple information thumbnails respectively.

[0037] Optionally, the response module is used to:

[0038] Determining a selected target information thumbnail from among the plurality of information thumbnails;

[0039] The information detail interfaces corresponding to the multiple information thumbnails are displayed respectively, and the information detail interface corresponding to the target information thumbnail is displayed on top.

[0040] Optionally, the first type of interface element includes a list or a pop-up window, and the plurality of second type of interface elements include a plurality of options;

[0041] The response module is used to switch from the currently displayed first option to the second option among the multiple options.

[0042] Optionally, the determining module is used to:

[0043] For any user gesture image among the multiple user gesture images, detecting a hand type corresponding to the user's hand in the any user gesture image, wherein the hand type includes a back of a hand or a palm;

[0044] If there is a user gesture image in which the corresponding hand type is switched among the multiple user gesture images, it is determined that the user's gesture operation is the hand flip operation.

[0045] Optionally, the determining module is used to:

[0046] For any user gesture image among the multiple user gesture images, determining a detection frame corresponding to the user's hand in the any user gesture image;

[0047] Get the width of the detection frame;

[0048] If there is a user gesture image in which the corresponding hand type is switched among the multiple user gesture images, and the width of the detection frame corresponding to each of the multiple user gesture images meets the preset conditions according to the change trend of the image acquisition time sequence, then it is determined that the user's gesture operation is the flip hand operation.

[0049] In a third aspect, an embodiment of the present invention provides an electronic device, comprising a processor and a memory, wherein the memory stores executable code, and when the executable code is executed by the processor, the processor can at least implement the interactive control method in the first aspect.

[0050] In a fourth aspect, an embodiment of the present invention provides a non-temporary machine-readable storage medium having executable code stored thereon. When the executable code is executed by a processor of an electronic device, the processor can at least implement the interactive control method in the first aspect.

[0051] By adopting the present invention, a new type of air gesture operation, namely, the flipping operation, is defined, and the interaction mode in the human-computer interaction process is increased. During the interaction process, when it is detected that the user has performed a flipping operation on the target interface element in the current application, the interaction response logic corresponding to the flipping operation can be obtained in response to the user's flipping operation. Based on the interaction response logic corresponding to the flipping operation set for the target interface element, the target interface element is displayed. In this way, it is possible to switch from the current display mode to another display mode to display the target interface element, thereby improving the display efficiency. Users can freely switch the display mode through the flipping operation according to actual needs. Compared with other types of air gesture operations, the flipping operation is simple to operate, which can reduce the user's fatigue and improve the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0053] Figure 1 A flowchart of an interactive control method provided by an embodiment of the present invention;

[0054] Figure 2 A schematic diagram of an interactive control scenario provided by an embodiment of the present invention;

[0055] Figure 3 A schematic diagram of another interactive control scenario provided by an embodiment of the present invention;

[0056] Figure 4 A schematic diagram of another interactive control scenario provided by an embodiment of the present invention;

[0057] Figure 5 A schematic diagram of another interactive control scenario provided by an embodiment of the present invention;

[0058] Figure 6 A schematic diagram of a process for identifying a hand flip operation provided by an embodiment of the present invention;

[0059] Figure 7 A schematic diagram of a coordinate system transformation corresponding to a flip operation provided by an embodiment of the present invention;

[0060] Figure 8 A schematic diagram of a detection frame change during a hand flip operation provided by an embodiment of the present invention;

[0061] Fig. 9 A schematic diagram of the structure of an interactive control device provided by an embodiment of the present invention;

[0062] Fig.10 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0064] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings, and "multiple" generally includes at least two.

[0065] As used herein, the words "if" and "if" may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0066] In addition, the step sequence in the following method embodiments is only an example and not a strict limitation.

[0067] Figure 1 The flowchart of an interactive control method provided by an embodiment of the present invention can be applied to electronic devices. Figure 1 As shown, the method comprises the following steps:

[0068] 101. Obtain multiple user gesture images triggered by the user for a target interface element in the current application.

[0069] 102. Determine whether the user's gesture operation is a hand flip operation based on multiple user gesture images.

[0070] 103. If the user's gesture operation is a flipping operation, the target interface element is displayed according to the interaction response logic set corresponding to the flipping operation.

[0071] With the development of science and technology, some electronic devices are becoming more and more intelligent, and the way of interacting with electronic devices has gradually developed from touch, voice commands, etc. to air gesture operations. Among them, air gesture operations can be the user completing gesture operations in the air without actually touching the electronic device to input corresponding instructions to the electronic device. Air gesture operations can be, for example, drag operations, fist drag operations, three-finger pinch drag operations, and so on. It is worth noting that a new air gesture operation, namely, a flip hand operation, is specifically proposed in an embodiment of the present invention.

[0072] In practical applications, flipping operations and other gesture operations in the air can be identified by computer vision technology. It is understandable that a camera can be installed in an electronic device to collect user gesture operations through the camera. For example, assuming that the electronic device is a smart TV, a front camera can be installed on the top of the smart TV. When the user makes some gesture operations facing the smart TV, multiple user gesture images of the user can be collected through the front camera. The multiple user gesture images can also be understood as user gesture videos, which are composed of multiple user gesture images.

[0073] By identifying multiple user gesture images, it is possible to determine whether the user has performed a gesture operation, and if the user has performed a gesture operation, what type of gesture operation the user has performed, such as a flip hand operation. The specific identification method for identifying whether the user has performed a flip hand operation will be described in detail in the following embodiments, and will not be described in detail here.

[0074] Assuming that there is a target interface element in the current application, and the detected user's flip operation is triggered by the target interface element, the interactive response logic corresponding to the flip operation can be determined at this time, that is, how to respond to the user's flip operation on the target interface element in the current application, that is, how to display the target interface element in the current application in response to the user's flip operation.

[0075] In some optional embodiments, the target interface element may include a first type interface element and a plurality of second type interface elements located within the first type interface element. It should be noted that the first type interface element and the second type interface element may be different elements in different scenarios, which will be described in detail later through different scenarios.

[0076] In actual applications, for a specific target interface element, i.e., an interface element including a first-type interface element and a plurality of second-type interface elements located within the first-type interface element, a flipping interaction event can be optionally registered in the operating system so that the flipping operation can be used as a system-level interaction method. By defining the flipping operation as a system-level interaction method, the flipping operation can be applied to various applications loaded in the operating system, thereby improving the versatility of this type of interaction method.

[0077] The flip operation behavior and the corresponding interaction response logic can be pre-defined in the flip interaction event. In this way, when it is detected that the user has performed a flip operation on the target interface element in the current application, the interaction response logic corresponding to the flip operation for the target interface element can be determined based on the flip interaction event registered in the operating system.

[0078] The following uses several specific scenarios to illustrate how to respond to the flip operation on the target interface element.

[0079] Scene 1

[0080] The first type interface element may include a list, and the plurality of second type interface elements may include a plurality of information thumbnails located in the list.

[0081] In this scenario, the summary information of multiple information can be displayed in the form of multiple information thumbnails. The multiple information thumbnails can be placed in the same list by item, and the user can roughly understand the multiple information currently displayed by viewing the list. When the user no longer wants to view the summary information of multiple information in the form of a list, but wants to have a deeper understanding of the specific content of each information, the user can perform a flipping operation. Accordingly, when the electronic device detects the user's flipping operation, the information details interface corresponding to the multiple information thumbnails can be displayed, so that the user can have a deeper understanding of the multiple information by viewing the information details interface.

[0082] Take movie information as an example. Figure 2 As shown, five movie information thumbnails can be displayed in the same movie information list, including the movie information thumbnails corresponding to Movie A, Movie B, Movie C, Movie D, and Movie E. Among them, the poster of the current movie can be displayed on the left side of each movie information thumbnail, the name of the current movie can be displayed on the upper right side of each movie information thumbnail, and the rating of the current movie can be displayed on the lower right side of each movie information thumbnail. Of course, the layout of the movie information thumbnails introduced here is only an example. In actual applications, the layout of the information thumbnails can be designed according to actual needs.

[0083] exist Figure 2In the example shown, when the user wants to learn more about some detailed information about these movies, the user can flip the hand to trigger the switch of the multiple movie information thumbnails currently displayed to the information details interface corresponding to each movie in the movie information list. The information details interface can be displayed to the user in the form of a card. Among them, in the information details interface corresponding to a certain movie, the poster of the current movie can be displayed at the top, and because the display space becomes larger, the poster of the current movie can be displayed in the form of a large picture. The name, rating, movie content brief information, and main actors participating in the current movie can be displayed in the middle of the information details interface. At the bottom of the information details interface, operation controls for the current information details interface can be displayed, such as save controls, comment controls, etc.

[0084] Each movie corresponds to an information details interface, and the information details interfaces corresponding to different movies can be superimposed at a preset distance. When switching from multiple movie information thumbnails to the information details interfaces corresponding to each movie for display, the information details interface corresponding to the movie with the first movie thumbnail in the movie information list can be displayed at the top by default, and the information details interfaces corresponding to the remaining movies can be displayed in the order in which they appear in the movie information list. When the user switches the information details interface corresponding to a currently displayed movie through relevant gesture operations, the information details interface corresponding to a movie that is ranked before or after the currently displayed movie in the movie information list can be obtained for display.

[0085] Optionally, when the user's hand flipping operation is detected again, it is also possible to jump back from the information details interface to display multiple information thumbnails displayed in a list for display.

[0086] Scene 2

[0087] Similar to the previous scenario, but different from the previous scenario, in the previous scenario, when the user switches the currently displayed multiple information thumbnails to the information details interface by flipping his hand, the information details interface corresponding to the first information thumbnail in the list is displayed on top by default, that is, after switching, the user first sees the information details interface corresponding to the first information thumbnail. In the current scenario, the user can select the target information thumbnail of interest in the list through relevant gesture operations, and when the user switches the currently displayed multiple information thumbnails to the information details interface by flipping his hand, the information details interface corresponding to the target information thumbnail is displayed on top.

[0088] It should be noted that, except for the information details interface corresponding to the target information thumbnail, the information details interfaces corresponding to the other information thumbnails can still be sorted and displayed according to their respective orders in the list. Among them, the first information thumbnail in the list to the information thumbnails sorted before the target information thumbnail can be sorted in sequence after the last information thumbnail in the list.

[0089] In addition, it should be noted that in some application scenarios, the user can select the target information thumbnail in the list in the following way: after the user's hand is detected, a mark that can move with the user's hand can be displayed on the screen corresponding to the position of the user's hand. The position of the user's hand can be continuously detected, and when the position of the user's hand changes, the above mark also moves with the user's hand, and the information thumbnail corresponding to the final position of the mark on the screen is the target information thumbnail selected by the user.

[0090] Still taking movie information as an example, Figure 3 As shown, five movie information thumbnails can be displayed in the same movie information list, including the movie information thumbnails corresponding to Movie A, Movie B, Movie C, Movie D, and Movie E. The user can move the mark on the screen to the target movie information thumbnail, such as the movie information thumbnail corresponding to Movie C, in the manner described above. Then, the user can perform a flip operation, so that the user can directly jump to the information details interface corresponding to Movie C so that the user can view the details of Movie C first.

[0091] Optionally, when the user's hand flipping operation is detected again, it is also possible to jump back from the information details interface to display multiple information thumbnails displayed in a list.

[0092] Scene 3

[0093] Suppose there is a long list, which includes multiple information. These information are sorted and displayed in the order of the long list, and only one information is displayed at a time. In this scenario, if the user wants to view the information in the middle or back position of the long list, it is necessary to slide over the information one by one, and then multiple sliding operations are required to locate the information to be viewed, resulting in low information viewing efficiency. In an embodiment of the present invention, by registering a flipping interaction event in the operating system, when it is detected that the user has performed a flipping operation, the long list can be split into multiple short lists according to preset rules, and each time the user flips his hand, it can directly jump to the first information in the next short list for display.

[0094] For example, if Figure 4As shown in the figure, assuming that a long list includes 100 pieces of information, if the user wants to view the 51st piece of information in the long list, it is necessary to perform 51 sliding operations to locate the information to be viewed. When the user flips his hand, the long list can be split into 10 short lists according to the rule of 10 pieces of information per group, and each short list includes 10 pieces of information. The first piece of information in the first short list can be displayed first, and when it is detected that the user flips his hand again, it can directly jump to the first piece of information in the next short list for display. In this way, the user can view the 51st piece of information in the long list with only 6 flip operations, which greatly improves the efficiency of information viewing.

[0095] Scene 4

[0096] The first type of interface element may include a list or a pop-up window, and the plurality of second type of interface elements may include a plurality of options. Accordingly, when a user's hand flip operation is detected, the currently displayed first option may be switched to a second option among the plurality of options.

[0097] Take the login verification page as an example. Figure 5 As shown, when the user opens the login verification page of the current application, the option of verifying by scanning the identification code by the client in the terminal can be displayed by default. If the user does not want to use this login verification method for login verification, the user can flip the hand. When the user's flipping operation is detected, the option can be switched from the currently displayed option to the option of verifying by receiving the terminal verification code. Of course, when the user flips his hand again, it can also switch to the option corresponding to other login verification methods.

[0098] Alternatively, taking the application or the commonly used component entrance of an application as an example, the application icons of multiple applications can be displayed to the user in a list by default. When the user's flip operation is detected, it can be switched to display the commonly used components corresponding to multiple applications. Among them, if the user clicks to select the application icon of an application, the application homepage interface of the application will be entered, and if the user clicks to select a commonly used component of an application, the component interface of the commonly used component of the application will be directly entered.

[0099] The above introduces multiple scenarios of the interactive control method implemented by flipping the hand. The following will introduce a method for determining whether a user's gesture operation is a flipping operation based on multiple user gesture images.

[0100] like Figure 6 FIG. 1 is a flow chart of determining whether a user's gesture operation is a hand flip operation provided by an embodiment of the present invention. Figure 6 As shown, the process may include:

[0101] 601. For any user gesture image among multiple user gesture images, detect a hand type corresponding to the user's hand in any user gesture image, where the hand type includes the back of a hand or a palm.

[0102] 602. If there is a user gesture image in which the corresponding hand type is switched among the plurality of user gesture images, determine that the user gesture operation is a hand flip operation.

[0103] In practical applications, any user gesture image can be detected to identify whether the hand type of the user captured in the any user gesture image is the back of the hand or the palm. When the hand type corresponding to the first few user gesture images among multiple user gesture images is one of the back of the hand or the palm, and the hand type corresponding to the last few user gesture images among the multiple user gesture images is the other of the back of the hand or the palm, it can be determined that the user has performed a hand flip operation.

[0104] For example, assuming that there are 10 user gesture images in total, by detecting the first 5 user gesture images, the corresponding hand type is identified as the palm, and by detecting the last 5 user gesture images, the corresponding hand type is identified as the back of the hand. It can be considered that the hand type has changed in the 10 user gesture images, and then it can be confirmed that the user has performed a hand flip operation.

[0105] It should be noted that the hand type corresponding to any user gesture image can be identified by a machine learning model, etc. Specifically, the machine learning model can be trained in advance with a large number of sample images of the palms or backs of hands of different users, so that the trained machine learning model can automatically identify the hand type corresponding to any subsequent user gesture image input therein.

[0106] It is worth noting that a three-dimensional coordinate system x, y, z can be established for the user's hand, where the x and y axes form the plane of the user's palm, and the z axis can be set to be perpendicular to the plane formed by the x and y axes, that is, the z axis is perpendicular to the palm. Figure 7 As shown, it can be considered that a rotation is performed around the y-axis, that is, the z-axis is flipped.

[0107] Optionally, in order to further improve the accuracy of determining whether the user's gesture operation is a flipping hand operation, the flipping hand operation can also be detected by the following method: for any user gesture image among multiple user gesture images, detect the hand type corresponding to the user's hand in any user gesture image, wherein the hand type includes the back of the hand or the palm; for any user gesture image among multiple user gesture images, determine the detection frame corresponding to the user's hand in any user gesture image; obtain the width of the detection frame; if there are user gesture images in which the corresponding hand type is switched among the multiple user gesture images, and the widths of the detection frames corresponding to the multiple user gesture images meet the preset conditions according to the changing trend of the image acquisition time sequence, then determine that the user's gesture operation is a flipping hand operation.

[0108] It is understandable that when the palm or back of the user's hand is parallel to the camera, the width of the detection frame of the user's hand that can be detected from the captured user gesture image is larger. When the user performs a hand-flipping operation, at a certain moment, the user's hand is facing the camera sideways, and at this time the width of the detection frame of the user's hand that can be detected from the captured user gesture image is smaller. When the user continues to perform the hand-flipping operation, the width of the detection frame that can be detected of the user's hand will continue to increase. Therefore, by judging whether the change trend of the width of the detection frame corresponding to each user gesture image meets the preset conditions, it can be determined whether the user's gesture operation is a hand-flipping operation.

[0109] Specifically, it can be determined whether the width of the detection box corresponding to each user gesture image gradually decreases and then increases in the order of image acquisition time. If the width of the detection box corresponding to each user gesture image gradually decreases and then increases in the order of image acquisition time, it can be determined that the user has performed a hand flip operation.

[0110] by Figure 8 Taking the example shown as an example, assuming that there are currently 4 user gesture images, the user's hand type is identified as the palm in the 1st and 2nd user gesture images, and the user's hand type is identified as the back of the hand in the 3rd and 4th user gesture images. First, it can be determined that the user's hand type has changed in the 4 user gesture images. Further, assuming that the width of the detection frame corresponding to the 1st user gesture image is x1, the width of the detection frame corresponding to the 2nd user gesture image is x2, the width of the detection frame corresponding to the 3rd user gesture image is x3, and the width of the detection frame corresponding to the 4th user gesture image is x4, and x1>x2>x3, x3<x4, satisfying the trend of the width of the detection frame first decreasing and then increasing, it can be finally determined that the user has performed a hand flip operation.

[0111] By adopting the present invention, a new type of air gesture operation, namely, the flipping operation, is defined, and the interaction mode in the human-computer interaction process is increased. During the interaction process, when it is detected that the user has performed a flipping operation on the target interface element in the current application, the interaction response logic corresponding to the flipping operation can be obtained in response to the user's flipping operation. Based on the interaction response logic corresponding to the flipping operation set for the target interface element, the target interface element is displayed. In this way, it is possible to switch from the current display mode to another display mode to display the target interface element, thereby improving the display efficiency. Users can freely switch the display mode through the flipping operation according to actual needs. Compared with other types of air gesture operations, the flipping operation is simple to operate, which can reduce the user's fatigue and improve the user experience.

[0112] The interactive control device of one or more embodiments of the present invention will be described in detail below. Those skilled in the art will appreciate that these interactive control devices can be configured using commercially available hardware components through the steps taught in this solution.

[0113] Fig. 9 A schematic diagram of the structure of an interactive control device provided by an embodiment of the present invention is shown in FIG. Fig. 9 As shown, the device comprises:

[0114] An acquisition module 91 is used to acquire multiple user gesture images triggered by the user for a target interface element in the current application;

[0115] A determination module 92, configured to determine whether the user's gesture operation is a hand flip operation based on the plurality of user gesture images;

[0116] The response module 93 is used to display the target interface element according to the set interactive response logic corresponding to the flipping operation if the user's gesture operation is the flipping operation.

[0117] Optionally, the target interface element includes a first type interface element and a plurality of second type interface elements located within the first type interface element;

[0118] The device further comprises a registration module, wherein the registration module is configured to:

[0119] A hand-flipping interaction event is registered for the target interface element in the operating system, wherein the hand-flipping interaction event includes a hand-flipping operation behavior and a corresponding interaction response logic.

[0120] Optionally, the first type interface element includes a list, and the plurality of second type interface elements include a plurality of information thumbnails located in the list;

[0121] The response module 93 is used to display the information detail interfaces corresponding to the multiple information thumbnails respectively.

[0122] Optionally, the response module 93 is used to:

[0123] Determining a selected target information thumbnail from among the plurality of information thumbnails;

[0124] The information detail interfaces corresponding to the multiple information thumbnails are displayed respectively, and the information detail interface corresponding to the target information thumbnail is displayed on top.

[0125] Optionally, the first type of interface element includes a list or a pop-up window, and the plurality of second type of interface elements include a plurality of options;

[0126] The responding module 93 is used to switch from the currently displayed first option to the second option among the multiple options.

[0127] Optionally, the determining module 92 is used to:

[0128] For any user gesture image among the multiple user gesture images, detecting a hand type corresponding to the user's hand in the any user gesture image, wherein the hand type includes a back of a hand or a palm;

[0129] If there is a user gesture image in which the corresponding hand type is switched among the multiple user gesture images, it is determined that the user's gesture operation is the hand flip operation.

[0130] Optionally, the determining module 92 is used to:

[0131] For any user gesture image among the multiple user gesture images, determining a detection frame corresponding to the user's hand in the any user gesture image;

[0132] Get the width of the detection frame;

[0133] If there is a user gesture image in which the corresponding hand type is switched among the multiple user gesture images, and the width of the detection frame corresponding to each of the multiple user gesture images meets the preset conditions according to the change trend of the image acquisition time sequence, then it is determined that the user's gesture operation is the flip hand operation.

[0134] Fig. 9 The device shown can perform the aforementioned Figures 1 to 8 The interactive control method provided in the illustrated embodiment, the detailed execution process and technical effects are described in the aforementioned embodiment and will not be repeated here.

[0135] In one possible design, the above Fig. 9The structure of the interactive control device shown can be implemented as an electronic device, such as Fig.10 As shown, the electronic device may include: a processor 101 and a memory 102. The memory 102 stores executable code, and when the executable code is executed by the processor 101, the processor 101 can at least implement the above-mentioned Figures 1 to 8 The interactive control method provided in the illustrated embodiment.

[0136] Optionally, the electronic device may further include a communication interface 103 for communicating with other devices.

[0137] In addition, an embodiment of the present invention provides a non-transitory machine-readable storage medium, wherein an executable code is stored on the non-transitory machine-readable storage medium. When the executable code is executed by a processor of an electronic device, the processor can at least implement the above-mentioned Figures 1 to 8 The interactive control method provided in the illustrated embodiment.

[0138] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. Those of ordinary skill in the art may understand and implement the present invention without creative effort.

[0139] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by adding a necessary general hardware platform, and of course can also be implemented by combining hardware and software. Based on such an understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a computer product, and the present invention can be in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0140] The interactive control method provided in the embodiment of the present invention can be executed by a certain program / software, and the program / software can be provided by the network side. The electronic device mentioned in the above embodiment can download the program / software to a local non-volatile storage medium, and when it needs to execute the above interactive control method, the program / software is read into the memory by the CPU, and then the CPU executes the program / software to implement the interactive control method provided in the above embodiment. The execution process can refer to the above Figures 1 to 8 Instructions in .

[0141] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An interactive control method, characterized in that: include: Obtain multiple user gesture images triggered by the user on the target interface element in the current application; Based on the multiple user gesture images, determining whether the user's gesture operation is a hand flip operation; If the user's gesture operation is the flipping operation, the target interface element is displayed according to the interaction response logic set corresponding to the flipping operation.

2. The method according to claim 1, characterized in that The target interface element includes a first type interface element and a plurality of second type interface elements located within the first type interface element; Before displaying the target interface element according to the interaction response logic set corresponding to the flip operation, the method further includes: A hand-flipping interaction event is registered for the target interface element in the operating system, wherein the hand-flipping interaction event includes a hand-flipping operation behavior and a corresponding interaction response logic.

3. The method according to claim 2, characterized in that The first type of interface element includes a list, and the plurality of second type of interface elements include a plurality of information thumbnails located in the list; The displaying the target interface element according to the set interactive response logic corresponding to the flip operation includes: Display the information detail interfaces corresponding to the multiple information thumbnails respectively.

4. The method according to claim 3, characterized in that: The information detail interface corresponding to each of the plurality of information thumbnails is displayed, comprising: Determining a selected target information thumbnail from among the plurality of information thumbnails; The information detail interfaces corresponding to the multiple information thumbnails are displayed respectively, and the information detail interface corresponding to the target information thumbnail is displayed on top.

5. The method according to claim 2, characterized in that: The first type of interface element includes a list or a pop-up window, and the plurality of second type of interface elements include a plurality of options; The displaying the target interface element according to the set interactive response logic corresponding to the flip operation includes: Switch from a currently displayed first option to a second option of the plurality of options.

6. The method according to claim 1, characterized in that The determining, based on the multiple user gesture images, whether the user's gesture operation is a hand flip operation includes: For any user gesture image among the multiple user gesture images, detecting a hand type corresponding to the user's hand in the any user gesture image, wherein the hand type includes a back of a hand or a palm; If there is a user gesture image in which the corresponding hand type is switched among the multiple user gesture images, it is determined that the user's gesture operation is the hand flip operation.

7. The method according to claim 6, characterized in that If there is a user gesture image in which the corresponding hand type is switched among the plurality of user gesture images, determining that the user gesture operation is the hand flip operation includes: For any user gesture image among the multiple user gesture images, determining a detection frame corresponding to the user's hand in the any user gesture image; Get the width of the detection frame; If there is a user gesture image in which the corresponding hand type is switched among the multiple user gesture images, and the width of the detection frame corresponding to each of the multiple user gesture images meets the preset conditions according to the change trend of the image acquisition time sequence, then it is determined that the user's gesture operation is the flip hand operation.

8. An interactive control device, characterized in that: include: An acquisition module is used to acquire multiple user gesture images triggered by the user for the target interface element in the current application; A determination module, configured to determine whether the user's gesture operation is a hand flip operation based on the plurality of user gesture images; A response module is used to display the target interface element according to the interaction response logic set corresponding to the flipping operation if the user's gesture operation is the flipping operation.

9. The device according to claim 8, characterized in that The target interface element includes a first type interface element and a plurality of second type interface elements located within the first type interface element; The device further comprises a registration module, wherein the registration module is configured to: A hand-flipping interaction event is registered for the target interface element in the operating system, wherein the hand-flipping interaction event includes a hand-flipping operation behavior and a corresponding interaction response logic.

10. The device according to claim 9, characterized in that The first type of interface element includes a list, and the plurality of second type of interface elements include a plurality of information thumbnails located in the list; The response module is used to display the information detail interfaces corresponding to the multiple information thumbnails respectively.

11. The device according to claim 10, characterized in that The response module is used to: Determining a selected target information thumbnail from among the plurality of information thumbnails; The information detail interfaces corresponding to the multiple information thumbnails are displayed respectively, and the information detail interface corresponding to the target information thumbnail is displayed on top.

12. The device according to claim 9, characterized in that The first type of interface element includes a list or a pop-up window, and the plurality of second type of interface elements include a plurality of options; The response module is used to switch from the currently displayed first option to the second option among the multiple options.

13. The device according to claim 8, characterized in that The determining module is used to: For any user gesture image among the multiple user gesture images, detecting a hand type corresponding to the user's hand in the any user gesture image, wherein the hand type includes a back of a hand or a palm; If there is a user gesture image in which the corresponding hand type is switched among the multiple user gesture images, it is determined that the user's gesture operation is the hand flip operation.

14. The device according to claim 13, characterized in that The determining module is used to: For any user gesture image among the multiple user gesture images, determining a detection frame corresponding to the user's hand in the any user gesture image; Get the width of the detection frame; If there is a user gesture image in which the corresponding hand type is switched among the multiple user gesture images, and the width of the detection frame corresponding to each of the multiple user gesture images meets the preset conditions according to the change trend of the image acquisition time sequence, then it is determined that the user's gesture operation is the flip hand operation.

15. An electronic device, characterized in that: include: A memory and a processor; wherein the memory stores executable code, and when the executable code is executed by the processor, the processor executes the interactive control method as described in any one of claims 1-7.

16. A non-transitory machine-readable storage medium, characterized in that: The non-transitory machine-readable storage medium stores executable code, and when the executable code is executed by a processor of an electronic device, the processor executes the interactive control method according to any one of claims 1 to 7.

Images