Interaction method, device, equipment, medium and product
By embedding gesture recognition function in the whiteboard program of large-screen devices, identifying user's gesture operations and performing corresponding functions, the problem of inconvenient operation on large-screen devices is solved, and the user's operation convenience and efficiency are improved.
Patent Information
- Application Number
- CN202410990478.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-23
- Publication Date
- 2024-11-08
AI Technical Summary
When using a whiteboard program to perform deletion, line breaking and other operations on large-screen devices, users need to frequently call virtual keyboards or handheld physical keyboards, resulting in inconvenient operation.
By embedding gesture recognition function in the whiteboard program of large-screen devices, identifying user gesture operations, determining target objects and function types, and directly executing corresponding functions to avoid relying on virtual keyboards or physical keyboards.
It improves user operation convenience, reduces dependence on the keyboard, and improves operation efficiency.
Smart Images

Figure CN118915912A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of human-computer interaction technology, and in particular to an interaction method, device, equipment, medium and product. Background Art
[0002] Large-screen devices are usually devices with larger screen sizes, such as screen sizes of 65 inches, 75 inches and above. Therefore, when a user uses a whiteboard program on a large-screen device to perform operations such as deletion and line break, it is necessary to call the virtual keyboard of the large-screen device or an external physical keyboard.
[0003] As for the virtual keyboard, since the size of the virtual keyboard is also large, the user needs to move back and forth when performing operations such as deletion and line change based on the virtual keyboard. As for the physical keyboard, since there is no structure to fix the physical keyboard on the large-screen device, the user needs to hold the physical keyboard during the operation.
[0004] It can be seen that the above two methods both have the problem of inconvenience in operation when the user performs operations such as deletion and line break. Summary of the invention
[0005] The embodiments of the present application provide an interactive method, apparatus, device, medium and product, which can improve the convenience of user operation when the user uses a whiteboard program on a large-screen device to perform operations such as deletion and line break.
[0006] In a first aspect, an embodiment of the present application provides an interaction method, which is applied to a large-screen device, in which a whiteboard program is installed, and the method includes:
[0007] Recognize the user's gesture operation on the interface display content in the whiteboard program to obtain a first gesture parameter and a second gesture parameter;
[0008] Determine, according to the first gesture parameter, a target object in the interface display content targeted by the gesture operation; and determine, according to the second gesture parameter, a function type corresponding to the gesture operation;
[0009] Executes the function corresponding to the function type on the target object.
[0010] In a second aspect, an embodiment of the present application provides an interactive device, which is applied to a large-screen device, in which a whiteboard program is installed, and the device includes:
[0011] A recognition module, used to recognize a user's gesture operation on the content displayed on the interface of the whiteboard program, and obtain a first gesture parameter and a second gesture parameter;
[0012] A determination module, configured to determine a target object in the interface display content targeted by the gesture operation according to the first gesture parameter; and to determine a function type corresponding to the gesture operation according to the second gesture parameter;
[0013] The execution module is used to execute the function corresponding to the function type on the target object.
[0014] In a third aspect, an embodiment of the present application provides an electronic device, including:
[0015] processor;
[0016] a memory for storing computer program instructions;
[0017] When the computer program instructions are executed by the processor, the method according to the first aspect is implemented.
[0018] In a fourth aspect, the present application provides a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, implements the method described in the first aspect.
[0019] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the method described in the first aspect.
[0020] The interactive method provided in the embodiment of the present application is applied to a large-screen device, in which a whiteboard program is installed. The first gesture parameter and the second gesture parameter are obtained by identifying the user's gesture operation on the interface display content in the whiteboard program; the target object in the interface display content targeted by the gesture operation is determined according to the first gesture parameter; and the function type corresponding to the gesture operation is determined according to the second gesture parameter; and the function corresponding to the function type is executed on the target object. That is, the embodiment of the present application embeds a gesture recognition function in the whiteboard program of the large-screen device, so that the whiteboard program can recognize the user's gesture operation, and then execute the function corresponding to the gesture operation on the target object in the whiteboard program, that is, the user can perform operations such as deletion and line break on the target object in the whiteboard program through gestures, without relying on a virtual keyboard or a physical keyboard, thus improving the convenience of user operation. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] In order to more clearly illustrate the technical solution of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0022] Figure 1 A flowchart of an interaction method provided in an embodiment of the present application;
[0023] Figure 2 A flowchart of another interaction method provided in an embodiment of the present application;
[0024] Figure 3 A flowchart of another interaction method provided in an embodiment of the present application;
[0025] Figure 4 A schematic diagram of selecting a target object provided in an embodiment of the present application;
[0026] Figure 5 A schematic diagram of deleting a target object provided in an embodiment of the present application;
[0027] Figure 6 A schematic diagram of inserting text between two character strings provided in an embodiment of the present application;
[0028] Figure 7 A schematic diagram of inserting a space between two adjacent characters in the same string provided in an embodiment of the present application;
[0029] Figure 8 A schematic diagram of deleting a space between two adjacent characters in the same string provided in an embodiment of the present application;
[0030] Fig. 9 A schematic diagram of wrapping a target object provided in an embodiment of the present application;
[0031] Fig.10 A structural diagram of an interactive device provided in an embodiment of the present application;
[0032] Fig.11 A structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0033] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and Examples. It should be understood that the specific embodiments described herein are only configured to explain the present application and are not configured to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.
[0034] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.
[0035] When a user uses a whiteboard program to write on a large-screen device, when the user needs to perform operations such as deleting, wrapping, and selecting the written content, it is usually necessary to call the virtual keyboard of the large-screen device or an external physical keyboard. As mentioned above, performing operations such as deleting, wrapping, and selecting the written content through the virtual keyboard or external physical keyboard will cause inconvenience to the user.
[0036] To this end, the embodiments of the present application provide an interactive method, device, equipment, medium and product, which can solve the problem of the inconvenience of using a virtual keyboard or an external physical keyboard to perform specific operations on written content in the related art.
[0037] The interactive method provided in the embodiment of the present application can be applied to a large-screen device, which can be an interactive device with a large screen size, such as a large-screen display and a host, an integrated large-screen device, a projector and a screen, etc. A whiteboard program is installed in the large-screen device, and a user can write in the whiteboard program using a writing pen.
[0038] In actual application, the whiteboard program can also be installed in a device supporting the Android system and / or the Windows system, or the interaction method can also be applied to a large-screen device supporting the Android system and / or the Windows system.
[0039] The following uses the example where the interaction method is applied to a large-screen device and a whiteboard program is installed in the large-screen device to illustrate the interaction method, device, equipment, medium and product provided in the embodiments of the present application.
[0040] Figure 1 A flowchart of an interactive method provided in an embodiment of the present application, such as Figure 1 As shown, the interaction method may include the following steps:
[0041] S110: Identify a user's gesture operation on the content displayed on the interface of the whiteboard program, and obtain a first gesture parameter and a second gesture parameter.
[0042] S120: Determine, according to the first gesture parameter, a target object in the interface display content targeted by the gesture operation.
[0043] S130. Determine a function type corresponding to the gesture operation according to the second gesture parameter.
[0044] S140: Execute the function corresponding to the function type on the target object.
[0045] The interactive method provided in the embodiment of the present application is applied to a large-screen device, in which a whiteboard program is installed. The first gesture parameter and the second gesture parameter are obtained by identifying the user's gesture operation on the interface display content in the whiteboard program; the target object in the interface display content targeted by the gesture operation is determined according to the first gesture parameter; and the function type corresponding to the gesture operation is determined according to the second gesture parameter; and the function corresponding to the function type is executed on the target object. That is, the embodiment of the present application embeds a gesture recognition function in the whiteboard program of the large-screen device, so that the whiteboard program can recognize the user's gesture operation, and then execute the function corresponding to the gesture operation on the target object in the whiteboard program, that is, the user can perform operations such as deletion and line break on the target object in the whiteboard program through gestures, without relying on a virtual keyboard or a physical keyboard, thus improving the convenience of user operation.
[0046] The above steps are described in detail below:
[0047] In S110 , the whiteboard program here may be a writing type whiteboard program, that is, the user can write in the whiteboard program.
[0048] The interface display content is the content displayed on the current interface of the whiteboard program, which may include but is not limited to images and text. When the interface display content includes text, the text may include at least one of handwritten text and printed text, that is, the embodiment of the present application can perform gesture operations not only on handwritten text, but also on printed text, thereby being applicable to more scenarios.
[0049] The gesture operation here may include but is not limited to touch screen gesture operation and air gesture operation, wherein the touch screen gesture operation may be a gesture operation in contact with the screen of the large-screen device, and the air gesture operation may be a gesture operation that does not contact the screen of the large-screen device. That is, the whiteboard program in the embodiment of the present application can not only recognize touch screen gestures, but also recognize air gestures, and can be applied to more scenarios. Regardless of which gesture operation is used, it no longer depends on the virtual keyboard of the large-screen device or the external physical keyboard, thereby improving the convenience of user operation.
[0050] The first gesture parameter may be a parameter capable of determining the position of the gesture, for example, may include the starting point and the end point of the gesture, and the position of the line between the starting point and the end point.
[0051] The second gesture parameter may be a parameter that can determine the function corresponding to the gesture operation, for example, it may include but is not limited to the gesture direction, gesture type, etc. The gesture direction may include a circular direction and a non-circular direction, and the non-circular direction may include, for example, from left to right, from right to left, from top to bottom, from bottom to top, etc. The gesture type may include, for example, the line type and number of lines of the gesture object.
[0052] In an embodiment of the present application, the whiteboard program can receive and identify the user's gesture operation on the interface display content, obtain the first gesture parameter and the second gesture parameter, and provide a data basis for the subsequent determination of the target object and function corresponding to the gesture operation.
[0053] In S120, the target object may be a partial object in the interface display content. For example, the target object may include an image, text, and the like.
[0054] Taking the target object including an image as an example, the target object may include one image or two images, Taking the target object including text as an example, the target object may include part or all of a string, two strings, etc.
[0055] The target object of the gesture operation can be determined according to the first gesture parameter. Taking the target object including text as an example, illustratively, when the starting point and the end point of the gesture correspond to the starting position and the ending position of a string respectively, the string can be determined as the target object. For example, the starting point of the gesture corresponds to the first character "h" of the string "hello", and the end point of the gesture corresponds to the last character "o" of the string "hello", then the string "hello" can be determined as the target object.
[0056] For another example, when the starting point of a gesture corresponds to the second character of a string and the end point of the gesture corresponds to the fifth character of the string, the second to fifth characters of the string can be determined as the target object. Taking the string "h1234ello" as an example, assuming that the starting point of the gesture corresponds to the second character "1" of the string "h1234ello" and the starting point of the gesture corresponds to the fifth character "4" of the string "h1234ello", "1234" can be determined as the target object, that is, a part of a string can be determined as the target object.
[0057] For another example, when the continuous line segment formed by the start and end points of a gesture is between two characters of the same string, the part before the previous character and the part after the next character can be determined as target objects. Taking the string "hello world" as an example, assuming that the continuous line segment formed by the start and end points of a gesture is between the characters "w" and "o" of the string "world", "hello w" and "orld" can be determined as target objects.
[0058] For another example, when the continuous line segment formed by the starting point and the end point of the gesture is between two adjacent character strings, the part before the previous character string and the part after the next character string can both be determined as the target object. ’ s information" as an example, assuming that the continuous line segment formed by the starting point and the end point of the gesture is between the string "Search" and the string "world ’ s", then you can put the string "Search" and the string "world ’ sinformation" is identified as the target object.
[0059] It should be noted that when the interface display content includes text, the text may be English text, Chinese text or text in other languages, and the embodiment of the present application does not limit the language type of the text.
[0060] In S130, the function types here may include, for example, deleting spaces, deleting text, wrapping, inserting spaces, inserting text, selecting, etc. These functions often need to be implemented with the help of a virtual keyboard of a large-screen device or an external physical keyboard, which makes it inconvenient for users to operate.
[0061] The embodiment of the present application can obtain the second gesture parameter by identifying the user's gesture operation, and then determine the function type corresponding to the gesture operation according to the second gesture parameter, which can reduce or even replace the use of the keyboard and improve the convenience of user operation.
[0062] Exemplarily, a correspondence between the second gesture parameter and the function type may be established in advance, and subsequently the corresponding function type may be obtained based on the actually recognized second gesture parameter in combination with the correspondence, which is simple and efficient.
[0063] The corresponding relationship may be a table relationship or a model relationship. That is, the function type corresponding to the gesture operation may be obtained by looking up a table, or the second gesture parameter may be input into a function type determination model to obtain the function type corresponding to the gesture operation. The function type determination model may be, for example, a neural network model, and the specific structure is not limited in the embodiments of the present application.
[0064] It should be noted that the embodiments of the present application do not limit the execution order of S120 and S130. For example, S120 may be executed first and then S130, or S130 may be executed first and then S120, or S120 and S130 may be executed simultaneously.
[0065] In S140 , after the function type corresponding to the gesture operation is determined, the function corresponding to the function type can be executed on the target object.
[0066] For example, if the function type is delete, and the target object of the gesture operation is string A, the whiteboard program can delete string A.
[0067] For another example, if the function type is to insert text and the gesture position is between string B and string C, the whiteboard program can fix the screen cursor between string B and string C, so that the user can insert the required text between string B and string C using a writing pen.
[0068] The embodiment of the present application embeds a gesture recognition function in a whiteboard program, so that when a user needs to perform operations such as deletion and line break on text, images, etc. in the whiteboard program, he only needs to perform gesture operations on the corresponding objects without the need to rely on a virtual keyboard or a physical keyboard, thereby improving the convenience of user operation.
[0069] For example, the second gesture parameter includes the gesture direction. Figure 2 As shown, Figure 2 A flowchart of another interaction method provided in an embodiment of the present application, Figure 2 and Figure 1 The difference is that Figure 1 The S130 in can be refined into Figure 2 S1301-S1303 in.
[0070] In S1301, at least one candidate function type of the gesture operation is determined according to the gesture direction.
[0071] In the embodiment of the present application, the same gesture direction may correspond to one or more candidate function types. The correspondence between different gesture directions and candidate function types may be predetermined or updated according to user needs.
[0072] The gesture direction may include, for example, a circular direction and a non-circular direction. The circular direction may be, for example, a direction of drawing a circle around the target object, and the circular direction may be a clockwise direction or a counterclockwise direction. The non-circular direction may include, for example, a horizontal direction and a vertical direction, and the horizontal direction may include, for example, from left to right or from right to left, and the vertical direction may include, for example, from top to bottom or from bottom to top.
[0073] Taking the case where the target object includes a target text as an example, illustratively, when the gesture direction is a horizontal direction, determining the candidate function type of the gesture operation includes at least one of the following: deleting the target text, selecting the target text;
[0074] In the case where the gesture direction is the vertical direction, determining that the candidate function type of the gesture operation includes at least one of the following: deleting a space in the target text, inserting a space in the target text, and inserting text in the target text;
[0075] When the gesture direction is a circular direction, determining the candidate function type of the gesture operation includes wrapping the target text.
[0076] In some embodiments, when the gesture direction is from left to right or from right to left, the candidate function type can be determined to be deleting the target text or selecting the target text. When the gesture direction is from top to bottom or from bottom to top, the candidate function type can be determined to be deleting a space in the target text, inserting a space in the target text, or inserting text in the target text. When the gesture direction is a circular direction, the candidate function type can be determined to wrap the target text.
[0077] In some embodiments, when the gesture direction is from left to right or from right to left, the candidate function types may be determined to be deleting a space in the target text, inserting a space in the target text, or inserting text in the target text. When the gesture direction is from top to bottom or from bottom to top, the candidate function types may be determined to be deleting the target text or selecting the target text. The correspondence between different gesture directions and candidate function types may be determined and updated according to scenarios, requirements, etc.
[0078] The embodiment of the present application can determine at least one candidate function type corresponding to the gesture direction according to the recognized gesture direction, thereby providing a basis for the user to independently select the required function type.
[0079] In S1302, at least one function type option is displayed.
[0080] After the whiteboard program determines the candidate function type corresponding to the gesture direction, the candidate function type options may be displayed for the user to select. One function type option corresponds to one candidate function type. For example, if the candidate function types include three, then the function type options also include three.
[0081] In S1303, when a selection input of at least one function type option is received from the user, the candidate function type corresponding to the selection input is determined as the function type corresponding to the gesture operation.
[0082] The selection input here can be a single click, double click, long press, slide, etc. of the user on the function type option. Exemplarily, after receiving the user's selection input on the function type option, the whiteboard program can determine the candidate function type corresponding to the selection input as the function type corresponding to the gesture operation.
[0083] The embodiment of the present application can determine at least one candidate function type based on the recognized gesture direction and display the corresponding function type options. The user can select the appropriate function type according to the needs, thereby meeting the user's personalized needs.
[0084] The second gesture parameter includes at least one of a gesture direction and a gesture type. Exemplarily, the above S130 may include the following steps:
[0085] The function type corresponding to the gesture operation is determined according to the second gesture parameter and the reference relationship. The reference relationship is used to indicate the association relationship between the second gesture parameter and the function type. Different second gesture parameters correspond to different function types.
[0086] The reference relationship here can be a table relationship or a model. Taking the table relationship as an example, illustratively, when the second gesture parameter includes the gesture direction, the reference relationship is used to characterize the association relationship between the gesture direction and the function type, and different gesture directions correspond to different function types.
[0087] Exemplarily, when the second gesture parameter includes a gesture type, the reference relationship is used to characterize the association relationship between the gesture type and the function type, and different gesture types correspond to different function types.
[0088] Exemplarily, when the second gesture parameter includes gesture direction and gesture type, the reference relationship is used to characterize the association relationship among gesture direction, gesture type and function type, and if either the gesture direction or the gesture type is different, the corresponding function type is also different.
[0089] Taking the reference relationship as a model as an example, illustratively, the second gesture parameter can be input into the corresponding model to obtain the corresponding function type.
[0090] When the second gesture parameter includes gesture direction and / or gesture type, the embodiment of the present application can automatically and uniquely determine the function type corresponding to the gesture operation based on the second gesture parameter and the reference relationship, which can further reduce the user's operation and improve the convenience of user operation.
[0091] For example, the second gesture parameter includes the gesture direction and gesture type, and the gesture type includes the line type and line number corresponding to the gesture. Figure 3 As shown, Figure 3 A flowchart of another interaction method provided in an embodiment of the present application. Figure 3 and Figure 1 The difference is that Figure 1 The S130 in can be refined into Figure 3 S1304-S1305 in.
[0092] In S1304, when the direction of the gesture is horizontal, the function type corresponding to the gesture operation is determined according to the line type corresponding to the gesture.
[0093] The line type here may include a straight line, a curve, etc. For example, when the whiteboard program recognizes that the gesture direction is horizontal and the line type is a straight line, it may determine that the function type corresponding to the gesture operation is to select a target object. Figure 4 As shown, the target object is "rld". If the user draws a horizontal line on the target object "rld" from left to right or from right to left, the whiteboard program can select the target object "rld".
[0094] For example, when the whiteboard program recognizes that the gesture direction is horizontal and the line type is a curve, it can determine that the function type corresponding to the gesture operation is to delete the target object. Figure 5 As shown, the target object is "Hello". If the user draws a wavy line on the target object "Hello" from left to right or from right to left, the whiteboard program can delete the target object "Hello".
[0095] In S1305 , when the direction of the gesture is the vertical direction, the function type corresponding to the gesture operation is determined according to the line type, the number of lines, and the continuous touch duration of the lines corresponding to the gesture.
[0096] Exemplarily, when there are multiple lines, the continuous touch duration of the lines here may be the continuous touch duration of the last line, that is, when drawing the last line, the long press duration of the gesture at the end point of the last line after the last line is drawn.
[0097] For example, when there is only one line, the continuous touch duration of the line here can be the long press duration of the gesture staying at the end point of the line after the line is drawn. The line can be used as both the starting line and the ending line.
[0098] Exemplarily, when the number of lines is a first number and the continuous touch duration of the last line is greater than or equal to the first duration, determining that the function type corresponding to the gesture operation is inserting the first object into the target object;
[0099] When the number of lines is the second number and the continuous touch duration of the last line is greater than or equal to the second duration, determining that the function type corresponding to the gesture operation is inserting the second object into the target object, or deleting the third object in the target object;
[0100] The second number is greater than the first number.
[0101] For example, the first number may be 1, and the second number may be 2, or other values, as long as the second number is greater than the first number. The first duration and the second duration may be the same or different, for example, both may be set to 0.6s.
[0102] Taking the first number being 1, the second number being 2, the first duration and the second duration being the same, both being 0.6s, as an example, when the number of lines is 1 and the long press is continued for greater than or equal to 0.6s after the line is drawn, it can be determined that the function type is to insert a first object into the target object. The first object can be, for example, text.
[0103] like Figure 6 As shown, the target objects are "Search" and "world ’ s information", the user draws a vertical line from top to bottom or from bottom to top between the characters "h" and "w", and continues to press and hold the end point for 0.8s≥0.6s, then the whiteboard program can fix the cursor between the characters "h" and "w", and the user can now select between "Search" and "world ’ sinformation" between the two. For example, you can insert the string "the" to get "Search the world ’ sinformation". The dotted box represents the screen cursor.
[0104] For example, when the number of lines is 2 and the pressing and holding continues for more than or equal to 0.6 seconds after the end line is drawn, the function type can be determined to be inserting a second object into the target object, or deleting a third object in the target object. For example, the second object and the third object can both be spaces or other objects. Of course, the second object and the third object can also be different.
[0105] Taking the example that both the second object and the third object are spaces, illustratively, if the gesture position acts between two adjacent characters of the same string, and there is no space between the two adjacent characters, then when the number of lines is 2 and the user continues to press and hold for greater than or equal to 0.6s after the last line is drawn, it can be determined that the function type is to insert a second object space between the target object, i.e., the two adjacent characters.
[0106] like Figure 7As shown, assuming that the gesture position is between the characters "w" and "o" of the character string "world", and there is no space between the characters "w" and "o", the whiteboard program can automatically insert a space between the characters "w" and "o" to obtain "w orld" when it recognizes that the user draws two vertical lines from top to bottom or from bottom to top between the characters "w" and "o", and continues to press and hold for 1s ≥ 0.6s after the second vertical line is drawn.
[0107] like Figure 8 As shown, assuming that the gesture position is still between the characters "w" and "o" of the character string "world", but there is a space between the characters "w" and "o", when the whiteboard program recognizes that the user draws two vertical lines from top to bottom or from bottom to top between the characters "w" and "o", and continues to press and hold for 1s ≥ 0.6s after the second vertical line is drawn, the space between the characters "w" and "o" can be automatically deleted to obtain "world".
[0108] The embodiments of the present application can also identify the function type corresponding to the gesture operation based on the gesture direction, the line type corresponding to the gesture, the number of lines, and the continuous touch duration of the lines. The factors considered are more comprehensive, and the recognition result can be more accurate. Moreover, by setting multiple factors, more function types can be covered to meet more user needs.
[0109] In some embodiments, Fig. 9 As shown, when the user circles the target object "world", that is, the gesture direction is a circular direction, the whiteboard program can wrap the target object "world".
[0110] Based on the same inventive concept, the present application embodiment also provides an interactive device. Fig.10 The interactive device provided in the embodiment of the present application is described in detail.
[0111] Fig.10 This is a structural diagram of an interactive device provided in an embodiment of the present application. Fig.10 As shown, the interactive device 1000 may include:
[0112] The recognition module 1001 is used to recognize the user's gesture operation on the interface display content in the whiteboard program, and obtain the first gesture parameter and the second gesture parameter;
[0113] The determination module 1002 is used to determine the target object in the interface display content targeted by the gesture operation according to the first gesture parameter; and determine the function type corresponding to the gesture operation according to the second gesture parameter;
[0114] The execution module 1003 is used to execute the function corresponding to the function type on the target object.
[0115] The interactive device provided in the embodiment of the present application is applied to a large-screen device, in which a whiteboard program is installed. The first gesture parameter and the second gesture parameter are obtained by identifying the user's gesture operation on the interface display content in the whiteboard program; the target object in the interface display content targeted by the gesture operation is determined according to the first gesture parameter; and the function type corresponding to the gesture operation is determined according to the second gesture parameter; and the function corresponding to the function type is executed on the target object. That is, the embodiment of the present application embeds a gesture recognition function in the whiteboard program of the large-screen device, so that the whiteboard program can recognize the user's gesture operation, and then execute the function corresponding to the gesture operation on the target object in the whiteboard program, that is, the user can perform operations such as deletion and line break on the target object in the whiteboard program through gestures, without relying on a virtual keyboard or a physical keyboard, thus improving the convenience of user operation.
[0116] In some embodiments, the second gesture parameter includes a gesture direction;
[0117] The determination module 1002 is specifically configured to:
[0118] Determine at least one candidate function type of the gesture operation according to the gesture direction;
[0119] In some embodiments, the interactive device 1000 may further include:
[0120] A display module, used for displaying at least one function type option, where one function type option corresponds to one candidate function type;
[0121] The determination module 1002 is specifically configured to:
[0122] In the case of receiving a selection input of at least one function type option from the user, the candidate function type corresponding to the selection input is determined as the function type corresponding to the gesture operation.
[0123] In some embodiments, the target object includes a target text;
[0124] The determination module 1002 is specifically configured to:
[0125] In the case where the gesture direction is a horizontal direction, determining that the candidate function type of the gesture operation includes at least one of the following: deleting a target text, selecting a target text;
[0126] In the case where the gesture direction is the vertical direction, determining that the candidate function type of the gesture operation includes at least one of the following: deleting a space in the target text, inserting a space in the target text, and inserting text in the target text;
[0127] When the gesture direction is a circular direction, determining the candidate function type of the gesture operation includes wrapping the target text.
[0128] In some embodiments, the second gesture parameter includes at least one of a gesture direction and a gesture type;
[0129] The determination module 1002 is specifically configured to:
[0130] The function type corresponding to the gesture operation is determined according to the second gesture parameter and the reference relationship. The reference relationship is used to indicate the association relationship between the second gesture parameter and the function type. Different second gesture parameters correspond to different function types.
[0131] In some embodiments, the second gesture parameter includes a gesture direction and a gesture type, and the gesture type includes a line type and a number of lines corresponding to the gesture;
[0132] The determination module 1002 is specifically configured to:
[0133] When the gesture direction is horizontal, the function type corresponding to the gesture operation is determined according to the line type corresponding to the gesture;
[0134] When the gesture direction is vertical, the function type corresponding to the gesture operation is determined according to the line type, the number of lines, and the continuous touch duration of the lines corresponding to the gesture.
[0135] In some embodiments, the determination module 1002 is specifically configured to:
[0136] When the number of lines is the first number and the continuous touch duration of the last line is greater than or equal to the first duration, determining that the function type corresponding to the gesture operation is inserting the first object into the target object;
[0137] When the number of lines is the second number and the continuous touch duration of the last line is greater than or equal to the second duration, determining that the function type corresponding to the gesture operation is inserting the second object into the target object, or deleting the third object in the target object;
[0138] The second number is greater than the first number.
[0139] In some embodiments, the gesture operation includes an air gesture operation and / or a touch screen gesture operation.
[0140] The embodiment of the present application embeds a gesture recognition function in the whiteboard program of the large-screen device, so that the whiteboard program can recognize the user's gesture operation, and then perform the function corresponding to the gesture operation on the target object in the whiteboard program. That is, the user can perform operations such as deletion and line break on the target object in the whiteboard program through gestures, without relying on a virtual keyboard or a physical keyboard, thereby improving the convenience of user operation.
[0141] Fig.10Each module in the device shown has the function of realizing Figure 1-Figure 3 The functions of each step in the process can achieve the corresponding technical effects, so for the sake of brevity, they will not be described in detail here.
[0142] Based on the same inventive concept, the embodiment of the present application further provides an electronic device, which may be a server. Fig.11 The electronic device provided in the embodiments of the present application is described in detail.
[0143] like Fig.11 As shown, the electronic device 1100 may include a processor 1110 and a memory 1120 for storing computer program instructions.
[0144] The processor 1110 may include a central processing unit (CPU) or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.
[0145] The memory 1120 may include a large capacity memory for data or instructions. For example, but not limitation, the memory 1120 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In one example, the memory 1120 may include a removable or non-removable (or fixed) medium, or the memory 1120 is a non-volatile solid-state memory. In one example, the memory 1120 may be a read-only memory (ROM). In one example, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM) or a flash memory or a combination of two or more of these.
[0146] The processor 1110 reads and executes the computer program instructions stored in the memory 1120 to implement Figure 1-Figure 3 The method in the embodiment shown in the figure is achieved Figure 1-Figure 3 The corresponding technical effects achieved by executing the method in the illustrated embodiment are not described in detail here for the sake of brevity.
[0147] In one example, the electronic device 1100 may further include a communication interface 1130 and a bus 1140. Fig.11As shown, the processor 1110, the memory 1120, and the communication interface 1130 are connected via a bus 1140 and communicate with each other.
[0148] The communication interface 1130 is mainly used to implement communication between various modules, devices and / or equipment in the embodiments of the present application.
[0149] Bus 1140 includes hardware, software or both, and each component of electronic device 1100 is coupled to each other. For example, but not limitation, bus 1140 may include accelerated graphics port (Accelerated Graphics Port, AGP) or other graphics bus, enhanced industry standard architecture (Extended Industry Standard Architecture, EISA) bus, front side bus (Front Side Bus, FSB), Hyper Transport (Hyper Transport, HT) interconnection, industry standard architecture (Industry Standard Architecture, ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnect (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 1140 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the present application considers any suitable bus or interconnection.
[0150] The electronic device can perform the interactive method in the embodiment of the present application after recognizing the user's gesture operation on the interface display content in the whiteboard program and obtaining the first gesture parameter and the second gesture parameter, thereby realizing the combination Figure 1-Figure 3 Describes the interaction method and Fig.10 Describe the interactive device.
[0151] In addition, in combination with the interaction method in the above embodiment, the embodiment of the present application can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when the computer program instructions are executed by a processor, any one of the interaction methods in the above embodiment is implemented.
[0152] In addition, an embodiment of the present application also provides a computer program product, including a computer program, which is executed by at least one processor to implement the various processes of the above-mentioned interaction method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0153] The above is only a specific implementation of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.
Claims
1. An interactive method, characterized in that: Applied to a large-screen device, in which a whiteboard program is installed, the method includes: Identifying a user's gesture operation on the content displayed on the interface of the whiteboard program to obtain a first gesture parameter and a second gesture parameter; Determining, according to the first gesture parameter, a target object in the interface display content targeted by the gesture operation; and determining, according to the second gesture parameter, a function type corresponding to the gesture operation; A function corresponding to the function type is executed on the target object.
2. The method according to claim 1, characterized in that The second gesture parameter includes a gesture direction; The determining, according to the second gesture parameter, a function type corresponding to the gesture operation includes: Determining at least one candidate function type of the gesture operation according to the gesture direction; Displaying at least one function type option, each function type option corresponds to a candidate function type; When a selection input of at least one of the function type options is received from the user, the candidate function type corresponding to the selection input is determined as the function type corresponding to the gesture operation.
3. The method according to claim 2, characterized in that The target object includes a target text; The determining, according to the gesture direction, at least one candidate function type of the gesture operation includes: In the case where the gesture direction is a horizontal direction, determining the candidate function type of the gesture operation includes at least one of the following: deleting the target text, selecting the target text; In the case where the gesture direction is a vertical direction, determining the candidate function type of the gesture operation includes at least one of the following: deleting a space in the target text, inserting a space in the target text, and inserting text in the target text; When the gesture direction is a circular direction, determining the candidate function type of the gesture operation includes wrapping the target text.
4. The method according to claim 1, characterized in that: The second gesture parameter includes at least one of a gesture direction and a gesture type; The determining, according to the second gesture parameter, a function type corresponding to the gesture operation includes: The function type corresponding to the gesture operation is determined according to the second gesture parameter and a reference relationship, wherein the reference relationship is used to indicate an association relationship between the second gesture parameter and the function type, and different second gesture parameters correspond to different function types.
5. The method according to claim 1, characterized in that The second gesture parameter includes a gesture direction and a gesture type, and the gesture type includes a line type and a number of lines corresponding to the gesture; The determining, according to the second gesture parameter, a function type corresponding to the gesture operation includes: When the direction of the gesture is horizontal, determining the function type corresponding to the gesture operation according to the line type corresponding to the gesture; When the direction of the gesture is a vertical direction, the function type corresponding to the gesture operation is determined according to the line type, the number of lines and the continuous touch duration of the lines corresponding to the gesture.
6. The method according to claim 5, characterized in that The determining the function type corresponding to the gesture operation according to the line type, the number of lines and the continuous touch duration of the lines corresponding to the gesture includes: When the number of lines is the first number and the continuous touch duration of the last line is greater than or equal to the first duration, determining that the function type corresponding to the gesture operation is to insert a first object into the target object; When the number of lines is the second number and the continuous touch duration of the last line is greater than or equal to the second duration, determining that the function type corresponding to the gesture operation is inserting a second object into the target object, or deleting a third object in the target object; Wherein, the second number is greater than the first number.
7. The method according to any one of claims 1 to 6, characterized in that: The gesture operation includes an air gesture operation and / or a touch screen gesture operation.
8. An interactive device, characterized in that: Applied to a large-screen device, in which a whiteboard program is installed, the device comprises: An identification module, used to identify a user's gesture operation on the interface display content in the whiteboard program, and obtain a first gesture parameter and a second gesture parameter; a determination module, configured to determine, according to the first gesture parameter, a target object in the interface display content targeted by the gesture operation; and to determine, according to the second gesture parameter, a function type corresponding to the gesture operation; An execution module is used to execute a function corresponding to the function type on the target object.
9. An electronic device, characterized in that: include: processor; a memory for storing computer program instructions; A touch screen for receiving and displaying hand-drawn lines; When the computer program instructions are executed by the processor, the method according to any one of claims 1 to 7 is implemented.
10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.
11. A computer program product, characterized in that The method comprises a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.
