A virtual ruler display method and related equipment

Abstract

The present application provides a method for displaying a virtual ruler, including: detecting a contact gesture with a touch screen, and displaying a virtual ruler on the touch screen based on the contact gesture indicating the contact between the side of the user's hand and the touch screen. The present application can trigger the display of a virtual ruler when a contact gesture indicating the contact between the side of the user's hand and the touch screen is detected. On the one hand, only one hand of the user is required to operate, and the operation difficulty and cost are very small. On the other hand, when the user's hand is unfolded, the contact area between the side of the hand and the touch screen is a flat long strip area, close to a straight line. Triggering the display of a virtual ruler based on this operation mode is more in line with the user's operation habits.

Description

Technical Field

[0001] The present application relates to the field of terminal software, and in particular to a virtual ruler display method and related equipment. Background Art

[0002] With the rapid development of the Internet and communication technology, office informatization has become an important part of the entire IT service system. Among them, interactive touch display devices (such as interactive whiteboards) that are closest to users are very popular and have become standard accessories for digital conferences. Figure 1 As shown, in order to achieve a better writing experience, an interactive touch display device is generally equipped with a stylus to facilitate users to write on the screen.

[0003] Electronic whiteboards are generally suitable for multi-person discussion scenarios, playing a role similar to that of a blackboard. In conference rooms or teaching scenarios, there are sometimes needs for drawing, table drawing, and pattern measurement, which require specific operations to call out a virtual ruler. The existing gestures for triggering the call out of a virtual ruler are relatively complex, with high difficulty and cost, and do not conform to the user's operating habits. Summary of the invention

[0004] The present application detects a contact gesture with the touch screen, indicates the contact between the side of the user's hand and the touch screen based on the contact gesture, and displays a virtual ruler on the touch screen according to the contact area of ​​the contact gesture. On the one hand, only one hand of the user is required for operation, and the operation difficulty and cost are very small. On the other hand, when the user's hand is spread out, the contact area between the side of the hand and the touch screen is a flat long strip area, close to a straight line. Triggering the display of the virtual ruler based on this operation mode is more in line with the user's operation habits.

[0005] The present application provides a virtual ruler display method, which is applied to an electronic device, wherein the electronic device includes a touch screen, and the method includes:

[0006] detecting a contact gesture with the touch screen;

[0007] The touch screen may include a touch-sensitive surface, a sensor or a sensor group for receiving input from a user based on tactile contact, and “detecting a contact gesture with the touch screen” may be understood as detecting a contact gesture with a touch-sensitive surface on the touch screen;

[0008] Based on the contact gesture indicating the contact between the side of the user's hand and the touch screen, a virtual ruler is displayed on the touch screen according to the contact area of ​​the contact gesture, wherein the contact area of ​​the contact gesture is a long strip area, and the virtual ruler is aligned with the long side of the long strip area.

[0009] The “contact area” may be understood as an area that can be detected and in contact with the touch-sensitive surface, which may be a continuous area or a discrete area consisting of densely distributed touch points (for example, the contact area may be a long strip area consisting of a group of densely distributed contact points);

[0010] The virtual ruler fits the long side of the long strip area, that is, in terms of display position, the virtual ruler and the long strip area need to be very close, and in terms of direction, the virtual ruler and the long side of the long strip area are parallel or nearly parallel;

[0011] In order to reduce the operational difficulty and cost of users triggering the virtual ruler display function, in an embodiment of the present application, when a contact gesture is detected indicating that the side of the user's hand is in contact with the touch screen, the display of the virtual ruler is triggered. On the one hand, only one hand of the user is required to operate, and the operation difficulty and cost are very small. On the other hand, when the user's hand is spread out, the contact area between the side of the hand and the touch screen is a flat long strip area, close to a straight line. Triggering the display of the virtual ruler based on this operation mode is more in line with the user's operating habits.

[0012] In a possible implementation, the hand side is the hand side of the user's hand located on the little finger side when the hand is in an unfolded state.

[0013] The hand side in the embodiment of the present application can be understood as the hand side of the user's hand located on the little finger side when the hand is in the unfolded state (for example, including at least one of the surface of the hypothenar part of the user's hand or the surface of the little finger part).

[0014] The unfolded state of the hand may be a state in which the user's fingers (or only the little finger) are in the same plane as the palm (or are close to being in the same plane).

[0015] In a possible implementation, the contact area includes: a contact area between the hypothenar part of the side of the user's hand and the touch screen; or a contact area between the little finger part of the side of the user's hand and the touch screen.

[0016] The side of the hand may include an area of ​​the hypothenar eminence and an area of ​​the little finger. When the side of the hand of the user contacts the touch screen, at least one of the following contact situations may occur due to the hand posture or the hand shape of the user:

[0017] The area of ​​the hypothenar eminence is in contact with the touch screen;

[0018] The area of ​​the pinky finger is in contact with the touch screen;

[0019] The area of ​​the hypothenar eminence and the area of ​​the little finger are in contact with the touch screen together.

[0020] In a possible implementation, the method also includes: determining that the contact gesture indicates contact between the side of the user's hand and the touch screen based on the contact area of ​​the contact gesture satisfying a preset condition; the preset condition includes: the contact area is a long strip area, and the shape and area size of the long strip area conform to the characteristics of the contact area when the hypothenar eminence and / or little finger area on the side of the user's hand contacts the touch screen.

[0021] The shape can be understood as the contour feature of the boundary line of the contact area. When the contour shape of the boundary line of the contact area is a flat long strip, it can be considered that the shape of the contact area conforms to the shape characteristics of the contact area when the hypothenar part and / or the little finger part of the side of the user's hand contacts the touch screen;

[0022] The shape can also be understood as the distribution characteristics of the contact points included in the contact area, and whether the above shape characteristics are met can be determined by detecting the distribution shape of the touch points. (For example, if a group of densely distributed long strip touch points are detected, it can be considered that the shape of the contact area meets the shape characteristics of the contact area when the hypothenar part and / or the little finger part of the side of the user's hand contacts the touch screen).

[0023] Among them, when the area size of the contact area is within a preset range (the maximum value in the range cannot be too large, and the minimum value in the range cannot be too small, and can be set based on the characteristics of the side of the human hand), it can be considered that the area size of the contact area conforms to the area size characteristics of the contact area when the hypothenar eminence and / or little finger on the side of the user's hand contacts the touch screen.

[0024] In a possible implementation, the method further includes:

[0025] Acquiring gesture data of the contact gesture;

[0026] Based on the gesture data, a neural network is used to determine that the contact gesture indicates contact between the side of the user's hand and the touch screen.

[0027] In a possible implementation, the virtual ruler is aligned with the long side of the long strip area, including:

[0028] The acute angle between the direction of the virtual ruler and the direction indicated by the long side of the long strip area is less than a preset value, which may be 1 degree, 2 degrees, 3 degrees, 4 degrees, 5 degrees, 10 degrees, etc.; and the virtual ruler and the long strip area satisfy one of the following conditions:

[0029] The virtual ruler overlaps with the elongated area; or, the virtual ruler is tangent to the elongated area; or, the distance between the virtual ruler and the elongated area is less than a preset value, which may be 1 cm, 2 cm, 3 cm, 4 cm, 5 cm, 10 cm, 15 cm, etc.

[0030] In a possible implementation, before displaying a virtual ruler on the touch screen according to the contact area of ​​the contact gesture, the method further includes: detecting that a maintenance time of the contact gesture is greater than a preset time.

[0031] Among them, the maintenance time of the contact gesture can be to start timing when the contact gesture is detected, or to start timing after detecting that the contact strength between the contact gesture and the touch screen is greater than a certain contact strength threshold (a threshold greater than 0); the maintenance time can be to end timing when the contact strength of the contact gesture is detected to be 0, or to end timing when the contact strength of the contact gesture is detected to be less than a certain contact strength threshold (a threshold greater than 0).

[0032] The duration of the contact gesture may be the duration of the contact gesture remaining in a static state (or moving less than a certain amplitude), for example, the duration of the contact gesture may be the duration of the static pressing gesture (or moving less than a certain amplitude). It should be understood that the static state may be understood as the static state of the contact area of ​​the contact gesture;

[0033] In a possible implementation, the method further includes: detecting movement of a contact area of ​​the contact gesture; and adjusting a display position of the virtual ruler so that the display position of the virtual ruler follows the contact area.

[0034] The so-called movement may be understood as a change in position and / or direction;

[0035] Among them, the so-called following can be understood as keeping the display position of the virtual ruler aligned with the contact area of ​​the contact gesture. As the contact area of ​​the contact gesture moves, the display position and direction of the virtual ruler will also change, and the display area of ​​the virtual ruler will always remain aligned with the long side of the contact area.

[0036] In a possible implementation, the method further includes: detecting a hand-raising gesture of the user; and hiding the display of the virtual ruler on the touch screen in response to the hand-raising gesture.

[0037] Among them, the hand-raising gesture can be understood as the side of the hand leaving the touch screen, or the contact strength between the side of the hand and the touch screen is less than a certain threshold, or the contact area between the side of the hand and the touch screen is less than a preset value.

[0038] In one possible implementation, indicating the contact between the side of the user's hand and the touch screen based on the contact gesture includes: indicating that the side of the user's hand has tapped the touch screen a preset number of times based on the contact gesture; after displaying the virtual ruler on the touch screen, the method further includes: detecting a hand-raising gesture of the user; and maintaining the display of the virtual ruler on the touch screen in response to the hand-raising gesture.

[0039] In some scenarios, when the user raises his hand, he still needs to use the virtual ruler, so it is necessary to trigger the fixed display of the virtual ruler on the touch screen based on a certain gesture;

[0040] Optionally, the preset number of taps needs to occur within a preset time, and the preset time is a shorter time (for example, 0.1S, 0.2S, 0.3S, 0.4S, 0.5S, etc.);

[0041] The tapping positions of the preset number of taps on the touch screen need to be kept consistent, or a slight deviation is allowed.

[0042] The preset number of times may be 2 times, 3 times, etc. For example, when the side of the hand is tapped twice in succession, the virtual ruler may be fixedly displayed on the touch screen.

[0043] In a possible implementation, the method further includes: detecting a user's selection instruction for the virtual ruler; and displaying a trigger control in response to the selection instruction, wherein the trigger control is used to instruct at least one of the following operations to be performed on the virtual ruler: a deletion operation, a position adjustment operation, and a rotation operation.

[0044] For example, the user can trigger the selection of the virtual ruler by clicking on a fixedly displayed virtual ruler. When selecting, the touch screen will pop up a trigger control. For example, the trigger control can be a prompt for indicating a deletion operation for the virtual ruler. The user can delete the virtual ruler by clicking on the "Delete" prompt or dragging the virtual ruler to the trash can icon).

[0045] For example, the user can trigger the selection of the virtual ruler by clicking on the fixedly displayed virtual ruler. When the ruler is selected, the touch screen will pop up a trigger control. For example, the trigger control can be a prompt for indicating the position adjustment operation for the virtual ruler. The user can click on the "posture adjustment" control to enter the posture adjustment mode for the virtual ruler. For example, in this mode, the user can adjust the display posture of the virtual ruler by contacting the side of the hand with the touch screen. The adjustment method can refer to the relevant description of the display position regulation of the virtual ruler in the above embodiment, which will not be repeated here. For example, if the virtual ruler needs to be rotated, two fingers can be used to click on two different positions (on the virtual ruler or within the preset distance of the virtual ruler), and the two touch points are rotated clockwise or counterclockwise at the same time to rotate the virtual ruler. Optionally, when operating the virtual ruler to rotate, the angle between the virtual rulers before and after the rotation can be displayed in real time.

[0046] In the embodiment of the present application, the virtual ruler is given a corresponding rotation or movement operation by rotating or moving the side of the hand, and gestures such as tapping to fix the display of the virtual ruler are in line with the user's operating habits and can greatly improve operating efficiency.

[0047] In a possible implementation, the virtual ruler is used to measure the length of a drawing straight line segment displayed on the touch screen; or, the virtual ruler is used to measure the distance between drawing points displayed on the touch screen.

[0048] In a possible implementation, the method further includes: detecting the presence of a drawing straight line segment on the touch screen that has a gesture association with the virtual ruler, and displaying the length value of the drawing straight line segment; the presence of the gesture association includes: the direction difference with the virtual ruler is less than a preset value; and / or the distance value between the virtual ruler and the virtual ruler is less than a preset value.

[0049] Among them, the so-called existence of posture association can be understood as a high degree of fit between the virtual ruler and the drawing straight line segment. For example, the posture association can be expressed by a direction difference and a distance value. When the direction difference between the drawing straight line segment and the virtual ruler is less than a preset value (for example, less than 20 degrees, 25 degrees, 30 degrees, 35 degrees, 40 degrees, 45 degrees, etc.); and / or the distance value between the drawing straight line segment and the virtual ruler is less than a preset value (for example, 1 cm, 2 cm, 3 cm, etc.), it can be considered that there is a drawing straight line segment on the touch screen that has a posture association with the virtual ruler.

[0050] The distance may be understood as the distance between the points on the virtual ruler closest to the straight line segment, or the average value of the distances between the points on the virtual ruler and the straight line segment, or other methods that can characterize the distance between the virtual ruler and the straight line segment may be used.

[0051] It should be understood that when the detection module detects that there are multiple drawing straight line segments with gesture associations with the virtual ruler on the touch screen, the length value of the drawing straight line segment with the greatest gesture association can be obtained and displayed.

[0052] In a possible implementation, the method further includes: adjusting the display position of the virtual ruler according to the position of the line segment to be measured so that the display position of the virtual ruler fits the drawing straight line. That is, a display effect similar to magnetic attraction is achieved. Adjusting the display position of the virtual ruler may include adjusting the position of the virtual ruler and adjusting the display direction.

[0053] In a possible implementation, after displaying the virtual ruler on the touch screen, the method further includes: detecting that a first intersection and a second intersection exist between the drawing line segment on the touch screen and the virtual ruler, and displaying a distance value between the first intersection and the second intersection.

[0054] In a possible implementation, the virtual ruler is used as a reference tool when drawing a straight line on the touch screen.

[0055] In a possible implementation, the method further includes: detecting a line drawing gesture on the touch screen; based on the virtual ruler being displayed on the touch screen and the distance between the line drawing position of the line drawing gesture and the virtual ruler being within a preset distance (for example, 1 cm, 2 cm, 3 cm, 4 cm, 5 cm, etc.), displaying a drawing straight line segment according to the line drawing position of the line drawing gesture, wherein the drawing straight line segment is parallel to the virtual ruler.

[0056] On the one hand, when writing and drawing within a certain range of the virtual ruler (without raising the hand), the writing and drawing range can be constrained not to exceed the virtual ruler, that is, writing and drawing can only be done on one side of the virtual ruler. When the writing and drawing range exceeds the virtual ruler, a straight line that fits the virtual ruler (the side where writing and drawing can be done) is displayed, which is equivalent to the virtual ruler playing a function similar to that of a ruler in physical space.

[0057] On the other hand, in response to the writing and drawing behavior within the predefined range of the virtual ruler, the drawing trajectory of the writing and drawing can be corrected to a straight line. In this way, when the user draws a line along the virtual ruler, a straight line will be automatically drawn.

[0058] In a possible implementation, the line drawing position includes a starting point position and an ending point position, and the drawing straight line segment is a line segment between the starting point position and the ending point position.

[0059] In a possible implementation, the line drawing position includes a starting point position and a real-time line drawing position, and the method further includes: displaying a line drawing length based on the real-time line drawing position, where the line drawing length is a distance value between the starting point position and the real-time line drawing position.

[0060] In order to obtain a line of expected length, when drawing along the ruler, the drawing line between the first and last drawing points can be calculated and displayed, and the length of the drawing line can be displayed. In a possible implementation, the drawing line position includes a starting point position and an ending point position, the drawing line segment is a line segment between the starting point position and the ending point position, and the detection module can display the drawing line length based on the real-time drawing line position, and the drawing line length is the distance value between the starting point position and the real-time drawing line position.

[0061] That is to say, if the length of the drawn line segment exceeds the expected value, you can draw it back, which will have a deletion effect. As long as the distance between the end point and the starting point is the expected value, the generated line segment is the expected value, and the drawn curve (the distance between the curve and the virtual ruler is within the preset range) will be automatically corrected to a straight line.

[0062] In the embodiments of the present application, the two functions of a ruler: measuring and drawing a straight line, are converted into intelligent operations suitable for a virtual interface. When measuring, the measured value is automatically displayed, and when drawing a straight line, the length value is displayed. This removes the limitations of the physical world, saves the user's reading time, and greatly improves user efficiency.

[0063] In a second aspect, the present application provides an object copying method, the method comprising: detecting a drag gesture for a target object displayed on a touch screen; in response to the drag gesture, displaying a mirror image of the target object on the touch screen, and updating the display position of the mirror image in real time according to the drag gesture so that the mirror image moves with the drag gesture; detecting a hand-raising gesture; in response to the hand-raising gesture, fixing the mirror image at the display position where the mirror image is located.

[0064] The method of copying and pasting by dragging gestures is simple and intuitive, which reduces the number of steps required for existing copying and pasting and shortens the process of copying and pasting.

[0065] In a possible implementation, before detecting the drag gesture for the target object displayed on the touch screen, the method further includes:

[0066] When at least one of the following gestures is detected, a copy function for the target object is enabled:

[0067] A long press gesture directed to the target object is detected; or,

[0068] A single-tap gesture directed to the target object is detected.

[0069] In a possible implementation, the detecting a long press gesture directed to the target object includes:

[0070] A long press gesture is detected in which a contact area with the touch screen covers the target object or is within a preset distance around the target object.

[0071] In a possible implementation, the drag gesture is a gesture of keeping the long press gesture in contact with the touch screen and moving on the touch screen.

[0072] In a possible implementation, the long press gesture is a two-finger long press gesture.

[0073] In a possible implementation, the detecting a single-click gesture directed to the target object includes:

[0074] A single-click gesture is detected in which a contact area with the touch screen covers the target object.

[0075] In a third aspect, an embodiment of the present application provides a virtual ruler display device, which is applied to an electronic device, wherein the electronic device includes a touch screen, and the device includes:

[0076] A detection module, used to detect a contact gesture with the touch screen;

[0077] The display module is used to indicate the contact between the side of the user's hand and the touch screen based on the contact gesture, and display a virtual ruler on the touch screen according to the contact area of ​​the contact gesture, wherein the contact area of ​​the contact gesture is a long strip area, and the virtual ruler is aligned with the long side of the long strip area.

[0078] In a possible implementation, the hand side is the hand side of the user's hand located on the little finger side when the hand is in an unfolded state.

[0079] In a possible implementation, the contact area includes:

[0080] The contact area between the hypothenar part of the side of the user's hand and the touch screen; or,

[0081] The contact area between the little finger of the user's hand and the touch screen.

[0082] In a possible implementation, the device further includes:

[0083] A determination module is used to determine that the contact gesture indicates contact between the side of the user's hand and the touch screen based on the contact area of ​​the contact gesture satisfying a preset condition; the preset condition includes:

[0084] The contact area is a long strip area, and the shape and area size of the long strip area conform to the characteristics of the contact area when the hypothenar part and / or the little finger part of the side of the user's hand contacts the touch screen.

[0085] In a possible implementation, the device further includes:

[0086] An acquisition module, used for acquiring gesture data of the contact gesture;

[0087] The determination module is further used to determine, based on the gesture data and through a neural network, that the contact gesture indicates contact between the side of the user's hand and the touch screen.

[0088] In a possible implementation, the virtual ruler is aligned with the long side of the long strip area, including:

[0089] The acute angle between the direction of the virtual ruler and the direction indicated by the long side of the long strip area is smaller than a preset value; and the virtual ruler and the long strip area satisfy one of the following conditions:

[0090] The virtual ruler overlaps with the long strip area; or, the virtual ruler is tangent to the long strip area; or, the distance between the virtual ruler and the long strip area is less than a preset value.

[0091] In a possible implementation, the detection module is configured to detect that a maintenance time of the contact gesture is greater than a preset time before a virtual ruler is displayed on the touch screen in the contact area according to the contact gesture.

[0092] In a possible implementation, the detection module is used to detect movement of the contact area of ​​the contact gesture;

[0093] The display module is used to adjust the display position of the virtual ruler so that the display position of the virtual ruler follows the contact area.

[0094] In a possible implementation, the detection module is used to detect a hand-raising gesture of the user;

[0095] The display module is used to hide the display of the virtual ruler on the touch screen in response to the hand-raising gesture.

[0096] In a possible implementation, indicating the contact between the side of the user's hand and the touch screen based on the contact gesture includes:

[0097] The side of the hand of the user taps the touch screen a preset number of times based on the contact gesture indication;

[0098] After displaying the virtual ruler on the touch screen, the method further includes:

[0099] Detecting the user's hand-raising gesture;

[0100] In response to the hand-raising gesture, the display of the virtual ruler is maintained on the touch screen.

[0101] In a possible implementation, the detection module is used to detect a selection instruction of the user with respect to the virtual ruler;

[0102] The display module is used to display a trigger control in response to the selection instruction, and the trigger control is used to instruct to perform at least one of the following operations on the virtual ruler:

[0103] Deletion, repositioning, and rotation.

[0104] In a possible implementation, the virtual ruler is used to measure the length of a drawing straight line segment displayed on the touch screen; or,

[0105] The virtual ruler is used to measure the distance between drawing points displayed on the touch screen.

[0106] In a possible implementation, the detection module is used to detect that there is a drawing straight line segment on the touch screen that is associated with the virtual ruler in a posture;

[0107] The display module is used to display the length value of the drawing straight line segment; the existence posture association includes:

[0108] The direction difference with the virtual ruler is less than a preset value; and / or,

[0109] The distance value between the virtual ruler and the virtual ruler is smaller than a preset value.

[0110] In a possible implementation, the display module is used to adjust the display position of the virtual ruler according to the position of the line segment to be measured, so that the display position of the virtual ruler fits the drawing straight line.

[0111] In a possible implementation, the detection module is configured to detect, after the virtual ruler is displayed on the touch screen, that a first intersection point and a second intersection point exist between the drawing line segment on the touch screen and the virtual ruler;

[0112] The display module is used to display the distance value between the first intersection point and the second intersection point.

[0113] In a possible implementation, the virtual ruler is used as a reference tool when drawing a straight line on the touch screen.

[0114] In a possible implementation, the detection module is used to detect a line drawing gesture on the touch screen;

[0115] The display module is used to display a drawing straight line segment according to the drawing position of the line drawing gesture based on the virtual ruler being displayed on the touch screen and the distance between the drawing position of the line drawing gesture and the virtual ruler being within a preset distance, wherein the drawing straight line segment is parallel to the virtual ruler.

[0116] In a possible implementation, the line drawing position includes a starting point position and an ending point position, and the drawing straight line segment is a line segment between the starting point position and the ending point position.

[0117] In a possible implementation, the line drawing position includes a starting point position and a real-time line drawing position, and the method further includes:

[0118] Based on the real-time line drawing position, the line drawing length is displayed, and the line drawing length is the distance value between the starting point position and the real-time line drawing position.

[0119] In a fourth aspect, the present application provides an object replication device, the device comprising:

[0120] A detection module, configured to detect a drag gesture directed to a target object displayed on the touch screen; the detection module is further configured to detect a hand-raising gesture after the display module displays a mirror image of the target object;

[0121] The display module is used to display the mirror image of the target object on the touch screen in response to the drag gesture, and to update the display position of the mirror image in real time according to the drag gesture so that the mirror image moves with the drag gesture; the display module is also used to display the mirror image fixedly at the display position where the mirror image is located in response to the hand-raising gesture.

[0122] In a possible implementation, the device further includes:

[0123] The enabling module is configured to enable a copy function for the target object displayed on the touch screen when at least one of the following gestures is detected before the drag gesture for the target object displayed on the touch screen is detected:

[0124] A long press gesture directed to the target object is detected; or,

[0125] A single-tap gesture directed to the target object is detected.

[0126] In a possible implementation, the detecting a long press gesture directed to the target object includes:

[0127] A long press gesture is detected in which a contact area with the touch screen covers the target object or is within a preset distance around the target object.

[0128] In a possible implementation, the drag gesture is a gesture of keeping the long press gesture in contact with the touch screen and moving on the touch screen.

[0129] In a possible implementation, the long press gesture is a two-finger long press gesture.

[0130] In a possible implementation, the detecting a single-click gesture directed to the target object includes:

[0131] A single-click gesture is detected in which a contact area with the touch screen covers the target object.

[0132] In a fifth aspect, the present application provides an electronic device, comprising: a processor, a memory, a touch screen and a bus, wherein: the processor, the memory and the touch screen are connected via the bus;

[0133] The memory is used to store computer programs or instructions;

[0134] The processor is used to call or execute the program or instructions stored in the memory, and is also used to control the touch screen to implement the steps described in the first aspect and any possible implementation method of the first aspect, as well as the steps described in the second aspect and any possible implementation method of the second aspect.

[0135] In a sixth aspect, the present application provides a computer storage medium comprising computer instructions. When the computer instructions are executed on an electronic device or a server, the steps described in the first aspect and any possible implementation method of the first aspect are executed, as well as the steps described in the second aspect and any possible implementation method of the second aspect are executed.

[0136] In a seventh aspect, the present application provides a computer program product, which, when running on an electronic device or a server, executes the steps described in the first aspect and any possible implementation of the first aspect, as well as the steps described in the second aspect and any possible implementation of the second aspect.

[0137] In an eighth aspect, the present application provides a chip system, which includes a processor for supporting an execution device or a training device to implement the functions involved in the above aspects, such as sending or processing the data involved in the above methods; or information. In one possible design, the chip system also includes a memory, which is used to store program instructions and data necessary for the execution device or the training device. The chip system can be composed of chips, or it can include chips and other discrete devices.

[0138] The embodiment of the present application provides a method for displaying a virtual ruler, including: detecting a contact gesture with the touch screen; based on the contact gesture indicating the contact between the side of the user's hand and the touch screen, displaying a virtual ruler on the touch screen according to the contact area of ​​the contact gesture, wherein the contact area of ​​the contact gesture is a long strip area, and the virtual ruler is aligned with the long side of the long strip area. When the contact gesture indicates the contact between the side of the user's hand and the touch screen, the display of the virtual ruler is triggered. On the one hand, only one hand of the user is required to operate, and the operation difficulty and cost are very small. On the other hand, when the user's hand is unfolded, the contact area between the side of the hand and the touch screen is a flat long strip area, close to a straight line. Triggering the display of the virtual ruler based on this operation mode is more in line with the user's operation habits. BRIEF DESCRIPTION OF THE DRAWINGS

[0139] Figure 1 A schematic diagram of the product structure provided for the embodiment of the present application;

[0140] Figure 2 A structural block diagram of an electronic device according to an embodiment of the present application;

[0141] Figure 3 A schematic diagram of an embodiment of a virtual ruler display method provided in an embodiment of the present application;

[0142] Figure 4 A gesture in the embodiment of the present application;

[0143] Figure 5 A gesture in the embodiment of the present application;

[0144] Figure 6 A gesture in the embodiment of the present application;

[0145] Figure 7 A gesture in the embodiment of the present application;

[0146] Figure 8 A gesture in the embodiment of the present application;

[0147] Fig. 9 A gesture contact in an embodiment of the present application is shown;

[0148] Fig.10 A gesture contact in an embodiment of the present application is shown;

[0149] Fig.11 A schematic diagram of a terminal interface in an embodiment of the present application;

[0150] Fig.12 A schematic diagram of a terminal interface in an embodiment of the present application;

[0151] Fig.13 A schematic diagram of a terminal interface in an embodiment of the present application;

[0152] Fig.14 A schematic diagram of a terminal interface in an embodiment of the present application;

[0153] Fig.15 A schematic diagram of a terminal interface in an embodiment of the present application;

[0154] Fig.16 A schematic diagram of a terminal interface in an embodiment of the present application;

[0155] Fig.17 A schematic diagram of a terminal interface in an embodiment of the present application;

[0156] Fig.18 A schematic diagram of a terminal interface in an embodiment of the present application;

[0157] Fig.19 A schematic diagram of a terminal interface in an embodiment of the present application;

[0158] Fig. 20 A schematic diagram of a terminal interface in an embodiment of the present application;

[0159] Fig.21 A schematic diagram of a terminal interface in an embodiment of the present application;

[0160] Fig.22a A schematic diagram of a terminal interface in an embodiment of the present application;

[0161] Figure 22b A schematic diagram of a terminal interface in an embodiment of the present application;

[0162] Fig.22c A schematic diagram of a terminal interface in an embodiment of the present application;

[0163] Fig.22d A schematic diagram of a terminal interface in an embodiment of the present application;

[0164] Fig.23 A schematic diagram of establishing an index table in an embodiment of the present application;

[0165] Fig.24 A schematic diagram of a terminal interface in an embodiment of the present application;

[0166] Fig.25 A schematic diagram of a terminal interface in an embodiment of the present application;

[0167] Fig.26 A schematic diagram of a terminal interface in an embodiment of the present application;

[0168] Fig. 27 A schematic diagram of a terminal interface in an embodiment of the present application;

[0169] Fig.28 A schematic diagram of a terminal interface in an embodiment of the present application;

[0170] Fig.29 A schematic diagram of a terminal interface in an embodiment of the present application;

[0171] Fig.30 A schematic diagram of a terminal interface in an embodiment of the present application;

[0172] Fig.31 A schematic diagram of a terminal interface in an embodiment of the present application;

[0173] Fig.32 A schematic diagram of a terminal interface in an embodiment of the present application;

[0174] Fig.33 A schematic diagram of a terminal interface in an embodiment of the present application;

[0175] Fig.34 A schematic diagram of a terminal interface in an embodiment of the present application;

[0176] Fig.35 A schematic diagram of a terminal interface in an embodiment of the present application;

[0177] Fig.36 A schematic diagram of a terminal interface in an embodiment of the present application;

[0178] Fig.37a A schematic diagram of an embodiment of an object replication method provided in an embodiment of the present application;

[0179] Fig.37b A schematic diagram of a terminal interface in an embodiment of the present application;

[0180] Fig.38 A schematic diagram of a terminal interface in an embodiment of the present application;

[0181] Fig.39 A schematic diagram of a terminal interface in an embodiment of the present application;

[0182] Fig.40 A schematic diagram of a terminal interface in an embodiment of the present application;

[0183] Fig.41 A schematic diagram of a terminal interface in an embodiment of the present application;

[0184] Fig.42 A schematic diagram of a terminal interface in an embodiment of the present application;

[0185] Fig.43 A schematic diagram of a terminal interface in an embodiment of the present application;

[0186] Fig.44 A schematic diagram of a terminal interface in an embodiment of the present application;

[0187] Fig.45 A schematic diagram of a terminal interface in an embodiment of the present application;

[0188] Fig.46 A schematic diagram of a terminal interface in an embodiment of the present application;

[0189] Fig.47 A schematic diagram of a terminal interface in an embodiment of the present application;

[0190] Fig.48 A schematic diagram of a terminal interface in an embodiment of the present application;

[0191] Fig.49 A schematic diagram of the structure of a virtual ruler display device provided in an embodiment of the present application;

[0192] Fig.50 A schematic diagram of the structure of an object replication device provided in an embodiment of the present application;

[0193] Fig.51 A schematic diagram of the structure of a terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0194] The embodiments of the present invention are described below in conjunction with the accompanying drawings in the embodiments of the present invention. The terms used in the implementation mode of the present invention are only used to explain the specific embodiments of the present invention, and are not intended to limit the present invention.

[0195] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0196] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0197] First, we introduce the exemplary operating environment of this application.

[0198] The embodiment of the present application may be applied in a system 100 including a touch screen 103 .

[0199] Reference Figure 1 , Figure 1 A system 100 to which an embodiment of the present application is applied is shown, wherein the system 100 may include an electronic device 101 and a pen 102 associated with the electronic device 101 .

[0200] The electronic device 101 may be Figure 1 The electronic whiteboard (or electronic interactive smart board) shown in the figure, the electronic device 101 includes a touch screen 103. It should be understood that the electronic device 101 can also be a portable mobile device including a touch screen, such as but not limited to a mobile or portable computing device (such as a smart phone), a personal computer, a server computer, a handheld device (such as a tablet) or a laptop, a multi-processor system, a game console or controller, a microprocessor-based system, a set-top box, a programmable consumer electronic product, a mobile phone, a mobile computing and / or communication device with a wearable or accessory form factor (such as a watch, glasses, a headset or earplugs), a network PC, a minicomputer, a mainframe computer, a distributed computing environment including any of the above systems or devices, and the like.

[0201] The touch screen 103 is capable of recognizing a user's contact gesture.

[0202] In one implementation, the touch screen 103 can be an infrared touch screen. The infrared touch screen is composed of infrared emitting and receiving sensing elements mounted on the outer frame of the touch screen. On the surface of the screen, an infrared detection network is formed. Any touching object can change the infrared on the contact point to realize the touch screen operation. The implementation principle of the infrared touch screen is similar to that of the surface acoustic wave touch screen, which uses infrared emitting and receiving sensing elements. These elements form an infrared detection network on the surface of the screen. The object of the touch operation (such as the contact between the side of the hand and the touch screen 103 in the embodiment of the present application) can change the infrared of the contact point, which is then converted into the coordinate position of the touch, thereby realizing the recognition of the touch gesture.

[0203] In one implementation, the touch screen 103 can be a capacitive touch screen, which works by using the current induction of the human body. Exemplarily, the capacitive touch screen can be a four-layer composite glass screen, the inner surface and the interlayer of the glass screen are each coated with a layer of ITO (nano indium tin metal oxide), the outermost layer is a thin layer of silica glass protective layer, the interlayer ITO coating is used as the working surface, four electrodes are drawn out from the four corners, and the inner layer ITO is a shielding layer to ensure a good working environment. When the user's hand touches the metal layer (for example, the contact between the side of the hand and the touch screen 103 in the embodiment of the present application), due to the human body electric field, the user and the touch screen surface form a coupling capacitor. For high-frequency current, the capacitor is a direct conductor, so the finger absorbs a very small current from the contact point. This current flows out from the electrodes on the four corners of the touch screen, and the current flowing through the four electrodes is proportional to the distance from the finger to the four corners. The controller obtains the position of the touch point by accurately calculating the ratio of these four currents, and then the recognition of the touch gesture can be realized.

[0204] It should be understood that the touch screen 103 may also be other types of touch screens with contact gesture recognition capabilities, or may be replaced with a touch screen that only has a display function but can cooperate with other external devices (such as sensors) to implement contact gesture recognition, which is not limited here.

[0205] In one example, the pen 102 may also provide input to the electronic device 101 by contacting or otherwise interacting with the touch screen 103 .

[0206] The touch screen 103 can display patterns and texts, and can also provide a writing and drawing interface for users to write and draw freely, such as a whiteboard interface provided by an electronic whiteboard, a writing and drawing board provided by an application (APP), etc.

[0207] In the embodiment of the present application, the user may touch the touch screen 103 with the side of the hand, and the electronic device 101 may trigger the display of the virtual ruler after detecting the contact gesture of the user.

[0208] The above introduces the exemplary operating environment of the present application. Next, the internal structure of the electronic device 101 is described with reference to an example.

[0209] See also Figure 2 , Figure 2 It is a schematic diagram of the structure of the electronic device 101 provided in an embodiment of the present application.

[0210] Understandably, Figure 2The illustrated structure does not constitute a specific limitation on the electronic device 101. In other embodiments of the present application, the electronic device 101 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0211] The electronic device 101 may include an input / output controller 218, which may output information to one or more output devices 222 (e.g., a touch screen or a speaker) that are separate from or integrated with the electronic device 101. The input / output controller 218 may also be used to receive input from one or more input devices 220 (e.g., a keyboard, a microphone, or a touch screen). In one embodiment, the output device 222 may also serve as the input device 220. An example of such a device may be a touch screen. In some embodiments, a user may provide input to the input device 220 and / or receive output from the output device 222.

[0212] In the embodiment of the present application, the input device 220 may be a touch screen. The user provides gesture input to the input / output controller 218 by touching the touch screen with the side of the hand. The input / output controller 218 may transmit the gesture input to the processor for processing by the processor 204.

[0213] The electronic device 101 may include one or more processors 204, which may include one or more processing units, for example, the processor 204 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors.

[0214] The controller of the processor 204 can generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0215] The processor 204 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 204 is a cache memory. The memory may store instructions or data that the processor 204 has just used or cyclically used. If the processor 204 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 204, and thus improves the efficiency of the system.

[0216] In some embodiments, the processor 204 may include one or more interfaces, such as, but not limited to, a mobile industry processor interface (MIPI), an external memory interface, and / or a universal serial bus (USB) interface.

[0217] The MIPI interface can be used to connect the processor 204 to peripheral devices such as a touch screen. The MIPI interface can include a display serial interface (DSI), etc. The processor 204 and the touch screen communicate through the DSI interface to realize the display function of the touch screen.

[0218] It is understandable that the interface connection relationship between the modules illustrated in this embodiment is only a schematic illustration and does not constitute a structural limitation on the electronic device 101. In other embodiments of the present application, the electronic device 101 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0219] Electronic device 101 can realize display function through GPU, touch screen, and application processor etc. GPU is a microprocessor for image processing, connected to touch screen and application processor. GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 204 may include one or more GPUs, which execute program instructions to generate or change display information. Specifically, one or more GPUs in processor 204 can realize the rendering task of image (such as rendering tasks related to drawing virtual ruler, distance value, length value, etc. in this application, and the rendering result is passed to application processor or other display driver, and the display screen is triggered by application processor or other display driver to display virtual ruler, distance value, length value, etc.).

[0220] A touch screen may include a display screen and associated sensors (eg, pressure sensors and touch sensors).

[0221] The display screen is used to display images, videos, etc. The display screen includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light emitting diode (QLED), etc.

[0222] The pressure sensor is used to sense the pressure signal and can convert the pressure signal into an electrical signal. There are many types of pressure sensors, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. A capacitive pressure sensor can be a parallel plate including at least two conductive materials. When a force acts on the pressure sensor, the capacitance between the electrodes changes. The electronic device 101 can determine the intensity of the pressure based on the change in capacitance. When a touch operation acts on the touch screen, the electronic device 101 can detect the intensity of the touch operation based on the pressure sensor. The electronic device 101 can also calculate the position of the touch based on the detection signal of the pressure sensor. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities can correspond to different operation instructions.

[0223] A touch sensor is also called a "touch control device". The touch sensor can be arranged on a touch screen, and the touch screen is composed of a touch sensor and a display screen. The touch sensor is used to detect a touch operation acting on or near it. The touch sensor can pass the detected touch operation to an application processor to determine the type of touch event. A visual output related to the touch operation can be provided through the display screen. In other embodiments, the touch sensor can also be arranged on the surface of the electronic device 101, which is different from the position of the display screen.

[0224] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. NPU can realize applications such as intelligent cognition of electronic device 101, for example, touch gesture category recognition tasks can be realized based on NPU.

[0225] The external memory interface can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 101. The external memory card communicates with the processor 204 through the external memory interface to implement a data storage function. For example, files such as music and videos can be stored in the external memory card.

[0226] The memory 214 can be used to store computer executable program codes, which include instructions. The memory 214 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system 206, application software 208 required for at least one function (such as an image playback function, etc.), etc. The data storage area may store data (such as image data, etc.) created during the use of the electronic device 101, etc. In addition, the memory 214 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 204 executes various functional applications and data processing of the electronic device 101 by running instructions stored in the memory 214 and / or instructions stored in a memory provided in the processor.

[0227] Among them, the method described in the embodiment of the present application can be a code stored in the memory 214 (or an external memory), and the processor 110 can obtain the code in the memory to implement the method provided by the embodiment of the present application.

[0228] The electronic device 101 may also interact with other electronic devices via the communication device 216 .

[0229] It should be understood that, alternatively or additionally, the methods described herein may be at least partially performed by one or more hardware logic components. For example, but not limitation, illustrative types of hardware logic components that may be used include field programmable gate arrays (FPGA), application specific integrated circuits (ASIC), application specific standard parts (ASSP), system on chip (SOC), and programmable logic devices (PLD). For example, functions such as detection of contact gestures and determination of gesture categories may be implemented based on hardware logic components.

[0230] In addition, at least part of the functionality of the various elements in the drawings may be distributedly performed by other elements in the drawings or entities not shown in the drawings (e.g., processors, web services, servers, applications, computing devices, etc.), without limitation here.

[0231] For ease of understanding, the virtual ruler display method provided in the embodiment of the present application is specifically described in combination with the accompanying drawings and application scenarios.

[0232] Reference Figure 3 , Figure 3 A flow chart of a virtual ruler display method provided in an embodiment of the present application is shown as follows: Figure 3 As shown, the virtual ruler display method provided in the embodiment of the present application includes:

[0233] 301. Detect a contact gesture with the touch screen.

[0234] In product usage scenarios such as conference rooms or teaching, there is sometimes a need for drawing, table drawing, and pattern measurement, so it is necessary to call out a virtual ruler on the touch screen through a specific gesture operation (such as the contact gesture in the embodiment of the present application).

[0235] In the embodiment of the present application, the electronic device 101 can detect a contact gesture with the touch screen.

[0236] Among them, the touch screen may include a touch-sensitive surface, sensor or sensor group that receives input from the user based on tactile contact, and "detecting a contact gesture with the touch screen" can be understood as detecting a contact gesture with the touch-sensitive surface on the touch screen.

[0237] First, devices and modules related to detecting contact gestures on the touch screen are described.

[0238] In a possible implementation, the touch screen can cooperate with the detection module to detect contact gestures on the touch screen. The detection module can be a program, data structure or a subset thereof stored in a memory and related to contact gesture detection, or a part of it exists in the memory in the form of a program, data structure or a subset thereof, and a part of it is in the form of a hardware logic module. The touch screen can capture contact data, and the detection module can perform various actions related to contact gesture detection according to the contact data.

[0239] In one possible implementation, the touch screen can capture the contact data (for example, an electrical signal) between the user and the touch-sensitive surface in real time, and transmit the contact data to the detection module. The detection module can perform various actions related to contact gesture detection based on the contact data.

[0240] Next, the detection module is described to realize the detection of contact gestures on the touch screen.

[0241] In one implementation, the detection module can determine the intensity and / or its changes when the user contacts the touch screen, and the size and / or its changes of the contact area based on the contact data, and then determine the gesture type of the contact gesture based on the above information.

[0242] Here, "intensity" can be understood as the force or pressure (force per unit area) of a contact (e.g., contact by the side of a hand) on the touch-sensitive surface of the touch screen. The intensity of the contact can be determined using a variety of methods and a variety of sensors or a combination of sensors. For example, below the touch-sensitive surface, the force at different points on the touch-sensitive surface of the touch screen is measured using one or more sensors adjacent to the touch-sensitive surface. In some specific implementations, the force measurements from multiple sensors can be combined (e.g., weighted average) to determine the intensity of the contact. In addition, the size of the contact area detected on the touch-sensitive surface and / or its changes, the capacitance of the touch-sensitive surface near the contact and / or its changes, and / or the resistance of the touch-sensitive surface near the contact and / or its changes can also be used to indirectly represent the force or pressure of the contact on the touch-sensitive surface.

[0243] Here, the "contact area" can be understood as the detectable area in contact with the touch-sensitive surface, which can be a continuous area or a discrete area composed of dense touch points (for example, the contact area can be a long strip area composed of a group of densely distributed contact points).

[0244] The detection module can perform various actions related to contact gesture detection based on the contact data, and further determine the gesture type of the contact gesture, the movement of the contact gesture, the stop status of the contact gesture, and the like.

[0245] For example, the detection module can determine whether contact has occurred and the type of contact gesture (for example, detecting a finger press event, or a contact event on the side of the hand), determine whether there is movement of contact and track the movement on the touch-sensitive surface (for example, detecting one or more finger drag events, or a drag event on the side of the hand), and determine whether the contact has stopped (for example, detecting a finger lift event, or detecting a lift event on the side of the hand).

[0246] Among them, the above-mentioned "movement of the contact gesture" can also be referred to as the movement of the contact area of ​​the contact gesture, and the data related to the movement of the contact area may include but is not limited to the rate (magnitude), speed (magnitude and direction) and / or acceleration (change in magnitude and / or direction) of the movement of the contact area.

[0247] Furthermore, the detection module can perform various actions related to contact gesture detection based on the contact data to determine the gesture type indicated by the contact gesture, where the gesture type includes but is not limited to clicking, double-clicking, long pressing, dragging, static contact between the side of the hand and the touch screen, dragging the side of the hand, rotating the side of the hand, double-clicking the side of the hand, and the like.

[0248] In a possible implementation, the detection module may be a pre-trained neural network model, which has the ability to identify the gesture category indicated by the contact gesture based on the contact data of the contact gesture.

[0249] 302. Based on the contact gesture indicating the contact between the side of the user's hand and the touch screen, a virtual ruler is displayed on the touch screen according to the contact area of ​​the contact gesture, wherein the contact area of ​​the contact gesture is a long strip area, and the virtual ruler is aligned with the long side of the long strip area.

[0250] In the embodiment of the present application, the detection module may determine the gesture type indicated by the contact gesture. In one implementation, the detection module may determine that the contact gesture indicates contact between the side of the user's hand and the touch screen.

[0251] In order to reduce the operational difficulty and cost of users triggering the virtual ruler display function, in an embodiment of the present application, when a contact gesture is detected indicating that the side of the user's hand is in contact with the touch screen, the display of the virtual ruler is triggered. On the one hand, only one hand of the user is required to operate, and the operation difficulty and cost are very small. On the other hand, when the user's hand is spread out, the contact area between the side of the hand and the touch screen is a flat long strip area, close to a straight line. Triggering the display of the virtual ruler based on this operation mode is more in line with the user's operating habits.

[0252] Next, the contact between the side of the user's hand and the touch screen is described with reference to the accompanying drawings.

[0253] 1. Side of the hand

[0254] The hand side in the embodiment of the present application can be understood as the hand side of the user's hand located on the little finger side when the hand is in the unfolded state (for example, including at least one of the surface of the hypothenar part of the user's hand or the surface of the little finger part).

[0255] The unfolded state of the hand may be a state in which the user's fingers (or only the little finger) are in the same plane as the palm (or are close to being in the same plane).

[0256] Reference Figure 4 , Figure 4 A user's hand gesture is shown in Figure 4In the hand gesture shown, the lower side of the hand is the side of the user's hand, which includes the surface of the little finger and the hypothenar part of the user's hand. Figure 5 , Figure 5 Another user's hand gesture is shown in Figure 5 In the hand posture shown, the lower right side of the hand is the side of the user's hand, which includes the surface of the little finger and the hypothenar part of the user's hand.

[0257] In the embodiment of the present application, the contact between the side of the user's hand and the touch screen is equivalent to the contact between the hypothenar part of the side of the user's hand and the touch screen and / or the contact between the little finger part of the side of the user's hand and the touch screen.

[0258] 2. Contact between the side of the hand and the touch screen

[0259] In the embodiment of the present application, when the contact strength and contact area of ​​the contact with the touch screen meet certain conditions, it can be considered that the side of the user's hand is in contact with the touch screen.

[0260] Reference Figure 6 and Figure 7 ,in, Figure 6 This is an illustration of the user's hand touching the touch screen from the perspective of looking directly at the touch screen. Figure 7 This is an illustration of when the side of the user's hand contacts the touch screen from the perspective of the side view of the touch screen. From the user's behavior, based on the touch strength and contact area, the side of the user's hand may be lightly swiping the touch screen (small contact area, very low contact strength), or lightly pressing the touch screen (large contact area, medium contact strength), or heavily pressing the touch screen (large contact area, large contact strength). In the embodiment of the present application, when the side of the user's hand swipes the touch screen, it can be considered that the user has accidentally touched the touch screen, and when the side of the user's hand lightly presses the touch screen or heavily presses the touch screen, it can be considered that the user needs to trigger the display of the virtual ruler.

[0261] In one implementation, when the user presses the touch screen lightly or heavily with the side of his hand for more than a certain period of time, it can be considered that the user needs to trigger the display of the virtual ruler. Figure 8 When the user presses the touch screen lightly or heavily with the side of his hand for more than X seconds, it can be considered that the user needs to trigger the display of the virtual ruler.

[0262] From the perspective of the detection module, it is necessary to perform processing and analysis based on the contact strength of the contact gesture and the contact area between the contact gesture and the touch screen. When certain conditions are met, it can be considered that the side of the user's hand is in contact with the touch screen.

[0263] Next, we introduce the conditions that the contact area needs to meet (including shape conditions and area size conditions):

[0264] In one implementation, the detection module can obtain the contact area of ​​the contact gesture, and based on the contact area satisfying preset conditions, determine that the contact gesture indicates contact between the side of the user's hand and the touch screen; the preset conditions include: the contact area is a long strip area, and the shape and area size of the long strip area conform to the characteristics of the contact area when the hypothenar eminence and / or little finger area on the side of the user's hand contacts the touch screen.

[0265] The side of the hand may include an area of ​​the hypothenar eminence and an area of ​​the little finger. When the side of the hand of the user contacts the touch screen, at least one of the following contact situations may occur due to the hand posture or the hand shape of the user:

[0266] The area of ​​the hypothenar eminence is in contact with the touch screen;

[0267] The area of ​​the pinky finger is in contact with the touch screen;

[0268] The area of ​​the hypothenar eminence and the area of ​​the little finger are in contact with the touch screen together;

[0269] Reference Fig.10 ,When the area of ​​the hypothenar part contacts the touch screen, the contact area is a flat long strip area;

[0270] When the area of ​​the little finger contacts the touch screen, the contact area is a flat long strip area, and the area size may be smaller than the area of ​​the hypothenar eminence contacting the touch screen;

[0271] Reference Fig. 9 When the area of ​​the hypothenar eminence and the area of ​​the little finger contact the touch screen together, the contact area is two long strip areas arranged in the same direction, and the long sides of the two long strip areas are in the same direction.

[0272] The detection module can obtain data of the contact area, and then analyze the shape and area size of the contact area. When the shape and area size of the contact area are consistent with the characteristics of the contact area when the hypothenar eminence and / or little finger area on the side of the user's hand contacts the touch screen, it can be determined that the contact gesture indicates contact between the side of the user's hand and the touch screen.

[0273] The shape can be understood as the contour feature of the boundary line of the contact area. When the contour shape of the boundary line of the contact area is a flat long strip, it can be considered that the shape of the contact area conforms to the shape characteristics of the contact area when the hypothenar part and / or the little finger part of the side of the user's hand contacts the touch screen;

[0274] The shape can also be understood as the distribution characteristics of the contact points included in the contact area, and whether the above shape characteristics are met can be determined by detecting the distribution shape of the touch points. (For example, if a group of densely distributed long strip touch points are detected, it can be considered that the shape of the contact area meets the shape characteristics of the contact area when the hypothenar part and / or the little finger part of the side of the user's hand contacts the touch screen).

[0275] Among them, when the area size of the contact area is within a preset range (the maximum value in the range cannot be too large, and the minimum value in the range cannot be too small, and can be set based on the characteristics of the side of the human hand), it can be considered that the area size of the contact area conforms to the area size characteristics of the contact area when the hypothenar eminence and / or little finger on the side of the user's hand contacts the touch screen.

[0276] Next, we introduce the conditions that need to be met for contact strength:

[0277] In some scenarios, the user may accidentally touch or perform a touch gesture with the touch screen, but the touch gesture is not used to trigger the virtual ruler. The display of the virtual ruler can be avoided in the above scenarios by filtering the contact strength of the touch gesture.

[0278] Specifically, when the user performs a light swiping touch on the touch screen (a touch with very small contact intensity), it can usually be considered that the user has made an accidental touch. When it is determined that the contact intensity of the touch gesture is too small, the display of the virtual ruler may not be triggered. However, when the contact gesture is a pressing gesture, the display of the virtual ruler may be triggered (provided that the contact area of ​​the contact gesture meets the above-mentioned shape and area size conditions).

[0279] Among them, the press gesture may include light press and deep press. When the contact strength increases from an intensity lower than the light press strength threshold to an intensity between the light press strength threshold and the deep press strength threshold, it can be called detecting a "light press" input on the touch surface. The contact strength increases from an intensity lower than the deep press strength threshold to an intensity higher than the deep press strength threshold, which can be called detecting a "deep press" input on the touch surface. The contact strength increases from an intensity lower than the contact detection strength threshold to an intensity between the contact detection strength threshold and the light press strength threshold, which can be called detecting a "swipe touch" input on the touch surface. When the contact strength decreases from an intensity higher than the contact detection strength threshold to an intensity lower than the contact detection strength threshold, it can be called detecting the contact being lifted off the touch surface (that is, a hand-raising gesture). In some embodiments, the contact detection strength threshold is zero. In some embodiments, the contact detection strength threshold may be greater than zero.

[0280] In one possible implementation, the detection module may be a pre-trained neural network model having the ability to identify the gesture category indicated by the contact gesture based on the contact data of the contact gesture. In an embodiment of the present application, the detection module may obtain the gesture data of the contact gesture, and determine, through a neural network, based on the gesture data, that the contact gesture indicates contact between the side of the user's hand and the touch screen.

[0281] In a possible implementation, in addition to being considered a false touch when the user swipes the touch screen, a contact gesture that is maintained for too short a time may also be considered a false touch, in which case the display of the virtual ruler may not be triggered.

[0282] In an embodiment of the present application, the maintenance time of the contact gesture can also be used as a condition for triggering the display of the virtual ruler, that is, it is necessary to detect that the maintenance time of the contact gesture is greater than a preset time. For example, the preset time can be set to 0.1S, 0.15S, 0.2S, 0.25S, 0.3S, 0.35S, 0.4S, 0.45S, 1S, or 2S.

[0283] Among them, the maintenance time of the contact gesture can be to start timing when the contact gesture is detected, or to start timing after detecting that the contact strength between the contact gesture and the touch screen is greater than a certain contact strength threshold (a threshold greater than 0); the maintenance time can be to end timing when the contact strength of the contact gesture is detected to be 0, or to end timing when the contact strength of the contact gesture is detected to be less than a certain contact strength threshold (a threshold greater than 0).

[0284] The holding time of the contact gesture may be the holding time when the contact gesture remains in a stationary state (or moves less than a certain amplitude), for example, the holding time of the contact gesture may be the holding time when the pressing gesture is in a stationary state (or moves less than a certain amplitude). It should be understood that the above-mentioned stationary state may be understood as the stationary state of the contact area of ​​the contact gesture.

[0285] In the embodiment of the present application, if it is determined that the contact gesture indicates contact between the side of the user's hand and the touch screen, a virtual ruler may be displayed on the touch screen according to the contact area of ​​the contact gesture.

[0286] Next, the virtual ruler is described:

[0287] The virtual ruler is an indication displayed on the touch screen. The display shape of the virtual ruler is similar to that of a ruler (for example, a flat rectangle or a line segment), which may include a length marking reference. The user can measure the length of a drawing straight line segment displayed on the touch screen based on the virtual ruler, or measure the distance between drawing points displayed on the touch screen. The user can also use the virtual ruler as a reference tool when drawing a straight line on the touch screen. The specific process is similar to the process of a user drawing a line with a ruler in a physical real space.

[0288] Next, the relationship between the display position of the virtual ruler and the contact area of ​​the contact gesture is described:

[0289] In order to enable the user to adjust the display position of the virtual ruler based on his or her own intention, it is necessary to establish an association between the user's contact gesture and the display position of the virtual ruler, that is, the display position of the virtual ruler needs to be based on the contact area of ​​the user's contact gesture with the touch screen as a reference.

[0290] In one implementation, since the contact gesture is the contact between the side of the hand after unfolding and the touch screen, the shape of the contact area is a flat long strip, and the direction indicated by the long side of the long strip is parallel (or nearly parallel) to the direction indicated by the four fingers (or only the little finger) of the user's hand. Fig.11 , the display direction of the virtual ruler can be made parallel or nearly parallel to the direction indicated by the long side of the strip. This display method of the virtual ruler conforms to the user's operating habits.

[0291] Among them, the display direction of the virtual ruler (which may also be referred to as the direction of the virtual ruler in the embodiment of the present application) can be understood as the direction of the edge of the virtual ruler used for measurement or line drawing reference.

[0292] In the embodiment of the present application, the virtual ruler is aligned with the long side of the elongated area, that is, in terms of display position, the virtual ruler and the elongated area need to be very close, and in terms of direction, the virtual ruler is parallel or nearly parallel to the long side of the elongated area.

[0293] Specifically, in terms of display position, the virtual ruler and the long strip area satisfy one of the following conditions:

[0294] The virtual ruler overlaps with the long strip area (for example Fig.13 or, the virtual ruler is tangent to the long strip area (for example Fig.12 or, the distance between the virtual ruler and the long strip area is less than a preset value (for example Fig.14 and Fig.15 As shown, Fig.15: shows the case when the virtual ruler is a line segment), the preset value can be 1cm, 2cm, 3cm, 4cm, 5cm, 10cm, 15cm, etc.

[0295] Specifically, in the display direction, the acute angle between the direction of the virtual ruler and the direction indicated by the long side of the long strip area is less than a preset value, and the preset value may be 1 degree, 2 degrees, 3 degrees, 4 degrees, 5 degrees, 10 degrees, etc.

[0296] Next, the display rules of the virtual ruler (including display position control, hidden triggering, fixed display, posture adjustment after fixed display, etc.) are described and explained separately:

[0297] 1. About the display position adjustment of the virtual ruler:

[0298] In the embodiment of the present application, after the display of the virtual ruler is triggered, the posture of the virtual ruler can be adjusted through specific gestures, for example, the posture (position and angle) of the virtual ruler can be changed so that the virtual ruler can measure different elements (the length of a drawing line or the distance between drawing points). In this embodiment, in combination with the user's contact gesture, the movement of the virtual ruler can be driven by the movement of the contact area on the side of the user's hand.

[0299] Specifically, the detection module may detect the movement of the contact area of ​​the contact gesture, and adjust the display position of the virtual ruler so that the display position of the virtual ruler follows the contact area.

[0300] The so-called movement may be understood as a change in position and / or direction;

[0301] Among them, the so-called following can be understood as keeping the display position of the virtual ruler aligned with the contact area of ​​the contact gesture. As the contact area of ​​the contact gesture moves, the display position and direction of the virtual ruler will also change, and the display area of ​​the virtual ruler will always remain aligned with the long side of the contact area.

[0302] Reference Fig.16 and Fig.17 , the contact area of ​​the touch gesture moves, and then the display position and direction of the virtual ruler will also change along with the contact area.

[0303] 2. About cancelling the display of the virtual ruler:

[0304] In some scenarios, when the user does not need to use the virtual ruler, a certain gesture can be used to trigger the touch screen to hide the display of the virtual ruler. The gesture can be a hand-raising gesture. In the embodiment of the present application, refer to Fig.18The detection module can detect a hand-raising gesture of the user, and hide the display of the virtual ruler on the touch screen in response to the hand-raising gesture of the user, wherein the hand-raising gesture can be understood as the side of the hand leaving the touch screen, or the contact strength between the side of the hand and the touch screen is less than a certain threshold, or the contact area between the side of the hand and the touch screen is less than a preset value.

[0305] 3. Fixed display of virtual ruler

[0306] In some scenarios, when the user raises his hand, he still needs to use the virtual ruler, so it is necessary to trigger the fixed display of the virtual ruler on the touch screen based on a certain gesture.

[0307] In a possible implementation, the user can trigger the fixed display of the virtual ruler on the touch screen by tapping the touch screen with the side of the hand for a preset number of times, referring to Fig.30 As shown, the detection module can indicate that the side of the user's hand has tapped the touch screen for a preset number of times based on the contact gesture, and after detecting the user's hand-raising gesture, maintain the display of the virtual ruler on the touch screen (for example, referring to Fig.31 shown).

[0308] Optionally, the preset number of taps needs to occur within a preset time, and the preset time is a shorter time (for example, 0.1S, 0.2S, 0.3S, 0.4S, 0.5S, etc.);

[0309] The tapping positions of the preset number of taps on the touch screen need to be kept consistent, or a slight deviation is allowed.

[0310] The preset number of times may be 2 times, 3 times, etc. For example, when the side of the hand is tapped twice in succession, the virtual ruler may be fixedly displayed on the touch screen.

[0311] 4. About the posture adjustment after the virtual ruler is fixed

[0312] In one implementation, when the virtual ruler is fixedly displayed on the touch screen, the display posture of the virtual ruler can be adjusted through certain gesture operations (the display posture can include display position and / or display direction). The detection module can detect a user's selection instruction for the virtual ruler, and in response to the selection instruction, a trigger control can be displayed, and the trigger control is used to instruct at least one of the following operations to be performed on the virtual ruler: a deletion operation, a position adjustment operation, and a rotation operation.

[0313] For example, refer to Fig.32 The user can trigger the selection of the virtual ruler by clicking on the fixedly displayed virtual ruler. When the selection is made, the touch screen will pop up a trigger control. For example, the trigger control may be a prompt for indicating a delete operation for the virtual ruler (for example Fig.33 The "Delete" prompt shown in the Fig.35 The user can delete the virtual ruler by clicking the “Delete” prompt or dragging the virtual ruler to the trash can icon).

[0314] For example, the user can trigger the selection of the virtual ruler by clicking on the fixedly displayed virtual ruler. When the selection is made, the touch screen will pop up a trigger control, for example, the trigger control may be a prompt for indicating the position adjustment operation for the virtual ruler (for example, Fig.33 The user can click the "posture adjustment" control to enter the posture adjustment mode for the virtual ruler. For example, in this mode, the user can adjust the display posture of the virtual ruler by touching the side of the hand with the touch screen. The adjustment method can refer to the relevant description of the display position control of the virtual ruler in the above embodiment, which will not be repeated here. For example, refer to Fig.34 If you need to rotate the virtual ruler, you can use two fingers to click on two different positions (on the virtual ruler or within a preset distance of the virtual ruler), and rotate the two touch points clockwise or counterclockwise at the same time to rotate the virtual ruler. Optionally, when operating the virtual ruler to rotate, the angle between the virtual ruler before and after the rotation can be displayed in real time.

[0315] It should be understood that after the user selects the virtual ruler, the display posture of the virtual ruler can be adjusted directly through gesture operations.

[0316] In one implementation, referring to Fig.36 In the case where the virtual ruler is not selected, if a contact gesture between the side of the user's hand and the touch screen is detected (satisfying the conditions for triggering the display of the virtual ruler), another virtual ruler can be displayed additionally, and the new virtual ruler can also move with the translation of the receiving area on the side of the hand. In addition, the new virtual ruler can also be fixedly displayed by a continuous tapping gesture.

[0317] In one implementation, if the touch screen exits the writing and drawing interface and enters the display interface (for example, displaying the projected content of other electronic devices), the virtual ruler displayed on the writing and drawing interface will also disappear with the exit of the writing and drawing interface and will not affect the display of the projected content.

[0318] In the embodiment of the present application, the virtual ruler is given a corresponding rotation or movement operation by rotating or moving the side of the hand, and gestures such as tapping to fix the display of the virtual ruler are in line with the user's operating habits and can greatly improve operating efficiency.

[0319] Next, the functional implementation of the virtual ruler (including length measurement, distance measurement, and line drawing) is described in combination with the interaction between the user and the virtual ruler:

[0320] 1. Length measurement function

[0321] In the embodiment of the present application, the virtual ruler can be used to measure the length of a drawing straight line segment displayed on the touch screen.

[0322] In one possible implementation, refer to Fig.19 At least one drawing straight line segment can be displayed on the touch screen. When the user wants to measure the length of one of the drawing straight line segments, the display position of the virtual ruler can be adjusted by moving the contact gesture on the touch screen, so that the display position of the virtual ruler is close to or accurately fits the drawing straight line segment to be measured.

[0323] In one implementation, referring to Fig. 20 When the display position of the virtual ruler accurately fits the drawing straight line segment to be measured, the detection module can obtain the length value of the drawing straight line segment to which the virtual ruler fits and display the length value.

[0324] However, when the user may not be able to accurately fit the drawing straight line segment to be measured, or the operation required to fit the drawing straight line segment to be measured is difficult, the length of the drawing straight line segment can be measured and the length value displayed when the virtual ruler is located near the drawing straight line segment to be measured.

[0325] Specifically, when the detection module detects that there is a drawing straight line segment on the touch screen that has a posture association with the virtual ruler, the length value of the drawing straight line segment is displayed. The so-called posture association can be understood as that the virtual ruler and the drawing straight line segment are highly fitted. For example, the posture association can be expressed by a direction difference and a distance value. When the direction difference between the drawing straight line segment and the virtual ruler is less than a preset value (for example, less than 20 degrees, 25 degrees, 30 degrees, 35 degrees, 40 degrees, 45 degrees, etc.); and / or the distance value between the drawing straight line segment and the virtual ruler is less than a preset value (for example, 1 cm, 2 cm, 3 cm, etc.), it can be considered that there is a drawing straight line segment on the touch screen that has a posture association with the virtual ruler.

[0326] The distance may be understood as the distance between the points on the virtual ruler closest to the straight line segment, or the average value of the distances between the points on the virtual ruler and the straight line segment, or other methods that can characterize the distance between the virtual ruler and the straight line segment may be used.

[0327] It should be understood that when the detection module detects that there are multiple drawing straight line segments with gesture associations with the virtual ruler on the touch screen, the length value of the drawing straight line segment with the greatest gesture association can be obtained and displayed.

[0328] For example, there are multiple drawing straight line segments (drawing straight line segment A, drawing straight line segment B) associated with postures, the distance value between the virtual ruler and the drawing straight line segment A is A1, the distance value between the virtual ruler and the drawing straight line segment B is B1, the direction difference between the virtual ruler and the drawing straight line segment A is A2, and the direction difference between the virtual ruler and the drawing straight line segment B is B2. When A1 is equal to A2 and B1 is greater than B2, the degree of association between the drawing straight line segment B and the virtual ruler in posture association is the largest. When A1 is greater than A2 and B1 is equal to B2, the degree of association between the drawing straight line segment B and the virtual ruler in posture association is the largest. In addition, by calculating the distance value and the direction difference, a feature quantity that can characterize the distance value and the direction difference can be obtained. The feature quantity is positively correlated with the distance value and the direction difference. The larger the feature quantity, the smaller the degree of association of the posture association.

[0329] In addition, when the detection module detects that there is a drawing straight line segment on the touch screen that is associated with the virtual ruler in a posture, and the display position of the virtual ruler does not accurately fit the drawing straight line segment, the display position of the virtual ruler can be adjusted based on the position of the drawing straight line segment so that the display position of the virtual ruler fits the drawing straight line, that is, a display effect similar to magnetic attraction is achieved.

[0330] Wherein, adjusting the display position of the virtual ruler may include adjusting the position and display direction of the virtual ruler.

[0331] Specifically, when measuring with the virtual ruler, the drawing straight line segment within the predefined range of the virtual ruler has a magnetic effect on the virtual ruler. Under the influence of the magnetic effect, the virtual ruler will automatically change its posture and automatically fit the drawing straight line segment.

[0332] For example, refer to Fig.21 , Fig.21 The touch screen shows two straight lines. Fig.22a , the contact area between the side of the user's hand and the touch screen moves to the left drawing straight line segment (hereinafter referred to as drawing straight line segment 1), and the virtual ruler moves with the user's gesture. Figure 22b When the virtual ruler moves to the left of the drawing straight line segment, the virtual ruler will automatically adjust its display position to fit the drawing straight line segment 1. Fig.22c , when the user's hand side continues to move to the right in contact with the touch screen, the virtual ruler will cancel the magnetic effect, that is, it will not fit the drawing straight line segment 1, but continue to move with the user's gesture. Fig.22d , the contact area between the side of the user's hand and the touch screen moves to the lower right. When the virtual ruler moves to the vicinity of the lower right drawing straight line segment (drawing straight line segment 2), the virtual ruler will automatically adjust the display position to a position that fits the drawing straight line segment 2.

[0333] In an embodiment of the present application, if the drawing line is near the virtual ruler, the drawing line will have a magnetic effect on the virtual ruler and automatically measure the length of the drawing line, thereby avoiding the process of manually fitting the virtual ruler and the drawing line, reducing the difficulty of measurement, and avoiding the problem of inaccurate fitting during manual fitting, thereby improving the accuracy of the measurement.

[0334] The above introduces the length measurement function of the virtual ruler. Next, we will introduce the distance measurement function of the virtual ruler:

[0335] 2. Distance measurement function

[0336] In the embodiment of the present application, the virtual ruler can be used to measure the distance between drawing points displayed on the touch screen.

[0337] In one possible implementation, refer to Fig.23 At least one drawing line segment may be displayed on the touch screen. When the user wants to measure the distance between two points on the drawing line segment, the user may adjust the display position of the virtual ruler by moving the touch gesture on the touch screen so that the virtual ruler intersects with the two points (or multiple points) on the drawing line segment. Fig.24 The detection module can obtain the distance value between two intersection points (or at least two intersection points among multiple intersection points) and display the distance value. Specifically, the detection module can detect that the drawing line segment on the touch screen has a first intersection point and a second intersection point with the virtual ruler, and display the distance value between the first intersection point and the second intersection point.

[0338] The above introduces the distance measurement function of the virtual ruler. Next, we will introduce the line drawing function of the virtual ruler:

[0339] 3. Line drawing function

[0340] In the embodiment of the present application, the virtual ruler is used as a reference tool when drawing a straight line on the touch screen.

[0341] Similar to the process of drawing a line with a ruler in physical space, the virtual ruler can be used as a reference tool when the user draws a straight line on the touch screen (refer to Fig.25 ), wherein the so-called reference tool can be understood as when the drawing trajectory fits the ruler, the drawing trajectory is restricted to a straight line due to the obstruction of the ruler.

[0342] Unlike a ruler in the physical space, a ruler in the physical space can be used as a reference tool when drawing a straight line based on its own structural characteristics (when placed on a drawing plane, when a user uses a brush to paint and the drawing trajectory intersects with the ruler, the drawing trajectory will be restricted to a straight line by the ruler, and a straight line can be drawn), while a virtual ruler is merely an indication displayed on the touch screen. Therefore, it is necessary to identify the positional relationship between the user's drawing trajectory (or drawing position) and the virtual ruler, and restrict the drawing trajectory based on the positional relationship between the drawing position and the virtual ruler.

[0343] Next, we will describe how to limit the line drawing trajectory based on the positional relationship between the line drawing position and the virtual ruler, thereby realizing the function of drawing a straight line based on the virtual ruler:

[0344] On the one hand, when writing or drawing within a certain range of the virtual ruler (without raising the hand), the range of writing or drawing can be constrained not to exceed the virtual ruler, that is, writing or drawing can only be done on one side of the virtual ruler. When the writing or drawing range exceeds the virtual ruler, a straight line that fits the virtual ruler (the side where writing or drawing is possible) is displayed, which is equivalent to the virtual ruler playing a similar function to a ruler in physical space (refer to Fig. 27 As shown, Fig. 27 The left picture shows the actual line drawing trajectory of the user, and the right picture shows the drawing straight line after correction).

[0345] On the other hand, the drawing trajectory can be corrected to a straight line in response to the drawing behavior within the predefined range of the virtual ruler (refer to Fig.26 As shown, Fig.26 The left picture in the figure shows the actual drawing trajectory of the user, and the right picture shows the drawing straight line after correction). In the above method, when the user draws along the virtual ruler, a straight line will be automatically drawn.

[0346] Specifically, the detection module can detect a line drawing gesture on the touch screen, based on that the virtual ruler is displayed on the touch screen, and the distance between the line drawing position of the line drawing gesture and the virtual ruler is within a preset distance (for example, 1 cm, 2 cm, 3 cm, 4 cm, 5 cm, etc.), a drawing straight line segment is displayed according to the line drawing position of the line drawing gesture, wherein the drawing straight line segment is parallel to the virtual ruler.

[0347] In order to obtain a line of expected length, when drawing along the ruler, the drawing straight line between the first and last drawing points can be calculated and displayed, and the length of the drawing straight line can be displayed. In a possible implementation, the drawing line position includes a starting point position and an ending point position, and the drawing straight line segment is a line segment between the starting point position and the ending point position (refer to Fig.28 As shown, Fig.28The left picture in the figure is the actual line drawing trajectory of the user, and the right picture is the drawing straight line displayed after correction), and the detection module can display the line drawing length based on the real-time line drawing position, and the line drawing length is the distance value between the starting point position and the real-time line drawing position.

[0348] That is to say, if the length of the drawn line segment exceeds the expected value, you can draw it back, which will have a deletion effect. As long as the distance between the end point and the starting point is the expected value, the generated line segment is the expected value, and the drawn curve (the distance between the curve and the virtual ruler is within the preset range) will be automatically corrected to a straight line.

[0349] like Fig.29 As shown in the figure, suppose that when drawing, the user needs to draw a line segment with a length of 15 cm (that is, the expected length of the line segment is 15 cm), starting from the starting point. However, when the user draws a line segment on the screen, it is not possible to draw a 15 cm line segment as accurately as drawing a line on a piece of white paper along a ruler. It is very likely that the length of the drawn line segment exceeds 15 cm. For example, after the pen trajectory is corrected (implemented in the background, the user sees a straight line), the length of the line segment A1 is 25 (as shown in the figure). Fig.29 ).

[0350] At this time, the general solution is to erase part of it or redraw it, but it is difficult to get a line segment of accurate length even by erasing or redrawing it. Fig.29 As shown in the right picture in , if you move the stylus in the opposite direction, the end point will move back (the starting point is the starting point of the stylus drawing, and the end point is the final position of the stylus when drawing. During the drawing process, the end point can be regarded as dynamically moving until the drawing is completed, and the position where the stylus leaves is the final end point).

[0351] exist Fig.29 In the right figure in , when the stylus moves in the opposite direction, the virtual ruler will correct the straight line / curve drawn by the stylus to a straight line. Therefore, in the line segment A2 part, it is equivalent to drawing twice. The first time (forward drawing) draws the trajectory line. When the second time (reverse drawing) repeats the first drawing, the repeated line segment A2 is deleted, that is, part of the first straight line is erased, and finally only the line segment A3 between the starting point position and the end point position is retained. When marking the length within the line segment, only the length of the line segment A3 between the starting point position and the end point position is marked. After deleting the line segment, the stylus can be redrawn when it is redrawn. Therefore, the expected length of 15cm can be obtained by (repeatedly) adjusting the position of the end point.

[0352] It should be understood that in order to avoid the intersection of two line segments being deleted when drawing geometric figures (for example, the intersection of the diagonals of a square will be drawn twice when drawing two different diagonals), the deletion effect of repeated line drawing can be enhanced by adding a time judgment mechanism, and only when the same line segment is drawn twice within a predetermined time will the deletion effect be achieved. Or it can be achieved through other judgment mechanisms, such as the direction of the stylus movement twice is opposite to meet the deletion judgment and other directional mechanisms.

[0353] In the embodiments of the present application, the two functions of a ruler: measuring and drawing a straight line, are converted into intelligent operations suitable for a virtual interface. When measuring, the measured value is automatically displayed, and when drawing a straight line, the length value is displayed. This removes the limitations of the physical world, saves the user's reading time, and greatly improves user efficiency.

[0354] The embodiment of the present application provides a method for displaying a virtual ruler, including: detecting a contact gesture with the touch screen; based on the contact gesture indicating the contact between the side of the user's hand and the touch screen, displaying a virtual ruler on the touch screen according to the contact area of ​​the contact gesture, wherein the contact area of ​​the contact gesture is a long strip area, and the virtual ruler is aligned with the long side of the long strip area. When the contact gesture indicates the contact between the side of the user's hand and the touch screen, the display of the virtual ruler is triggered. On the one hand, only one hand of the user is required to operate, and the operation difficulty and cost are very small. On the other hand, when the user's hand is unfolded, the contact area between the side of the hand and the touch screen is a flat long strip area, close to a straight line. Triggering the display of the virtual ruler based on this operation mode is more in line with the user's operation habits.

[0355] Reference Fig.37a , Fig.37a A schematic diagram of the process of an object replication method provided in an embodiment of the present application is shown as follows: Fig.37a As shown, an object replication method provided in an embodiment of the present application includes:

[0356] 3701. A drag gesture directed to a target object displayed on the touch screen is detected.

[0357] There are gestures related to copy operations on mainstream terminal devices (such as smartphones and tablets). For example, the existing operation logic includes pinching three fingers together to complete copying and releasing three fingers to paste. At the same time, the industry has a set of basic logic for copying and pasting: select the content, right-click or use a special button to enter the semantic menu, click the copy button, move to the target location and call up the semantic menu, and click the paste button. It can be seen that the mainstream method must specifically select the element as a prerequisite before copying and pasting. There are many operation steps, and the operation cost of copying and pasting is very high.

[0358] In the embodiment of the present application, when performing a copy and paste operation, the copied object can be dragged to the location where it needs to be pasted, and the location where the hand is raised is the pasting location.

[0359] The following describes how to select the copy object:

[0360] 1. Through long press gesture:

[0361] In the embodiment of the present application, when a long press gesture directed to the target object is detected, a copy function directed to the target object can be enabled. The long press gesture can be a two-finger long press gesture, for example, Fig.37b .

[0362] The target object is a drawing object displayed on the touch screen, for example, a character, a pattern, a stroke, etc.

[0363] Among them, the long press gesture may include light press and deep press, and when the contact strength increases from an intensity lower than the light press strength threshold to an intensity between the light press strength threshold and the deep press strength threshold, it can be called detecting a "light press" input on the touch surface. The contact strength increases from an intensity lower than the deep press strength threshold to an intensity higher than the deep press strength threshold, which can be called detecting a "deep press" input on the touch surface. The contact strength increases from an intensity lower than the contact detection strength threshold to an intensity between the contact detection strength threshold and the light press strength threshold, which can be called detecting a "swipe touch" input on the touch surface. When the contact strength decreases from an intensity higher than the contact detection strength threshold to an intensity lower than the contact detection strength threshold, it can be called detecting the contact being lifted off the touch surface (that is, a hand-raising gesture). In some embodiments, the contact detection strength threshold is zero. In some embodiments, the contact detection strength threshold may be greater than zero.

[0364] In a possible implementation, in addition to being considered a false touch when the user swipes the touch screen, a long press gesture that is maintained for too short a time may also be considered a false touch, in which case the display of the virtual ruler may not be triggered.

[0365] In an embodiment of the present application, the maintenance time of the long press gesture can also be used as a condition to trigger the copy function for the target object, that is, it is necessary to detect that the maintenance time of the long press gesture is greater than the preset time. For example, the preset time can be set to 0.1S, 0.15S, 0.2S, 0.25S, 0.3S, 0.35S, 0.4S, 0.45S, 1S, 2S.

[0366] Among them, the maintenance time of the long press gesture can be to start timing when the long press gesture is detected, or to start timing after detecting that the contact strength between the long press gesture and the touch screen is greater than a certain contact strength threshold (a threshold greater than 0); the maintenance time can be to end timing when the contact strength of the long press gesture is detected to be 0, or to end timing when the contact strength of the long press gesture is detected to be less than a certain contact strength threshold (a threshold greater than 0).

[0367] The holding time of the long press gesture may be the holding time when the long press gesture remains stationary (or moves less than a certain amplitude), for example, the holding time of the long press gesture may be the holding time when the pressing gesture is stationary (or moves less than a certain amplitude). It should be understood that the above-mentioned stationary may be understood as the contact area of ​​the long press gesture not moving or moving very little.

[0368] Among them, the so-called enabling of the copy function for the target object can be understood as activating the copy state. When the copy state is not activated, the display elements on the writing and drawing interface cannot be copied. When other gesture operations are used, they are operations on the display elements themselves. Only after enabling the copy function for the display elements, the copy state of the display elements will be activated and can they respond to the subsequent copy and paste related operations.

[0369] The so-called long press gesture for the target object may be understood as the contact area of ​​the long press gesture covering the target object.

[0370] In a possible implementation, when the detection module detects a long press gesture, if there is no display element in the contact area of ​​the long press gesture, the copy instruction will not be triggered (that is, the copy function for the display element will not be enabled). If there is a display element (such as a target object), the display element can be selected and the display elements within the range are placed in a copy activation state (that is, the copy function for the target object is enabled).

[0371] Taking the long press gesture as a two-finger long press gesture as an example, if the touch gesture is a two-finger long press gesture and there is a display element in the contact area, the display element is selected and placed in a copy activation state (that is, the copy function for the target object is enabled).

[0372] In one implementation, the so-called long press gesture directed to the target object may also be understood as a contact area of ​​the long press gesture being within a preset distance around the target object.

[0373] In the embodiments of the present application, the contact area for a long-press gesture may be associated with a fixed predefined range, or may be extended based on the predefined range. For example, if a pattern is found within a preset distance from the boundary of the predefined range, the range can be extended to the newly discovered pattern, and the newly discovered pattern can also apply to this extension. For example, referring to Fig.43 , when long-pressing the character "ox" with two fingers, a circular area is determined according to the predefined range, and the patterns within the circular area are selected, that is, the character "ox" is selected. For example, referring to Fig.44 , boundary vicinity determination can also be performed. Within a preset distance (such as 5 mm) from the circular boundary, check if there is a pattern, then select at least one stroke of the character "year", then select the entire character "year", and then select the characters "big" and "lucky" in sequence. In this way, the four characters "happy year of the ox" can be selected at once. When performing range extension, the extended shape can be adapted to the predefined shape. For example, perform a translation of the circle to finally obtain an ellipse. It can also be adapted to the preset distance of the boundary determination. For example, an irregular pattern adapted to the boundary of the character.

[0374] It should be understood that when selecting a pattern that does not need to be copied, this extension can be interrupted. For example, referring to Fig.45 , an extra finger can be used to slide to divide the selected area into two parts, and the part of the area relative to the two fingers is deselected. As shown in the following figure, after splitting, the character "lucky" is not selected, so when the two fingers move, only "big year of the ox" is copied.

[0375] 2. Through a single-click gesture:

[0376] In the embodiments of the present application, when a single-click gesture for the target object is detected, the copy function for the target object can be enabled.

[0377] In a possible implementation, referring to Fig.40 , detecting a single-click gesture for the target object can be understood as detecting a single-click gesture whose contact area with the touch screen covers the target object.

[0378] The difference between single-click selection for copying and long-press selection for copying is that single-click selection selects one object, while two-finger selection selects the display elements within the predefined range. This is crucial for copying on a writing board. When writing or drawing on a writing board, since characters are composed of strokes and the strokes are independent during writing or drawing, one stroke is a pattern, and a character is composed of several patterns (strokes). Therefore, when selecting by single-click, only one stroke (ignoring the case of connected strokes) can be selected, and only the selected stroke (such as the horizontal stroke in the above figure) can be copied, and the entire character cannot be copied completely. When selecting with two fingers, the patterns within the two-finger area are selected, and the entire character can be selected, so the entire character can be copied completely.

[0379] In addition, refer to Fig.46 and Fig.47 You can also select the target object for copying based on the selection control displayed on the touch screen (which can be physical or virtual). After selecting the button, you can select the icon. After selecting the icon, you can select the target object, thereby enabling the copy function for the target object.

[0380] The above describes how to enable the copy function for the target object. Next, we will describe how to copy the target object:

[0381] In the embodiment of the present application, the user performs a drag gesture on the touch screen for the target object, and the end point of the drag gesture (that is, the position of the hand-raising gesture) can be the position where the paste operation for the target object needs to be performed. The drag gesture can be a two-finger drag gesture.

[0382] In a possible implementation, the user triggers a copy function for the target object through a long press gesture, and the drag gesture may be a gesture of keeping the long press gesture in contact with the touch screen and moving on the touch screen.

[0383] In one possible implementation, refer to Fig.41 The user triggers the copy function for the target object through a click gesture, and then can use the drag gesture to perform the copy operation, that is, the selection operation and the copy operation are separated. At this time, the starting point of the touch area of ​​the drag gesture may not be (or may be) near the display position of the target object, but can be dragged at any display position on the touch screen, which can make the copied target object to be copied and drag the target object to slide along the direction of the two-finger sliding.

[0384] 3702. In response to the drag gesture, display a mirror image of the target object on the touch screen, and update a display position of the mirror image in real time according to the drag gesture so that the mirror image moves along with the drag gesture;

[0385] Reference Fig.38 In an embodiment of the present application, after the detection module detects a drag gesture directed to the target object, the target object has a tendency to be dragged away. After detecting the instruction of the drag gesture, a copy operation of the target object is performed to copy the target object to generate a copy (or mirror image), and the mirror image moves with the movement of the drag gesture, and the main body of the target object remains in the original position without sliding.

[0386] In a possible implementation, in order to indicate a duplicate, the mirror image of the target object may be presented in a semi-virtualized state.

[0387] 3703. A hand-raising gesture is detected.

[0388] For the specific description of the hand-raising gesture, please refer to the relevant description in the above embodiment, which will not be repeated here.

[0389] 3704. In response to the hand-raising gesture, the mirror image is fixedly displayed at the display position where the mirror image is located.

[0390] Reference Fig.39 and Fig.42 In an embodiment of the present application, after the drag gesture slides to the expected position, a hand-raising gesture is performed, that is, the hand and the display screen are out of contact or the contact strength is less than a threshold value, triggering the paste operation, that is, pasting the target object at the position of the hand-raising gesture, thereby completing the copy and paste process.

[0391] In addition, the above-mentioned gesture rules for copying operations are not only applicable to the writing and drawing elements on the writing and drawing interface, but also applicable to the copying of text on the display interface. Fig.48 , after selecting the text, long press the selected text (for example Fig.48 Drag to create a copy, and when you release the copy, it appears where your fingers left.

[0392] The embodiment of the present application provides an object copying method, the method comprising: detecting a drag gesture for a target object displayed on a touch screen; in response to the drag gesture, displaying a mirror image of the target object on the touch screen, and updating the display position of the mirror image in real time according to the drag gesture so that the mirror image moves with the drag gesture; detecting a hand-raising gesture; in response to the hand-raising gesture, displaying the mirror image fixedly at the display position where the mirror image is located. The method of copying and pasting by dragging gesture is simple and intuitive, which reduces the number of steps required for existing copying and pasting and shortens the process of copying and pasting.

[0393] In addition, the copy activation state is stimulated based on a long press gesture, so that the copying method can be performed. The gesture is simple, commonly used, easy to operate and consistent with user cognition.

[0394] Reference Fig.49 , Fig.49 A schematic diagram of the structure of a virtual ruler display device provided in an embodiment of the present application is applied to an electronic device, wherein the electronic device includes a touch screen, and the device 4900 includes:

[0395] A detection module 4901, configured to detect a contact gesture with the touch screen;

[0396] For the specific description of detection module 4901, please refer to the description of step 301, which will not be repeated here.

[0397] Display module 4902 is used to indicate the contact between the side of the user's hand and the touch screen based on the contact gesture, and display a virtual ruler on the touch screen according to the contact area of ​​the contact gesture, wherein the contact area of ​​the contact gesture is a long strip area, and the virtual ruler is aligned with the long side of the long strip area.

[0398] For the detailed description of the display module 4902, please refer to the description of step 302, which will not be repeated here.

[0399] In a possible implementation, the hand side is the hand side of the user's hand located on the little finger side when the hand is in an unfolded state.

[0400] In a possible implementation, the contact area includes:

[0401] The contact area between the hypothenar part of the side of the user's hand and the touch screen; or,

[0402] The contact area between the little finger of the user's hand and the touch screen.

[0403] In a possible implementation, the device further includes:

[0404] The determination module 4904 is used to determine that the contact gesture indicates contact between the side of the user's hand and the touch screen based on the contact area of ​​the contact gesture satisfying a preset condition; the preset condition includes:

[0405] The contact area is a long strip area, and the shape and area size of the long strip area conform to the characteristics of the contact area when the hypothenar part and / or the little finger part of the side of the user's hand contacts the touch screen.

[0406] In a possible implementation, the device further includes:

[0407] An acquisition module 4903, used for acquiring gesture data of the contact gesture;

[0408] The determination module 4904 is further configured to determine, based on the gesture data and through a neural network, that the contact gesture indicates contact between the side of the user's hand and the touch screen.

[0409] In a possible implementation, the virtual ruler is aligned with the long side of the long strip area, including:

[0410] The acute angle between the direction of the virtual ruler and the direction indicated by the long side of the long strip area is smaller than a preset value; and the virtual ruler and the long strip area satisfy one of the following conditions:

[0411] The virtual ruler overlaps with the long strip area; or, the virtual ruler is tangent to the long strip area; or, the distance between the virtual ruler and the long strip area is less than a preset value.

[0412] In a possible implementation, the detection module 4901 is configured to detect that the duration of the contact gesture is greater than a preset time before a virtual ruler is displayed on the touch screen in the contact area according to the contact gesture.

[0413] In a possible implementation, the detection module 4901 is used to detect the movement of the contact area of ​​the contact gesture;

[0414] The display module 4902 is used to adjust the display position of the virtual ruler so that the display position of the virtual ruler follows the contact area.

[0415] In a possible implementation, the detection module 4901 is used to detect a hand-raising gesture of the user;

[0416] The display module is used to hide the display of the virtual ruler on the touch screen in response to the hand-raising gesture.

[0417] In a possible implementation, indicating the contact between the side of the user's hand and the touch screen based on the contact gesture includes:

[0418] The side of the hand of the user taps the touch screen a preset number of times based on the contact gesture indication;

[0419] After displaying the virtual ruler on the touch screen, the method further includes:

[0420] Detecting the user's hand-raising gesture;

[0421] In response to the hand-raising gesture, the display of the virtual ruler is maintained on the touch screen.

[0422] In a possible implementation, the detection module 4901 is used to detect a selection instruction of the user for the virtual ruler;

[0423] The display module is used to display a trigger control in response to the selection instruction, and the trigger control is used to instruct to perform at least one of the following operations on the virtual ruler:

[0424] Deletion, repositioning, and rotation.

[0425] In a possible implementation, the virtual ruler is used to measure the length of a drawing straight line segment displayed on the touch screen; or,

[0426] The virtual ruler is used to measure the distance between drawing points displayed on the touch screen.

[0427] In a possible implementation, the detection module 4901 is used to detect that there is a drawing straight line segment on the touch screen that is associated with the virtual ruler in a gesture;

[0428] The display module is used to display the length value of the drawing straight line segment; the existence posture association includes:

[0429] The direction difference with the virtual ruler is less than a preset value; and / or,

[0430] The distance value between the virtual ruler and the virtual ruler is smaller than a preset value.

[0431] In a possible implementation, the display module 4902 is used to adjust the display position of the virtual ruler according to the position of the line segment to be measured, so that the display position of the virtual ruler fits the drawing straight line.

[0432] In a possible implementation, the detection module 4901 is configured to detect, after the virtual ruler is displayed on the touch screen, that a first intersection point and a second intersection point exist between the drawing line segment on the touch screen and the virtual ruler;

[0433] The display module 4902 is used to display the distance value between the first intersection point and the second intersection point.

[0434] In a possible implementation, the virtual ruler is used as a reference tool when drawing a straight line on the touch screen.

[0435] In a possible implementation, the detection module 4901 is used to detect a line drawing gesture on the touch screen;

[0436] The display module 4902 is used to display a drawing straight line segment according to the drawing position of the line drawing gesture based on the virtual ruler being displayed on the touch screen and the distance between the drawing position of the line drawing gesture and the virtual ruler being within a preset distance, wherein the drawing straight line segment is parallel to the virtual ruler.

[0437] In a possible implementation, the line drawing position includes a starting point position and an ending point position, and the drawing straight line segment is a line segment between the starting point position and the ending point position.

[0438] In a possible implementation, the line drawing position includes a starting point position and a real-time line drawing position, and the method further includes:

[0439] Based on the real-time line drawing position, the line drawing length is displayed, and the line drawing length is the distance value between the starting point position and the real-time line drawing position.

[0440] The embodiment of the present application provides a virtual ruler display device, including: a detection module 4901, for detecting a contact gesture with the touch screen; a display module 4902, for displaying a virtual ruler on the touch screen according to the contact area of ​​the contact gesture indicating the contact between the side of the user's hand and the touch screen, wherein the contact area of ​​the contact gesture is a long strip area, and the virtual ruler is aligned with the long side of the long strip area. When the contact gesture indicates the contact between the side of the user's hand and the touch screen, the display of the virtual ruler is triggered. On the one hand, only one hand of the user is required to operate, and the operation difficulty and cost are very small. On the other hand, when the user's hand is unfolded, the contact area between the side of the hand and the touch screen is a flat long strip area, close to a straight line. Triggering the display of the virtual ruler based on this operation mode is more in line with the user's operation habits.

[0441] Reference Fig.50 , Fig.50 A schematic diagram of the structure of an object replication device provided in an embodiment of the present application is shown as follows: Fig.50 As shown, the device 5000 includes:

[0442] The detection module 5001 is used to detect a drag gesture for a target object displayed on the touch screen; the detection module is also used to detect a hand-raising gesture after the display module displays a mirror image of the target object;

[0443] For the specific description of the detection module 5001, please refer to the description of step 3701 and step 3703, which will not be repeated here.

[0444] The display module 5002 is used to display the mirror image of the target object on the touch screen in response to the dragging gesture, and to update the display position of the mirror image in real time according to the dragging gesture so that the mirror image moves along with the dragging gesture; the display module is also used to display the mirror image fixedly at the display position where the mirror image is located in response to the hand-raising gesture.

[0445] For the specific description of the display module 5002, please refer to the description of step 3702 and step 3704, which will not be repeated here.

[0446] In a possible implementation, the device further includes:

[0447] The enabling module 5003 is configured to enable a copy function for the target object when at least one of the following gestures is detected before the drag gesture for the target object displayed on the touch screen is detected:

[0448] A long press gesture directed to the target object is detected; or,

[0449] A single-tap gesture directed to the target object is detected.

[0450] In a possible implementation, the detection module 5001 is used to detect a long press gesture in which the contact area with the touch screen covers the target object or is within a preset distance around the target object.

[0451] In a possible implementation, the drag gesture is a gesture of keeping the long press gesture in contact with the touch screen and moving on the touch screen.

[0452] In a possible implementation, the long press gesture is a two-finger long press gesture.

[0453] In a possible implementation, the detection module 5001 is used to detect a single-click gesture in which the contact area with the touch screen covers the target object.

[0454] The embodiment of the present application provides an object copying device, including: a detection module 5001, which is used to detect a drag gesture for a target object displayed on a touch screen; the detection module is also used to detect a hand-raising gesture after the display module displays the mirror image of the target object; a display module 5002, which is used to display the mirror image of the target object on the touch screen in response to the drag gesture, and update the display position of the mirror image in real time according to the drag gesture so that the mirror image moves with the drag gesture; the display module is also used to display the mirror image fixedly at the display position where the mirror image is located in response to the hand-raising gesture. The method of copying and pasting by dragging gesture is simple and intuitive, which reduces the number of steps required for existing copying and pasting and shortens the process of copying and pasting.

[0455] Next, a terminal device provided in an embodiment of the present application is introduced. The terminal device can be Fig.49 as well as Fig.50 For the device described in Fig.51 , Fig.51 A schematic diagram of the structure of a terminal device provided in an embodiment of the present application, the terminal device 5100 can be specifically an electronic whiteboard, a virtual reality VR device, a mobile phone, a tablet, a laptop computer, a smart wearable device, etc., which is not limited here. Specifically, the terminal device 5100 includes: a receiver 5101, a transmitter 5102, a processor 5103 and a memory 5104 (wherein the number of processors 5103 in the terminal device 5100 can be one or more, Fig.51In the example of FIG. 5 , a processor 5103 may include an application processor 51031 and a communication processor 51032. In some embodiments of the present application, the receiver 5101, the transmitter 5102, the processor 5103 and the memory 5104 may be connected via a bus or other means.

[0456] The memory 5104 may include a read-only memory and a random access memory, and provides instructions and data to the processor 5103. A portion of the memory 5104 may also include a non-volatile random access memory (NVRAM). The memory 5104 stores processor and operation instructions, executable modules or data structures, or subsets thereof, or extended sets thereof, wherein the operation instructions may include various operation instructions for implementing various operations.

[0457] The processor 5103 controls the operation of the terminal device. In a specific application, the various components of the terminal device are coupled together through a bus system, wherein the bus system includes not only a data bus but also a power bus, a control bus, and a status signal bus, etc. However, for the sake of clarity, various buses are referred to as bus systems in the figure.

[0458] The method disclosed in the above embodiment of the present application can be applied to the processor 5103, or implemented by the processor 5103. The processor 5103 can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 5103. The above processor 5103 can be a general processor, a digital signal processor (digital signal processing, DSP), a microprocessor or a microcontroller, and can further include an application specific integrated circuit (application specific integrated circuit, ASIC), a field programmable gate array (field-programmable gate array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The processor 5103 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to execute, or the hardware and software modules in the decoding processor can be combined and executed. The software module may be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 5104, and the processor 5103 reads the information in the memory 5104, and completes the steps of the above method in combination with its hardware. Specifically, the processor 5103 can read the information in the memory 5104, and complete the steps related to the display of the virtual ruler in steps 301 to 302 in the above embodiment in combination with its hardware, and the steps related to the copying method in steps 3701 to 3704 in the above embodiment.

[0459] The receiver 5101 can be used to receive input digital or character information and generate signal input related to the relevant settings and function control of the terminal device. The transmitter 5102 can be used to output digital or character information through the first interface; the transmitter 5102 can also be used to send instructions to the disk group through the first interface to modify the data in the disk group; the transmitter 5102 can also include a display device such as a touch screen.

[0460] The present application also provides a computer program product which, when executed on a computer, enables the computer to execute the above-mentioned embodiments. Figure 3 as well as Fig.37a The steps of the method described in the corresponding embodiment.

[0461] A computer-readable storage medium is also provided in an embodiment of the present application. The computer-readable storage medium stores a program for performing signal processing. When the program is run on a computer, the computer executes the steps of the image processing method in the method described in the above embodiment.

[0462] The image display device provided in the embodiment of the present application may specifically be a chip, and the chip includes: a processing unit and a communication unit, wherein the processing unit may be, for example, a processor, and the communication unit may be, for example, an input / output interface, a pin or a circuit, etc. The processing unit may execute the computer execution instructions stored in the storage unit so that the chip in the execution device executes the data processing method described in the above embodiment, or so that the chip in the training device executes the data processing method described in the above embodiment. Optionally, the storage unit is a storage unit in the chip, such as a register, a cache, etc. The storage unit may also be a storage unit located outside the chip in the wireless access device, such as a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM), etc.

[0463] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0464] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including application-specific integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented with corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a magnetic disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0465] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0466] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, a computer, a server, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or a data center that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

Claims

1. A virtual ruler display method, characterized in that: Applied to an electronic device, the electronic device includes a touch screen, and the method includes: detecting a contact gesture with the touch screen; Based on the contact gesture indicating the contact between the side face of the user's hand and the touch screen, a virtual ruler is displayed on the touch screen according to the contact area of ​​the contact gesture, wherein the contact area of ​​the contact gesture is a long strip area, the virtual ruler is aligned with the long side of the long strip area, the side face of the hand is the side face of the user's hand on the little finger side in an unfolded state, and the display direction of the virtual ruler is parallel to the direction indicated by the long side of the long strip area or the acute angle between the display direction of the virtual ruler and the direction indicated by the long side of the long strip area is less than a preset value.

2. The method according to claim 1, characterized in that: The contact area includes: The contact area between the hypothenar part of the side of the user's hand and the touch screen; or, The contact area between the little finger of the user's hand and the touch screen.

3. The method according to any one of claims 1 to 2, characterized in that: The method further comprises: Based on the contact area of ​​the contact gesture satisfying a preset condition, determining that the contact gesture indicates contact between the side of the user's hand and the touch screen; the preset condition includes: The contact area is a long strip area, and the shape and area size of the long strip area conform to the characteristics of the contact area when the hypothenar part and / or the little finger part of the side of the user's hand contacts the touch screen.

4. The method according to any one of claims 1 to 2, characterized in that: The method further comprises: Acquiring gesture data of the contact gesture; Based on the gesture data, a neural network is used to determine that the contact gesture indicates contact between the side of the user's hand and the touch screen.

5. The method according to any one of claims 1 to 2, characterized in that: The acute angle between the display direction of the virtual ruler and the direction indicated by the long side of the long strip area is smaller than a preset value; and the virtual ruler and the long strip area meet one of the following conditions: The virtual ruler overlaps with the long strip area; or, the virtual ruler is tangent to the long strip area; or, the distance between the virtual ruler and the long strip area is less than a preset value.

6. The method according to any one of claims 1 to 2, characterized in that: Before displaying a virtual ruler on the touch screen according to the contact area of ​​the contact gesture, the method further includes: It is detected that the maintenance time of the contact gesture is greater than a preset time.

7. The method according to any one of claims 1 to 2, characterized in that: The method further comprises: detecting movement of a contact area of ​​the contact gesture; The display position of the virtual ruler is adjusted so that the display position of the virtual ruler follows the contact area.

8. The method according to any one of claims 1 to 2, characterized in that: The method further comprises: Detecting the user's hand-raising gesture; In response to the hand-raising gesture, the display of the virtual ruler on the touch screen is hidden.

9. The method according to any one of claims 1 to 2, characterized in that: The step of indicating the contact between the side of the user's hand and the touch screen based on the contact gesture comprises: The side of the hand of the user taps the touch screen a preset number of times based on the contact gesture indication; After displaying the virtual ruler on the touch screen, the method further includes: Detecting the user's hand-raising gesture; In response to the hand-raising gesture, the display of the virtual ruler is maintained on the touch screen.

10. The method according to claim 9, characterized in that The method further comprises: detecting a user's selection instruction for the virtual ruler; In response to the selection instruction, a trigger control is displayed, where the trigger control is used to instruct at least one of the following operations to be performed on the virtual ruler: Deletion, repositioning, and rotation.

11. The method according to any one of claims 1 to 2, characterized in that: The virtual ruler is used to measure the length of a drawing straight line segment displayed on the touch screen; or, The virtual ruler is used to measure the distance between drawing points displayed on the touch screen.

12. The method according to any one of claims 1 to 2, characterized in that: The method further comprises: Detecting that there is a drawing straight line segment on the touch screen that is associated with the virtual ruler in a gesture, and displaying the length value of the drawing straight line segment; the gesture association includes: The direction difference with the virtual ruler is less than a preset value; and / or, The distance value between the virtual ruler and the virtual ruler is smaller than a preset value.

13. The method according to claim 12, characterized in that The method further comprises: According to the position of the drawing straight line segment, the display position of the virtual ruler is adjusted so that the display position of the virtual ruler fits the drawing straight line segment.

14. The method according to any one of claims 1 to 2, characterized in that: After displaying the virtual ruler on the touch screen, the method further includes: It is detected that a first intersection point and a second intersection point exist between the drawing line segment on the touch screen and the virtual ruler, and a distance value between the first intersection point and the second intersection point is displayed.

15. The method according to any one of claims 1 to 2, characterized in that: The virtual ruler is used as a reference tool when drawing a straight line on the touch screen.

16. The method according to any one of claims 1 to 2, characterized in that: The method further comprises: detecting a line drawing gesture on the touch screen; Based on the virtual ruler being displayed on the touch screen and the distance between the line drawing position of the line drawing gesture and the virtual ruler being within a preset distance, a drawing straight line segment is displayed according to the line drawing position of the line drawing gesture, wherein the drawing straight line segment is parallel to the virtual ruler.

17. The method according to claim 16, characterized in that The line drawing position includes a starting point position and an ending point position, and the drawing straight line segment is a line segment between the starting point position and the ending point position.

18. The method according to claim 16, characterized in that The line drawing position includes a starting point position and a real-time line drawing position, and the method further includes: Based on the real-time line drawing position, the line drawing length is displayed, and the line drawing length is the distance value between the starting point position and the real-time line drawing position.

19. A virtual ruler display device, characterized in that: Applied to an electronic device, the electronic device includes a touch screen, and the device includes: A detection module, used to detect a contact gesture with the touch screen; A display module is used to indicate the contact between the side of the user's hand and the touch screen based on the contact gesture, and display a virtual ruler on the touch screen according to the contact area of ​​the contact gesture, wherein the contact area of ​​the contact gesture is a long strip area, the virtual ruler is in contact with the long side of the long strip area, the hand side is the side of the user's hand on the little finger side in the unfolded state, and the display direction of the virtual ruler is parallel to the direction indicated by the long side of the long strip area or the acute angle between the display direction of the virtual ruler and the direction indicated by the long side of the long strip area is less than a preset value.

20. The device according to claim 19, characterized in that The contact area includes: The contact area between the hypothenar part of the side of the user's hand and the touch screen; or, The contact area between the little finger of the user's hand and the touch screen.

21. The device according to any one of claims 19 to 20, characterized in that The device also includes: A determination module is used to determine that the contact gesture indicates contact between the side of the user's hand and the touch screen based on the contact area of ​​the contact gesture satisfying a preset condition; the preset condition includes: The contact area is a long strip area, and the shape and area size of the long strip area conform to the characteristics of the contact area when the hypothenar part and / or the little finger part of the side of the user's hand contacts the touch screen.

22. The device according to any one of claims 19 to 20, characterized in that The device also includes: An acquisition module, used for acquiring gesture data of the contact gesture; The determination module is used to determine, based on the gesture data and through a neural network, that the contact gesture indicates contact between the side of the user's hand and the touch screen.

23. The device according to any one of claims 19 to 20, characterized in that The acute angle between the display direction of the virtual ruler and the direction indicated by the long side of the long strip area is smaller than a preset value; and the virtual ruler and the long strip area meet one of the following conditions: The virtual ruler overlaps with the long strip area; or, the virtual ruler is tangent to the long strip area; or, the distance between the virtual ruler and the long strip area is less than a preset value.

24. The device according to any one of claims 19 to 20, characterized in that The detection module is used to detect that the contact gesture is maintained for a time greater than a preset time before a virtual ruler is displayed on the touch screen in the contact area according to the contact gesture.

25. The device according to any one of claims 19 to 20, characterized in that The detection module is used to detect the movement of the contact area of ​​the contact gesture; The display module is used to adjust the display position of the virtual ruler so that the display position of the virtual ruler is aligned with the contact area.

26. The device according to any one of claims 19 to 20, characterized in that The detection module is used to detect the user's hand-raising gesture; The display module is used to hide the display of the virtual ruler on the touch screen in response to the hand-raising gesture.

27. The device according to any one of claims 19 to 20, characterized in that The step of indicating the contact between the side of the user's hand and the touch screen based on the contact gesture comprises: The side of the hand of the user taps the touch screen a preset number of times based on the contact gesture indication; After the virtual ruler is displayed on the touch screen, the detection module is further used to detect a hand-raising gesture of the user; The display module is further configured to maintain display of the virtual ruler on the touch screen in response to the hand-raising gesture.

28. The device according to claim 27, characterized in that The detection module is used to detect a selection instruction of the user for the virtual ruler; The display module is used to display a trigger control in response to the selection instruction, and the trigger control is used to instruct to perform at least one of the following operations on the virtual ruler: Deletion, repositioning, and rotation.

29. The device according to any one of claims 19 to 20, characterized in that The virtual ruler is used to measure the length of a drawing straight line segment displayed on the touch screen; or, The virtual ruler is used to measure the distance between drawing points displayed on the touch screen.

30. The device according to any one of claims 19 to 20, characterized in that The detection module is used to detect the presence of a drawing straight line segment on the touch screen that is associated with the virtual ruler; The display module is used to display the length value of the drawing straight line segment; The presence posture association includes: The direction difference with the virtual ruler is less than a preset value; and / or, The distance value between the virtual ruler and the virtual ruler is smaller than a preset value.

31. The device according to claim 30, characterized in that The display module is used to adjust the display position of the virtual ruler according to the position of the drawing straight line segment, so that the display position of the virtual ruler fits the drawing straight line segment.

32. The device according to any one of claims 19 to 20, characterized in that The detection module is used to detect, after the virtual ruler is displayed on the touch screen, that a first intersection point and a second intersection point exist between the drawing line segment on the touch screen and the virtual ruler; The display module is used to display the distance value between the first intersection point and the second intersection point.

33. The device according to any one of claims 19 to 20, characterized in that The virtual ruler is used as a reference tool when drawing a straight line on the touch screen.

34. The device according to any one of claims 19 to 20, characterized in that The detection module is used to detect a line drawing gesture on the touch screen; The display module is used to display a drawing straight line segment according to the drawing position of the line drawing gesture based on the virtual ruler being displayed on the touch screen and the distance between the drawing position of the line drawing gesture and the virtual ruler being within a preset distance, wherein the drawing straight line segment is parallel to the virtual ruler.

35. The device according to claim 34, characterized in that The line drawing position includes a starting point position and an ending point position, and the drawing straight line segment is a line segment between the starting point position and the ending point position.

36. The device according to claim 34, characterized in that The line drawing position includes a starting point position and a real-time line drawing position. The display module is further used to display a line drawing length based on the real-time line drawing position. The line drawing length is a distance value between the starting point position and the real-time line drawing position.

37. An electronic device, characterized in that: The device comprises a processor, a memory, a touch screen and a bus, wherein: The processor, the memory and the touch screen are connected via the bus; The memory is used to store computer programs or instructions; The processor is used to call or execute the program or instruction stored in the memory, and is also used to control the touch screen to implement the method steps described in any one of claims 1-18.

38. A computer-readable storage medium comprising a program, which, when executed on a computer, causes the computer to execute the method according to any one of claims 1 to 18.

39. A computer program product comprising instructions, characterized in that When the computer program product is executed on a terminal, the terminal is enabled to execute the method according to any one of claims 1 to 18.

Images