Gesture recognition method and related device
By acquiring the number of touch points and physical parameters of the touchscreen, and calculating the physical data of the touch points, the problem of gesture recognition error on touchscreens of different resolutions and sizes is solved, and higher precision gesture judgment is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING HITEVISION AIXUE EDUCATION TECH CO LTD
- Filing Date
- 2022-11-11
- Publication Date
- 2026-05-12
AI Technical Summary
On touchscreens of different resolutions and sizes, errors can easily occur when judging gestures based on pixel values, resulting in gestures failing to be triggered or being triggered accidentally.
By acquiring the number of touch points, pixel data, and physical parameters of the touchscreen, the physical data of the touch points is calculated, and gestures are recognized based on preset thresholds. The pixel data is then converted into actual physical data for judgment.
It effectively reduces the error of gesture recognition on touchscreens of different sizes and resolutions, and improves the accuracy of judgment.
Smart Images
Figure CN115826838B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of gesture recognition, and in particular to a gesture recognition method and related device. BACKGROUND
[0002] When using a touch screen, gesture operation is very convenient for users. Whether a gesture is formed is generally determined by analyzing user multi-finger moving track and speed, and contact area information. Since application software obtains user touch data based on pixel values, gestures are generally processed and determined based on pixel values. However, when gestures are determined based on pixel values, a determination condition in pixel units needs to be set. Different resolutions can be set for a touch screen, and the touch screen itself also has various sizes. This can result in different pixel values of the same finger operation on different resolutions or different sizes of the touch screen. When gestures are determined, pixel value errors are prone to be too large, which can result in some gestures being unable to be triggered or being triggered by mistake. SUMMARY
[0003] Therefore, the present application aims to provide a gesture recognition method and related device.
[0004] To achieve the above purpose, the present application provides a gesture recognition method, comprising:
[0005] acquiring a number of touch points of a touch screen, and acquiring pixel data of the touch points according to the number of the touch points;
[0006] calculating physical data of the touch points according to the number of the touch points, the pixel data of the touch points, acquired physical parameters of the touch screen, and acquired resolution of the touch screen;
[0007] obtaining a gesture corresponding to the touch points according to the physical data of the touch points and a preset threshold value.
[0008] In a possible implementation, the number of the touch points is one, and the pixel data of the touch points comprises a maximum pixel width value of a touch area of the touch points and a maximum pixel height value of the touch area of the touch points.
[0009] In a possible implementation, the physical parameters of the touch screen comprise a physical width value of the touch screen and a physical height value of the touch screen.
[0010] The resolution of the touch screen comprises a resolution width of the touch screen and a resolution height of the touch screen.
[0011] The physical data of the touch points comprises a physical maximum width value of the touch points and a physical maximum height value of the touch points.
[0012] The step of calculating the physical data of the touch points based on the number of touch points, the pixel data of the touch points, the acquired physical parameters of the touch screen, and the acquired resolution of the touch screen includes:
[0013] The maximum physical width of the touch point is calculated based on the maximum pixel width of the touch area of the touch point, the physical width of the touch screen, and the resolution width of the touch screen.
[0014] The maximum physical height of the touch point is calculated based on the maximum pixel height of the touch area, the physical height of the touch screen, and the resolution height of the touch screen.
[0015] In one possible implementation, the number of touch points is greater than one, and the pixel data of the touch points includes: the horizontal pixel coordinate value and the vertical pixel coordinate value of the center pixel of the touch point; the center pixel is located at the center of the touch area corresponding to the touch point.
[0016] In one possible implementation, the physical parameters of the touchscreen include: the physical width value and the physical height value of the touchscreen;
[0017] The resolution of the touchscreen includes: the width of the touchscreen and the height of the touchscreen;
[0018] The physical data of the touch point includes the physical horizontal coordinate value and the physical vertical coordinate value of the touch point;
[0019] The step of calculating the physical data of the touch points based on the number of touch points, the pixel data of the touch points, the acquired physical parameters of the touchscreen, and the acquired resolution of the touchscreen includes:
[0020] The physical horizontal coordinate value of the touch point is calculated based on the pixel horizontal coordinate value of the center pixel of the touch point, the physical width value of the touch screen, and the resolution width of the touch screen.
[0021] The physical coordinate value of the touch point is calculated based on the pixel ordinate value of the center pixel of the touch point, the physical height value of the touch screen, and the resolution height of the touch screen.
[0022] In one possible implementation, the gesture corresponding to the touch point includes: erasing with the back of the hand, normal writing, and fine pen writing; the preset threshold includes: a first threshold and a second threshold; the second threshold is less than the first threshold;
[0023] The step of obtaining the gesture corresponding to the touch point based on the physical data of the touch point and a preset threshold includes:
[0024] When the physical data of any of the touch points is greater than the first threshold, the gesture corresponding to the touch point is determined to be a back-of-hand wipe.
[0025] When the physical data of all the touch points is less than the first threshold and the physical data of any touch point is greater than the second threshold, the gesture corresponding to the touch point is obtained as normal writing.
[0026] When the physical data of all the touch points are less than the second threshold, the gesture corresponding to the touch point is obtained as fine pen writing.
[0027] In one possible implementation, the gestures corresponding to the touch point include: two-finger zoom and rotate, two-finger pan, and five-finger wipe; the preset thresholds include: a third threshold and a fourth threshold; the physical horizontal coordinate and the physical vertical coordinate of the touch point are used as the physical coordinates of the touch point;
[0028] The step of obtaining the gesture corresponding to the touch point based on the physical data of the touch point and a preset threshold includes:
[0029] When there are two touch points, calculate the first physical distance between the two touch points;
[0030] When the first physical distance is greater than the third threshold, the gesture corresponding to the touch point is obtained as two-finger scaling and rotation;
[0031] When the first physical distance is less than the third threshold, the gesture corresponding to the touch point is obtained as a two-finger translation;
[0032] When the number of touch points is five, calculate the second physical distance between the physical coordinates of every two adjacent touch points;
[0033] When all the second physical distances are less than the fourth threshold, the gesture corresponding to the touch point is obtained as five-finger wiping.
[0034] In one possible implementation, the preset threshold can be automatically adjusted based on the physical data.
[0035] Based on the same inventive concept, embodiments of this application also provide a gesture recognition device, including:
[0036] The acquisition module is configured to acquire the number of touch points on the touch screen and acquire the pixel data of the touch points based on the number of touch points;
[0037] The calculation module is configured to calculate the physical data of the touch points based on the number of touch points, the pixel data of the touch points, the physical parameters of the touch screen, and the resolution of the touch screen.
[0038] The judgment module is configured to obtain the gesture corresponding to the touch point based on the physical data of the touch point and a preset threshold.
[0039] In one possible implementation, the number of touch points is one, and the pixel data of the touch points includes: the maximum pixel width value of the touch area of the touch point and the maximum pixel height value of the touch area of the touch point.
[0040] In one possible implementation, the physical parameters of the touchscreen include: the physical width value and the physical height value of the touchscreen.
[0041] The resolution of the touchscreen includes: the width of the touchscreen and the height of the touchscreen;
[0042] The physical data of the touch point includes: the maximum physical width value and the maximum physical height value of the touch point;
[0043] The computing module is further configured as follows:
[0044] The maximum physical width of the touch point is calculated based on the maximum pixel width of the touch area of the touch point, the physical width of the touch screen, and the resolution width of the touch screen.
[0045] The maximum physical height of the touch point is calculated based on the maximum pixel height of the touch area, the physical height of the touch screen, and the resolution height of the touch screen.
[0046] In one possible implementation, the number of touch points is greater than one, and the pixel data of the touch points includes: the horizontal pixel coordinate value and the vertical pixel coordinate value of the center pixel of the touch point; the center pixel is located at the center of the touch area corresponding to the touch point.
[0047] In one possible implementation,
[0048] The physical parameters of the touchscreen include: the physical width value and the physical height value of the touchscreen;
[0049] The resolution of the touchscreen includes: the width of the touchscreen and the height of the touchscreen;
[0050] The physical data of the touch point includes the physical horizontal coordinate value and the physical vertical coordinate value of the touch point;
[0051] The computing module is further configured as follows:
[0052] The physical horizontal coordinate value of the touch point is calculated based on the pixel horizontal coordinate value of the center pixel of the touch point, the physical width value of the touch screen, and the resolution width of the touch screen.
[0053] The physical coordinate value of the touch point is calculated based on the pixel ordinate value of the center pixel of the touch point, the physical height value of the touch screen, and the resolution height of the touch screen.
[0054] In one possible implementation, the gesture corresponding to the touch point includes: erasing with the back of the hand, normal writing, and fine pen writing; the preset threshold includes: a first threshold and a second threshold; the second threshold is less than the first threshold;
[0055] The judgment module is further configured as follows:
[0056] When the physical data of any of the touch points is greater than the first threshold, the gesture corresponding to the touch point is determined to be a back-of-hand wipe.
[0057] When the physical data of all the touch points is less than the first threshold and the physical data of any touch point is greater than the second threshold, the gesture corresponding to the touch point is obtained as normal writing.
[0058] When the physical data of all the touch points are less than the second threshold, the gesture corresponding to the touch point is obtained as fine pen writing.
[0059] In one possible implementation, the gestures corresponding to the touch point include: two-finger zoom and rotate, two-finger pan, and five-finger wipe; the preset thresholds include: a third threshold and a fourth threshold; the physical horizontal coordinate and the physical vertical coordinate of the touch point are used as the physical coordinates of the touch point;
[0060] The judgment module is further configured as follows:
[0061] When there are two touch points, calculate the first physical distance between the two touch points;
[0062] When the first physical distance is greater than the third threshold, the gesture corresponding to the touch point is obtained as two-finger scaling and rotation;
[0063] When the first physical distance is less than the third threshold, the gesture corresponding to the touch point is obtained as a two-finger translation;
[0064] When the number of touch points is five, calculate the second physical distance between the physical coordinates of every two adjacent touch points;
[0065] When all the second physical distances are less than the fourth threshold, the gesture corresponding to the touch point is obtained as five-finger wiping.
[0066] In one possible implementation, the preset threshold can be automatically adjusted based on the physical data.
[0067] Based on the same inventive concept, embodiments of this application also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the gesture recognition method as described in any of the above.
[0068] As can be seen from the above description, the gesture recognition method and related equipment provided in this application acquire the number of touch points on the touchscreen, acquire pixel data of the touch points based on the number of touch points, calculate physical data of the touch points based on the number of touch points, the pixel data of the touch points, the acquired physical parameters of the touchscreen, and the acquired resolution of the touchscreen, and obtain the gesture corresponding to the touch point based on the physical data of the touch point and a preset threshold. This application effectively converts the pixel data corresponding to the touch action into actual physical data, and then uses the actual physical data to further recognize the gesture corresponding to the touch action, effectively preventing large errors in gesture judgment and effectively improving the accuracy of gesture judgment. Attached Figure Description
[0069] To more clearly illustrate the technical solutions in this application or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0070] Figure 1 This is a flowchart of a gesture recognition method according to an embodiment of this application;
[0071] Figure 2 This is a schematic diagram of a gesture recognition device according to an embodiment of this application;
[0072] Figure 3 This is a schematic diagram of the electronic device structure according to an embodiment of this application. Detailed Implementation
[0073] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with specific embodiments and the accompanying drawings.
[0074] It should be noted that, unless otherwise defined, the technical or scientific terms used in the embodiments of this application should have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms "first," "second," and similar terms used in the embodiments of this application do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are only used to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.
[0075] As described in the background section, gesture recognition solutions in related technologies are generally based on pixel values. However, when judging gestures based on pixel values, it is necessary to set judgment conditions in units of pixels. Since touchscreens can be set to different resolutions and have various sizes, the final pixel values responded to under the same finger operation will be different on touchscreens of different resolutions or sizes. This will lead to excessive pixel value errors when judging gestures based on the aforementioned pixel values, resulting in the inability to recognize the correct gestures.
[0076] In light of the above considerations, this application proposes a gesture recognition method. The method obtains pixel data based on the number of touch points on the touchscreen. It then calculates the physical data of the corresponding touch points based on the pixel data, the obtained physical parameters of the touchscreen, and the obtained resolution of the touchscreen. Finally, it determines the gesture corresponding to the touch point based on the relationship between the physical data and a preset threshold. This method effectively measures the pixel values corresponding to touch actions on touchscreens of different sizes and resolutions using actual physical dimensions. Based on these actual physical dimensions, it allows for precise judgment of touch actions and accurate recognition of their corresponding gestures.
[0077] The technical solutions of the embodiments of this application will be described below through specific examples.
[0078] refer to Figure 1 The gesture recognition method of this application includes the following steps:
[0079] Step S101: Obtain the number of touch points on the touch screen, and obtain the pixel data of the touch points based on the number of touch points;
[0080] Step S102: Calculate the physical data of the touch points based on the number of touch points, the pixel data of the touch points, the acquired physical parameters of the touch screen, and the acquired resolution of the touch screen.
[0081] Step S103: Based on the physical data of the touch point and a preset threshold, obtain the gesture corresponding to the touch point.
[0082] For step S101, first obtain the physical parameters and resolution of the touch screen, then obtain the number of touch points on the touch screen, and then obtain the pixel data corresponding to each touch point.
[0083] The following example uses one touch point, where the pixel data corresponding to that touch point is the maximum pixel width and maximum pixel height of the touch area. In general, when a user touches the touchscreen with their finger or the back of their hand, the resulting touch area is an irregular shape. Therefore, directly taking the maximum pixel width and maximum pixel height of this irregular shape effectively saves the step of secondary calculation while ensuring accuracy.
[0084] Furthermore, regarding step S102, based on the previously obtained physical parameters and resolution of the touchscreen, and combined with the pixel data of the touch point, the physical data of the touch point is calculated. Specifically, the obtained physical parameters of the touchscreen include the physical width and physical height values of the touchscreen, and the touchscreen resolution includes the resolution width and resolution height of the touchscreen. Further, based on the aforementioned maximum width value of the touch area, the physical width value of the touchscreen, and the resolution width of the touchscreen, the physical width value of the touch point is calculated. The physical width value of the touch point is the actual physical size of the touch area corresponding to the touch point.
[0085] Specifically, the physical width of the touch point is calculated using the following formula:
[0086]
[0087] Among them, W p P represents the physical width value of the touch point. w W represents the maximum pixel width value. s D represents the physical width value of the touchscreen. w This indicates the width value of the touchscreen's resolution.
[0088] Furthermore, based on the maximum pixel height value of the touch area, the physical height value of the touch screen, and the resolution height of the touch screen mentioned above, the physical height value of the touch point is calculated. The physical height value of the touch point is the actual physical size of the touch area corresponding to the touch point.
[0089] Specifically, the physical height value of the touch point is calculated using the following formula:
[0090]
[0091] Among them, H p P represents the physical height value of the touch point. h H represents the maximum pixel height value. s D represents the physical height value of the touchscreen. h This indicates the resolution height value of the touchscreen.
[0092] After obtaining the physical width and height values of the touch point, in step S103, it is necessary to obtain the gesture corresponding to the touch point based on the physical data of the touch point and a preset threshold. In this embodiment, the physical data of the touch point refers to the physical width and height values of the touch area corresponding to the touch point. The preset threshold includes a first threshold and a second threshold, where the second threshold is less than the first threshold.
[0093] Specifically, when either the physical width or physical height of the touch point is greater than a first threshold, the gesture corresponding to that touch point is determined to be back-of-hand erasure. When both the physical width and physical height of the touch point are less than the first threshold and either the physical width or physical height is greater than a second threshold, the gesture corresponding to that touch point is determined to be normal writing. When both the physical width and physical height of the touch point are less than the second threshold, the gesture corresponding to that touch point is determined to be fine pen writing.
[0094] It should be noted that the first threshold and the second threshold involved in the above embodiments can be set according to the needs, and can also be automatically adjusted during the user's use. The specific automatic adjustment technical solution should be able to be implemented by those skilled in the art after adaptive adjustment based on the technology in the field, so it will not be described in detail here.
[0095] In an optional embodiment, the touchscreen is 60 inches in size, corresponding to a physical width of 1328 mm, a physical height of 747 mm, and a resolution of 3840*2160, corresponding to a resolution width of 3840 pixels and a resolution height of 2160 pixels. A preset first threshold is 40 mm, and a second threshold is 5 mm. At this time, there is one touch point. The maximum pixel height of the contact area corresponding to this touch point is 144 pixels, and the maximum pixel width is 160 pixels. Correspondingly, the physical height of the touch point is calculated to be 49.8 mm, and the physical width is 55.2 mm. Further, it is determined that both the calculated physical height and physical width of the touch point are greater than 40 mm, indicating that the gesture is a back-of-hand wiping.
[0096] In another optional embodiment, under the same touch conditions, the touchscreen size is changed to 96 inches, corresponding to a physical width of 2162 mm, a physical height of 1217 mm, and a resolution of 1920*1080, with a resolution width of 1920 and a resolution height of 1080. The obtained maximum pixel height of the contact area corresponding to this touch point is 44 pixels, and the maximum pixel width is 49 pixels. Comparing with the above embodiment, it can be seen that the pixel value corresponding to the same gesture varies greatly when the actual physical size and resolution of the touchscreen are different. Obviously, using pixel values to judge gestures will introduce a large error. However, using the technical solution of this application, the pixel value is converted into the actual physical size. No matter how the size of the touchscreen and the pixel value change, the physical size of the actual touch area will not change, effectively reducing the error of gesture recognition.
[0097] It should be noted that the above embodiments only use a portion of gestures when the touch point is a single point as examples. Those skilled in the art can undoubtedly apply the technical solutions of this application to the recognition of other gestures when the touch point is a single point. The steps for converting the touch point from pixel data to physical data during the recognition process can be the same as the above steps. Those skilled in the art can also set thresholds to limit the recognition of other gestures.
[0098] In an optional embodiment, taking two touch points as an example, based on the previously obtained physical parameters and resolution of the touchscreen, the pixel coordinates of the center pixel of each of the two touch points are obtained. The center pixel of a touch point is located at the center of the touch area corresponding to the touch point, that is, the intersection of the center line in the height direction and the center line in the width direction of the touch area. The pixel coordinates of the center pixel include the horizontal coordinate and the vertical coordinate of the center pixel. After obtaining the pixel coordinates of the center pixel, the physical horizontal coordinate of the touch point is calculated based on the horizontal coordinate of the center pixel, the physical width of the touchscreen, and the resolution width of the touchscreen.
[0099] Specifically, the physical x-coordinate value of the touch point is calculated using the following formula:
[0100]
[0101] Where X represents the physical x-coordinate value of the touch point, P x W represents the x-coordinate of the center pixel. s D represents the physical width value of the touchscreen. w This indicates the width value of the touchscreen's resolution.
[0102] Furthermore, the physical coordinate value of the touch point is calculated based on the pixel ordinate value of the center pixel, the physical height value of the touch screen, and the resolution height of the touch screen.
[0103] Specifically, the physical ordinate value of the touch point is calculated using the following formula:
[0104]
[0105] Where Y represents the physical vertical coordinate value of the touch point, P y H represents the ordinate value of the center pixel. s D represents the physical height value of the touchscreen. h This indicates the resolution height value of the touchscreen.
[0106] After obtaining the physical coordinates of the two touch points, the first physical distance between them is calculated, with a preset threshold set as the third threshold. The first physical distance can be calculated using any method; it is the straight-line distance between the center pixels of the two touch points. The specific calculation process will not be detailed here, but those skilled in the art should understand it. Furthermore, the relationship between the first physical distance and the third threshold is determined. Generally, during translation, the distance between the two fingers is closer, while during scaling and rotation, the distance is farther. Therefore, when the first physical distance is greater than the third threshold, the gesture corresponding to the touch point is two-finger scaling and rotation; when the first physical distance is less than the third threshold, the gesture corresponding to the touch point is two-finger translation.
[0107] Similarly, as should be noted in the above embodiments, this embodiment only uses a portion of gestures when the touch point is two points as examples. Those skilled in the art can undoubtedly apply the technical solution of this application to the recognition of other gestures when the touch point is two points. The step of converting the touch point from pixel data to physical data during the recognition process can be the same as the above steps. Those skilled in the art can also set thresholds to limit the recognition of other gestures. Adaptive modifications to the thresholds of other gestures according to the embodiments of this application do not require creative effort, so they will not be elaborated here.
[0108] In an optional embodiment, taking five touch points as an example, based on the previously obtained physical parameters and resolution of the touchscreen, the pixel coordinates of the center pixel of each of the five touch points are obtained. The center pixel of a touch point is located at the center of the touch area corresponding to the touch point, that is, the intersection of the center line in the height direction and the center line in the width direction of the touch area. The pixel coordinates of the center pixel include the horizontal and vertical coordinates of the center pixel. After obtaining the pixel coordinates of the center pixel, the physical horizontal coordinate of the touch point is calculated based on the horizontal coordinate of the center pixel, the physical width of the touchscreen, and the resolution width of the touchscreen.
[0109] Specifically, the physical x-coordinate value of the touch point is calculated using the following formula:
[0110]
[0111] Where X represents the physical x-coordinate value of the touch point, P x W represents the x-coordinate of the center pixel. s D represents the physical width value of the touchscreen. w This indicates the width value of the touchscreen's resolution.
[0112] Furthermore, the physical coordinate value of the touch point is calculated based on the pixel ordinate value of the center pixel, the physical height value of the touch screen, and the resolution height of the touch screen.
[0113] Specifically, the physical ordinate value of the touch point is calculated using the following formula:
[0114]
[0115] Where Y represents the physical vertical coordinate value of the touch point, P y H represents the ordinate value of the center pixel. s D represents the physical height value of the touchscreen. h This indicates the resolution height value of the touchscreen.
[0116] After obtaining the physical coordinates of the five touch points, the second physical distance between every two touch points is calculated, with a preset threshold set as the fourth threshold. The second physical distance can be calculated using any method; it is the straight-line distance between the two center pixels corresponding to every two touch points. The specific calculation process is not detailed here, but should be understood by those skilled in the art. Further, the relationship between the second physical distance and the fourth threshold is determined. Generally, considering that multiple people may operate the large screen simultaneously, if five fingers are being used by different people in different areas of the screen to avoid triggering a five-finger erase gesture, at least two fingers will have a relatively large distance between them. Therefore, when any second physical distance is greater than the fourth threshold, the gesture corresponding to the touch point is a writing operation; when all second physical distances are less than the fourth threshold, the gesture is a five-finger erase. In general, the fourth threshold is set to 200 mm. Of course, those skilled in the art should undoubtedly understand that the fourth threshold can be adjusted as needed.
[0117] Similarly, as should be noted in the above embodiments, this embodiment only uses a portion of gestures when the touch point is five points as an example. Those skilled in the art can undoubtedly apply the technical solution of this application to the recognition of other gestures when the touch point is five points. The step of converting the pixel data of the touch point to the physical data of the touch point during the recognition process can be the same as the above steps. Those skilled in the art can also set thresholds to limit the recognition of other gestures. Adapting the thresholds of other gestures according to the embodiments of this application does not require creative effort, so it will not be elaborated here.
[0118] As can be seen from the above embodiments, the gesture recognition method described in this application calculates the physical data of the touch point by acquiring the pixel data of the touch point, as well as the physical data and resolution of the touch screen. This effectively converts the pixel data of the touch point into the actual physical data of the pixel. Furthermore, the physical data of the pixel is used to further recognize the touch gesture, effectively solving the problem of differences in pixel data under different screen sizes and resolutions. This prevents large errors in judging gestures on screens of different sizes and resolutions, and effectively improves the accuracy of gesture judgment.
[0119] It should be noted that the method in this embodiment can be executed by a single device, such as a computer or server. The method can also be applied in a distributed scenario, where multiple devices cooperate to complete the task. In such a distributed scenario, one of these devices may execute only one or more steps of the method in this embodiment, and the multiple devices will interact with each other to complete the method described.
[0120] It should be noted that the above description describes some embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the above embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0121] Based on the same inventive concept, corresponding to any of the above embodiments, this application also provides a gesture recognition device.
[0122] refer to Figure 2 The gesture recognition device includes:
[0123] The acquisition module 21 is configured to acquire the number of touch points on the touch screen and acquire the pixel data of the touch points based on the number of touch points;
[0124] The calculation module 22 is configured to calculate the physical data of the touch points based on the number of touch points, the pixel data of the touch points, the physical parameters of the touch screen, and the resolution of the touch screen.
[0125] The judgment module 23 is configured to obtain the gesture corresponding to the touch point based on the physical data of the touch point and a preset threshold.
[0126] In one possible implementation, the number of touch points is one, and the pixel data of the touch points includes: the maximum pixel width value of the touch area of the touch point and the maximum pixel height value of the touch area of the touch point.
[0127] In one possible implementation, the physical parameters of the touchscreen include: the physical width value and the physical height value of the touchscreen;
[0128] The resolution of the touchscreen includes: the width of the touchscreen and the height of the touchscreen;
[0129] The physical data of the touch point includes: the maximum physical width value and the maximum physical height value of the touch point;
[0130] The computing module is further configured as follows:
[0131] The maximum physical width of the touch point is calculated based on the maximum pixel width of the touch area of the touch point, the physical width of the touch screen, and the resolution width of the touch screen.
[0132] The maximum physical height of the touch point is calculated based on the maximum pixel height of the touch area, the physical height of the touch screen, and the resolution height of the touch screen.
[0133] In one possible implementation, the number of touch points is greater than one, and the pixel data of the touch points includes: the horizontal pixel coordinate value and the vertical pixel coordinate value of the center pixel of the touch point; the center pixel is located at the center of the touch area corresponding to the touch point.
[0134] In one possible implementation, the physical parameters of the touchscreen include: the physical width value and the physical height value of the touchscreen;
[0135] The resolution of the touchscreen includes: the width of the touchscreen and the height of the touchscreen;
[0136] The physical data of the touch point includes the physical horizontal coordinate value and the physical vertical coordinate value of the touch point;
[0137] The computing module is further configured as follows:
[0138] The physical horizontal coordinate value of the touch point is calculated based on the pixel horizontal coordinate value of the center pixel of the touch point, the physical width value of the touch screen, and the resolution width of the touch screen.
[0139] The physical coordinate value of the touch point is calculated based on the pixel ordinate value of the center pixel of the touch point, the physical height value of the touch screen, and the resolution height of the touch screen.
[0140] In one possible implementation, the gesture corresponding to the touch point includes: erasing with the back of the hand, normal writing, and fine pen writing; the preset threshold includes: a first threshold and a second threshold; the second threshold is less than the first threshold;
[0141] The judgment module is further configured as follows:
[0142] When the physical data of any of the touch points is greater than the first threshold, the gesture corresponding to the touch point is determined to be a back-of-hand wipe.
[0143] When the physical data of all the touch points is less than the first threshold and the physical data of any touch point is greater than the second threshold, the gesture corresponding to the touch point is obtained as normal writing.
[0144] When the physical data of all the touch points are less than the second threshold, the gesture corresponding to the touch point is obtained as fine pen writing.
[0145] In one possible implementation, the gestures corresponding to the touch point include: two-finger zoom and rotate, two-finger pan, and five-finger wipe; the preset thresholds include: a third threshold and a fourth threshold; the physical horizontal coordinate and the physical vertical coordinate of the touch point are used as the physical coordinates of the touch point;
[0146] The judgment module is further configured as follows:
[0147] When there are two touch points, calculate the first physical distance between the two touch points;
[0148] When the first physical distance is greater than the third threshold, the gesture corresponding to the touch point is obtained as two-finger scaling and rotation;
[0149] When the first physical distance is less than the third threshold, the gesture corresponding to the touch point is obtained as a two-finger translation;
[0150] When the number of touch points is five, calculate the second physical distance between the physical coordinates of every two adjacent touch points;
[0151] When all the second physical distances are less than the fourth threshold, the gesture corresponding to the touch point is obtained as five-finger wiping.
[0152] In one possible implementation, the preset threshold can be automatically adjusted based on the physical data.
[0153] For ease of description, the above devices are described in terms of function, divided into various modules. Of course, in implementing this application, the functions of each module can be implemented in one or more software and / or hardware.
[0154] The apparatus of the above embodiments is used to implement the corresponding gesture recognition method in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.
[0155] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the gesture recognition method described in any of the above embodiments.
[0156] Figure 3 This embodiment illustrates a more specific hardware structure of an electronic device, which may include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, memory 1020, input / output interface 1030, and communication interface 1040 are interconnected internally via the bus 1050.
[0157] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.
[0158] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.
[0159] The input / output interface 1030 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.
[0160] The communication interface 1040 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).
[0161] Bus 1050 includes a pathway for transmitting information between various components of the device, such as processor 1010, memory 1020, input / output interface 1030, and communication interface 1040.
[0162] It should be noted that although the above-described device only shows the processor 1010, memory 1020, input / output interface 1030, communication interface 1040, and bus 1050, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.
[0163] The electronic devices described above are used to implement the corresponding gesture recognition methods in any of the foregoing embodiments and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.
[0164] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of this application (including the claims) is limited to these examples; within the framework of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the embodiments of this application as described above, which are not provided in the details for the sake of brevity.
[0165] Additionally, to simplify the description and discussion, and to avoid obscuring the embodiments of this application, the well-known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided drawings. Furthermore, the apparatus may be shown in block diagram form to avoid obscuring the embodiments of this application, and this also takes into account the fact that the details of the implementation of these block diagram apparatuses are highly dependent on the platform on which the embodiments of this application will be implemented (i.e., these details should be fully understood by those skilled in the art). While specific details (e.g., circuits) have been set forth to describe exemplary embodiments of this application, it will be apparent to those skilled in the art that the embodiments of this application can be implemented without these specific details or with variations thereof. Therefore, these descriptions should be considered illustrative rather than restrictive.
[0166] Although this application has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.
[0167] The embodiments of this application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the embodiments of this application should be included within the protection scope of this application.
Claims
1. A gesture recognition method, characterized in that, include: Obtain the number of touch points on the touchscreen, and based on the number of touch points, obtain the pixel data of the touch points; The physical data of the touch points is calculated based on the number of touch points, the pixel data of the touch points, the acquired physical parameters of the touchscreen, and the acquired resolution of the touchscreen. This includes: responding to a touch point count of one, calculating the maximum physical width of the touch point based on the maximum pixel width of the touch area, the physical width of the touchscreen, and the resolution width of the touchscreen; calculating the maximum physical height of the touch point based on the maximum pixel height of the touch area, the physical height of the touchscreen, and the resolution height of the touchscreen. The pixel data of the touch point includes: the maximum pixel width of the touch area and the maximum pixel height of the touch area. The physical parameters of the touchscreen include: the physical width and the physical height of the touchscreen. The resolution of the touchscreen includes: the resolution width and the resolution height of the touchscreen. The physical data of the touch point includes: the maximum physical width and the maximum physical height of the touch point. The gesture corresponding to the touch point is obtained based on the physical data of the touch point and a preset threshold.
2. The method according to claim 1, characterized in that, The number of touch points is greater than one, and the pixel data of the touch points includes: the horizontal pixel coordinate value and the vertical pixel coordinate value of the center pixel of the touch point; the center pixel is located at the center of the touch area corresponding to the touch point.
3. The method according to claim 2, characterized in that, The physical parameters of the touchscreen include: the physical width value and the physical height value of the touchscreen; The resolution of the touchscreen includes: the width of the touchscreen and the height of the touchscreen; The physical data of the touch point includes the physical horizontal coordinate value and the physical vertical coordinate value of the touch point; The step of calculating the physical data of the touch points based on the number of touch points, the pixel data of the touch points, the acquired physical parameters of the touchscreen, and the acquired resolution of the touchscreen includes: The physical horizontal coordinate value of the touch point is calculated based on the pixel horizontal coordinate value of the center pixel of the touch point, the physical width value of the touch screen, and the resolution width of the touch screen. The physical coordinate value of the touch point is calculated based on the pixel ordinate value of the center pixel of the touch point, the physical height value of the touch screen, and the resolution height of the touch screen.
4. The method according to claim 1, characterized in that, The gestures corresponding to the touch points include: erasing with the back of the hand, normal writing, and fine pen writing; the preset thresholds include: a first threshold and a second threshold; the second threshold is less than the first threshold; The step of obtaining the gesture corresponding to the touch point based on the physical data of the touch point and a preset threshold includes: When the physical data of any of the touch points is greater than the first threshold, the gesture corresponding to the touch point is determined to be a back-of-hand wipe. When the physical data of all the touch points is less than the first threshold and the physical data of any touch point is greater than the second threshold, the gesture corresponding to the touch point is obtained as normal writing. When the physical data of all the touch points are less than the second threshold, the gesture corresponding to the touch point is obtained as fine pen writing.
5. The method according to claim 3, characterized in that, The gestures corresponding to the touch point include: two-finger zoom and rotate, two-finger pan, and five-finger erase; the preset thresholds include: a third threshold and a fourth threshold; the physical horizontal coordinate and the physical vertical coordinate of the touch point are used as the physical coordinates of the touch point; The step of obtaining the gesture corresponding to the touch point based on the physical data of the touch point and a preset threshold includes: When there are two touch points, calculate the first physical distance between the two touch points; When the first physical distance is greater than the third threshold, the gesture corresponding to the touch point is obtained as two-finger scaling and rotation; When the first physical distance is less than the third threshold, the gesture corresponding to the touch point is obtained as a two-finger translation; When the number of touch points is five, calculate the second physical distance between the physical coordinates of every two adjacent touch points; When all the second physical distances are less than the fourth threshold, the gesture corresponding to the touch point is obtained as five-finger wiping.
6. The method according to any one of claims 1 to 5, characterized in that, The preset threshold can be automatically adjusted based on the physical data.
7. A gesture recognition device, characterized in that, include: The acquisition module is configured to acquire the number of touch points on the touch screen and acquire the pixel data of the touch points based on the number of touch points; The calculation module is configured to calculate the physical data of the touch points based on the number of touch points, the pixel data of the touch points, the acquired physical parameters of the touchscreen, and the acquired resolution of the touchscreen. This includes: in response to the number of touch points being one, calculating the maximum physical width of the touch point based on the maximum pixel width of the touch area, the physical width of the touchscreen, and the resolution width of the touchscreen; and calculating the maximum physical height of the touch point based on the maximum pixel height of the touch area, the physical height of the touchscreen, and the resolution height of the touchscreen. The pixel data of the touch point includes: the maximum pixel width of the touch area and the maximum pixel height of the touch area; the physical parameters of the touchscreen include: the physical width and the physical height of the touchscreen; the resolution of the touchscreen includes: the resolution width and the resolution height of the touchscreen; and the physical data of the touch point includes: the maximum physical width and the maximum physical height of the touch point. The judgment module is configured to obtain the gesture corresponding to the touch point based on the physical data of the touch point and a preset threshold.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 6.