Method, apparatus and device for determining triggering position and medium

Abstract

Embodiments of the present disclosure relate to a method and apparatus for determining a trigger position, and a device and medium, wherein the method comprises: obtaining a current hand position change amount when a user performs a gesture operation, and obtaining a movement direction parameter value corresponding to the current hand position change amount; determining a first reference hand position change amount and a corresponding first time that satisfy a preset tremor condition; obtaining a second time of a last second reference hand position change amount that satisfies the preset tremor condition before the first time; and if a time interval between the first time and the second time is greater than a first preset time interval, updating a current trigger position on an operation screen according to the current hand position change amount. Thus, the effectiveness of the gesture operation is determined in combination with the time interval and the movement direction, avoiding misjudgment caused by hand shaking or tremor, ensuring the accuracy of the trigger operation position, ensuring the stability of the trigger position during gesture tracking, and improving the user's interaction experience.

Description

Technical Field

[0001] This disclosure relates to the field of computer application technology, and in particular to a method, apparatus, device and medium for determining a trigger location. Background Technology

[0002] Gesture tracking is a technology that uses image recognition to obtain the coordinate changes of key points of the hand in the two-dimensional space of the image, and maps these coordinate changes to the corresponding cursor position on the screen of a computer or other device, thereby realizing motion control of the computer or other specific devices.

[0003] In related technologies, image recognition technology identifies changes in the horizontal and vertical coordinates of the hand, which are then converted into movements along the horizontal and vertical axes on the display screen of devices such as computers. This allows for triggering operations on displayed content based on changes in hand position.

[0004] However, in actual operation, users' hands may unconsciously shake or tremble due to tension or fatigue. The changes in hand coordinates caused by such shaking or trembling will also be reflected on the display screen, thus causing changes in the trigger position and affecting the accuracy of trigger position positioning. Summary of the Invention

[0005] To solve the above-mentioned technical problems, or at least partially solve them, this disclosure provides a method for determining the trigger position. This method combines two dimensions—time interval and movement direction—to determine whether a gesture operation is effective, avoiding misjudgments caused by hand tremors or shaking, ensuring the accuracy of the trigger position, guaranteeing the stability of the trigger position during gesture tracking, and improving the user's interactive experience.

[0006] This disclosure provides a method for determining a trigger position. The method includes: acquiring the change in the current hand position when a user performs a gesture operation, and acquiring a movement direction parameter value corresponding to the change in the current hand position; determining a first reference hand position change that satisfies a preset vibration condition based on the movement direction parameter value, and determining a first time of the first reference hand position change; acquiring a second time of the previous second reference hand position change that satisfies the preset vibration condition before the first time; and updating the current trigger position on the operation screen based on the current hand position change if the time interval between the first time and the second time is greater than a first preset time interval.

[0007] This disclosure also provides a trigger position determination device, the device comprising: a first acquisition module, configured to acquire the current hand position change amount when a user performs a gesture operation, and acquire a movement direction parameter value corresponding to the current hand position change amount; a determination module, configured to determine a first reference hand position change amount that satisfies a preset vibration condition based on the movement direction parameter value, and determine a first time of the first reference hand position change amount; a second acquisition module, configured to acquire a second time of the previous second reference hand position change amount that satisfies the preset vibration condition before the first time; and a position update module, configured to update the current trigger position on the operation screen based on the current hand position change amount when the time interval between the first time and the second time is greater than a first preset time interval.

[0008] This disclosure also provides an electronic device, the electronic device comprising: a processor; a memory for storing executable instructions of the processor; the processor being configured to read the executable instructions from the memory and execute the instructions to implement the trigger location determination method provided in this disclosure.

[0009] This disclosure also provides a computer-readable storage medium storing a computer program for executing the method for determining a trigger location as provided in this disclosure.

[0010] The technical solution provided in this disclosure has the following advantages compared with the prior art:

[0011] The trigger position determination scheme provided in this embodiment obtains the change in the current hand position when the user performs a gesture operation, and obtains the movement direction parameter value corresponding to the change in the current hand position. Then, based on the movement direction parameter value, it determines the first time when the hand position change meets the preset tremor condition, and obtains the second time when the hand position change that previously met the preset tremor condition is obtained. If the time interval between the first time and the second time is greater than a first preset time interval, the current trigger position on the operation screen is updated based on the current hand position change. Therefore, by combining the time interval and movement direction dimensions to determine whether the gesture operation is effective, it avoids misjudgments caused by hand tremors or shaking, ensures the accuracy of the trigger operation position, guarantees the stability of the trigger position during gesture tracking, and improves the user's interactive experience. Attached Figure Description

[0012] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0013] Figure 1 A flowchart illustrating a method for determining a trigger location provided in an embodiment of this disclosure;

[0014] Figure 2 A schematic diagram of a scenario for determining a trigger location according to an embodiment of this disclosure;

[0015] Figure 3 A flowchart illustrating another method for determining a trigger location provided in an embodiment of this disclosure;

[0016] Figure 4 A schematic diagram of a scenario for determining another trigger location provided in an embodiment of this disclosure;

[0017] Figure 5 A flowchart illustrating another method for determining a trigger location provided in an embodiment of this disclosure;

[0018] Figure 6 A flowchart illustrating another method for determining a trigger location provided in an embodiment of this disclosure;

[0019] Figure 7 A flowchart illustrating another method for determining a trigger location provided in an embodiment of this disclosure;

[0020] Figure 8 A flowchart illustrating another method for determining a trigger location provided in an embodiment of this disclosure;

[0021] Figure 9 A schematic diagram of a scenario for determining another trigger location provided in an embodiment of this disclosure;

[0022] Figure 10 A schematic diagram of a scenario for determining another trigger location provided in an embodiment of this disclosure;

[0023] Figure 11 A flowchart illustrating another method for determining a trigger location provided in an embodiment of this disclosure;

[0024] Figure 12 A schematic diagram of a trigger position determination device provided in an embodiment of this disclosure;

[0025] Figure 13 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0026] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0027] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0028] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0029] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0030] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0031] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0032] To address the aforementioned issues, this disclosure provides a method for determining the trigger position. This method is based on the ability to detect hand tremors caused by user fatigue or tension during gesture tracking in real time and eliminate the resulting drift in the trigger position. This ensures the stability of the trigger position tracked during gesture recognition. This embodiment utilizes real-time coordinate information of the hand in the image as the data source for tremor elimination, combined with movement direction parameters in two-dimensional space and time intervals to effectively identify and eliminate tremors. It can effectively distinguish between the user's subjective movements and unconscious tremors, ensuring that the trigger position during gesture tracking is not affected by misjudgments of hand tremors. The solution is transferable and adaptable.

[0033] The method will be described below with reference to specific embodiments.

[0034] Figure 1 This is a flowchart illustrating a method for determining a trigger position according to an embodiment of this disclosure. This method can be executed by a trigger position determining device, which can be implemented in software and / or hardware, and is generally integrated into an electronic device. Figure 1 As shown, the method includes:

[0035] Step 101: Obtain the change in the current hand position when the user performs a gesture operation, and obtain the movement direction parameter value corresponding to the change in the current hand position.

[0036] In this embodiment, the movement direction parameter value can be understood as any parameter that reflects the current hand movement direction, such as cosine similarity, sine similarity, movement angle, etc., which will not be listed here.

[0037] The change in hand position reflects the change between the hand's position in the current frame and the hand's position in the previous frame. This change in hand position is a positive value. For example, in some possible embodiments, if the hand position is identified by pixel coordinates, i.e., when capturing an image of the user's hand, pixels are typically used as the unit of description for the hand's position coordinate information. The horizontal axis represents the number of pixels from the left edge of the image data captured by the current camera, and the vertical axis represents the number of pixels from the top edge of the image data captured by the current camera. Therefore, all coordinate information is positive. In this case, such as... Figure 2 As shown, the hand position change flow is determined by the number of pixels on the horizontal and vertical axes of the hand coordinates A in the current frame and the hand coordinates B in the previous frame.

[0038] It is easy to understand that if the gesture operation is caused by the tremor or shaking of the user's hand, the direction of movement of such a gesture operation usually changes significantly. However, if the gesture operation is a related behavior of the user, the change in the direction of movement is obviously relatively small, such as a change in the horizontal or vertical direction. Therefore, in this embodiment, after obtaining the change in the current hand position when the user performs a gesture operation, the movement direction parameter value corresponding to the change in the current hand position is obtained, so as to preliminarily determine whether the gesture operation is a conscious action based on the change in the movement direction.

[0039] When obtaining the change in the current hand position when the user performs a gesture operation, the position information of the current key points of the hand can be identified based on image recognition technology as the current hand position, and the change in the current hand position can be obtained by comparing the current hand position with the previous hand position.

[0040] Step 102: Determine the change in position of the first reference hand that meets the preset tremor condition based on the movement direction parameter value, and determine the first time of the change in position of the first reference hand.

[0041] It is easy to understand that even if the direction of movement conforms to the pattern of tremor or shaking, it does not mean that the current gesture operation is caused by tremor or shaking. Therefore, in order to avoid misjudgment, the time interval is further combined to determine whether the gesture operation is valid.

[0042] In this embodiment, the amount of hand position change that meets the preset tremor condition is determined based on the movement direction parameter value, and the first time of the hand position change is determined. This first time can be the system time when the hand position change is collected, etc. Obviously, in this embodiment, it is necessary to mark the time when the hand position change meets the preset detection condition.

[0043] In one embodiment of this disclosure, such as Figure 3 As shown, the first time to determine the change in hand position that meets the preset tremor condition based on the movement direction parameter value includes:

[0044] Step 301: Determine whether the movement direction parameter value meets the preset vibration condition.

[0045] In this embodiment, it is determined whether the movement direction parameter value corresponding to the change in the current hand position meets the preset vibration condition. For example, it is determined whether the movement direction parameter value is greater than the preset movement direction parameter threshold. If it is not greater than the preset movement direction parameter threshold, it is considered that the preset vibration condition is met.

[0046] Step 302: If the preset tremor condition is met, then the current hand position change is determined as the first reference hand position change.

[0047] In one embodiment of this disclosure, if a preset tremor condition is met, it means that the movement direction parameter value corresponding to the current hand position change conforms to the tremor pattern. Therefore, the current hand position change is determined as the first reference hand position change, and the current time corresponding to the current hand position change is determined as the first time. The time corresponding to the current hand position change is used as a judgment time to further determine whether the hand is actually trembling.

[0048] In one embodiment of this disclosure, if the preset tremor condition is not met, the previous reference hand position change that met the preset tremor condition before the current hand position change is obtained as the first reference hand position change, that is, the reference hand position change that met the preset tremor condition in the previous frame is obtained, and the time corresponding to the reference hand position change is determined as the first time.

[0049] In one embodiment of this disclosure, in order to improve the efficiency of determining the first time, during the actual execution process, the change in hand position that meets the preset tremor condition is timestamped and the timestamps are stored in the form of a list, so that the corresponding timestamps can be read directly.

[0050] For example, after obtaining the movement direction parameter value corresponding to the change in the current hand position, such as Figure 4 As shown, if the movement direction parameter value meets the preset tremor condition, the timestamp s corresponding to the current hand position change is recorded and stored in the last position of the list. The list stores the timestamps of hand position changes that meet the preset tremor condition in the storage order.

[0051] Step 103: Obtain the second time of the second reference hand position change that met the preset tremor condition before the first time.

[0052] It's easy to understand that trembling behavior is usually relatively continuous in time, rather than suddenly occurring sporadically in a single frame. Therefore, we obtain the second time of the previous hand position change that met the preset trembling conditions before the first time. That is, if the movement direction parameter value corresponding to the current hand position change does not meet the preset trembling conditions, we obtain the second time of the previous hand position change that met the preset trembling conditions before the hand position change that is closest to the current hand position change. If the movement direction parameter value corresponding to the current hand position change does not meet the preset trembling conditions, we obtain the second time of the hand position change that is closest to the current hand position change and meets the preset trembling conditions. This second time can be obtained by reading the timestamps in the list, etc.

[0053] Step 104: If the time interval between the first time and the second time is greater than the first preset time interval, then update the current trigger position on the operation screen according to the current hand position change.

[0054] The first preset time interval is determined based on experimental data.

[0055] In this embodiment, the time interval between the first time and the second time is calculated. If the time interval is greater than the first preset time interval, it indicates that the current change in hand position may not be a continuation of the tremor behavior. Therefore, the current trigger position on the operation screen is updated according to the current change in hand position. For example, the coordinate movement position corresponding to the current change in hand position is calculated, and the current trigger position is moved according to the coordinate movement position to achieve tracking and movement of the trigger position.

[0056] To enable those skilled in the art to more clearly understand the method for determining the trigger position in the embodiments of this disclosure, specific examples are provided below for illustration:

[0057] like Figure 5 As shown, hand image frames are acquired according to a preset acquisition frequency. The hand image frame of the current frame is analyzed to obtain the hand position information of the current frame. Based on the hand information of the current frame and the hand position information obtained in the previous frame, the change in the current hand position is calculated. Then, the movement direction parameter value corresponding to the change in the current hand position is obtained.

[0058] The system determines whether the movement direction parameter value is greater than a preset movement direction parameter threshold. If it is not greater than the preset movement direction parameter threshold, the preset tremor condition is met, and it is considered that the hand may have trembled once. Therefore, a timestamp of the current hand position change is generated and stored based on the current first time. If the preset tremor condition is not met, the system further obtains the previous reference hand position change that met the preset tremor condition before the current hand position change, and determines the time corresponding to the reference hand position change as the first time. Then, the system obtains the second time of the previous hand position change that met the preset tremor condition before the first time. The time interval between the first time and the second time is calculated. If the time interval between the first time and the second time is greater than a first preset time interval, the current gesture operation is considered to be a conscious behavior rather than a tremor behavior. Therefore, the current trigger position on the operation screen is updated based on the current hand position change, such as updating the cursor position on the operation screen.

[0059] In summary, the trigger position determination method of this disclosure obtains the change in the current hand position when the user performs a gesture operation, and obtains the movement direction parameter value corresponding to the current hand position change. Then, based on the movement direction parameter value, it determines a first time when the hand position change meets a preset tremor condition, and obtains a second time when the previous hand position change meeting the preset tremor condition was obtained before the first time. If the time interval between the first time and the second time is greater than a first preset time interval, the current trigger position on the operation screen is updated based on the current hand position change. Therefore, by combining the time interval and movement direction dimensions to determine whether the gesture operation is effective, misjudgments caused by hand tremors or shaking are avoided, ensuring the accuracy of the trigger operation position, ensuring the stability of the trigger position during gesture tracking, and improving the user's interactive experience.

[0060] To more clearly illustrate how the movement direction parameter value is obtained, the following example uses cosine similarity as the movement direction parameter value.

[0061] In one embodiment of this disclosure, such as Figure 6As shown, the movement direction parameter value corresponding to the change in the current hand position is obtained, including:

[0062] Step 601: Obtain the change in hand position in the previous frame when the user performs a gesture operation.

[0063] In the embodiments of this disclosure, the latest two frames of hand position change can always be stored in a preset database. Therefore, after obtaining the current hand position change, the previous frame of hand position change when the user performed the gesture operation can be obtained by querying the preset database.

[0064] Step 602: Calculate the cosine similarity between the current hand position change and the hand position change in the previous frame.

[0065] Step 603: Determine the cosine similarity as the moving direction parameter value.

[0066] In the embodiments of this disclosure, in order to determine the change in the direction of hand movement, the cosine similarity between the current change in hand position and the change in hand position in the previous frame is calculated, and the cosine similarity is determined as the movement direction parameter value.

[0067] In one embodiment of this disclosure, the obtained current hand position change and the previous frame hand position change can be input into a pre-trained computational model to obtain the cosine similarity output by the computational model.

[0068] In another embodiment of this disclosure, such as Figure 7 As shown, if the hand position change includes both horizontal and vertical position changes, then the cosine similarity between the current hand position change and the hand position change in the previous frame is calculated, including:

[0069] Step 701: Calculate the first product of the first horizontal position change corresponding to the current hand position change and the second horizontal position change corresponding to the hand position change in the previous frame.

[0070] The first horizontal position change can be understood as the change in the X-axis coordinates from the current hand position to the hand position in the previous frame, and the second horizontal position change can be understood as the change in the X-axis coordinates from the hand position in the previous frame to the hand position in the frame before that.

[0071] In this embodiment, the first product of the first horizontal position change corresponding to the current hand position change and the second horizontal position change corresponding to the hand position change in the previous frame is calculated.

[0072] Step 702: Calculate the second product of the first vertical position change corresponding to the current hand position change and the second vertical position change corresponding to the hand position change in the previous frame.

[0073] The first vertical position change can be understood as the change in the Y-axis coordinate between the current hand position and the hand position in the previous frame, and the second vertical position change can be understood as the change in the Y-axis coordinate between the hand position in the previous frame and the hand position in the frame before that.

[0074] In this embodiment, the second product of the first vertical position change corresponding to the current hand position change and the second vertical position change corresponding to the hand position change in the previous frame is calculated.

[0075] Step 703: Summing the first product value and the second product value to obtain the first sum value.

[0076] Step 704: Calculate the first square root of the sum of the squares of the first horizontal position change and the first vertical position change, and calculate the second square root of the sum of the squares of the second horizontal position change and the second vertical position change.

[0077] In this embodiment, the square value of the first horizontal position change and the square value of the first vertical position change can be calculated, the two square values ​​can be summed to obtain a sum of squares, and the square root of the sum of squares can be obtained to obtain the first square root value.

[0078] Similarly, we can calculate the square of the second horizontal position change and the square of the second vertical position change, sum the two squares to get the sum of squares, and take the square root of the sum of squares to get the second square root value.

[0079] Step 705: Summing the first square root value and the second square root value to obtain the second sum value.

[0080] Step 706: Calculate the ratio of the first summation value to the second summation value to obtain the cosine similarity.

[0081] In this embodiment, if Δh represents the first horizontal position change, Δv represents the first vertical position change, prevΔh represents the second horizontal position change, prevΔv represents the second vertical position change, and α represents the cosine similarity, then in this embodiment, the corresponding cosine similarity can be calculated using the following formula (1):

[0082]

[0083] Therefore, the two-dimensional cosine similarity of pixel coordinate changes takes values ​​between [-1, 1], reflecting the degree of change in the direction of hand movement in two-dimensional space; the smaller the value, the greater the change in the direction of hand movement. Since hand tremors caused by user fatigue, tension, etc., are usually manifested as changes in pixel coordinates, the cosine similarity of pixel coordinate changes can be used as a basis for tremor judgment. In addition, since pixel two-dimensional coordinates can most directly reflect the planar state of the current key points of the hand, it can achieve a high recognition rate and low false positive rate for hand tremors, and has stronger portability and adaptability to different hardware devices and solutions, improving the efficiency of judgment and reducing the complexity of the judgment algorithm.

[0084] For example, in this embodiment, if the two-dimensional cosine similarity α of the pixel coordinate change is less than a preset threshold, it is considered that the hand has experienced an unrelated tremor behavior, and the behavior timestamp is recorded at this time.

[0085] In summary, the trigger position determination method of this disclosure uses the change in pixel coordinates of the hand in two-dimensional space as the data source required for tremor elimination judgment. Combined with the tremor discrimination method based on cosine similarity detection of two-dimensional vectors, it can more accurately distinguish between unconscious tremor actions and conscious actions of the user. It has strong transferability and adaptability, can effectively improve the recognition rate of hand tremors, and ensure the stability of the tracking cursor during gesture tracking.

[0086] Based on the above embodiments, if the time interval between the first time and the second time is less than or equal to the first preset time interval, it is not directly determined that the current hand is trembling. Therefore, further verification is required. That is, if the time interval between the two most recent trembling behaviors exceeds the first preset time interval, it is considered that the key points of the hand have not undergone high-frequency posture changes, and the current gesture tracking is determined to be in a conscious behavior state. Conversely, if the time interval between the two most recent trembling behaviors is less than or equal to the first preset time interval, it is considered that the interval between the two recent trembling behaviors is very short, and it may still be in trembling, requiring further verification.

[0087] In one embodiment of this disclosure, such as Figure 8 As shown, the method also includes:

[0088] Step 801: If the time interval between the first time and the second time is less than or equal to the first preset time interval, then determine the current time corresponding to the current change in hand position.

[0089] In this embodiment, if the time interval between the first time and the second time is less than or equal to the first preset time interval, the current time corresponding to the current change in hand position is determined so as to further determine whether the hand operation is in tremor based on the current time.

[0090] If the current hand position change data includes time information, then the time information can be extracted as the first time. If it does not include time information, then the system time at which the current hand position change data was obtained is determined as the first time.

[0091] Step 802: Determine whether the time interval between the current time and the first time is greater than the second preset time interval.

[0092] In this embodiment, to determine whether the current hand operation is a continuation of the previous tremor, it is determined whether the time interval between the current time and the first time is greater than a second preset time interval. The second preset time interval can be calibrated based on experimental data.

[0093] Step 803: If the time interval is greater than the second preset time interval, the current trigger position is determined based on the change in the current hand position.

[0094] In this embodiment, if the time interval is greater than the second preset time interval threshold, it is considered that the hand has stopped high-frequency posture changes, and the current gesture tracking is determined to be in a conscious behavior state. Therefore, the current trigger position is determined based on the current hand position change.

[0095] Conversely, if the change is less than or equal to the second preset time interval, it is determined whether there is a third reference hand position change that satisfies the preset valid operation conditions between the first time and the current time. If there is a third reference hand position change that satisfies the preset valid operation conditions, the current trigger position on the operation screen is determined based on the current hand position change.

[0096] If there is no third reference hand position change that meets the preset valid operation conditions, the trigger position will not move. For example, the current trigger position will remain unchanged, or the movement of the trigger position can be limited to create a visual effect of no movement. The ratio of the current hand position change to a preset compression ratio (a preset value greater than 1) is calculated to reduce the hand position change, and then the current trigger position is updated based on this ratio.

[0097] For example, such as Figure 9 As shown, if the point P1 where the current hand position change is located is the change in X-axis coordinate x1 and the change in Y-axis coordinate y1, if the time interval between the first time and the second time is less than or equal to the first preset time interval, and the time interval between the current time corresponding to the current hand position change and the first time is greater than the second preset time interval, then the trigger position on the operation screen can be moved from S1 to S2, and the user can intuitively see the movement from S1 to S2.

[0098] Conversely, such as Figure 10As shown, if the time interval between the current time corresponding to the current hand position change and the first time is less than or equal to the second preset time interval, it is determined whether there is a third reference hand position change that meets the preset valid operation conditions between the first time and the current time. If there is a third reference hand position change, the current trigger position is determined according to the current hand position change, that is, the trigger position on the operation screen can be moved from S1 to S2. The user can intuitively see the movement from S1 to S2. If there is no third reference hand position change, the ratio of the current hand position change to the preset compression ratio is calculated, that is, the current hand position change is compressed. The compression ratio can be the ratio of the current hand key point coordinate change to the preset threshold. The coordinates in the horizontal and vertical directions are compressed proportionally, and the final result is applied to the actual coordinate update on the operation screen. The trigger position is updated from S1 to S3 according to the ratio. At this time, the displacement of S3 relative to S1 is not obvious and is difficult for the user to observe visually, thereby achieving tremor elimination.

[0099] If the time interval is less than the second preset time interval, it is considered that the current timestamp is close enough to the most recent tremor event and may still be in the tremor state. The system continues to check whether any obvious action has occurred within the time interval between the current time and the most recent tremor timestamp. If any obvious action has occurred within the time interval between the current time and the most recent tremor timestamp, the current gesture tracking is determined to be in a conscious action state. Conversely, if no obvious action has occurred within the second preset time interval, it is considered that the current event is close enough to the most recent tremor time and the current gesture operation is determined to be in an unconscious tremor state.

[0100] Based on the above description, during the tracking of gesture operations, it is also possible to identify whether each current hand position change meets the preset valid operation conditions, and mark the timestamp of the hand position change that meets the preset valid operation conditions. The corresponding timestamps can be stored in a list for easy subsequent querying.

[0101] Before determining whether there is a third reference hand position change that meets the preset valid operation conditions between the first time and the current time, the movement distance corresponding to the current hand position change is calculated. If the current hand position change is a change in the X-axis and Y-axis coordinates, the square root of the change in the X-axis and Y-axis coordinates is calculated as the corresponding movement distance. If the movement distance is greater than the preset movement distance threshold, it is determined that the current hand position change meets the preset valid operation conditions. Thus, the corresponding timestamp can be determined and stored.

[0102] Therefore, when determining whether the change in the current hand position meets the tremor condition, the square root of the change in the horizontal and vertical two-dimensional coordinates corresponding to the change in the current hand position is simultaneously calculated as a measure of the overall hand coordinate change. If this change exceeds a preset threshold, it is considered that a significant hand movement has occurred, and a timestamp indicating that the preset valid operation condition is met is recorded; if the change does not exceed the preset threshold, it is considered that no significant movement has occurred. Thus, using the change in pixel coordinates of the current hand position in two-dimensional space as the data source required for tremor elimination judgment, combined with a tremor discrimination method based on cosine similarity detection of two-dimensional vectors, it can accurately distinguish between unconscious tremor movements and conscious movements of the user. It has strong transferability and adaptability, can effectively improve the recognition rate of hand tremors, and ensure the stability of the tracking cursor during gesture tracking.

[0103] To make the process of determining the trigger position in the embodiments of this disclosure clearer to those skilled in the art, the following examples illustrate the process. In this example, the change in the current hand position is the change in the coordinates of the horizontal and vertical axes, and the movement direction parameter is the cosine similarity. The explanation is as follows:

[0104] In this embodiment, as Figure 11 As shown, image frames of the user's hand performing a gesture are captured by a camera device. The current horizontal and vertical coordinates of the hand are identified based on the hand image frames. The change in the current hand position is calculated using the horizontal and vertical coordinates of the hand in the previous adjacent hand image frame. Then, a cosine similarity is calculated based on the change in the current hand position. If the cosine similarity is greater than a preset cosine similarity threshold, the time interval between the two most recent reference hand position changes that meet the preset tremor condition is obtained. If the cosine similarity is not greater than the preset cosine similarity threshold, the timestamp corresponding to the current hand position change is recorded as the timestamp that meets the preset tremor condition, and the current position change is recorded as the first reference hand position change. The second time of the previous second reference hand position change that meets the preset tremor condition and is adjacent to the current hand position change is obtained, which is also the time interval between the two most recent reference hand position changes that meet the preset tremor condition.

[0105] Synchronously, after calculating the change in the current hand position using the horizontal and vertical coordinates of the hand in the previous adjacent hand image frame, the system calculates the movement distance corresponding to the change in the current hand position and determines whether the movement distance is greater than a preset movement distance threshold. If it is greater than the preset movement distance threshold, the system records the timestamp of the current hand position change meeting the preset valid operation conditions.

[0106] Furthermore, after calculating the time interval corresponding to the two most recent satisfying preset tremor conditions, it is compared whether the time interval is greater than the first preset time interval. If it is greater than the first preset time interval, the current trigger position on the operation screen is updated according to the current hand position change, and gesture tracking continues.

[0107] If the time interval is less than or equal to the first preset time interval (i.e., the tremor interval is very short), to determine whether the current gesture operation is still in tremor mode, the current time corresponding to the current hand position change is determined, and it is determined whether the time interval between the current time and the first time interval is greater than the second preset time interval. If it is greater than the second preset time interval, the current trigger position is determined based on the current hand position change, and gesture tracking continues.

[0108] If the time interval is less than or equal to the second preset time interval, that is, the current time of the current gesture operation is close enough to the time of the most recent tremor gesture operation, it is necessary to further determine whether there is a third reference hand position change that meets the preset valid operation conditions between the first time and the current time. If there is a third reference hand position change, the current trigger position is determined based on the current hand position change, and gesture tracking continues.

[0109] Otherwise, if there is no third reference hand position change, the current trigger position is not updated, or the current hand position change is compressed (not shown in the figure), and the current trigger position is updated according to the compressed ratio to achieve the effect of tremor elimination visually.

[0110] In summary, the trigger position determination method of this disclosure uses the coordinate change of the current hand position change in a two-dimensional image as the data source required for judging and eliminating tremors. It has strong transferability and adaptability. Based on two-dimensional spatial cosine similarity detection, it distinguishes between hand tremors and subjective movements, and achieves effective judgment and targeted elimination of tremor signal frames while ensuring subjective movement. This ensures that the stability of following the cursor is not affected by unconscious hand tremors during the user's gesture tracking process, and improves the overall stability of gesture tracking.

[0111] To implement the above embodiments, this disclosure also proposes a trigger position determination device. Figure 12 This is a schematic diagram of a trigger position determination device provided in an embodiment of the present disclosure. This device can be implemented by software and / or hardware, and is generally integrated into an electronic device for determining the trigger position. Figure 12 As shown, the device includes: a first acquisition module 1210, a determination module 1220, a second acquisition module 1230, and a location update module 1240, wherein,

[0112] The first acquisition module 1210 is used to acquire the change in the current hand position when the user performs a gesture operation, and to acquire the movement direction parameter value corresponding to the change in the current hand position.

[0113] The determination module 1220 is used to determine the change in position of the first reference hand that meets the preset tremor condition based on the movement direction parameter value, and to determine the first time of the change in position of the first reference hand.

[0114] The second acquisition module 1230 is used to acquire the second time of the previous second reference hand position change that met the preset tremor condition before the first time.

[0115] The position update module 1240 is used to update the current trigger position on the operation screen according to the current hand position change when the time interval between the first time and the second time is greater than the first preset time interval.

[0116] The trigger position determination device provided in this disclosure can execute the trigger position determination method provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects of the execution method. The implementation principle is similar and will not be repeated here.

[0117] To implement the above embodiments, this disclosure also proposes a computer program product, including a computer program / instruction, which, when executed by a processor, implements the method for determining the trigger position in the above embodiments.

[0118] Figure 13 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure.

[0119] The following is a detailed reference. Figure 13 The diagram illustrates a structural schematic suitable for implementing the electronic device 1300 in the embodiments of this disclosure. The electronic device 1300 in the embodiments of this disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 13 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0120] like Figure 13As shown, the electronic device 1300 may include a processor (e.g., a central processing unit, a graphics processing unit, etc.) 1301, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 1302 or a program loaded from memory 1308 into random access memory (RAM) 1303. The RAM 1303 also stores various programs and data required for the operation of the electronic device 1300. The processor 1301, ROM 1302, and RAM 1303 are interconnected via a bus 1304. An input / output (I / O) interface 1305 is also connected to the bus 1304.

[0121] Typically, the following devices can be connected to I / O interface 1305: input devices 1306 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 1307 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 1308 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1309. Communication device 1309 allows electronic device 1300 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 13 An electronic device 1300 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0122] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 1309, or installed from memory 1308, or installed from ROM 1302. When the computer program is executed by processor 1301, it performs the functions defined in the method for determining the trigger location of embodiments of this disclosure.

[0123] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0124] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0125] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0126] The aforementioned computer-readable medium carries one or more programs. When these programs are executed by the electronic device, the electronic device: acquires the change in hand position when the user performs a gesture operation, acquires the movement direction parameter value corresponding to the current hand position change, then determines a first time when the hand position change meets a preset tremor condition based on the movement direction parameter value, and acquires a second time when the previous hand position change met the preset tremor condition before the first time. If the time interval between the first time and the second time is greater than a first preset time interval, the current trigger position on the operation screen is updated based on the current hand position change. Thus, by combining the time interval and movement direction dimensions to determine the validity of the gesture operation, misjudgments caused by hand tremors or shaking are avoided, ensuring the accuracy of the trigger position, the stability of the trigger position during gesture tracking, and improving the user's interactive experience.

[0127] Electronic devices can be programmed with computer program code in one or more programming languages ​​or combinations thereof to perform the operations of this disclosure. These programming languages ​​include, but are not limited to, object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as "C" or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0128] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0129] The units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the units are not, in some cases, intended to limit the specific unit.

[0130] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0131] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0132] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0133] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0134] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A method for determining a trigger position, characterized in that, Includes the following steps: Obtain the change in the current hand position when the user performs a gesture operation, and obtain the movement direction parameter value corresponding to the change in the current hand position; When the movement direction parameter value meets the preset vibration condition, the current hand position change is determined as the first reference hand position change. When the movement direction parameter value does not meet the preset vibration condition, the previous hand position change that met the preset vibration condition is obtained as the first reference hand position change. The first time to determine the change in the position of the first reference hand; Obtain the second time before the first time that the second reference hand position change that met the preset tremor condition was obtained; If the time interval between the first time and the second time is greater than the first preset time interval, the current trigger position on the operation screen is updated according to the current hand position change.

2. The method as described in claim 1, characterized in that, The step of obtaining the movement direction parameter value corresponding to the current hand position change includes: Get the change in hand position in the previous frame when the user performs a gesture operation; Calculate the cosine similarity between the current hand position change and the hand position change in the previous frame; The cosine similarity is determined as the value of the movement direction parameter.

3. The method as described in claim 2, characterized in that, If the change in hand position includes both horizontal and vertical position changes, then calculating the cosine similarity between the current hand position change and the hand position change in the previous frame includes: Calculate the first product of the first horizontal position change corresponding to the current hand position change and the second horizontal position change corresponding to the hand position change in the previous frame; Calculate the second product of the first vertical position change corresponding to the current hand position change and the second vertical position change corresponding to the hand position change in the previous frame; Summing the first product value and the second product value yields a first summation value; Calculate the first square root of the sum of the squares of the first horizontal position change and the first vertical position change, and calculate the second square root of the sum of the squares of the second horizontal position change and the second vertical position change. Summing the first square root value and the second square root value yields a second summation value; The ratio of the first summation value to the second summation value is calculated to obtain the cosine similarity.

4. The method as described in claim 1, characterized in that, The step of determining whether the movement direction parameter value meets the preset vibration condition includes: Determine whether the value of the movement direction parameter is greater than a preset movement direction parameter threshold. If the value is not greater than the preset movement direction parameter threshold, then the preset tremor condition is determined to be met.

5. The method as described in claim 1, characterized in that, Also includes: If the time interval between the first time and the second time is less than or equal to the first preset time interval, then the current time corresponding to the current hand position change is determined; Determine whether the time interval between the current time and the first time is greater than a second preset time interval; If the time interval is greater than the second preset time interval, the current trigger position is determined based on the change in the current hand position.

6. The method as described in claim 5, characterized in that, Also includes: If it is less than or equal to the second preset time interval, then determine whether there is a third reference hand position change that satisfies the preset valid operation conditions between the first time and the current time; If the third reference hand position change exists, the current trigger position is determined based on the current hand position change.

7. The method as described in claim 6, characterized in that, If the third reference hand position change does not exist, then calculate the ratio of the current hand position change to the preset compression ratio; Update the current trigger position based on the ratio.

8. The method of claim 6, wherein prior to the feature, Before determining whether there is a third reference hand position change that satisfies preset valid operating conditions between the first time and the current time, the method further includes: Calculate the distance moved corresponding to the change in the current hand position; If the moving distance is greater than a preset moving distance threshold, then the change in the current hand position is determined to meet the preset valid operation conditions.

9. A device for determining a trigger position, characterized in that, include: The first acquisition module is used to acquire the change in the current hand position when the user performs a gesture operation, and to acquire the movement direction parameter value corresponding to the change in the current hand position; The determination module is used to determine the current hand position change as the first reference hand position change when the movement direction parameter value meets the preset vibration condition, and to obtain the previous hand position change that meets the preset vibration condition as the first reference hand position change when the movement direction parameter value does not meet the preset vibration condition, and to determine the first time of the first reference hand position change. The second acquisition module is used to acquire the second time of the previous second reference hand position change that met the preset tremor condition before the first time. The position update module is used to update the current trigger position on the operation screen according to the current hand position change when the time interval between the first time and the second time is greater than the first preset time interval.

10. An electronic device, characterized in that, The electronic device includes: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the executable instructions to implement the method for determining the trigger position as described in any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program for executing the method for determining the trigger position as described in any one of claims 1-8.

Images