Air gesture control method and device, vehicle and storage medium

Abstract

The invention provides an air gesture control method and device, a vehicle and a storage medium, the method is applied to the field of vehicles, and the method comprises the steps that a target air gesture is recognized, and a finger moving distance corresponding to the target air gesture is acquired; determining the current movable range of the virtual control on the screen; determining the number of intervals matched with the movable range; dividing the maximum distance of one-way movement of the finger into a plurality of continuous finger distance intervals according to the number of the intervals; in a plurality of continuous finger distance intervals, determining a target finger distance interval where the finger moving distance is located; and controlling the virtual control to move based on the control moving distance corresponding to the target finger distance interval, and executing an operation instruction corresponding to the movement of the virtual control. By means of the method, the user can conduct gesture control with a small motion range, and therefore the feeling of fatigue caused by long-time gesture control is effectively avoided.

Description

Technical Field

[0001] The present application relates to the field of vehicles, and more specifically, to a method, device, vehicle and storage medium for air gesture control in the field of vehicles. Background Art

[0002] In the prior art, although the in-vehicle sensors have achieved a large coverage range, they are insufficient in terms of gesture operation response. Users need to raise their arms and swing them more significantly to ensure that the sensors can recognize gestures more accurately. However, this operation method of keeping the arms raised for a long time and performing large-scale movements causes users to easily feel fatigued during long-term operation. Therefore, when users perform gesture control operations for a long time, how to effectively improve their fatigue has become a technical problem that needs to be solved urgently. Summary of the invention

[0003] The present application provides a method, device, vehicle and storage medium for air gesture control. Through this method, a user can perform gesture control with a smaller movement amplitude, thereby effectively avoiding fatigue caused by long-term gesture control.

[0004] In a first aspect, a method for air gesture control is provided, characterized in that the method includes: identifying a target air gesture, and obtaining a finger movement distance corresponding to the target air gesture; determining a current movable range of a virtual control on the screen; determining the number of intervals matching the movable range; dividing the maximum distance of a finger's one-way movement into a plurality of continuous finger distance intervals according to the number of intervals; wherein each of the finger distance intervals corresponds to a control movement distance, and the control movement distance is determined based on the movable range and the number of intervals; among the plurality of continuous finger distance intervals, determining a target finger distance interval in which the finger movement distance is located; based on the control movement distance corresponding to the target finger distance interval, controlling the movement of the virtual control, and executing an operation instruction corresponding to the movement of the virtual control.

[0005] In the above technical solution, the change of the movable range of the virtual control on the screen will affect the change of the number of intervals matching the movable range, so that when the maximum distance of the finger's one-way movement is divided into multiple continuous finger distance intervals according to the number of intervals, different finger distance intervals can be obtained based on different numbers of intervals, that is, when the movable range of the virtual control on the screen changes dynamically, the interval range of the finger distance interval is also in dynamic change. Because each finger distance interval corresponds to its own control movement distance, and the control movement distance is determined based on the above movable range and the above number of intervals, that is, the control movement distance corresponding to each finger distance interval also changes with the movable range. Therefore, by determining the finger distance interval in which the finger movement distance is located, the control movement distance corresponding to the finger distance interval can be obtained, and the movement of the virtual control is controlled based on the control movement distance, which means that even if the finger movement distance is the same, if the movable range of the virtual control on the screen is different, the same finger movement distance may also correspond to different control movement distances, avoiding not considering the movable range of the virtual control on the screen, and determining the control movement distance corresponding to the finger movement distance based on a fixed corresponding relationship in any case, resulting in low control accuracy. In the above technical solution, it is equivalent to dynamically adjusting the correspondence between the finger distance interval and the control movement distance based on the current movable range of the virtual control on the screen, so that the correspondence can adapt to the current movable range of the virtual control on the screen, which is conducive to improving the control accuracy of the virtual control through the target air gesture. When the user is performing gesture control, even if the finger movement distance is very small, it can be accurately recognized and converted into the movement of the virtual control. The user can control the virtual control with a smaller gesture movement amplitude, which significantly reduces the burden brought by long-term operation. At the same time, by recognizing air gestures and directly converting them into control movement and corresponding operation instructions, the intermediate steps such as clicking and sliding in traditional touch operations are reduced, which improves the comfort of the user experience and effectively avoids the fatigue caused by long-term gesture control.

[0006] In combination with the first aspect, in certain implementations of the first aspect, the maximum distance of a finger's one-way movement is divided into multiple continuous finger distance intervals according to the number of intervals, including: calculating multiple gesture thresholds based on the maximum distance of a finger's one-way movement and the number of intervals; wherein the number of the multiple gesture thresholds is the same as the number of intervals; constructing multiple continuous finger distance intervals based on the multiple gesture thresholds; wherein the upper limit value of each of the finger distance intervals is one of the gesture thresholds.

[0007] In combination with the first aspect and the above-mentioned implementation methods, in some implementation methods of the first aspect, the target air gesture corresponds to a finger moving direction, and controlling the movement of the virtual control based on the control moving distance corresponding to the target finger distance interval includes: determining the target object used by the virtual control to control in the screen; determining the control moving direction based on the finger moving direction and the target object; controlling the movement of the virtual control based on the control moving distance corresponding to the target finger distance interval and the control moving direction.

[0008] In combination with the first aspect and the above-mentioned implementations, in some implementations of the first aspect, determining the current movable range of the virtual control on the screen includes: determining the current movable range of the virtual control on the screen along the direction of movement of the control.

[0009] In combination with the first aspect and the above-mentioned implementation methods, in some implementation methods of the first aspect, determining the movement direction of the control based on the finger movement direction and the target object includes: when the finger movement direction does not move along the specified baseline, converting the finger movement direction into a target direction parallel to the specified baseline, and determining the target direction as the movement direction of the control; obtaining the finger movement distance corresponding to the target air gesture includes: obtaining the finger movement curve corresponding to the target air gesture, and determining the projection distance of the finger movement curve in the target direction as the finger movement distance corresponding to the target air gesture.

[0010] In combination with the first aspect and the above-mentioned implementation methods, in some implementation methods of the first aspect, obtaining the finger movement distance corresponding to the target air gesture includes: identifying the finger key point corresponding to the target air gesture, and obtaining multiple three-dimensional coordinates of the finger key point during the finger movement process; wherein the multiple three-dimensional coordinates include the coordinates of the finger key point before the movement starts and the coordinates of the finger key point after the movement ends; converting the multiple three-dimensional coordinates into multiple two-dimensional coordinates; performing curve fitting based on the multiple two-dimensional coordinates to obtain a finger movement curve; and determining the finger movement distance based on the finger movement curve.

[0011] The above technical solution converts multiple three-dimensional coordinates into multiple two-dimensional coordinates. This method can effectively reduce the complexity of data processing and calculation. Then, curve fitting is performed based on the multiple two-dimensional coordinates to obtain the finger movement curve. When performing curve fitting, a suitable curve can be selected for fitting, so as to obtain the trajectory of the finger movement more accurately.

[0012] In combination with the first aspect and the above-mentioned implementation methods, in some implementation methods of the first aspect, converting the multiple three-dimensional coordinates into multiple two-dimensional coordinates includes: if it is detected that the finger is not moving parallel to the screen, performing spatial coordinate transformation on the multiple three-dimensional coordinates to obtain multiple three-dimensional coordinates of the finger parallel to the screen, and converting the multiple three-dimensional coordinates of the finger parallel to the screen into multiple two-dimensional coordinates.

[0013] The above technical solution, through spatial rectangular coordinate transformation, can still realize air gesture control when the finger is not moving parallel to the screen, which is conducive to realizing that the hand can be placed in any detectable position. There is no need to require the user's fingers to face and be parallel to the screen to realize air gesture control, which improves the convenience of air gesture control and facilitates user operation.

[0014] In a second aspect, a device for air gesture control is provided, the device comprising:

[0015] An acquisition module is used to identify a target air gesture and acquire the finger movement distance corresponding to the target air gesture; a first determination module is used to determine the current movable range of the virtual control on the screen; a second determination module is used to determine the number of intervals matching the movable range; a division module is used to divide the maximum distance of a finger's one-way movement into a plurality of continuous finger distance intervals according to the number of intervals; wherein each of the finger distance intervals corresponds to a control movement distance, and the control movement distance is determined based on the movable range and the number of intervals; a third determination module is used to determine the target finger distance interval in which the finger movement distance is located in the plurality of continuous finger distance intervals; a control module is used to control the movement of the virtual control based on the control movement distance corresponding to the target finger distance interval, and execute operation instructions corresponding to the movement of the virtual control.

[0016] In combination with the second aspect and the above-mentioned implementation methods, in some implementation methods of the second aspect, the division module is specifically used to: calculate multiple gesture thresholds based on the maximum distance of one-way movement of the finger and the number of intervals; wherein the number of the multiple gesture thresholds is the same as the number of intervals; construct multiple continuous finger distance intervals based on the multiple gesture thresholds; wherein the upper limit value of each of the finger distance intervals is one of the gesture thresholds.

[0017] In combination with the second aspect and the above-mentioned implementation methods, in some implementation methods of the second aspect, the device also includes: a fourth determination module is used to: determine the control movement direction based on the finger movement direction; the control module is specifically used to: control the movement of the virtual control based on the control movement distance corresponding to the target finger distance interval and the control movement direction.

[0018] In combination with the second aspect and the above-mentioned implementation manners, in some implementation manners of the second aspect, the first determination module is specifically used to: determine the current movable range of the virtual control on the screen along the moving direction of the control.

[0019] In combination with the second aspect and the above-mentioned implementation methods, in some implementation methods of the second aspect, the fourth determination module is specifically used to: when the finger movement direction does not move along the specified baseline, convert the finger movement direction into a target direction parallel to the specified baseline, and determine the target direction as the movement direction of the control; an acquisition module is specifically used to obtain the finger movement curve corresponding to the target air gesture, and determine the projection distance of the finger movement curve in the target direction as the finger movement distance corresponding to the target air gesture.

[0020] In combination with the second aspect, in certain implementations of the second aspect, the acquisition module includes an acquisition unit, a conversion unit, a fitting unit and a determination unit, the acquisition unit being used to: identify the finger key points corresponding to the target air gesture, and obtain multiple three-dimensional coordinates of the finger key points during the finger movement process; wherein the multiple three-dimensional coordinates include the coordinates of the finger key points before the movement starts and the coordinates of the finger key points after the movement ends; the conversion unit is used to: convert the multiple three-dimensional coordinates into multiple two-dimensional coordinates; the fitting unit is used to: perform curve fitting based on the multiple two-dimensional coordinates to obtain a finger movement curve; the determination unit is used to: determine the finger movement distance based on the finger movement curve.

[0021] In combination with the second aspect, in certain implementations of the second aspect, the acquisition module is specifically used to: if it is detected that the finger is not moving parallel to the screen, perform spatial coordinate transformation on the multiple three-dimensional coordinates to obtain multiple three-dimensional coordinates of the finger parallel to the screen, and convert the multiple three-dimensional coordinates of the finger parallel to the screen into multiple two-dimensional coordinates.

[0022] In a third aspect, a vehicle is provided, comprising a memory and a processor. The memory is used to store executable program code, and the processor is used to call and run the executable program code from the memory, so that the vehicle executes the method in the first aspect or any possible implementation of the first aspect.

[0023] In a fourth aspect, a computer program product is provided, comprising: a computer program code, which, when executed on a computer, enables the computer to execute the method in the first aspect or any possible implementation of the first aspect.

[0024] In a fifth aspect, a computer-readable storage medium is provided, which stores a computer program code. When the computer program code runs on a computer, the computer executes the method in the above-mentioned first aspect or any possible implementation manner of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 is a structural schematic diagram of a vehicle provided in an embodiment of the present application;

[0026] Figure 2 is a schematic diagram of an air gesture provided in an embodiment of the present application;

[0027] Figure 3 is a schematic flow chart of a method for air gesture control provided by an embodiment of the present application;

[0028] Figure 4 is a schematic diagram of a gesture sample library provided in an embodiment of the present application;

[0029] Figure 5 is a schematic diagram of a coordinate system provided by an embodiment of the present application;

[0030] Figure 6 This is a schematic diagram of the maximum number of intervals corresponding to an interface layout provided in an embodiment of the present application;

[0031] Figure 7 It is a schematic diagram of a movable range of a virtual control currently on a vehicle screen provided by an embodiment of the present application;

[0032] Figure 8 is a schematic diagram of another movable range of a virtual control currently on a vehicle screen provided by an embodiment of the present application;

[0033] Fig. 9 This is a schematic diagram of the number of current intervals corresponding to an interface layout provided in an embodiment of the present application;

[0034] Fig.10 is a structural schematic diagram of a device for air gesture control provided by an embodiment of the present application;

[0035] Fig.11 It is a schematic diagram of the hardware structure of a vehicle provided in an embodiment of the present application. DETAILED DESCRIPTION

[0036] The technical solution in the present application will be described clearly and in detail below in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B: "and / or" in the text is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0037] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features.

[0038] At present, although the in-car sensors have achieved a large coverage range, they are still insufficient in terms of gesture operation response. Users need to raise their arms and swing them more significantly to ensure that the sensors can recognize gestures more accurately. However, this operation method of keeping the arms raised for a long time and performing large-scale movements causes users to easily feel fatigue during long-term operation. Therefore, when users perform gesture control operations for a long time, how to effectively improve their fatigue has become a technical problem that needs to be solved urgently.

[0039] Based on the problems existing in the above-mentioned related technologies, an embodiment of the present application provides a method of air gesture control. This method combines the current movable range of the virtual control on the screen and corresponds the finger movement distance to the control movement distance, so that the user can control the movement of the virtual control with subtle movements of the fingertips, effectively avoiding the fatigue caused by long-term and large-scale gesture control.

[0040] This method can be applied to vehicles with gesture recognition function and vehicle screen display function. Figure 1 It should be understood that the method of air gesture control provided by the embodiment of the present application can be applied to Figure 1 The vehicle 100 shown may include Figure 1 The vehicle 100 shown may have more or fewer components, which is not limited in the present embodiment. Figure 1 As shown, the vehicle 100 has a vehicle screen 101 and a gesture recognition device 102 .

[0041] Optionally, the gesture recognition device 102 may be a camera configured on the vehicle, and the camera may be integrated on the vehicle screen 101 , which is not limited in the embodiment of the present application.

[0042] Alternatively, the gesture recognition device 102 may be a sensor with a gesture recognition function configured on the vehicle, which is not limited in the embodiment of the present application.

[0043] In the following embodiments, a camera integrated on the vehicle screen 101 is used as the gesture recognition device 102 to introduce the method of air gesture control provided by the embodiments of the present application.

[0044] It should be understood that in the embodiment of the present application, the gestures collected by the gesture recognition device 102 are air gestures. Air gestures are an interactive method that uses technologies such as ultrasound or camera sensors to recognize user gestures. It allows users to accurately control a larger area on the car screen through only small-range movements without directly touching the car screen, thereby realizing intelligent interaction between man and machine.

[0045] Figure 2 is a schematic diagram of an air gesture provided by an embodiment of the present application, Figure 2 In the embodiment, the user can control a larger area 202 on the vehicle screen by only a small range of movements 201 without directly touching the vehicle screen, which significantly improves the user experience and effectively avoids fatigue caused by long-term gesture control.

[0046] Figure 3 It is a schematic flowchart of a method for air gesture control provided in an embodiment of the present application.

[0047] For example, Figure 3 As shown, the method 300 includes:

[0048] 301, identifying a target air gesture, and obtaining a finger movement distance corresponding to the target air gesture.

[0049] 302 , determining a current movable range of the virtual control on the screen.

[0050] 303, determine the number of intervals matching the movable range.

[0051] 304, dividing the maximum distance of a finger's one-way movement into a plurality of continuous finger distance intervals according to the number of intervals; wherein each finger distance interval corresponds to a control movement distance, and the control movement distance is determined based on the movable range and the number of intervals.

[0052] 305 , determining a target finger distance interval in which the finger moving distance is located among a plurality of continuous finger distance intervals.

[0053] 306 , based on the control movement distance corresponding to the target finger distance interval, control the movement of the virtual control, and execute the operation instruction corresponding to the movement of the virtual control.

[0054] In the embodiment of the present application, the change of the movable range of the virtual control on the screen will affect the change of the number of intervals matching the movable range, so that when the maximum distance of the finger's one-way movement is divided into multiple continuous finger distance intervals according to the number of intervals, different finger distance intervals can be obtained based on different numbers of intervals, that is, when the movable range of the virtual control on the screen changes dynamically, the interval range of the finger distance interval is also in dynamic change. Because each finger distance interval corresponds to its own control movement distance, and the control movement distance is determined based on the movable range and the number of intervals, that is, the control movement distance corresponding to each finger distance interval also changes with the movable range. Therefore, by determining the finger distance interval in which the finger movement distance is located, the control movement distance corresponding to the finger distance interval can be obtained, and the movement of the virtual control is controlled based on the control movement distance, which means that even if the finger movement distance is the same, if the movable range of the virtual control on the screen is different, the same finger movement distance may also correspond to different control movement distances, avoiding not considering the movable range of the virtual control on the screen, and determining the control movement distance corresponding to the finger movement distance based on a fixed corresponding relationship in any case, resulting in low control accuracy. In the above technical solution, it is equivalent to dynamically adjusting the correspondence between the finger distance interval and the control movement distance based on the current movable range of the virtual control on the screen, so that the correspondence can adapt to the current movable range of the virtual control on the screen, which is conducive to improving the control accuracy of the virtual control through the target air gesture. When the user is performing gesture control, even if the finger movement distance is very small, it can be accurately recognized and converted into the movement of the virtual control. The user can control the virtual control with a smaller gesture movement amplitude, which significantly reduces the burden brought by long-term operation. At the same time, by recognizing air gestures and directly converting them into control movement and corresponding operation instructions, the intermediate steps such as clicking and sliding in traditional touch operations are reduced, which improves the comfort of the user experience and effectively avoids the fatigue caused by long-term gesture control.

[0055] Below Figure 3 The implementation method of each step in the embodiment shown is specifically described:

[0056] For 301, it can be understood that the above-mentioned air gesture refers to a gesture or hand movement change made by the user without touching the car screen with the hand, but at a certain distance from the car screen. The target air gesture is an air gesture used to control the movement of virtual controls among the air gestures, such as an air left swipe gesture, an air right swipe gesture, an air up swipe gesture, an air down swipe gesture, and other air gestures with a swiping distance and swiping direction.

[0057] For example, the embodiment of the present application may predefine a gesture sample library, such as Figure 4 As shown, Figure 4 400 includes gestures such as finger sliding up and down 401, finger sliding left and right 402, finger tapping 403, and finger circling 404. Therefore, the above-mentioned recognition of the target air gesture may include: tracking the user's finger through a camera or a sensor to obtain a three-dimensional motion trajectory, mapping the three-dimensional motion trajectory to the gesture sample library, and determining which gesture sample in the gesture sample library the gesture formed by the three-dimensional motion trajectory is by calculating the matching degree between the gesture formed by the three-dimensional motion trajectory and each gesture sample in the gesture sample library. If it is determined that the gesture formed by the three-dimensional motion trajectory is a finger sliding up and down 401 or a finger sliding left and right 402, it is determined that the target air gesture is recognized.

[0058] In some embodiments, the above step of obtaining the finger movement distance corresponding to the target air gesture includes the following steps S11 to S14:

[0059] S11: Identify the finger key points corresponding to the target air gesture, and obtain multiple three-dimensional coordinates of the finger key points during the finger movement process.

[0060] Among them, the multiple three-dimensional coordinates include the coordinates of the finger key points before the movement starts and the coordinates of the finger key points after the movement ends. The above-mentioned finger key points are pre-set key points that can represent the movement of fingers. The finger key points may include but are not limited to the thumb and index finger joints. The above-mentioned multiple three-dimensional coordinates are collected by the gesture recognition device 102 during the user's finger movement. The multiple three-dimensional coordinates include the coordinates of the finger key points before the movement starts (recorded as the initial three-dimensional coordinates) and the coordinates of the finger key points after the finger movement ends (recorded as the final three-dimensional coordinates), and the positions of the above-mentioned coordinates are displayed through the three axes of xyz.

[0061] In the specific implementation, in order to facilitate the subsequent accurate finger movement curve, more three-dimensional coordinates of the finger key points during the movement process can also be obtained, that is, the above multiple three-dimensional coordinates can also include multiple three-dimensional coordinates of the finger key points between the start and end of the movement.

[0062] S12: Convert the multiple three-dimensional coordinates into multiple two-dimensional coordinates.

[0063] Specifically, the above-mentioned multiple two-dimensional coordinates refer to the conversion of multiple three-dimensional coordinates into a two-dimensional plane, so as to quantify the movement of the finger in the three-dimensional space into the movement on the two-dimensional plane through the dimensionality reduction processing of the three-dimensional coordinates, the x-axis displacement represents the left and right movement, the y-axis displacement represents the up and down movement, and the displacement on the z-axis can be ignored through algorithm compensation. The above-mentioned multiple two-dimensional coordinates include the initial two-dimensional coordinates obtained by dimensionality reduction processing of the initial three-dimensional coordinates of the key points of the finger before the movement begins, and the final two-dimensional coordinates obtained by dimensionality reduction processing of the final three-dimensional coordinates of the key points of the finger after the finger movement ends.

[0064] In some embodiments, the above-mentioned conversion of multiple three-dimensional coordinates into multiple two-dimensional coordinates includes: if it is detected that the finger is not moving parallel to the screen, the multiple three-dimensional coordinates are transformed into spatial coordinates to obtain multiple three-dimensional coordinates of the finger parallel to the screen, and the multiple three-dimensional coordinates of the finger parallel to the screen are converted into multiple two-dimensional coordinates.

[0065] It can be understood that within the recognition area covered by the gesture recognition device 102, the user can perform gesture operations at a comfortable angle and position. If the user's finger is not currently operating parallel to the vehicle screen, that is, there is a certain angle between the finger and the vehicle screen, the multiple three-dimensional coordinates collected by the gesture recognition device 102 will be transformed into spatial coordinates. Through the spatial rectangular coordinate transformation formula, the multiple three-dimensional coordinates collected when the finger is not moving parallel to the vehicle screen are converted into multiple three-dimensional coordinates when the finger moves parallel to the vehicle screen, and then the multiple three-dimensional coordinates when the finger moves parallel to the vehicle screen are converted into multiple two-dimensional coordinates.

[0066] For example, Figure 5 As shown, Figure 5 This is a schematic diagram of a coordinate system provided by an embodiment of the present application. In the figure, the metacarpophalangeal joint of the index finger is set as the coordinate point O, the direction in which the index finger slides to the right is set as the positive direction of the i-axis, the direction in which the index finger slides upward is set as the positive direction of the j-axis, and the direction in which the index finger slides closer to the user is set as the positive direction of the k-axis. If the finger is not moving toward the vehicle screen at this time, when performing the spatial coordinate transformation, the finger is rotated by the above-mentioned certain angle until the finger is parallel to the vehicle screen.

[0067] S13: Perform curve fitting based on the multiple two-dimensional coordinates to obtain a finger movement curve.

[0068] Specifically, after obtaining a plurality of two-dimensional coordinates, a curve fitting may be performed on the plurality of two-dimensional coordinates by a preset fitting algorithm to obtain a finger movement curve. The preset fitting algorithm may be a polynomial formula fitting method.

[0069] It is understandable that the above finger movement curve presents nonlinear continuity characteristics, that is, a continuous motion curve. Finger movement curves, especially in the context of discussing human motion or biomechanics, usually present nonlinear continuity characteristics. This is because human motion is complex and is affected by many factors, including changes in muscle strength, joint flexibility, nerve reaction time, and external environmental factors, etc. These factors work together to make it difficult to simplify the finger movement curve into a simple linear relationship.

[0070] For example, assume that the polynomial formula is: f(x) = a5x 5 +a4x 4 +a3x 3 +a2x 2 +a1x+a0, substitute the above two-dimensional coordinates into the fifth-order polynomial formula, and solve it through the Vandermonde matrix to obtain the coefficient values ​​a of the fifth-order polynomial formula i , the coefficient values ​​a i Substituting it into the polynomial formula, the finger movement curve can be obtained. In the embodiment of the present application, the solution method through the Vandermonde matrix includes but is not limited to Gaussian elimination method, QR decomposition (QRDecomposition, orthogonal triangular decomposition method), LU decomposition (LU Decomposition, LU decomposition method) and SVD decomposition (Singular Value Decomposition, singular value decomposition), which is not limited in the embodiment of the present application.

[0071] S14: Determine the finger movement distance based on the finger movement curve.

[0072] Specifically, according to the finger movement curve determined by the above calculation, the above initial two-dimensional coordinates and the final two-dimensional coordinates are projected onto the finger movement curve, and the finger movement distance is calculated by the two coordinates projected onto the finger movement curve through the coordinate formula between two points.

[0073] For 302, the virtual control may be a cursor on the screen. It is understandable that the movable range of the virtual control on the vehicle screen is determined according to the current position of the virtual control on the screen and the current interface layout of the screen. The movable range represents the distance that the virtual control can be moved under the current interface layout.

[0074] For 303, the movable range of the virtual control on the screen is variable. In this step, the number of intervals matching the movable range can be determined based on the current movable range of the virtual control on the screen. Different movable ranges can correspond to different numbers of intervals.

[0075] Exemplarily, the number of intervals matching the current movable range can be determined based on the current interface layout of the screen. Specifically, the maximum number of intervals corresponding to the current interface layout can be determined first, that is, different interface layouts correspond to different maximum numbers of intervals. The maximum number of intervals can be understood as the number of intervals corresponding to the maximum movable range of the virtual control on the screen.

[0076] According to the maximum number of intervals corresponding to the current interface layout and the maximum movement range corresponding to the current interface layout, the number of intervals that match the current movable range of the virtual control on the screen can be determined. The ratio between the maximum number of intervals and the maximum movement range is a first ratio, and the ratio between the current movable range of the virtual control on the screen and the number of intervals that match it is a second ratio. According to the relationship that the first ratio and the second ratio are equal, the number of intervals that match the current movable range of the virtual control on the screen can be determined.

[0077] For example, Figure 6 As shown, Figure 6 is a schematic diagram of the maximum number of intervals corresponding to an interface layout provided in an embodiment of the present application, Figure 6 601 is an interface 601 displayed on the vehicle screen after the air gesture control is awakened. One or more application icons may be displayed on the interface 601, such as phone, music, information, vehicle, weather map, reading, photo album, and set vehicle application icons. Figure 6 602 is a virtual control on the interface 601.

[0078] like Figure 6 As shown in interface 601 (a), when the current position of the virtual control 602 in the interface 601 is position A, the movable range of the virtual control 602 on the screen is the largest. The maximum number of intervals in the horizontal direction is determined according to the number of application icons in the horizontal direction in the interface 601. At this time, the maximum number of intervals 603 of the virtual control 602 in the horizontal direction on the interface 601 is 4 (such as the 4 intervals from interval 1 to interval 4 in the figure).

[0079] like Figure 6 (b) Figure 6 (b) shows an interface 604 for adjusting the brightness of the car screen. When the current position of the virtual control 602 in the interface 604 is position B, the movable range of the virtual control 602 on the screen is the largest. The number of preset brightness levels in the interface 604 (for example, 10 levels) is determined as the maximum number of intervals in the horizontal direction. At this time, the maximum number of intervals 605 of the virtual control 602 in the horizontal direction on the interface 604 is 10 (for example, the 10 intervals from interval 1 to interval 10 in the figure).

[0080] For example, Figure 7 As shown, Figure 7 Schematic diagram of the movable range of a virtual control currently on the vehicle screen provided by an embodiment of the present application. Figure 7 As shown in the interface 701, it is assumed that the maximum movable range of the virtual control 602 in the horizontal direction is 40 cm, and the maximum movable range in the vertical direction is 30 cm on the interface 701. The current position of the virtual control 602 in the interface 701 is position C.

[0081] like Figure 7 As shown, if the virtual control 602 needs to be moved vertically upward from position C, the current movable range of the virtual control 602 on the screen is 18 cm.

[0082] like Figure 7 As shown, if the virtual control 602 needs to be moved vertically downward from position C, the current movable range of the virtual control 602 on the screen is 12 cm.

[0083] like Figure 7 As shown, when the virtual control 602 needs to be moved horizontally to the left from position C, the current movable range of the virtual control 602 to the left on the screen is 10 cm.

[0084] like Figure 7 As shown, when the virtual control 602 needs to be moved horizontally to the right from position C, the current movable range of the virtual control 602 to the right on the screen is 30 cm.

[0085] For example, Figure 8 As shown, Figure 8 This is a schematic diagram of another movable range of a virtual control currently on the vehicle screen provided in an embodiment of the present application. Figure 8 801 in FIG. 8 is an interface 801 for adjusting the brightness of the vehicle screen. Figure 8 As shown in the interface 801, it is assumed that the maximum movable range of the virtual control 602 in the horizontal direction on the interface 801 is 20 cm. The current position of the virtual control 602 on the interface 801 is position D.

[0086] like Figure 8 As shown, when the virtual control 602 needs to be moved horizontally to the left from position D, the current movable range of the virtual control 602 to the left on the screen is 2 cm.

[0087] like Figure 8 As shown, when the virtual control 602 needs to be moved horizontally to the right from position D, the current movable range of the virtual control 602 to the right on the screen is 18 cm.

[0088] Exemplarily, the maximum movable range of the virtual control on the screen in the reference direction is recorded as A1, and the maximum number of intervals of the virtual control on the screen in the reference direction is recorded as A2. The current movable range of the virtual control on the screen along the reference direction is recorded as A3, and the number of intervals matching the current movable range is recorded as A4. From the above A1, A2, A3 and A4, the corresponding relationship that can be obtained is: A1 / A2=A3 / A4. Therefore, when the above A1, A2 and A3 are known, A4 can be calculated based on the corresponding relationship. The above reference direction can be a horizontal direction or a vertical direction.

[0089] For example, Figure 6 As shown in the interface 601 of (a), the maximum movable range of the virtual control 602 in the horizontal direction on the interface 601 is A1=40 cm, and the maximum number of intervals 603 of the virtual control 602 in the horizontal direction on the interface 601 is A2=4.

[0090] like Fig. 9 As shown, Fig. 9 1 is a schematic diagram of the current number of intervals corresponding to an interface layout provided in an embodiment of the present application. The current number of intervals is the number of intervals that matches the current movable range of the virtual control on the screen.

[0091] like Fig. 9 As shown in (a), the current position of the virtual control 602 on the screen is position C. If the virtual control 602 moves to the left along the horizontal direction, combined with Figure 7 It can be seen that the movable range of the virtual control 602 on the screen in the left direction is A3 = 10 cm. According to the above correspondence relationship, the number of intervals matching the movable range can be calculated to be A4 = 1. Figure 7 It can be known that the current movable range of the virtual control 602 on the screen extending to the right is A3=30 cm. According to the above correspondence relationship, it can be calculated that the number of intervals matching the movable range is A4=3.

[0092] For example, Figure 6 As shown in the interface 604 of (b), the maximum movable range of the virtual control 602 in the horizontal direction on the interface 604 is A1=20 cm, and the maximum number of intervals 605 of the virtual control 602 in the horizontal direction on the interface 604 is A2=10.

[0093] like Fig. 9 As shown in (b), the current position of the virtual control 602 on the screen is position D. If the virtual control 602 moves to the left along the horizontal direction, then Figure 8It can be known that the movable range of the virtual control 602 on the screen in the left direction is A3 = 2 cm. According to the above correspondence relationship, the number of intervals corresponding to the movable range can be calculated to be A4 = 1. If the virtual control 602 moves to the right in the horizontal direction, then Figure 8 It can be known that the movable range of the virtual control 602 on the screen extending to the right is A3=18 cm. According to the above correspondence relationship, it can be calculated that the number of intervals corresponding to the movable range is A4=9.

[0094] As for 304, it can be understood that the maximum distance of the finger moving in one direction is obtained under the premise of being ergonomic, for example, it can be 2 cm. The finger distance interval is dynamically divided according to the maximum distance of the finger moving in one direction and the number of the intervals. The control moving distance is determined according to the movable range of the virtual control 602 on the screen and the number of the intervals.

[0095] Exemplarily, an equal-width discretization method may be used to divide the maximum distance of a finger's one-way movement into a plurality of continuous and equal-width finger movement intervals according to the number of intervals.

[0096] In some embodiments, the maximum distance of a finger's one-way movement is divided into multiple continuous finger distance intervals according to the number of intervals, including: calculating multiple gesture thresholds based on the maximum distance of a finger's one-way movement and the number of intervals; wherein the number of the multiple gesture thresholds is the same as the number of intervals; constructing multiple continuous finger distance intervals based on the multiple gesture thresholds; wherein the upper limit value of each finger distance interval is a gesture threshold.

[0097] Specifically, the width of each finger distance interval is calculated according to the acquired maximum distance of one-way movement of the finger and the number of intervals, and the calculated finger distance interval widths are superimposed to obtain multiple gesture thresholds.

[0098] For example, Figure 6 As shown in the interface 601 of (a), when the current position of the virtual control 602 in the interface 601 is position A, according to the number of application icons in the horizontal direction in the interface 601, it is determined that the maximum number of intervals 603 of the virtual control 602 in the horizontal direction on the interface 601 is 4, and the maximum distance of a finger moving in one direction is 2 cm. The maximum distance of a finger moving in one direction is divided into 4 intervals by using the equal width discrete method, that is: w = (x max -x min ) / k, where w is the width of each finger distance interval, x max is the maximum distance a finger can move in one direction (2 cm), x minis the minimum distance of a finger's one-way movement (0 cm), k is the number of intervals (4 in this case), and the calculated width of each finger distance interval is 0.5 cm. The calculated finger distance interval widths are superimposed, and the four gesture thresholds obtained include: 0.2 cm, 1.0 cm, 1.5 cm, and 2.0 cm.

[0099] like Figure 6 (b) Figure 6 (b) shows an interface 604 for adjusting the brightness of the vehicle screen. When the current position of the virtual control 602 in the interface 604 is position B, according to the number of preset brightness levels in the interface 604, the maximum number of intervals 605 of the virtual control 602 in the horizontal direction on the interface 604 is determined to be 10, and the maximum distance of a finger moving in one direction is 2 cm. The maximum distance of a finger moving in one direction is divided into 10 intervals by using the equal width discrete method, that is, w = (x max -x min ) / k, where w is the width of each finger distance interval, x max is the maximum distance a finger can move in one direction (2 cm), x min is the minimum distance of one-way movement of a finger (0 cm), k is the number of intervals (10 in this case), and the width of each finger distance interval is calculated to be 0.2 cm. The calculated finger distance interval widths are superimposed, and the 10 gesture thresholds obtained include: 0.2 cm, 0.4 cm, 0.6 cm, 0.8 cm, 1.0 cm, 1.2 cm, 1.4 cm, 1.6 cm, 1.8 cm, and 2.0 cm.

[0100] After obtaining multiple gesture thresholds, multiple finger distance intervals can be constructed based on the multiple gesture thresholds, and one finger distance interval can be constructed for every two adjacent gesture thresholds. For example, the four finger distance intervals constructed based on the four gesture thresholds can be referred to in the following Table 1. Table 1 is also the multiple finger distance intervals obtained by dividing the above interface 601.

[0101] Table 1

[0102] Finger distance range Gesture Threshold 0~0.5cm 0.5cm 0.5~1.0cm 1.0cm 1.0~1.5cm 1.5cm 1.5~2.0cm 2.0cm

[0103] For example, the 10 finger distance intervals constructed according to the above 10 gesture thresholds can be referred to in the following Table 2. Table 2 is a plurality of finger distance intervals obtained by dividing the above interface 604 .

[0104] Table 2

[0105] Finger distance range Gesture Threshold 0~0.2cm 0.2cm 0.2~0.4cm 0.4cm 0.4~0.6cm 0.6cm 0.6~0.8cm 0.8cm 0.8~1.0cm 1.0cm 1.0~1.2cm 1.2cm 1.2~1.4cm 1.4cm 1.4~1.6cm 1.6cm 1.6~1.8cm 1.8cm 1.8~2.0cm 2.0cm

[0106] Specifically, each finger distance interval corresponds to its own control movement distance, and the above control movement distance is determined based on the above movable range and the above number of intervals. According to the obtained above movable range and the above number of intervals, the width of each control movement distance interval is calculated, and the calculated widths of the control movement distance intervals are superimposed to obtain the control movement distances corresponding to multiple finger distance intervals.

[0107] For example, Figure 6 As shown in the interface 601 of (a), when the current position of the virtual control 602 in the interface 601 is position A, according to the number of application icons in the horizontal direction in the interface 601, it is determined that the maximum number of intervals 603 of the virtual control 402 in the horizontal direction on the interface 601 is 4, and the maximum movable range of the virtual control 602 in the horizontal direction on the interface 601 is 40 cm. Since the movement of the virtual control 602 in the interface 601 is not linear and presents a discrete characteristic, the equal-width discrete method is used to divide the maximum movable range into 4 intervals, so that the calculated width of the moving distance interval of each control is 10 cm. The calculated control moving distance interval widths are superimposed, and the 4 control moving distances obtained include: 10 cm, 20 cm, 30 cm, and 40 cm.

[0108] like Figure 6 As shown in the interface 604 of (b), when the virtual control 602 is at the current position of the interface 604, the number of brightness levels preset in the interface 604 (for example, 10 levels) is determined as the maximum number of intervals 605 of the virtual control 602 in the horizontal direction on the interface 604 is 10, that is, the brightness has 10 levels, and the maximum movable range of the virtual control 602 in the horizontal direction on the interface 604 is 20 cm. Since the movement of the virtual control 602 in the interface 604 is not linear and presents a discrete characteristic, the equal-width discrete method is used to divide the maximum movable range into 10 intervals, so as to calculate the width of the moving distance interval of each control to be 2 cm. The calculated control moving distance interval widths are superimposed, and the resulting 10 control moving distances include: 2 cm, 4 cm, 6 cm, 8 cm, 10 cm, 12 cm, 14 cm, 16 cm, 18 cm, and 20 cm.

[0109] After a plurality of finger movement intervals are divided and a plurality of control movement distances are calculated, each of the finger distance intervals is matched with each of the control movement distances.

[0110] Exemplarily, according to the above interface 601, the corresponding relationship between each finger distance interval and the moving distance of each control is shown in Table 3 below.

[0111] Table 3

[0112] Finger distance range Control moving distance Gesture Threshold 0~0.5cm 10cm 0.5cm 0.5~1.0cm 20cm 1.0cm 1.0~1.5cm 30cm 1.5cm 1.5~2.0cm 40cm 2.0cm

[0113] Exemplarily, according to the above interface 604, the corresponding relationship between each finger distance interval and the moving distance of each control is shown in Table 4 below.

[0114] Table 4

[0115] Finger distance range Control moving distance brightness Gesture Threshold 0~0.2cm 2cm 1st gear 0.2cm 0.2~0.4cm 4m 2nd gear 0.4cm 0.4~0.6cm 6cm 3rd gear 0.6cm 0.6~0.8cm 8cm 4 gears 0.8cm 0.8~1.0cm 10cm 5 gears 1.0cm 1.0~1.2cm 12cm 6 gears 1.2cm 1.2~1.4cm 14cm 7 gears 1.4cm 1.4~1.6cm 16cm 8 gears 1.6cm 1.6~1.8cm 18cm 9 gears 1.8cm 1.8~2.0cm 20cm 10 levels 2.0cm

[0116] For example, Fig. 9 As shown in the interface 701 of (a), the current position of the virtual control 602 on the screen is position C. If the virtual control 602 moves to the right in the horizontal direction, combined with Figure 7 It can be seen that the current movable range of the virtual control 602 on the screen extending to the right is A3=30cm. At this time, the number of intervals matching the movable range can be calculated from the above correspondence relationship as A4=3. Since the movement of the virtual control 602 in the interface 701 is not linear and presents a discrete characteristic, the equal-width discrete method is used to divide the movable range into 3 intervals, so that the calculated width of the moving distance interval of each control is 10cm. The calculated control moving distance interval widths are superimposed, and the resulting 3 control moving distances include: 10cm, 20cm, and 30cm.

[0117] Assuming that the maximum distance a finger can move in one direction is 2 cm, the equal-width discretization method is used again to divide the maximum distance a finger can move in one direction into three equal intervals, namely: w = (x max -x min ) / k, where w is the width of each finger distance interval, x max is the maximum distance a finger can move in one direction (2 cm), x min is the minimum distance of a finger's one-way movement (0 cm), k is the number of intervals (3 in this case), and the calculated width of each finger distance interval is approximately 0.67 cm. The calculated finger distance interval widths are superimposed, and the three gesture thresholds obtained include: 0.67 cm, 1.34 cm, and 2.00 cm.

[0118] Exemplarily, the finger distance intervals divided according to the above interface 701 are shown in Table 5 below.

[0119] Table 5

[0120] Finger distance range Control moving distance Gesture Threshold 0~0.67cm 10cm 0.67cm 0.67~1.34cm 20cm 1.34cm 1.34~2.00cm 30cm 2.00cm

[0121] At this time, if the finger moves 0.5 cm to the right, it can be seen that the control movement distance corresponding to the finger distance interval of the finger movement distance is 10 cm. Therefore, when the finger moves 0.5 cm to the right, the virtual control moves 10 cm to the right on the interface 701.

[0122] For example, Fig. 9 As shown in the interface 801 of (b), according to the current position of the virtual control 602 on the screen as position D, if the virtual control 602 moves to the right along the horizontal direction, combined with Figure 8 It can be seen that the current movable range of the virtual control 602 on the screen extending to the right is A3=18cm. At this time, the number of intervals corresponding to the movable range can be calculated from the above correspondence relationship to be A4=9. Since the movement of the virtual control 602 in the interface 801 is not linear and presents a discrete characteristic, the equal-width discrete method is used to divide the movable range into 9 intervals, so that the calculated width of the moving distance interval of each control is 2cm. The calculated control moving distance interval widths are superimposed, and the 9 control moving distances obtained include: 2cm, 4cm, 6cm, 8cm, 10cm, 12cm, 14cm, 16cm, and 18cm.

[0123] Assuming that the maximum distance a finger can move in one direction is 2 cm, the equal-width discretization method is used again to divide the maximum distance a finger can move in one direction into 9 equal intervals, namely: w = (x max -x min ) / k, where w is the width of the interval of distance moved by each finger, x max is the maximum distance a finger can move in one direction (20 cm), x min is the minimum distance moved by the virtual control (0cm), k is the number of intervals (9 in this case), and the calculated width of each finger distance interval is approximately 0.22cm. The calculated finger distance interval widths are superimposed, and the 9 gesture thresholds obtained include: 0.22cm, 0.44cm, 0.67cm, 0.89cm, 1.11cm, 1.33cm, 1.56cm, 1.78cm, and 2.00cm.

[0124] Exemplarily, the finger distance intervals divided according to the above interface 801 are shown in Table 6 below.

[0125] Table 6

[0126]

[0127]

[0128] Since the current screen brightness is level 1, if the finger moves 1.0 cm to the right, it can be seen that the control movement distance corresponding to the finger distance interval of the finger movement distance is 10 cm. Therefore, when the finger moves 1.0 cm to the right, the virtual control moves 10 cm to the right on the interface 801, and the screen brightness is level 6 at this time.

[0129] As for 305 , it can be understood that the target finger distance interval is: the finger distance interval in which the finger movement distance obtained in step 301 is located among the multiple continuous finger distance intervals obtained in step 304 .

[0130] Specifically, since the multiple finger distance intervals in the embodiments of the present application are divided based on the number of intervals that match the current movable range of the virtual control on the screen, even for the same finger movement distance, the corresponding finger movement intervals and gesture thresholds may be different under different movable ranges of different interfaces. For example, if the finger movement distance is 0.7cm, it can be seen from Table 3 above that the finger movement interval of the finger movement distance is 0.5-1.0cm, the corresponding gesture threshold is 1.0cm, and the control movement distance is 20cm; it can be seen from Table 4 above that the finger movement interval of the finger movement distance is 0.6-0.8cm, the corresponding gesture threshold is 0.8cm, and the control movement distance is 8cm. For another example, if the finger movement distance is 1.2 cm, it can be seen from Table 5 above that the finger movement distance at this time is in the finger movement interval of 0.67 to 1.34 cm, the corresponding gesture threshold is 1.34 cm, and the control movement distance is 20 cm; it can be seen from Table 6 above that the finger movement distance at this time is in the finger movement interval of 1.11 to 1.33 cm, the corresponding gesture threshold is 1.33 cm, and the control movement distance is 12 cm.

[0131] For 306, it can be understood that the moving distance of the virtual control is controlled according to the control moving distance corresponding to the target finger distance interval, so that the virtual control moves from the current position to the target position. Among them, the interval distance between the target position and the current position is the control moving distance corresponding to the target finger distance interval. After the virtual control moves to the above-mentioned target position, the operation instruction corresponding to the movement of the virtual control can be executed.

[0132] For example, Figure 6 As shown in the interface 601 of (a), if the virtual control 602 is moved based on the control movement distance, the virtual control 602 can be moved to different application icons to execute the operation instruction of clicking the application icon. Figure 6 As shown in the interface 604 of (b), if the movement of the virtual control 602 is controlled based on the control movement distance, the virtual control 602 can be used to adjust the brightness of the vehicle screen. When the current position of the virtual control 602 in the interface 604 is position B, if the virtual control 602 moves 12 cm to the right, the brightness is adjusted to level 6.

[0133] In some other embodiments, when controlling the movement of a virtual control through a target air gesture, the movement direction of the virtual control can be further controlled in combination with the finger movement direction corresponding to the target air gesture. For example, the control movement direction can be determined according to the finger movement direction corresponding to the target air gesture, and then the movement direction of the virtual control can be controlled according to the control movement direction. The following describes a scheme for controlling the movement of a virtual control in combination with the finger movement direction.

[0134] In some embodiments, the control movement of the virtual control based on the control movement distance corresponding to the target finger distance interval as described in the embodiments of the present application includes: determining the control movement direction based on the finger movement direction; controlling the movement of the virtual control based on the control movement distance and the control movement direction corresponding to the target finger distance interval.

[0135] It is understandable that the finger movement direction can be horizontal or vertical, or diagonal. The finger movement direction can be determined by: determining the finger movement direction according to the change in the movement of the finger on each coordinate axis; or by: in a three-dimensional coordinate system, determining the direction of the line connecting the initial three-dimensional coordinate and the final three-dimensional coordinate as the finger movement direction.

[0136] Exemplarily, when the finger moves in a horizontal direction, the control moves in a horizontal direction. When the finger moves in a vertical direction, the control moves in a vertical direction. When the finger moves in an oblique direction, the control moves in an oblique direction. In other words, the finger moves in the same direction as the control moves.

[0137] Therefore, in an embodiment of the present application, the above-mentioned control movement distance and control movement direction can be combined to control the distance and direction of movement of the virtual control on the screen.

[0138] In some embodiments, determining the control movement direction based on the finger movement direction as described in the embodiments of the present application includes: when the finger movement direction does not move along the specified baseline, converting the finger movement direction into a target direction parallel to the specified baseline, and determining the target direction as the control movement direction.

[0139] It is understandable that the above-mentioned designated reference line can be horizontal or vertical. The designated reference line is related to the target object to be controlled by the virtual control. If the target object is a slider, the designated reference line is horizontal; if the target object is each card or application icon displayed on the screen, the designated reference line can include horizontal and vertical directions.

[0140] In one possible implementation, the direction of the specified baseline is horizontal. If the finger moves in the horizontal direction, the finger moves in the direction of the specified baseline. Therefore, the target direction is the finger moving direction. The target direction is then determined as the control moving direction, that is, the control moving direction is horizontal.

[0141] In one possible implementation, the direction of the specified baseline is horizontal. If the finger moves in an oblique direction, the finger movement direction needs to be converted into a direction parallel to the specified baseline. At this time, the target direction is a horizontal direction parallel to the specified baseline. The target direction is then determined as the control movement direction, that is, the control movement direction is horizontal.

[0142] In a possible implementation, the specified reference line direction includes the horizontal direction and the vertical direction. If the finger moves in an oblique direction, it is necessary to obtain the angle between the finger moving direction and the horizontal direction θ1 (θ1 must be less than 90°) and the angle between the finger moving direction and the vertical direction θ2 (θ2 must be less than 90°). If θ1 is less than θ2, the finger moving direction is converted to a direction parallel to the horizontal direction. At this time, the target direction is the horizontal direction, and the target direction is determined as the control moving direction, that is, the control moving direction is the horizontal direction. If θ1 is greater than θ2, the finger moving direction is converted to a direction parallel to the vertical direction. At this time, the target direction is the vertical direction, and the target direction is determined as the control moving direction, that is, the control moving direction is the vertical direction; if θ1 is equal to θ2, the finger moving direction needs to be obtained again.

[0143] In one possible implementation, the direction of the specified reference line is the vertical direction. If the finger moves in the vertical direction, the finger moves in the direction of the specified reference line. Therefore, the target direction is the finger moving direction, and the target direction is determined as the control moving direction, that is, the control moving direction is the vertical direction. If the finger moves in an oblique direction, the finger moving direction needs to be converted into a direction parallel to the specified reference line. In this case, the target direction is the vertical direction parallel to the specified reference line, and the target direction is determined as the control moving direction, that is, the control moving direction is the vertical direction.

[0144] Based on the change in the finger movement direction, the finger movement distance will also change. In some embodiments, obtaining the finger movement distance corresponding to the target air gesture includes: obtaining a finger movement curve corresponding to the target air gesture, and determining the projection distance of the finger movement curve in the target direction as the finger movement distance corresponding to the target air gesture.

[0145] Specifically, if the finger movement direction does not extend along the specified baseline direction, then when calculating the finger movement distance, the finger movement distance calculated by the coordinate formula between the two points needs to be projected to the target direction, and the projection distance obtained after the projection is determined as the finger movement distance corresponding to the target air gesture.

[0146] In some embodiments, determining the current movable range of the virtual control on the screen as described in the embodiments of the present application includes: determining the current movable range of the virtual control on the screen along the direction of control movement.

[0147] It can be understood that the above-mentioned control movement direction refers to the movement direction of the virtual control on the vehicle screen; the virtual control on the vehicle screen can move horizontally or vertically, so the above-mentioned control movement direction can include horizontal movement or vertical movement direction, specifically including vertical upward movement, vertical downward movement, horizontal left movement, horizontal right movement and other directions.

[0148] For example, Figure 7 As shown in the interface 701, if the current position of the virtual control 602 in the interface 701 is position C. When the control moving direction is upward, that is, when the virtual control 602 moves upward, it is determined that the movable range of the virtual control 602 along the control moving direction is 18cm. When the control moving direction is downward, that is, when the virtual control 602 moves downward, it is determined that the movable range of the virtual control 602 along the control moving direction is 12cm. When the control moving direction is left, that is, the virtual control 602 moves to the left, it is determined that the movable range of the virtual control 602 along the control moving direction is 10cm. When the control moving direction is right, that is, the virtual control 602 moves to the right, it is determined that the movable range of the virtual control 602 along the control moving direction is 30cm.

[0149] For example, Figure 8 As shown in the interface 801, if the current position of the virtual control 602 in the interface 801 is position D. When the control moving direction is left, that is, the virtual control 602 moves to the left, the movable range of the virtual control 602 along the control moving direction is determined to be 2 cm. When the control moving direction is right, that is, the virtual control 602 moves to the right, the movable range of the virtual control 602 along the control moving direction is determined to be 18 cm.

[0150] In some embodiments, controlling the movement of a virtual control based on the control movement distance corresponding to a target finger distance interval as described in the embodiments of the present application also includes: determining a target object for the virtual control to control on the screen; determining a control movement direction based on the finger movement direction and the target object; controlling the movement of the virtual control based on the control movement distance and the control movement direction corresponding to the target finger distance interval.

[0151] It can be understood that the above-mentioned target objects refer to the various cards displayed on the screen, the above-mentioned application icons or sliding bars, etc.; the above-mentioned finger movement direction can be horizontal or vertical, or diagonal; the above-mentioned control movement direction can be horizontal or vertical.

[0152] Specifically, the control movement direction is related to the target object to be controlled by the virtual control. If the target object is each card displayed on the screen or the application icon, the finger movement direction is the control movement direction; if the target object is a slider, if the finger movement direction is diagonal, it is also necessary to convert the finger movement direction to horizontal, and the control movement direction is the converted horizontal direction.

[0153] The above technical solution can dynamically adjust the correspondence between the finger distance interval and the control movement distance based on the current movable range of the virtual control on the screen, so that the correspondence can adapt to the current movable range of the virtual control on the screen, which is conducive to improving the control accuracy of the virtual control through the target air gesture. When the user is performing gesture control, even if the finger movement distance is very small, it can be accurately recognized and converted into the movement of the virtual control. The user can control the virtual control with a smaller gesture movement amplitude, which significantly reduces the burden caused by long-term operation. At the same time, by recognizing air gestures and directly converting them into control movement and corresponding operation instructions, the intermediate steps such as clicking and sliding in traditional touch operations are reduced, which improves the comfort of the user experience and effectively avoids the fatigue caused by long-term gesture control.

[0154] Fig.10 It is a structural schematic diagram of a device for air gesture control provided in an embodiment of the present application.

[0155] For example, Fig.10 As shown, the device 1000 includes:

[0156] The acquisition module 1001 is used to identify the target air gesture and obtain the finger movement distance corresponding to the target air gesture.

[0157] A first determining module 1002 is used to determine the current movable range of the virtual control on the screen;

[0158] The second determining module 1003 is used to determine the number of intervals matching the movable range.

[0159] The division module 1004 is used to divide the maximum distance of a finger's one-way movement into multiple continuous finger distance intervals according to the number of intervals; wherein each finger distance interval corresponds to a control movement distance, and the control movement distance is determined based on the movable range and the number of intervals.

[0160] The third determining module 1005 is used to determine the target finger distance interval in which the finger moving distance is located among a plurality of continuous finger distance intervals.

[0161] The control module 1006 is used to control the movement of the virtual control based on the control movement distance corresponding to the target finger distance interval, and execute the operation instruction corresponding to the movement of the virtual control.

[0162] In one possible implementation, the division module is specifically used to: calculate multiple gesture thresholds based on the maximum distance of one-way movement of a finger and the number of intervals; wherein the number of multiple gesture thresholds is the same as the number of intervals; construct multiple continuous finger distance intervals based on the multiple gesture thresholds; wherein the upper limit value of each finger distance interval is a gesture threshold.

[0163] In a possible implementation, the device also includes: a fourth determination module for determining a control movement direction based on a finger movement direction; and the control module is specifically used for controlling the movement of the virtual control based on a control movement distance and a control movement direction corresponding to a target finger distance interval.

[0164] In a possible implementation, the first determination module is specifically used to determine a movable range of the virtual control on the screen along a moving direction of the control.

[0165] In one possible implementation, the fourth determination module is specifically used to: when the finger movement direction does not move along the specified baseline, convert the finger movement direction into a target direction parallel to the specified baseline, and determine the target direction as the control movement direction; the acquisition module is specifically used to obtain the finger movement curve corresponding to the target air gesture, and determine the projection distance of the finger movement curve in the target direction as the finger movement distance corresponding to the target air gesture.

[0166] In one possible implementation, the acquisition module includes an acquisition unit, a conversion unit, a fitting unit and a determination unit. The acquisition unit is specifically used to: identify the finger key points corresponding to the target air gesture, and obtain multiple three-dimensional coordinates of the finger key points during the finger movement process; wherein the multiple three-dimensional coordinates include the coordinates of the finger key points before the movement starts and the coordinates of the finger key points after the movement ends; the conversion unit is specifically used to: convert the multiple three-dimensional coordinates into multiple two-dimensional coordinates; the fitting unit is specifically used to: perform curve fitting based on the multiple two-dimensional coordinates to obtain a finger movement curve; the determination unit is specifically used to: determine the finger movement distance based on the finger movement curve.

[0167] In one possible implementation, the acquisition module is also specifically used to: if it is detected that the finger is not moving parallel to the screen, perform spatial coordinate transformation on multiple three-dimensional coordinates to obtain multiple three-dimensional coordinates of the finger parallel to the screen, and convert the multiple three-dimensional coordinates of the finger parallel to the screen into multiple two-dimensional coordinates.

[0168] Fig.11 It is a schematic diagram of the hardware structure of a vehicle provided in an embodiment of the present application.

[0169] For example, Fig.11 As shown, the vehicle 1100 includes: a memory 1101 and a processor 1102, wherein the memory 1101 stores an executable program code 11011, and the processor 1102 is used to call and execute the executable program code 11011 to perform a method of air gesture control.

[0170] In addition, an embodiment of the present application also protects a device, which may include a memory and a processor, wherein the memory stores executable program code, and the processor is used to call and execute the executable program code to perform a method of air gesture control provided by an embodiment of the present application.

[0171] In this embodiment, the functional modules of the device can be divided according to the above method example. For example, each functional module can be corresponded, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware. It should be noted that the division of modules in this embodiment is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0172] In the case of dividing each functional module according to each function, the device may further include an acquisition module, a first determination module, a second determination module, a division module, a third determination module, and a control module, etc. It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module, and will not be repeated here.

[0173] It should be understood that the device provided in this embodiment is used to execute the above-mentioned method of air gesture control, and thus can achieve the same effect as the above-mentioned implementation method.

[0174] In the case of an integrated unit, the device may include a processing module and a storage module. When the device is applied to a vehicle, the processing module may be used to control and manage the actions of the vehicle. The storage module may be used to support the vehicle in executing related program codes, etc.

[0175] The processing module may be a processor or a controller, which may implement or execute various exemplary logic blocks, modules and circuits shown in conjunction with the disclosure of the present application. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc. The storage module may be a memory.

[0176] In addition, the device provided in the embodiments of the present application may specifically be a chip, component or module, and the chip may include a connected processor and memory; wherein the memory is used to store instructions, and when the processor calls and executes the instructions, the chip can execute a method of air gesture control provided in the above embodiments.

[0177] This embodiment also provides a computer-readable storage medium, which stores computer program code. When the computer program code runs on a computer, the computer executes the above-mentioned related method steps to implement a method of air gesture control provided by the above embodiment.

[0178] This embodiment also provides a computer program product. When the computer program product runs on a computer, it enables the computer to execute the above-mentioned related steps to implement a method of air gesture control provided by the above embodiment.

[0179] Among them, the device, computer-readable storage medium, computer program product or chip provided in this embodiment is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above, and will not be repeated here.

[0180] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0181] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0182] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for air gesture control, characterized in that: The method comprises: Identify a target air gesture and obtain a finger movement distance corresponding to the target air gesture; Determine the current movable range of the virtual control on the screen; determining the number of intervals matching the movable range; According to the number of intervals, the maximum distance of a finger moving in one direction is divided into a plurality of continuous finger distance intervals; wherein each of the finger distance intervals corresponds to a control moving distance, and the control moving distance is determined based on the movable range and the number of intervals; Determine a target finger distance interval in which the finger moving distance is located among the multiple continuous finger distance intervals; Based on the control movement distance corresponding to the target finger distance interval, the movement of the virtual control is controlled, and an operation instruction corresponding to the movement of the virtual control is executed.

2. The method according to claim 1, characterized in that The maximum distance of a finger moving in one direction is divided into a plurality of continuous finger distance intervals according to the number of intervals, including: Calculating a plurality of gesture thresholds based on the maximum distance of a finger's one-way movement and the number of intervals; wherein the number of the plurality of gesture thresholds is the same as the number of intervals; According to the multiple gesture thresholds, multiple continuous finger distance intervals are constructed; wherein the upper limit value of each of the finger distance intervals is one of the gesture thresholds.

3. The method according to claim 1, characterized in that The target air gesture corresponds to a finger movement direction, and the control of the movement of the virtual control based on the control movement distance corresponding to the target finger distance interval includes: Determine the control movement direction based on the finger movement direction; The movement of the virtual control is controlled based on the control movement distance and the control movement direction corresponding to the target finger distance interval.

4. The method according to claim 3, characterized in that The determining the current movable range of the virtual control on the screen includes: Determine the current movable range of the virtual control on the screen along the moving direction of the control.

5. The method according to claim 3, characterized in that: The determining the control movement direction based on the finger movement direction includes: In the case that the finger moving direction does not move along the specified reference line, converting the finger moving direction into a target direction parallel to the specified reference line, and determining the target direction as the control moving direction; The obtaining of the finger movement distance corresponding to the target air gesture includes: A finger movement curve corresponding to the target air gesture is obtained, and a projection distance of the finger movement curve in the target direction is determined as the finger movement distance corresponding to the target air gesture.

6. The method according to claim 1, characterized in that The obtaining of the finger movement distance corresponding to the target air gesture includes: Identify the finger key point corresponding to the target air gesture, and obtain multiple three-dimensional coordinates of the finger key point during the finger movement process; wherein the multiple three-dimensional coordinates include the coordinates of the finger key point before the movement starts and the coordinates of the finger key point after the movement ends; Converting the plurality of three-dimensional coordinates into a plurality of two-dimensional coordinates; Performing curve fitting based on the multiple two-dimensional coordinates to obtain a finger movement curve; Based on the finger movement curve, the finger movement distance is determined.

7. The method according to claim 6, characterized in that The converting the multiple three-dimensional coordinates into multiple two-dimensional coordinates comprises: If it is detected that the finger does not move parallel to the screen, the multiple three-dimensional coordinates are spatially transformed to obtain multiple three-dimensional coordinates of the finger parallel to the screen, and the multiple three-dimensional coordinates of the finger parallel to the screen are converted into multiple two-dimensional coordinates.

8. A device for air gesture control, characterized in that: The device comprises: An acquisition module, used to identify a target air gesture and acquire a finger movement distance corresponding to the target air gesture; A first determination module, used to determine the current movable range of the virtual control on the screen; A second determination module, used to determine the number of intervals matching the movable range; A division module, used to divide the maximum distance of a finger's one-way movement into a plurality of continuous finger distance intervals according to the number of intervals; wherein each of the finger distance intervals corresponds to a control movement distance, and the control movement distance is determined based on the movable range and the number of intervals; A third determining module, configured to determine a target finger distance interval in which the finger moving distance is located among a plurality of continuous finger distance intervals; The control module is used to control the movement of the virtual control based on the control movement distance corresponding to the target finger distance interval, and execute the operation instruction corresponding to the movement of the virtual control.

9. A vehicle, characterized in that: The vehicle comprises: A memory for storing executable program codes; A processor, configured to call and run the executable program code from the memory, so that the vehicle executes the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed, the method according to any one of claims 1 to 7 is implemented.

Images