Page control method and device for air gesture, vehicle and electronic equipment

Abstract

The invention discloses an air gesture page control method and device, a vehicle and electronic equipment, and the method comprises the steps: responding to the triggering of a first air gesture, and starting a gesture control window; determining a movement offset in a gesture control window according to the detected displacement of the hand, and judging a target configurable function item corresponding to the movement offset; and in response to triggering of a second air gesture, executing the target configurable function item, and controlling the page based on the target configurable function item. Compared with a rolling control strategy along the x axis and the y axis in the related technology, the air gesture page control method provided by the embodiment of the invention has the advantages that the mode of selecting the target configurable function item through hand displacement is more flexible, the configurable function item can be customized and configured according to the requirement, the number of functions which can be realized is more, and the user experience is improved. The problem that the control strategy of the air gesture in the related technology is too limited is solved.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular to a page control method and device using air gestures, a vehicle, and an electronic device. Background Art

[0002] With the development of technology, applications such as in-vehicle navigation systems and in-vehicle entertainment systems are gradually increasing, and there are more and more ways of human-computer interaction. In order to facilitate user operation and enhance user experience, human-computer interaction methods based on air gestures are gradually being widely used.

[0003] Air gesture is a method in which an electronic device uses technology such as cameras or ultrasound to detect and identify gestures performed by users in three-dimensional space, thereby realizing air-operated operations on some functions in the electronic device and achieving intelligent interaction.

[0004] In the related art, air gesture technology can realize scrolling control along the x-axis and y-axis, and this scrolling control strategy is written into electronic devices, making the air gesture control strategy too limited. Summary of the invention

[0005] The present disclosure provides a page control method and device for air gesture, a vehicle and an electronic device, the main purpose of which is to solve the problem that the scrolling control strategy along the x-axis and y-axis written into the electronic device makes the control strategy of the air gesture too limited.

[0006] According to a first aspect of the present disclosure, a page control method using air gestures is provided, comprising:

[0007] In response to a trigger of a first air gesture, a gesture control window is started, wherein the gesture control window includes at least two configurable function items, and the first air gesture is used to start the gesture control window;

[0008] Determine a movement offset in the gesture control window according to the detected displacement of the hand, and determine a target configurable function item corresponding to the movement offset, wherein the movement offset includes a target movement direction and a target movement distance;

[0009] In response to triggering of a second air gesture, the target configurable function item is executed, and the page is controlled based on the target configurable function item, and the second air gesture is used to execute the target configurable function item.

[0010] Optionally, starting the gesture control window includes:

[0011] At a cursor position in the page corresponding to the first air gesture, controlling to start the gesture control window;

[0012] Controlling the focus position to be the preset position of the gesture control window;

[0013] Record the first coordinate of the first air gesture in the gesture control window.

[0014] Optionally, determining the movement offset in the gesture control window according to the detected displacement of the hand, and determining the target configurable function item corresponding to the movement offset includes:

[0015] According to the displacement of the hand, obtaining a second coordinate of the first air gesture in the gesture control window, and calculating a movement offset of the second coordinate relative to the first coordinate;

[0016] Determining a target focus position according to the preset position and the movement offset, wherein the target focus position is a position offset relative to the preset position based on the movement offset;

[0017] According to the target focus position, the target configurable function item corresponding to the target focus position in the gesture control window is determined.

[0018] Optionally, after triggering the target configurable function item and controlling the page based on the target configurable function item, the method includes:

[0019] Controlling the gesture control window to remain in display state;

[0020] In response to triggering of the second air gesture, if it is determined that the triggered target configurable function item is closed, the gesture control window is controlled to execute closing.

[0021] Optionally, the recognizing the first air gesture or the recognizing the second air gesture includes:

[0022] Based on the acquired hand image, extracting a preset number of joint points in the hand image;

[0023] The preset gesture recognition algorithm is called to identify the joint points, and the gesture category and hand coordinates corresponding to the first air gesture or the second air gesture are obtained respectively.

[0024] Optionally, the page control method of the air gesture further includes:

[0025] receiving a configuration instruction for the configurable function item in the gesture control window;

[0026] The configuration of the gesture control window is completed according to the configuration instruction.

[0027] According to a second aspect of the present disclosure, a page control device using air gestures is provided, comprising:

[0028] a start-up unit, configured to start a gesture control window in response to a trigger of a first air gesture, wherein the gesture control window includes at least two configurable function items, and the first air gesture is used to start the gesture control window;

[0029] A judging unit, configured to determine a moving offset in the gesture control window according to the detected displacement of the hand, and to judge a target configurable function item corresponding to the moving offset, wherein the moving offset includes a target moving direction and a target moving distance;

[0030] An execution unit is used to execute the target configurable function item in response to the triggering of a second air gesture, and control the page based on the target configurable function item, wherein the second air gesture is used to execute the target configurable function item.

[0031] Optionally, the starting unit includes:

[0032] A starting module, controlling to start the gesture control window at a cursor position in the page corresponding to the first air gesture;

[0033] A control module, controlling the focus position to be a preset position of the gesture control window;

[0034] A recording module records the first coordinate of the first air gesture in the gesture control window.

[0035] Optionally, the judging unit includes:

[0036] a calculation module, configured to obtain a second coordinate of the first air gesture in the gesture control window according to the displacement of the hand, and calculate a movement offset of the second coordinate relative to the first coordinate;

[0037] A determination module, configured to determine a target focus position according to the preset position and the movement offset, wherein the target focus position is a position offset relative to the preset position based on the movement offset;

[0038] The judgment module is used to judge the target configurable function item corresponding to the target focus position in the gesture control window according to the target focus position.

[0039] Optionally, the page control device of the air gesture further includes:

[0040] A display unit, used for controlling the gesture control window to remain in a display state;

[0041] A closing unit is used to respond to the triggering of the second air gesture, and if it is determined that the triggered target configurable function item is closed, control the gesture control window to execute closing.

[0042] Optionally, the device further includes an identification unit, wherein the identification unit includes:

[0043] An extraction module, used for extracting a preset number of joint points in the hand image based on the acquired hand image;

[0044] A recognition module is used to call the preset gesture recognition algorithm to identify the joint points, and obtain the gesture category and hand coordinates corresponding to the first air gesture or the second air gesture respectively.

[0045] Optionally, the window page control device of the air gesture further includes:

[0046] A receiving unit, configured to receive a configuration instruction for the configurable function item in the gesture control window;

[0047] A configuration unit is used to complete the configuration of the gesture control window according to the configuration instruction.

[0048] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0049] at least one processor; and

[0050] a memory communicatively connected to the at least one processor; wherein,

[0051] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the first aspect.

[0052] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method described in the first aspect.

[0053] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the method as described in the first aspect above.

[0054] The present disclosure provides a page control method and device for air gesture, a vehicle and an electronic device. In response to the triggering of a first air gesture, a gesture control window is started, the gesture control window contains at least two configurable function items, and the first air gesture is used to start the gesture control window; the movement offset in the gesture control window is determined according to the detected displacement of the hand, and the target configurable function item corresponding to the movement offset is judged, and the movement offset includes the target movement direction and the target movement distance; in response to the triggering of a second air gesture, the target configurable function item is executed, and the page is controlled based on the target configurable function item, and the second air gesture is used to execute the target configurable function item. The page control method for air gesture provided by the embodiment of the present disclosure starts the gesture control window after detecting the first air gesture, and can display all configurable function items, and selects and activates the target configurable function item through the displacement of the hand and the second air gesture. Compared with the scrolling control function along the x-axis and y-axis, the method of selecting the target configurable function item through the displacement of the hand is more flexible, and the configurable function item can be customized according to the needs, and the number of functions that can be realized is greater, which solves the problem that the control strategy of air gesture in the related art is too limited.

[0055] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0057] Figure 1 A schematic diagram of a flow chart of a page control method using air gestures provided in an embodiment of the present disclosure;

[0058] Figure 2 A schematic diagram of a gesture control window provided in an embodiment of the present disclosure;

[0059] Figure 3 A schematic diagram of another gesture control window provided in an embodiment of the present disclosure;

[0060] Figure 4 A schematic diagram of the effect of a gesture control window provided by an embodiment of the present disclosure;

[0061] Figure 5 A schematic diagram of hand image recognition provided by an embodiment of the present disclosure;

[0062] Figure 6 A schematic diagram of a process for determining a target configurable function item corresponding to coordinate information of hand movement provided by an embodiment of the present disclosure;

[0063] Figure 7 A schematic diagram of the structure of a page control device using air gestures provided in an embodiment of the present disclosure;

[0064] Figure 8 A schematic diagram of the structure of another page control device using air gestures provided in an embodiment of the present disclosure;

[0065] Fig. 9 A schematic block diagram of an exemplary electronic device 400 provided for an embodiment of the present disclosure. DETAILED DESCRIPTION

[0066] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0067] The following describes the page control method and device, vehicle and electronic device of the air gesture according to the embodiments of the present disclosure with reference to the accompanying drawings.

[0068] Figure 1 A flow chart of a page control method using air gestures provided in an embodiment of the present disclosure.

[0069] like Figure 1 As shown, the method comprises the following steps:

[0070] Step 101, in response to the triggering of a first air gesture, start a gesture control window, the gesture control window including at least two configurable function items, and the first air gesture is used to start the gesture control window.

[0071] In some embodiments, the gesture control window is a window for controlling the computer system according to the air gestures made by the user in three-dimensional space. The window refers to a rectangular area in the computer operating system, which usually includes a title bar, menu bar, toolbar, status bar and other interface tools for displaying the interface of the application. The user can operate the application through the window. In order to facilitate a better understanding of the start gesture control window, as shown in FIG. Figure 2 As shown, Figure 2A schematic diagram of a gesture control window provided in an embodiment of the present application. The figure is an explanation of an example of launching the gesture control window through a first air gesture: after detecting the first air gesture, at the cursor position in the page corresponding to the first air gesture, the gesture control window is controlled to be launched and the first coordinate of the first air gesture in the gesture control window is recorded, and all configurable function items of the gesture control window are displayed. The gesture control window contains at least two positions that can be focused, and each position corresponds to a configurable function item. Therefore, the gesture control window contains at least two configurable function items, and the configurable function items at each position can be customized by the user according to his or her own preferences. After the gesture control window is launched at the cursor position in the page, the focus position is controlled to be the preset position of the gesture control window. The preset position can also be customized according to the user's preferences. For example, the preset position can be controlled to be the center position, or the preset position can be controlled to be the upper left corner position, which is not limited to the specific embodiments of the present application. For a better understanding of the preset position, please refer to Figure 2 , Figure 2 Taking the preset position as the center position as an example, Figure 2 The configurable function item corresponding to the center position is "Close the gesture control window".

[0072] To better understand that the gesture control window includes at least two configurable function items, please continue to refer to Figure 2 , Figure 2 The provided gesture control window diagram includes seven configurable function items, and the functions corresponding to the configurable function items included in the gesture control window are: close the control window, scroll the page to the left, scroll the page to the right, scroll the page down, scroll the page up, return to the previous page, and return to the HOME page. The configurable function items can be configured according to the user's preferences and habits through configuration instructions to meet the user's needs.

[0073] However, it should be clear that this description is not intended to limit the number of configurable function items of the gesture control window, nor is it intended to limit the functions of the configurable function items of the gesture control window.

[0074] Step 102, determining a movement offset in the gesture control window according to the detected displacement of the hand, and determining a target configurable function item corresponding to the movement offset, wherein the movement offset includes a target movement direction and a target movement distance.

[0075] In the embodiment provided in the present application, the movement offset is determined according to the displacement of the hand and the preset position, including the target movement direction and the target movement distance relative to the preset position. After the focus position is moved based on the preset position according to the movement offset, the target focus position can be obtained, and the configurable function item corresponding to the target focus position is determined as the target configurable function item.

[0076] In order to facilitate a better understanding of determining the movement offset in the gesture control window according to the detected displacement of the hand, and determining the target configurable function item corresponding to the movement offset, as shown in FIG. Figure 3 As shown, Figure 3 A schematic diagram of another gesture control window provided in an embodiment of the present application. According to the displacement of the hand, the movement offset of the second coordinate relative to the first coordinate is calculated, including: recording the first coordinate, the first coordinate is the coordinate of the first air gesture in the gesture control window recorded when the gesture control window is started; recording the second coordinate, the second coordinate is the coordinate of the first air gesture in the gesture control window after the hand moves, and the coordinate after the movement is recorded as the second coordinate; calculating the vector difference between the second coordinate and the first coordinate, and determining the vector difference as the movement offset, the movement offset is a vector, including the target movement direction and the target movement distance.

[0077] In order to better understand the determination of the target focus position according to the preset position and the movement offset and the determination of the target configurable function item corresponding to the target focus position in the gesture control window, please continue to refer to Figure 2 and Figure 3 , compared to Figure 2 The preset position selected for the middle focus, Figure 3 The gesture control window shown focuses on the configurable function item on the right side of the preset position. This position is the target focus position obtained by offsetting the preset position according to the movement offset. The configurable function item corresponding to the target focus position is the target configurable function item, such as Figure 3 The function of the target configurable function item corresponding to the middle focus selection position is "scroll the page to the left".

[0078] Step 103, in response to the triggering of the second air gesture, execute the target configurable function item, and control the page based on the target configurable function item, and the second air gesture is used to execute the target configurable function item.

[0079] In one embodiment provided in the present application, the second air gesture is different from the first air gesture in gesture type and is used to execute the target configurable function item. In order to facilitate a better understanding of the execution of the target configurable function item in response to the triggering of the second air gesture, as shown in FIG. Figure 4 As shown, Figure 4 A schematic diagram of the effect of a gesture control window after triggering a second air gesture provided by an embodiment of the present application. Figure 3 The target configurable function item corresponding to the focus position in the gesture control window shown in FIG. 1 activates the function corresponding to the target configurable function item, and the corresponding effect is as follows: Figure 4 As shown, the page implements corresponding operations under the control of the function corresponding to the target configurable function item. While triggering the focus-selected target configurable function item, the gesture control window is controlled to remain displayed.

[0080] The present disclosure provides a page control method of air gesture, in response to the triggering of a first air gesture, a gesture control window is started, the gesture control window contains at least two configurable function items, the first air gesture is used to start the gesture control window; according to the detected displacement of the hand, the movement offset in the gesture control window is determined, and the target configurable function item corresponding to the movement offset is judged, the movement offset includes the target movement direction and the target movement distance; in response to the triggering of a second air gesture, the target configurable function item is executed, and the page is controlled based on the target configurable function item, the second air gesture is used to execute the target configurable function item. The page control method of air gesture provided by the embodiment of the present disclosure starts the gesture control window after detecting the first air gesture, and can display all configurable function items, and selects and activates the target configurable function item through the displacement of the hand and the second air gesture. Compared with the scrolling control function along the x-axis and y-axis, the method of selecting the target configurable function item through the displacement of the hand is more flexible, and the configurable function item can be customized according to the needs, and the number of functions that can be realized is greater, which solves the problem that the control strategy of the air gesture in the related art is too limited.

[0081] In some embodiments, in order to obtain a hand image for the recognition of an air gesture, the following method may be used but should not be limited to, for example, a video stream of a human hand in a three-dimensional space is obtained through a camera, key frames of the air gesture in the video stream are captured, and an image with a resolution of 640×480 and color channels of red (R), green (G), and blue (B) is generated as hand image data. However, it should be clear that this method is not intended to limit the way of obtaining a hand image, and the acquisition of a hand image may also be carried out by other methods, nor is it intended to limit the attributes of the hand image. The resolution of the hand image can be flexibly set according to demand, such as 640×480, 960×540, or other values. Similarly, the color space of the hand image can use a red, green, and blue (RGB) color space, or a hue H (Hue), saturation S (Saturation), and brightness V (Value) color space, which can be flexibly set according to demand.

[0082] In order to realize the recognition of the first air gesture or the second air gesture, the following method may be used but should not be limited to: calling a preset gesture recognition algorithm to recognize the first air gesture or the second air gesture, and obtaining the gesture category and hand coordinates corresponding to the first air gesture or the second air gesture respectively. Figure 5 As shown, Figure 5 FIG. 1 is a schematic diagram of hand image recognition. The hand information of the hand image is recognized by using a trained neural network model, and a preset number of joint points in the hand image are extracted. Figure 5 As shown, Figure 5 21 joint points in the human hand image are extracted. Then the preset gesture recognition algorithm is called to identify the joint points, and the gesture category and hand coordinates corresponding to the first air gesture or the second air gesture are obtained respectively. According to the position of the joint points, the preset gesture recognition algorithm is called to identify the joint points, and then the type of air gesture is determined, and then matched with the preset air gesture to trigger the corresponding air gesture. According to the position coordinates of the joint points, the preset algorithm is called to calculate the hand coordinates corresponding to the preset number of joint points. For the calculation of hand coordinates, the average value of the coordinate point values ​​of each preset number of joint points can be used as the hand coordinate method corresponding to the preset number of joint points, such as calculating Figure 5 The average value of the coordinate point values ​​corresponding to the 21 joint points shown is taken as the hand coordinates shown in the figure. Other methods can also be used to calculate the hand coordinates. In some embodiments, the type of air gesture and the shape of the gesture are also related to the method of calculating the hand coordinates. The specific calculation method is not limited in this embodiment.

[0083] In order to start the gesture control window, the following implementation methods may be used but should not be limited to, for example, after the camera detects the first air gesture, the gesture control window is started, and at the cursor position in the page corresponding to the first air gesture, the gesture control window is controlled to start, and all configurable function items of the gesture control window are displayed. Please continue to refer to Figure 2 , control the focus position to be the preset position of the gesture control window, and record the first coordinate of the first air gesture in the gesture control window.

[0084] In order to record the first coordinate of the first air gesture in the gesture control window, the following method may be used but is not limited to it. Please continue to refer to Figure 5 , through the trained neural network model, recognize the human hand information and extract Figure 5 The joint points shown are calculated according to a preset algorithm to obtain hand coordinates, and the hand coordinates are determined as first coordinates and recorded.

[0085] As a refinement of step 102, according to the detected coordinate information of the hand movement, determining the target configurable function item corresponding to the coordinate information of the hand movement can be implemented in the following manners, but should not be limited to: Figure 6 As shown, the method includes:

[0086] Step 201: acquiring the second coordinate of the first air gesture in the gesture control window according to the displacement of the hand, and calculating the movement offset of the second coordinate relative to the first coordinate.

[0087] In order to obtain the movement offset of the second coordinate relative to the first coordinate, to determine the coordinate information of the hand movement, and to further determine the target configurable function item focused in the gesture control window. The following method can be used but should not be limited to, for example, after triggering the first air gesture and recording the first coordinate, the displacement of the hand and the coordinates of the position of the hand after the movement are obtained through the camera, and the coordinates of the position of the hand after the displacement are determined as the second coordinate.

[0088] According to the displacement of the hand and the second coordinate, the movement offset of the second coordinate relative to the first coordinate is calculated, and the calculation formula is as follows:

[0089] Δx=xx c (1)

[0090] Δy=yy c (2)

[0091] Wherein, Δx is the offset of the hand coordinate on the x-axis in the rectangular coordinate system, Δy is the offset of the hand coordinate on the y-axis in the rectangular coordinate system; x is the value of the abscissa of the first coordinate in the rectangular coordinate system, y is the value of the ordinate of the first coordinate in the rectangular coordinate system; x c is the value of the horizontal coordinate of the second coordinate in the rectangular coordinate system, y c is the value of the ordinate of the second coordinate in the rectangular coordinate system.

[0092] It should be noted that the movement offset includes a target movement direction and a target movement distance, and the target movement direction and the target movement distance are determined by the magnitude and positive and negative of Δx and Δy.

[0093] The above formula is not intended to limit the calculation of the movement offset of the second coordinate relative to the first coordinate to only be implemented by the above formula, and other methods are not limited in the embodiments of the present application.

[0094] Step 202: determining a target focus position according to the preset position and the movement offset, wherein the target focus position is a position offset relative to the preset position based on the movement offset.

[0095] According to the movement offset calculated by formula (1) and formula (2) and the preset position, the target focus position obtained after the offset relative to the preset position in the gesture control window can be determined, and the target configurable function item corresponding to the target focus position can be determined according to the target focus position.

[0096] Step 203: determining the target configurable function item corresponding to the target focus position in the gesture control window according to the target focus position.

[0097] According to the movement offsets Δx and Δy calculated by formula (1) and formula (2), the coordinate information of the hand movement is determined. According to the coordinate information of the hand movement, the target configurable function item focused in the gesture control window is determined. For example, if Δx is positive and Δy is 0, it means that the coordinate information of the gesture movement is located on the right half axis of the x-axis of the rectangular coordinate system, then the schematic diagram of the target configurable function item focused in the gesture control window is as follows: Figure 3 shown.

[0098] In some embodiments, after the target configurable function item is triggered and the page is controlled based on the target configurable function item, the control window needs to be closed. The following methods may be used but should not be limited to, for example, after the target configurable function item is triggered, the gesture control window is controlled to remain displayed, and through the displacement of the hand, the focus function is the target configurable function item for closing the gesture control window, and when the second air gesture is detected, the control of the gesture control window is triggered to close.

[0099] In the embodiments of the present application, Figure 3 The configurable function items in the gesture control window shown can be configured according to the user's preferences and habits, and can adopt but should not be limited to the following methods, for example, receiving configuration instructions for the configurable function items in the gesture control window, and completing the configuration of the gesture control window according to the configuration instructions.

[0100] In summary, the embodiments of the present disclosure can achieve the following effects:

[0101] 1. After detecting the first air gesture, the gesture control window is started to display all configurable function items, and the target configurable function item is selected and activated through hand displacement and the second air gesture. Compared with the scrolling control function along the x-axis and y-axis, the method of selecting the target configurable function item through hand displacement is more flexible.

[0102] 2. The function items displayed in the gesture control window can be configured according to the user's preferences and habits, which can realize more functions and bring convenience to user operation.

[0103] 3. By moving the hand, the focus target can be configured with function items, which is not limited to realizing the scrolling function along the x-axis and y-axis, and can realize more control functions of the page, solving the problem that the scrolling control strategy along the x-axis and y-axis written into the electronic device is too limited.

[0104] Corresponding to the above-mentioned air gesture page control method, the present invention also proposes an air gesture page control device. Since the device embodiment of the present invention corresponds to the above-mentioned method embodiment, the details not disclosed in the device embodiment can be referred to the above-mentioned method embodiment, and will not be repeated in the present invention.

[0105] Figure 7 A schematic diagram of a page control device for air gestures provided in an embodiment of the present disclosure is shown in FIG. Figure 7 As shown, it includes: a starting unit 31, a judging unit 32, and an executing unit 33.

[0106] A starting unit 31, configured to start a gesture control window in response to a trigger of a first air gesture, wherein the gesture control window includes at least two configurable function items, and the first air gesture is used to start the gesture control window;

[0107] A judging unit 32, configured to determine a moving offset in the gesture control window according to the detected displacement of the hand, and to judge a target configurable function item corresponding to the moving offset, wherein the moving offset includes a target moving direction and a target moving distance;

[0108] The execution unit 33 is used to execute the target configurable function item in response to the triggering of the second air gesture, and control the page based on the target configurable function item, and the second air gesture is used to execute the target configurable function item.

[0109] The present disclosure provides a page control method and device for air gesture, a vehicle and an electronic device. In response to the triggering of a first air gesture, a gesture control window is started, the gesture control window contains at least two configurable function items, and the first air gesture is used to start the gesture control window; the movement offset in the gesture control window is determined according to the detected displacement of the hand, and the target configurable function item corresponding to the movement offset is judged, and the movement offset includes the target movement direction and the target movement distance; in response to the triggering of a second air gesture, the target configurable function item is executed, and the page is controlled based on the target configurable function item, and the second air gesture is used to execute the target configurable function item. The page control method for air gesture provided by the embodiment of the present disclosure starts the gesture control window after detecting the first air gesture, and can display all configurable function items, and selects and activates the target configurable function item through the displacement of the hand and the second air gesture. Compared with the scrolling control function along the x-axis and y-axis, the method of selecting the target configurable function item through the displacement of the hand is more flexible, and the configurable function item can be customized according to the needs, and the number of functions that can be realized is greater, which solves the problem that the control strategy of air gesture in the related art is too limited.

[0110] Furthermore, in a possible implementation of this embodiment, as Figure 8 As shown, the starting unit 31 includes:

[0111] A starting module 311 controls starting the gesture control window at a cursor position in the page corresponding to the first air gesture;

[0112] A control module 312 controls the focus position to be a preset position of the gesture control window;

[0113] The recording module 313 records the first coordinate of the first air gesture in the gesture control window.

[0114] Furthermore, in a possible implementation of this embodiment, as Figure 8 As shown, the judging unit 32 includes:

[0115] A calculation module 321, configured to obtain a second coordinate of the first air gesture in the gesture control window according to the displacement of the hand, and calculate a movement offset of the second coordinate relative to the first coordinate;

[0116] A determination module 322, configured to determine a target focus position according to the preset position and the movement offset, wherein the target focus position is a position offset relative to the preset position based on the movement offset;

[0117] The determination module 323 is configured to determine, according to the target focus position, the target configurable function item corresponding to the target focus position in the gesture control window.

[0118] Furthermore, if Figure 8 As shown, the page control device of the air gesture further includes:

[0119] A display unit 34, used to control the gesture control window to remain in a display state;

[0120] The closing unit 35 is used to respond to the triggering of the second air gesture, and if it is determined that the triggered target configurable function item is closed, control the gesture control window to execute closing.

[0121] Furthermore, if Figure 8 As shown, the device further includes an identification unit 36, which is used to:

[0122] An extraction module 361 is used to extract a preset number of joint points in the hand image based on the acquired hand image;

[0123] The recognition module 362 is used to call the preset gesture recognition algorithm to identify the joint points, and obtain the gesture category and hand coordinates corresponding to the first air gesture or the second air gesture respectively.

[0124] Furthermore, if Figure 8 As shown, the window page control device of the air gesture further includes:

[0125] A receiving unit 37, configured to receive a configuration instruction for the configurable function item in the gesture control window;

[0126] The configuration unit 38 is used to complete the configuration of the gesture control window according to the configuration instruction.

[0127] It should be noted that the above explanation of the method embodiment is also applicable to the device of this embodiment, and the principle is the same, which is not limited in this embodiment.

[0128] According to an embodiment of the present disclosure, the present disclosure also provides a vehicle, an electronic device, a readable storage medium, and a computer program product.

[0129] Fig. 9 A schematic block diagram of an example electronic device 400 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0130] like Fig. 9 As shown, the device 400 includes a computing unit 401, which can perform various appropriate actions and processes according to a computer program stored in a ROM (Read-Only Memory) 402 or a computer program loaded from a storage unit 408 to a RAM (Random Access Memory) 403. In the RAM 403, various programs and data required for the operation of the device 400 can also be stored. The computing unit 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An I / O (Input / Output) interface 405 is also connected to the bus 404.

[0131] A number of components in the device 400 are connected to the I / O interface 405, including: an input unit 406, such as a keyboard, a mouse, etc.; an output unit 407, such as various types of displays, speakers, etc.; a storage unit 408, such as a disk, an optical disk, etc.; and a communication unit 409, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 409 allows the device 400 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0132] The computing unit 401 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, a CPU (Central Processing Unit), a GPU (Graphic Processing Units), various dedicated AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, a DSP (Digital Signal Processor), and any appropriate processor, controller, microcontroller, etc. The computing unit 401 performs the various methods and processes described above, such as the page control method of the air gesture. For example, in some embodiments, the page control method of the air gesture may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 408. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 400 via the ROM 402 and / or the communication unit 409. When the computer program is loaded into the RAM 403 and executed by the computing unit 401, one or more steps of the method described above may be performed. Alternatively, in other embodiments, the computing unit 401 may be configured to execute the aforementioned page control method of air gestures in any other appropriate manner (for example, by means of firmware).

[0133] Various embodiments of the systems and techniques described above herein may be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application Specific Standard Products), SOCs (System On Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that may be executed and / or interpreted on a programmable system including at least one programmable processor that may be a dedicated or general-purpose programmable processor that may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0134] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0135] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include electrical connections based on one or more lines, portable computer disks, hard disks, RAM, ROM, EPROM (Electrically Programmable Read-Only-Memory) or flash memory, optical fiber, CD-ROM (Compact Disc Read-Only Memory), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0136] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0137] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: LAN (Local Area Network), WAN (Wide Area Network), the Internet, and blockchain networks.

[0138] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0139] It should be noted that artificial intelligence is a discipline that studies how computers can simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.), and includes both hardware-level and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies mainly include computer vision technology, speech recognition technology, natural language processing technology, as well as machine learning / deep learning, big data processing technology, knowledge graph technology, and other major directions.

[0140] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0141] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A page control method using air gestures, characterized in that: include: In response to a trigger of a first air gesture, a gesture control window is started, wherein the gesture control window includes at least two configurable function items, and the first air gesture is used to start the gesture control window; Determine a movement offset in the gesture control window according to the detected displacement of the hand, and determine a target configurable function item corresponding to the movement offset, wherein the movement offset includes a target movement direction and a target movement distance; In response to triggering of a second air gesture, the target configurable function item is executed, and the page is controlled based on the target configurable function item, and the second air gesture is used to execute the target configurable function item.

2. The method according to claim 1, characterized in that The starting gesture control window comprises: At a cursor position in the page corresponding to the first air gesture, controlling to start the gesture control window; Controlling the focus position to be the preset position of the gesture control window; Record the first coordinate of the first air gesture in the gesture control window.

3. The method according to claim 2, characterized in that The step of determining the movement offset in the gesture control window according to the detected displacement of the hand, and determining the target configurable function item corresponding to the movement offset includes: According to the displacement of the hand, obtaining a second coordinate of the first air gesture in the gesture control window, and calculating a movement offset of the second coordinate relative to the first coordinate; Determining a target focus position according to the preset position and the movement offset, wherein the target focus position is a position offset relative to the preset position based on the movement offset; According to the target focus position, the target configurable function item corresponding to the target focus position in the gesture control window is determined.

4. The method according to claim 1, characterized in that: After triggering the target configurable function item and controlling the page based on the target configurable function item, the method includes: Controlling the gesture control window to remain in display state; In response to triggering of the second air gesture, if it is determined that the triggered target configurable function item is closed, the gesture control window is controlled to execute closing.

5. The method according to any one of claims 1 to 4, characterized in that Recognizing the first air gesture or recognizing the second air gesture includes: Based on the acquired hand image, extracting a preset number of joint points in the hand image; The preset gesture recognition algorithm is called to identify the joint points, and the gesture category and hand coordinates corresponding to the first air gesture or the second air gesture are obtained respectively.

6. The method according to claim 1, characterized in that The method comprises: receiving a configuration instruction for the configurable function item in the gesture control window; The configuration of the gesture control window is completed according to the configuration instruction.

7. A page control device using air gestures, characterized in that: include: a start-up unit, configured to start a gesture control window in response to a trigger of a first air gesture, wherein the gesture control window includes at least two configurable function items, and the first air gesture is used to start the gesture control window; A judging unit, configured to determine a moving offset in the gesture control window according to the detected displacement of the hand, and to judge a target configurable function item corresponding to the moving offset, wherein the moving offset includes a target moving direction and a target moving distance; An execution unit is used to execute the target configurable function item in response to the triggering of a second air gesture, and control the page based on the target configurable function item, wherein the second air gesture is used to execute the target configurable function item.

8. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-6.

10. A computer program product, characterized in that The invention comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 6.

Images