Cursor control method, control device, computer equipment and program product

Abstract

The invention discloses a cursor control method and device, computer equipment and a program product, and belongs to the technical field of man-machine interaction. The method comprises the following steps: acquiring a target image collected by a camera; and obtaining position coordinates of the manipulator in the target image. And determining a target position coordinate according to the position coordinate of the manipulator in the target image, wherein the target position coordinate is an average position coordinate of the manipulator in the target image and continuous n-1 frames of images collected by the camera before the target image. And determining a first manipulator displacement according to the target position coordinate and a historical position coordinate, wherein the historical position coordinate is an average position coordinate of a manipulator in continuous n frames of images collected by the camera before the target image. And controlling a cursor in the screen to move according to the first manipulator displacement. According to the method, the fluctuation quantity of the position coordinates of the operator in the whole image can be reduced, so that cursor jitter can be reduced, and the user operation experience is improved.

Description

Technical Field

[0001] The present application relates to the field of human-computer interaction technology, and in particular to a cursor control method, a control device, a computer device and a program product. Background Art

[0002] With the continuous development of science and technology, gesture control technology has gradually been applied to various fields. For example, users can control the cursor on the screen through air gestures.

[0003] Currently, electronic devices can analyze the captured images, determine the hand making the access gesture as the operating hand, and then control the cursor to move accordingly according to the movement trajectory of the operating hand.

[0004] However, when performing air gesture control, the user's hand is likely to shake, which will cause the cursor to shake more when moving, reducing the user's operating experience. Summary of the invention

[0005] The present application provides a cursor control method, a control device, a computer device and a program product, which can improve the smoothness of cursor movement. The technical solution is as follows:

[0006] In a first aspect, a cursor control method is provided, the method comprising:

[0007] Get the target image captured by the camera;

[0008] Acquire the position coordinates of the operator in the target image;

[0009] Determine the target position coordinates according to the position coordinates of the operator in the target image, wherein the target position coordinates are the average position coordinates of the operator in the target image and the continuous n-1 frames of images collected by the camera before the target image, where n is an integer greater than or equal to 2;

[0010] Determine the displacement of the first operator according to the target position coordinates and the historical position coordinates, wherein the historical position coordinates are the average position coordinates of the operator in n consecutive frames of images captured by the camera before the target image;

[0011] The cursor movement on the screen is controlled according to the displacement of the first operating hand.

[0012] In the present application, since the target position coordinates and the historical position coordinates can reflect the movement trend of the operator in the most recent n frames of images, and the target position coordinates and the historical position coordinates are the average position coordinates of the operator in the continuous n frames of images, when the position coordinates of the operator fluctuate greatly, the embodiment of the present application can distribute the fluctuation of the position coordinates of the operator to the n frames of images. In this way, the fluctuation of the position coordinates of the operator in the overall image can be reduced, thereby reducing the cursor jitter and improving the user operation experience.

[0013] Optionally, before determining the target position coordinates according to the position coordinates of the operator in the target image, the method further includes:

[0014] Determine whether the number of position coordinates in the historical detection coordinate library is n;

[0015] The step of determining the target position coordinates according to the position coordinates of the operator in the target image includes:

[0016] If the number of position coordinates in the historical detection coordinate library is n, then deleting the position coordinates in the historical detection coordinate library with the earliest addition time, and adding the position coordinates of the operator in the target image to the historical detection coordinate library;

[0017] The average value of all position coordinates in the historical detection coordinate library is determined as the target position coordinate.

[0018] Optionally, after determining whether the number of position coordinates in the historical detection coordinate library is n, the method further includes:

[0019] If the number of position coordinates in the historical detection coordinate library is 0, the position coordinates of the operator in the target image are added to the historical detection coordinate library; the initial position coordinates of the cursor in the screen are determined according to the position coordinates of the operator in the target image;

[0020] If the number of position coordinates in the historical detection coordinate library is greater than 0 and less than n, the position coordinates of the operator in the target image are added to the historical detection coordinate library; and the displacement of the first operator is determined to be 0.

[0021] Optionally, controlling the movement of a cursor on a screen according to the displacement of the first operator includes:

[0022] Determining the size of the operator's hand in the target image;

[0023] Adjusting the displacement of the first manipulator according to the size of the manipulator in the target image to obtain the displacement of the second manipulator;

[0024] The cursor movement is controlled according to the displacement of the second operating hand.

[0025] Optionally, the size of the operator in the target image includes a width and a height of a detection frame of the operator in the target image, and the first operator displacement includes a lateral displacement and a longitudinal displacement;

[0026] The adjusting the displacement of the first operator according to the size of the operator in the target image to obtain the displacement of the second operator includes:

[0027] The lateral displacement in the first operator displacement is divided by the width of the detection frame and then multiplied by a preset coefficient to obtain the lateral displacement in the second operator displacement;

[0028] The longitudinal displacement in the first operator displacement is divided by the height of the detection frame and then multiplied by the preset coefficient to obtain the longitudinal displacement in the second operator displacement.

[0029] Optionally, before controlling the movement of the cursor according to the displacement of the second operating hand, the method further includes:

[0030] If the lateral displacement in the second operator displacement is greater than the lateral displacement threshold, updating the lateral displacement in the second operator displacement to the lateral displacement threshold;

[0031] If the longitudinal displacement in the second operator displacement is greater than the longitudinal displacement threshold, the longitudinal displacement in the second operator displacement is updated to the longitudinal displacement threshold.

[0032] Optionally, after controlling the movement of a cursor on the screen according to the displacement of the first operator, the method further includes:

[0033] Predicting a first displacement amount according to three cursor position coordinates, wherein the three cursor position coordinates are the position coordinates to which the cursor is moved on the screen according to the three frames of images most recently captured by the camera, and the first displacement amount is the predicted displacement amount of the cursor on the screen controlled according to the next frame of image captured by the camera after the target image;

[0034] Dividing the first displacement by a preset number of interpolation frames to obtain a second displacement;

[0035] The cursor movement is controlled according to the second displacement.

[0036] Optionally, the three cursor position coordinates are sorted in order of movement time from early to late, and the predicting the first displacement amount according to the three cursor position coordinates includes:

[0037] Determine a first displacement vector and a second displacement vector according to the three cursor position coordinates, wherein the first displacement vector is a displacement vector from the first cursor position coordinate to the second cursor position coordinate among the three cursor position coordinates, and the second displacement vector is a displacement vector from the second cursor position coordinate to the third cursor position coordinate among the three cursor position coordinates;

[0038] Subtracting the first displacement vector from the second displacement vector to obtain an acceleration increment vector;

[0039] The second displacement vector is added to the acceleration increment vector to obtain a third displacement vector, and the elements of the third displacement vector are the first displacement amounts.

[0040] Optionally, controlling the movement of the cursor according to the second displacement includes:

[0041] Predicting a first duration, where the first duration is a predicted interval duration for controlling the movement of the cursor according to the image captured by the camera;

[0042] Divide the first duration by m to obtain a second duration, where m is the sum of the preset number of interpolated frames and 1;

[0043] The cursor movement is controlled according to the second displacement at every second time period.

[0044] In a second aspect, a cursor control device is provided, the device comprising:

[0045] A first acquisition module is used to acquire a target image captured by a camera;

[0046] A second acquisition module is used to acquire the position coordinates of the operator in the target image;

[0047] A first determination module is used to determine the target position coordinates according to the position coordinates of the operator in the target image, wherein the target position coordinates are the average position coordinates of the operator in the target image and the continuous n-1 frames of images collected by the camera before the target image, where n is an integer greater than or equal to 2;

[0048] A second determination module is used to determine the displacement of the first operator according to the target position coordinates and the historical position coordinates, wherein the historical position coordinates are the average position coordinates of the operator in the continuous n frames of images collected by the camera before the target image;

[0049] The first control module is used to control the movement of the cursor on the screen according to the displacement of the first operating hand.

[0050] In a third aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the cursor control method described in the first aspect when executed by the processor.

[0051] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the cursor control method described in the first aspect is implemented.

[0052] In a fifth aspect, a computer program product is provided. When the computer program product is run on a computer device, the computer device executes the cursor control method described in the first aspect.

[0053] It can be understood that the beneficial effects of the second, third, fourth and fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0055] Figure 1 is a schematic diagram of a cursor control system provided in an embodiment of the present application;

[0056] Figure 2 is a schematic diagram of hand feature points provided in an embodiment of the present application;

[0057] Figure 3 is a schematic diagram of a target image provided in an embodiment of the present application;

[0058] Figure 4 is a flow chart of a cursor control method provided by an embodiment of the present application;

[0059] Figure 5 is a flow chart for predicting a first displacement provided by an embodiment of the present application;

[0060] Figure 6 is a flowchart of a frame insertion process provided by an embodiment of the present application;

[0061] Figure 7 is a flowchart of another frame insertion process provided by an embodiment of the present application;

[0062] Figure 8is a flowchart of another frame insertion process provided by an embodiment of the present application;

[0063] Fig. 9 It is a flow chart of the process of building and operating a cursor control system provided by an embodiment of the present application;

[0064] Fig.10 is a flow chart of another cursor control method provided by an embodiment of the present application;

[0065] Fig.11 is a structural schematic diagram of a cursor control device provided in an embodiment of the present application;

[0066] Fig.12 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0067] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below in conjunction with the accompanying drawings.

[0068] It should be understood that the "multiple" mentioned in this application refers to two or more. In the description of this application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in order to facilitate the clear description of the technical solution of this application, the words "first" and "second" are used to distinguish between the same items or similar items with basically the same functions and effects. Those skilled in the art can understand that the words "first" and "second" do not limit the quantity and execution order, and the words "first" and "second" do not limit them to be different.

[0069] The phrases such as "one embodiment" or "some embodiments" described in the present application mean that the specific features, structures or characteristics described in the embodiment are included in one or more embodiments of the present application. Therefore, the phrases such as "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments" that appear in the differences in the present application do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. In addition, the terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0070] The application scenarios of the embodiments of the present application are described below.

[0071] Gesture control technology has been gradually applied to gaming, smart home, cursor control and other fields due to its features of remote control, diverse application scenarios, timely feedback and dynamic response. Currently, electronic devices can analyze the captured images, identify the hand that makes the access gesture as the operator, and then associate the operator with the cursor displayed by the electronic device and track the operator in real time to control the movement of the cursor.

[0072] In the related art, after an electronic device captures a frame of image, it can determine the position of the operator in the current image, and then determine the displacement of the operator based on the position of the operator in the current image and the position of the operator in the previous frame, and then control the cursor to move based on the displacement. However, when performing air gesture control, the user's hand is likely to shake. If the cursor movement is controlled based on the position of the operator in each frame of image, the cursor will shake greatly, which will reduce the user's operating experience.

[0073] To this end, an embodiment of the present application provides a cursor control method, which can reduce the jitter of the cursor when performing air gesture control, so that the cursor can move more smoothly, thereby improving the user's operating experience.

[0074] The cursor control system provided in the embodiment of the present application is exemplarily described below.

[0075] Figure 1 is a schematic diagram of a cursor control system provided by an embodiment of the present application. Figure 1 The cursor control system 10 may include a collection module 101 , a processing module 102 and a cursor module 103 .

[0076] It should be noted that the cursor control system 10 may be a system running in a computer device.

[0077] The acquisition module 101 may communicate with the processing module 102 via a wired connection or a wireless connection.

[0078] The processing module 102 may communicate with the cursor module 103 via a wired connection or a wireless connection.

[0079] The acquisition module 101 is used to acquire images and send the acquired images to the processing module 102. For example, the acquisition module 101 may be a camera.

[0080] The processing module 102 is used to identify the position coordinates and gesture type of the operator in the image, determine the displacement of the cursor according to the position coordinates of the operator, determine the operation event of the cursor according to the gesture type of the operator, and send the displacement of the cursor and the operation event of the cursor to the cursor module 103.

[0081] The gesture type is the type of gesture that can be recognized by the processing module 102. The gesture type can be preset. For example, the gesture type can include one or more of an access gesture, a cursor movement gesture, a left-click gesture, a right-click gesture, a left-double-click gesture, a right-double-click gesture, etc.

[0082] The access gesture is a gesture for entering a gesture control process (ie, controlling a cursor).

[0083] The cursor movement gesture is a gesture for controlling the movement of the cursor.

[0084] The left-click gesture is a gesture for controlling a cursor to perform a left-click.

[0085] The right-click gesture is a gesture for controlling a cursor to right-click.

[0086] The left-button double-click gesture is a gesture for controlling the cursor to double-click the left button.

[0087] The right-click double-click gesture is a gesture for controlling a cursor to perform a right-click double-click.

[0088] In some embodiments, the processing module 102 may include a gesture recognition model and a gesture recognition algorithm.

[0089] The gesture recognition model and the gesture recognition algorithm may be preset.

[0090] The gesture recognition model is used to identify multiple hand feature points of the operator in the image. For example, Figure 2 As shown, the gesture recognition model can identify 21 feature points of the hand in the image.

[0091] The gesture recognition algorithm is used to identify the gesture type of the operator according to the multiple hand feature points, and determine the position coordinates of the operator in the image. For example, Figure 3 As shown, the position coordinates (x, y) of the operator may be the pixel coordinates of the operator in the image, and the gesture type of the operator may be the type of gesture displayed by the operator in the image.

[0092] In this case, after receiving the image, the processing module 102 can input the image into the gesture recognition model to identify multiple hand feature points of the operator in the image, and then determine the gesture type of the operator and the position coordinates of the operator based on the gesture recognition algorithm and the identified multiple hand feature points, and finally determine the displacement of the operator based on the position coordinates of the operator.

[0093] It should be noted that, in the embodiment of the present application, the processing module 102 may also identify the position coordinates and gesture type of the operator in the image in other ways, and the embodiment of the present application is not limited to this.

[0094] Optionally, a first correspondence is set in the processing module 102, and the first correspondence is a correspondence between a hand gesture type and a cursor operation event. The first correspondence can be preset. For example, an access gesture can correspond to an event of controlling a cursor, a cursor movement gesture can correspond to an event of moving a cursor, a left-click gesture can correspond to an event of a cursor left-click, a right-click gesture can correspond to an event of a cursor right-click, etc., a left-click double-click gesture can correspond to an event of a cursor left-click double-click, a right-click double-click gesture can correspond to an event of a cursor right-click, etc., and the present application embodiment does not limit this.

[0095] In this case, after identifying the gesture type, the processing module 102 can determine the operation event that the cursor needs to perform from the first corresponding relationship according to the gesture type.

[0096] Optionally, a second corresponding relationship is further provided in the processing module 102, and the second corresponding relationship is a corresponding relationship between the displacement of the operator and the displacement of the cursor. The second corresponding relationship can be preset.

[0097] In this case, after the processing module 102 determines the displacement of the operator according to the position coordinates of the operator, it can determine the displacement of the cursor according to the second corresponding relationship and the displacement of the operator.

[0098] The cursor module 103 is used to control the cursor to move according to the displacement of the cursor, and to control the cursor to operate according to the operation event of the cursor.

[0099] In some embodiments, the processing module 102 may be a local service program, which may communicate with the operating system in the computer device before capturing the image, register the virtual cursor event in the operating system, and generate a virtual cursor node (i.e., cursor module 103) in the operating system. In this way, the coordinate conversion between the position coordinates of the operator's hand and the position coordinates of the cursor is completed.

[0100] In some embodiments, before the cursor control system 10 controls the cursor to move according to the position coordinates of the operator in the image, the operator in the image may be determined first.

[0101] For example, the cursor control system 10 acquires an image, identifies the user's hand and the gesture type of the user's hand in the image, to determine whether there is an access gesture in the image. If there is an access gesture in the image, the user's hand with the gesture type of the access gesture in the image is determined as the operating hand, and then the cursor control process is entered. After that, the cursor control system 10 continues to acquire new images, and continues to track the operating hand in the acquired images, and accordingly controls the cursor to move through the cursor control method provided below.

[0102] The cursor control method provided in the embodiment of the present application is explained in detail below.

[0103] Figure 4 1 is a flow chart of a cursor control method provided by an embodiment of the present application. The method can be applied to a computer device, for example, can be applied to a cursor control system 10 running in a computer device. Figure 4 , the method comprises the following steps:

[0104] Step 401: The computer device obtains the target image captured by the camera.

[0105] For example, the camera may be the acquisition module 101 described above.

[0106] The target image is a frame of image captured by the computer device after the operator is determined.

[0107] Step 402: The computer device obtains the position coordinates of the operator in the target image.

[0108] For example, the computer device may identify multiple hand feature points of the operator in the target image through a gesture recognition model, and then identify the position coordinates of the operator in the target image through a gesture recognition algorithm based on the multiple hand feature points.

[0109] Step 403: The computer device determines the target position coordinates according to the position coordinates of the operator in the target image.

[0110] The target position coordinates are the average position coordinates of the operator in the target image and the continuous n-1 frames of images collected by the camera before the target image, where n is an integer greater than or equal to 2.

[0111] Since the target image and the continuous n-1 frames of images collected before the target image can reflect the movement trend of the operator in the latest n frames of images, the target position coordinates determined based on them can more accurately represent the latest position of the operator. And since the target position coordinates are average position coordinates, when the position coordinates of the operator fluctuate greatly, the fluctuation of the position coordinates of the operator can be distributed to the n frames of images, so that the fluctuation of the position coordinates of the operator in the target image can be reduced, thereby reducing the cursor jitter.

[0112] In some embodiments, after step 402 , that is, after obtaining the position coordinates of the operator in the target image, the computer device may first determine whether the number of position coordinates in the historical detection coordinate library is n.

[0113] The historical detection coordinate library is used to store the position coordinates of the operator detected after entering the cursor control process. For example, the position coordinates in the historical detection coordinate library can be stored in chronological order. n is the maximum number of position coordinates that can be stored in the historical detection coordinate library.

[0114] Considering that in the first n frames of images after entering the cursor control process, the user may not control the operator to move, but may confirm whether the cursor control process has been successfully entered. However, in the n frames of images, the cursor may shake due to the shaking of the operator's hand. Therefore, after the computer device obtains the position coordinates of the operator's hand in the target image, it can first determine whether the number of position coordinates in the historical detection coordinate library is n, so as not to control the cursor movement in the first n frames of images, and then control the cursor movement after the n frames of images. This is explained below:

[0115] If the number of position coordinates in the historical detection coordinate library is 0, it means that the target image is the first frame image after entering the cursor control process, so the position coordinates of the operator in the target image can be added to the historical detection coordinate library, and then the initial position coordinates of the cursor on the screen are determined according to the position coordinates of the operator in the target image, and then the cursor is displayed at the initial position coordinates.

[0116] The initial position coordinates are the position coordinates of the cursor when it is first displayed on the screen after entering the cursor control process.

[0117] If the number of position coordinates in the historical detection coordinate library is greater than 0 and less than n, it means that the target image is one of the first n frames of images after entering the cursor control process, so the position coordinates of the operator in the target image can be added to the historical detection coordinate library, and the displacement of the first operator is determined to be 0, so as not to move the cursor in the first n frames of images. In this way, cursor jitter caused by operator jitter can be avoided, thereby improving the user experience.

[0118] If the number of position coordinates in the historical detection coordinate library is n, it means that the target image is an image frame after the first n frames of images after entering the cursor control process, so step 403 can be executed. Specifically, the position coordinates with the earliest addition time in the historical detection coordinate library can be deleted, the position coordinates of the operator in the target image can be added to the historical detection coordinate library, and then the average value of all position coordinates in the historical detection coordinate library is determined as the target position coordinates.

[0119] For example, the horizontal coordinate values ​​of all position coordinates in the historical detection coordinate library can be added and divided by n to obtain the horizontal coordinate value in the target position coordinate, and the vertical coordinate values ​​of all position coordinates in the historical detection coordinate library can be added and divided by n to obtain the vertical coordinate value in the target position coordinate.

[0120] Step 404: The computer device determines the displacement of the first operator according to the target position coordinates and the historical position coordinates.

[0121] The historical position coordinates are the average position coordinates of the operator in the continuous n frames of images captured by the camera before the target image. For example, the historical position coordinates may be the target position coordinates determined by the computer device when acquiring the last frame of image captured by the camera.

[0122] Since the historical position coordinates and the target position coordinates are both average position coordinates obtained after smooth jittering, and comparing the target position coordinates with the historical position coordinates can reflect the movement trend of the operator in the current frame image, the displacement of the operator after smooth jittering, that is, the first operator displacement, can be determined based on the target position coordinates and the historical position coordinates.

[0123] In some embodiments, the operation of the computer device determining the first operator displacement amount based on the target position coordinates and the historical position coordinates can be: the computer device subtracts the horizontal coordinate value of the historical position coordinates from the horizontal coordinate value of the target position coordinates to obtain the horizontal displacement amount in the first operator displacement amount, and subtracts the vertical coordinate value of the historical position coordinates from the vertical coordinate value of the target position coordinates to obtain the vertical displacement amount in the first operator displacement amount.

[0124] In some embodiments, after the computer device determines the displacement of the first operator, the target position coordinates determined in the current frame image may be saved as historical position coordinates so as to determine the displacement of the first operator in the next frame image accordingly.

[0125] Step 405: The computer device controls the movement of a cursor on the screen according to the displacement of the first operating hand.

[0126] In some embodiments, the operation of step 405 may be: the computer device determines the cursor displacement according to the first operator displacement through the second corresponding relationship, and then controls the movement of the cursor on the screen according to the cursor displacement.

[0127] In other embodiments, the operation of step 405 may be: the computer device determines the size of the operator in the target image; adjusts the first operator displacement according to the size of the operator in the target image to obtain the second operator displacement; and controls the cursor movement according to the second operator displacement.

[0128] It should be noted that the size of the operator captured by the camera will affect the speed of the operator's movement in the image. The closer the operator is to the camera, the larger the proportion of pixels occupied by the operator in the target image; the farther the operator is from the camera, the smaller the proportion of pixels occupied by the operator in the target image. When the operator moves the same distance, the displacement of the operator close to the camera in the image is greater than the displacement of the operator far from the camera in the image. This situation may cause the controlled cursor to move at different speeds, affecting the user experience. Therefore, it is necessary to adjust the displacement of the first operator according to the size of the operator in the target image so that the cursor movement speed tends to be consistent when the operator is at different distances from the camera.

[0129] In some embodiments, the size of the operator in the target image includes the width and height of the detection frame of the operator in the target image, and the first operator displacement includes a lateral displacement and a longitudinal displacement. The computer device adjusts the first operator displacement according to the size of the operator in the target image, and the operation of obtaining the second operator displacement may be: dividing the lateral displacement in the first operator displacement by the width of the detection frame and then multiplying by a preset coefficient to obtain the lateral displacement in the second operator displacement; dividing the longitudinal displacement in the first operator displacement by the height of the detection frame and then multiplying by a preset coefficient to obtain the longitudinal displacement in the second operator displacement.

[0130] For example, Figure 3 As shown, the detection frame of the operator can be the minimum circumscribed rectangle of the operator in the target image. For example, the size of the detection frame of the operator can be rect(w, h), where w is the width of the detection frame and h is the height of the detection frame.

[0131] The preset coefficient can be preset. For example, the preset coefficient can be set by a technician based on experience. The size of the preset coefficient is used to reflect the sensitivity of the operator when controlling the cursor movement.

[0132] For example, the computer device may determine the lateral displacement in the displacement of the second operating hand by using the following formula.

[0133] offset x1 =offset x ÷w×rate;

[0134] Among them, offset x1 is the lateral displacement of the second manipulator, offset x is the lateral displacement of the first operator, ω is the width of the detection frame of the operator, and rate is the preset coefficient.

[0135] For example, the computer device may determine the longitudinal displacement in the second operating hand displacement by the following formula.

[0136] offset y1 =offset y ÷h×rate;

[0137] Among them, offset y1 is the longitudinal displacement of the second manipulator, offset y is the longitudinal displacement of the first operator's displacement, h is the height of the detection frame of the operator, and rate is a preset coefficient.

[0138] In this way, the movement speed of the cursor when the operator is at different distances from the camera can be corrected, reducing the influence of the distance between the operator and the camera on the movement speed of the cursor, thereby improving the user experience.

[0139] In some embodiments, before the computer device controls the movement of the cursor according to the displacement of the second operator, the computer device may also adjust the displacement of the second operator.

[0140] For example, the computer device may determine whether the lateral displacement in the second operator's displacement is greater than a lateral displacement threshold, and determine whether the longitudinal displacement in the second operator's displacement is greater than a longitudinal displacement threshold.

[0141] The lateral displacement threshold is the maximum distance that the operator is allowed to move in the lateral direction each time. The lateral displacement threshold can be preset. For example, the lateral displacement threshold can be set by a technician according to needs.

[0142] The longitudinal displacement threshold is the maximum distance that the operator is allowed to move in the longitudinal direction each time. The longitudinal displacement threshold can be preset. For example, the longitudinal displacement threshold can be set by a technician according to needs.

[0143] When the displacement of the second operator hand is large, the position of the cursor display may change greatly, which may cause the cursor to "jump" during the display process, thereby reducing the user experience. In this case, the displacement of the second operator hand can be adjusted by the lateral displacement threshold and the longitudinal displacement threshold to improve the coherence of the cursor during the movement process.

[0144] If the lateral displacement in the second operator displacement is greater than the lateral displacement threshold, the lateral displacement in the second operator displacement can be updated to the lateral displacement threshold. If the longitudinal displacement in the second operator displacement is greater than the longitudinal displacement threshold, the longitudinal displacement in the second operator displacement can be updated to the longitudinal displacement threshold.

[0145] In some embodiments, the operation of the computer device controlling the cursor movement according to the displacement of the second operator hand may be: the computer device determines the cursor displacement according to the second operator hand displacement through a second corresponding relationship, and then controls the cursor movement on the screen according to the cursor displacement.

[0146] In some embodiments, after step 405, that is, after controlling the movement of the cursor on the screen according to the first operator displacement, the computer device can predict the first displacement according to the three cursor position coordinates; divide the first displacement by the preset number of interpolation frames to obtain the second displacement; and control the cursor movement according to the second displacement.

[0147] The three cursor position coordinates are the position coordinates to which the computer device moves the cursor in the screen according to the three frames of images most recently captured by the camera, that is, the position coordinates to which the cursor is moved in the screen according to the first operator displacement or the second operator displacement.

[0148] The first displacement is the predicted displacement of the cursor on the screen based on the next frame image captured by the camera after the target image, that is, the cursor displacement corresponding to the first operator displacement or the second operator displacement.

[0149] The preset number of interpolation frames can be preset. For example, the preset number of interpolation frames can be set to 1, 2, 3, etc., which is not limited in the embodiment of the present application.

[0150] The second displacement is the displacement that the cursor needs to move in each frame when inserting a preset number of virtual frames.

[0151] Since the three cursor position coordinates can reflect information such as the cursor's most recent moving distance and acceleration, the computer device can predict the first displacement accordingly.

[0152] It should be noted that the speed of determining the displacement of the first operator in the embodiment of the present application is affected by the processing speed of the computer device. The faster the processing speed of the computer device, the faster the speed of determining the displacement of the first operator, and the faster the frame rate of reporting the cursor displacement. The slower the processing speed of the computer device, the slower the speed of determining the displacement of the first operator, and the slower the frame rate of reporting the cursor displacement, which will cause the cursor to "jump" during movement, affecting the user experience. In this case, the frame rate of the cursor movement can be improved by interpolation, thereby improving the coherence of the cursor movement, and by predicting the position coordinates of the cursor to control the movement of the cursor, the cursor movement can be made more natural and smooth.

[0153] In some embodiments, the three cursor position coordinates are sorted in order of movement time from early to late, and the operation of the computer device predicting the first displacement amount based on the three cursor position coordinates can be: the computer device determines the first displacement vector and the second displacement vector based on the three cursor position coordinates; subtracts the first displacement vector from the second displacement vector to obtain an acceleration increment vector; adds the second displacement vector to the acceleration increment vector to obtain a third displacement vector, and the elements of the third displacement vector are the first displacement amounts.

[0154] The first displacement vector is a displacement vector from a first cursor position coordinate to a second cursor position coordinate among the three cursor position coordinates.

[0155] The second displacement vector is a displacement vector from the second cursor position coordinate to the third cursor position coordinate among the three cursor position coordinates.

[0156] The acceleration increment is the predicted velocity increment of the cursor at the next frame of the target image.

[0157] The transverse component value of the third displacement vector is the transverse displacement value of the first displacement amount, and the longitudinal component value of the third displacement vector is the longitudinal displacement value of the first displacement amount.

[0158] For example, the computer device may determine the third displacement by the following formula.

[0159] I x =R×(D x +A x );

[0160] I y =R×(D y +A y );

[0161] Among them, I x is the horizontal component value in the third displacement vector, that is, the horizontal displacement in the first displacement; R is the preset number of interpolation frames; D x is the lateral component value of the second displacement vector; A x is the lateral component value of the acceleration increment vector; I y is the longitudinal component value in the third displacement vector, that is, the longitudinal displacement in the first displacement; D y A is the longitudinal component value of the second displacement vector; y is the longitudinal component value of the acceleration increment vector.

[0162] For example, Figure 5As shown in Figure (a), assuming that the three cursor position coordinates are A(3, 8), B(8, 5) and C(11, 0), the first displacement vector a(5, -3) from point A to point B and the second displacement vector b(3, -5) from point B to point C can be determined. Figure 5 As shown in Figure (b), the acceleration increment vector c(-2, -2) is obtained by subtracting the first displacement vector a(5, -3) from the second displacement vector b(3, -5). Figure 5 As shown in Figure (c), the second displacement vector b(3, -5) is added to the acceleration increment vector c(-2, -2) to obtain the third displacement vector e(1, -7).

[0163] In some embodiments, the operation of controlling the cursor movement according to the second displacement by the computer device may include the following three methods:

[0164] The first method: the computer device predicts a first duration; divides the first duration by m to obtain a second duration, where m is the sum of a preset number of interpolated frames and 1; and controls the movement of the cursor according to a second displacement at every second duration.

[0165] The first duration is the predicted interval duration of controlling the movement of the cursor according to the image captured by the camera. For example, it can be the interval duration of the processing module 102 reporting the cursor displacement to the cursor module 103 according to each frame of image captured by the acquisition module 101.

[0166] It should be noted that in the first method, the computer device can predict the time of reporting the cursor displacement next time, so as to perform the frame insertion operation according to the predicted time.

[0167] The second duration is the interval duration of each reporting of the second displacement.

[0168] For example, the following combination Figure 6 The first method mentioned above is described as an example.

[0169] Figure 6 This is a flowchart of a frame insertion process provided by an embodiment of the present application, see Figure 6 , the process includes steps 601 to 603.

[0170] Step 601: the processing module 102 reports the cursor displacement corresponding to the first operating hand displacement to the cursor module 103 to control the movement of the cursor.

[0171] Step 602: The processing module 102 predicts a first duration, and determines a second duration according to the first duration.

[0172] Step 603: The processing module 102 reports the second displacement to the cursor module 103 every second time period to control the movement of the cursor.

[0173] The second method: the computer device divides the preset time length by m to obtain a third time length; and controls the movement of the cursor according to the second displacement every third time length.

[0174] The preset duration can be preset. The preset duration can be the interval duration of the computer device controlling the movement of the cursor according to the image collected by the camera, estimated by the technician based on experience.

[0175] It should be noted that in the second method, the time for the computer device to report the cursor displacement each time is estimated to be a fixed time. In this case, the frame insertion operation can be performed according to the fixed time.

[0176] For example, the following combination Figure 7 The second method mentioned above is described as an example.

[0177] Figure 7 This is a flowchart of a frame insertion process provided by an embodiment of the present application, see Figure 7 , the process includes step 701 to step 702.

[0178] Step 701: the processing module 102 reports the cursor displacement corresponding to the first operating hand displacement to the cursor module 103 to control the movement of the cursor.

[0179] Step 702: The processing module 102 reports the second displacement to the cursor module 103 every third time period to control the movement of the cursor.

[0180] A third method: the computer device controls the movement of the cursor according to the second displacement at fourth time intervals.

[0181] The fourth duration can be preset and can be set by a technician according to the requirements of frame insertion.

[0182] For example, the computer device may perform timing, and when the timing duration is equal to the fourth duration and no new cursor displacement is determined based on the image captured by the camera, control the cursor to move according to the second displacement.

[0183] For example, the following combination Figure 8 The third method mentioned above is described as an example.

[0184] Figure 8 This is a flowchart of a frame insertion process provided by an embodiment of the present application, see Figure 8 , the process includes steps 801 to 803.

[0185] Step 801: the processing module 102 reports the cursor displacement corresponding to the first operating hand displacement to the cursor module 103 to control the movement of the cursor.

[0186] Step 802: The processing module 102 determines whether a cursor displacement corresponding to a new operator displacement is determined.

[0187] If the processing module 102 determines the cursor displacement corresponding to the new operator displacement, step 801 is executed; if the processing module 102 does not determine the cursor displacement corresponding to the new operator displacement, step 803 is executed.

[0188] Step 803: The processing module 102 reports the second displacement to the cursor module 103 every fourth time period to control the movement of the cursor.

[0189] For ease of understanding, the following Fig. 9 The construction and operation process of the above cursor control system is exemplified.

[0190] Fig. 9 is a flowchart of the construction and operation process of a cursor control system provided in an embodiment of the present application, see Fig. 9 , the process includes steps 901 to 906.

[0191] Step 901: The computer device sets a first corresponding relationship.

[0192] The first corresponding relationship is the corresponding relationship between the hand gesture type and the cursor operation event.

[0193] Step 902: The computer device registers a virtual cursor node with the operating system.

[0194] Step 903: After acquiring the target image, the computer device inputs the target image into the gesture recognition model to obtain a plurality of hand feature points.

[0195] Step 904: The computer device determines a gesture type according to the plurality of hand feature points and a gesture recognition algorithm.

[0196] Step 905: The computer device determines whether the gesture type is the gesture type set in the first corresponding relationship.

[0197] If the gesture type is not the gesture type set in the first corresponding relationship, step 903 is executed to identify the gesture type in the next frame image; if the gesture type is the gesture type set in the first corresponding relationship, step 906 is executed.

[0198] Step 906: The computer device reports the cursor displacement to the virtual cursor node.

[0199] Combine the following Fig.10 The process of the computer device reporting the cursor displacement to the virtual cursor node is exemplified.

[0200] Fig.10 is a flowchart of a cursor control method provided in an embodiment of the present application, see Fig.10 , the process includes steps 1001 to 1012.

[0201] Step 1001: The computer device obtains the position coordinates of the operator in the target image.

[0202] Step 1002: The computer device determines whether the number of position coordinates in the historical detection coordinate library is n.

[0203] If the number of position coordinates in the historical detection coordinate library is not n, step 1003 is executed; if the number of position coordinates in the historical detection coordinate library is n, step 1007 is executed.

[0204] Step 1003: The computer device determines whether the number of position coordinates in the historical detection coordinate library is 0.

[0205] If the number of position coordinates in the historical detection coordinate library is 0, step 1004 is executed; if the number of position coordinates in the historical detection coordinate library is not 0, step 1005 is executed.

[0206] Step 1004 : the computer device adds the position coordinates of the operator in the target image to the historical detection coordinate library, determines the initial position coordinates of the cursor in the screen according to the position coordinates of the operator in the target image, and executes step 1006 .

[0207] Step 1005 : The computer device adds the position coordinates of the operator in the target image to the historical detection coordinate library, and executes step 1006 .

[0208] Step 1006: The computer device determines the average value of all position coordinates in the historical detection coordinate library as the target position coordinate, takes the target position coordinate as the historical position coordinate, determines the displacement of the first operating hand to be 0, and executes step 1011.

[0209] Step 1007: The computer device deletes the position coordinates with the earliest addition time in the historical detection coordinate library, and adds the position coordinates of the operator in the target image to the historical detection coordinate library.

[0210] Step 1008: The computer device determines the average value of all position coordinates in the historical detection coordinate library as the target position coordinates.

[0211] Step 1009: The computer device determines the displacement of the first operator according to the target position coordinates and the historical position coordinates.

[0212] Step 1010: The computer device uses the target location coordinates as historical location coordinates.

[0213] Step 1011: the computer device adjusts the displacement of the first operator according to the size of the operator in the target image to obtain the displacement of the second operator.

[0214] Step 1012: The computer device determines a cursor displacement according to a second operator displacement.

[0215] The computer device may report the cursor displacement to the virtual cursor node to control the cursor movement.

[0216] In an embodiment of the present application, the computer device obtains the target image captured by the camera, obtains the position coordinates of the operator in the target image, and then determines the target position coordinates according to the position coordinates of the operator in the target image, the target position coordinates being the average position coordinates of the operator in the target image and the continuous n-1 frames of images captured by the camera before the target image. Afterwards, the first operator displacement is determined according to the target position coordinates and the historical position coordinates, the historical position coordinates being the average position coordinates of the operator in the continuous n frames of images captured by the camera before the target image. Finally, the cursor movement in the screen is controlled according to the first operator displacement. Since the target position coordinates and the historical position coordinates can reflect the movement trend of the operator in the most recent n frames of images, and the target position coordinates and the historical position coordinates are both the average position coordinates of the operator in the continuous n frames of images, when the position coordinates of the operator fluctuate greatly, the embodiment of the present application can distribute the fluctuation of the position coordinates of the operator to the n frames of images. In this way, the fluctuation of the position coordinates of the operator in the overall image can be reduced, thereby reducing the cursor jitter and improving the user operation experience.

[0217] Fig.11 1 is a schematic diagram of a cursor control device provided in an embodiment of the present application. The device can be implemented as part or all of a computer device by software, hardware, or a combination of both. The computer device can be as follows Fig.12 Computer equipment shown. Fig.11 The device includes: a first acquisition module 1101, a second acquisition module 1102, a first determination module 1103, a second determination module 1104, and a first control module 1105.

[0218] The first acquisition module 1101 is used to acquire a target image captured by a camera;

[0219] The second acquisition module 1102 is used to acquire the position coordinates of the operator in the target image;

[0220] The first determination module 1103 is used to determine the target position coordinates according to the position coordinates of the operator in the target image, where the target position coordinates are the average position coordinates of the operator in the target image and the continuous n-1 frames of images collected by the camera before the target image, where n is an integer greater than or equal to 2;

[0221] A second determination module 1104 is used to determine the displacement of the first operator according to the target position coordinates and the historical position coordinates, where the historical position coordinates are the average position coordinates of the operator in n consecutive frames of images collected by the camera before the target image;

[0222] The first control module 1105 is used to control the movement of the cursor on the screen according to the displacement of the first operating hand.

[0223] Optionally, the device further comprises:

[0224] A third determination module is used to determine whether the number of position coordinates in the historical detection coordinate library is n;

[0225] Optionally, the first determining module 1103 is used to:

[0226] If the number of position coordinates in the historical detection coordinate library is n, the position coordinates added earliest in the historical detection coordinate library are deleted, and the position coordinates of the operator in the target image are added to the historical detection coordinate library;

[0227] The average value of all position coordinates in the historical detection coordinate library is determined as the target position coordinate.

[0228] Optionally, the device further comprises:

[0229] A first adding module, configured to add the position coordinates of the operator in the target image to the historical detection coordinate library if the number of position coordinates in the historical detection coordinate library is 0;

[0230] A fourth determination module, used to determine the initial position coordinates of the cursor in the screen according to the position coordinates of the operator in the target image;

[0231] Optionally, the device further comprises:

[0232] A second adding module is used to add the position coordinates of the operator in the target image to the historical detection coordinate library if the number of position coordinates in the historical detection coordinate library is greater than 0 and less than n;

[0233] The fifth determining module is used to determine that the displacement of the first operating hand is 0.

[0234] Optionally, the first control module 1105 is used to:

[0235] Determine the size of the manipulator's hand in the target image;

[0236] The displacement of the first manipulator is adjusted according to the size of the manipulator in the target image to obtain the displacement of the second manipulator;

[0237] The cursor movement is controlled according to the displacement of the second operating hand.

[0238] Optionally, the size of the operator in the target image includes a width and a height of a detection frame of the operator in the target image, and the first operator displacement includes a lateral displacement and a longitudinal displacement;

[0239] Optionally, the first control module 1105 is used to:

[0240] The lateral displacement in the first operator displacement is divided by the width of the detection frame and then multiplied by a preset coefficient to obtain the lateral displacement in the second operator displacement;

[0241] The longitudinal displacement in the first operator displacement is divided by the height of the detection frame and then multiplied by a preset coefficient to obtain the longitudinal displacement in the second operator displacement.

[0242] Optionally, the device further comprises:

[0243] A first updating module, configured to update the lateral displacement amount in the second operator displacement amount to the lateral displacement threshold if the lateral displacement amount in the second operator displacement amount is greater than the lateral displacement threshold;

[0244] The second updating module is used to update the longitudinal displacement amount in the displacement amount of the second manipulator to the longitudinal displacement threshold value.

[0245] Optionally, the device further comprises:

[0246] A prediction module, used for predicting a first displacement amount according to three cursor position coordinates, wherein the three cursor position coordinates are the position coordinates to which the cursor is moved in the screen according to the three frames of images most recently acquired by the camera, and the first displacement amount is the predicted displacement amount of the cursor in the screen controlled according to the next frame of image acquired by the camera after the target image;

[0247] An obtaining module, configured to obtain a second displacement by dividing the first displacement by a preset number of interpolation frames;

[0248] The second control module is used to control the movement of the cursor according to the second displacement.

[0249] Optionally, the three cursor position coordinates are sorted in order of movement time from early to late, and the prediction module is used to:

[0250] Determine a first displacement vector and a second displacement vector according to the three cursor position coordinates, the first displacement vector being a displacement vector from the first cursor position coordinate to the second cursor position coordinate among the three cursor position coordinates, and the second displacement vector being a displacement vector from the second cursor position coordinate to the third cursor position coordinate among the three cursor position coordinates;

[0251] Subtract the first displacement vector from the second displacement vector to obtain an acceleration increment vector;

[0252] The second displacement vector is added to the acceleration increment vector to obtain a third displacement vector, and the elements of the third displacement vector are the first displacement amounts.

[0253] Optionally, the second control module is used for:

[0254] Predicting a first duration, where the first duration is a predicted interval duration for controlling the movement of the cursor based on the image captured by the camera;

[0255] Divide the first duration by m to obtain a second duration, where m is the sum of the preset number of interpolated frames and 1;

[0256] The cursor movement is controlled according to the second displacement at every second time period.

[0257] In the embodiment of the present application, since the target position coordinates and the historical position coordinates can reflect the movement trend of the operator in the most recent n frames of images, and the target position coordinates and the historical position coordinates are the average position coordinates of the operator in the continuous n frames of images, when the position coordinates of the operator fluctuate greatly, the embodiment of the present application can distribute the fluctuation of the position coordinates of the operator to the n frames of images. In this way, the fluctuation of the position coordinates of the operator in the overall image can be reduced, thereby reducing the cursor jitter and improving the user operation experience.

[0258] It should be noted that: the cursor control device provided in the above embodiment only uses the division of the above-mentioned functional modules as an example to illustrate when controlling the cursor. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0259] The functional units and modules in the above embodiments may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit, and the above integrated units may be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present application.

[0260] The cursor control device and the cursor control method provided in the above embodiments belong to the same concept. The specific working process and technical effects brought about by the units and modules in the above embodiments can be found in the method embodiment part and will not be repeated here.

[0261] Fig.12 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. Fig.12 As shown, the computer device 12 includes: a processor 120, a memory 121, and a computer program 122 stored in the memory 121 and executable on the processor 120. When the processor 120 executes the computer program 122, the steps of the cursor control method in the above embodiment are implemented.

[0262] The computer device 12 may be a general-purpose computer device or a dedicated computer device. In a specific implementation, the computer device 12 may be a desktop computer, a portable computer, a network server, a PDA, a mobile phone, a tablet computer, a wireless terminal device, a communication device or an embedded device. The embodiment of the present application does not limit the type of the computer device 12. Those skilled in the art will appreciate that Fig.12 It is only an example of the computer device 12 and does not constitute a limitation on the computer device 12. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components, such as input and output devices, network access devices, etc.

[0263] The processor 120 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0264] In some embodiments, the memory 121 may be an internal storage unit of the computer device 12, such as a hard disk or memory of the computer device 12. In other embodiments, the memory 121 may also be an external storage device of the computer device 12, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device 12. Further, the memory 121 may also include both an internal storage unit of the computer device 12 and an external storage device. The memory 121 is used to store an operating system, an application program, a boot loader, data, and other programs. The memory 121 may also be used to temporarily store data that has been output or is to be output.

[0265] An embodiment of the present application also provides a computer device, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, and when the processor executes the computer program, the steps in any of the above-mentioned method embodiments are implemented.

[0266] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0267] An embodiment of the present application provides a computer program product, which, when executed on a computer, enables the computer to execute the steps in the above-mentioned method embodiments.

[0268] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above method embodiments, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above method embodiments can be implemented. Among them, the computer program includes computer program code, which can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the camera / terminal device, recording medium, computer memory, ROM (Read-Only Memory), RAM (Random Access Memory), CD-ROM (Compact Disc Read-Only Memory), magnetic tape, floppy disk and optical data storage device. The computer-readable storage medium mentioned in the present application can be a non-volatile storage medium, in other words, it can be a non-transient storage medium.

[0269] It should be understood that all or part of the steps to implement the above embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. The computer instructions can be stored in the above-mentioned computer readable storage medium.

[0270] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0271] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0272] In the embodiments provided in the present application, it should be understood that the disclosed devices / computer equipment and methods can be implemented in other ways. For example, the device / computer equipment embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0273] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0274] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with the relevant regulations and standards of relevant countries and regions, and corresponding operation entrances are provided for users to choose to authorize or refuse.

[0275] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A cursor control method, characterized in that: The method comprises: Get the target image captured by the camera; Acquire the position coordinates of the operator in the target image; Determine the target position coordinates according to the position coordinates of the operator in the target image, wherein the target position coordinates are the average position coordinates of the operator in the target image and the continuous n-1 frames of images collected by the camera before the target image, where n is an integer greater than or equal to 2; Determine the displacement of the first operator according to the target position coordinates and the historical position coordinates, wherein the historical position coordinates are the average position coordinates of the operator in n consecutive frames of images captured by the camera before the target image; The cursor movement on the screen is controlled according to the displacement of the first operating hand.

2. The method according to claim 1, characterized in that Before determining the target position coordinates according to the position coordinates of the operator in the target image, the method further includes: Determine whether the number of position coordinates in the historical detection coordinate library is n; The step of determining the target position coordinates according to the position coordinates of the operator in the target image includes: If the number of position coordinates in the historical detection coordinate library is n, then deleting the position coordinates in the historical detection coordinate library with the earliest addition time, and adding the position coordinates of the operator in the target image to the historical detection coordinate library; The average value of all position coordinates in the historical detection coordinate library is determined as the target position coordinate.

3. The method according to claim 2, characterized in that After determining whether the number of position coordinates in the historical detection coordinate library is n, the method further includes: If the number of position coordinates in the historical detection coordinate library is 0, the position coordinates of the operator in the target image are added to the historical detection coordinate library; the initial position coordinates of the cursor in the screen are determined according to the position coordinates of the operator in the target image; If the number of position coordinates in the historical detection coordinate library is greater than 0 and less than n, the position coordinates of the operator in the target image are added to the historical detection coordinate library; and the displacement of the first operator is determined to be 0.

4. The method according to claim 1, characterized in that The controlling the movement of the cursor on the screen according to the displacement of the first operator comprises: Determining the size of the operator's hand in the target image; Adjusting the displacement of the first manipulator according to the size of the manipulator in the target image to obtain the displacement of the second manipulator; The cursor movement is controlled according to the displacement of the second operating hand.

5. The method according to claim 4, characterized in that The size of the operator in the target image includes the width and height of the detection frame of the operator in the target image, and the displacement of the first operator includes the lateral displacement and the longitudinal displacement; The adjusting the displacement of the first operator according to the size of the operator in the target image to obtain the displacement of the second operator includes: The lateral displacement in the first operator displacement is divided by the width of the detection frame and then multiplied by a preset coefficient to obtain the lateral displacement in the second operator displacement; The longitudinal displacement in the first operator displacement is divided by the height of the detection frame and then multiplied by the preset coefficient to obtain the longitudinal displacement in the second operator displacement.

6. The method according to claim 4, characterized in that Before controlling the movement of the cursor according to the displacement of the second operating hand, the method further includes: If the lateral displacement in the second operator displacement is greater than the lateral displacement threshold, updating the lateral displacement in the second operator displacement to the lateral displacement threshold; If the longitudinal displacement in the second operator displacement is greater than the longitudinal displacement threshold, the longitudinal displacement in the second operator displacement is updated to the longitudinal displacement threshold.

7. The method according to any one of claims 1 to 6, characterized in that: After controlling the movement of the cursor on the screen according to the displacement of the first operator, the method further includes: Predicting a first displacement amount according to three cursor position coordinates, wherein the three cursor position coordinates are the position coordinates to which the cursor is moved on the screen according to the three frames of images most recently captured by the camera, and the first displacement amount is the predicted displacement amount of the cursor on the screen controlled according to the next frame of image captured by the camera after the target image; Dividing the first displacement by a preset number of interpolation frames to obtain a second displacement; The cursor movement is controlled according to the second displacement.

8. The method according to claim 7, characterized in that The three cursor position coordinates are sorted in order of movement time from early to late, and the first displacement is predicted according to the three cursor position coordinates, including: Determine a first displacement vector and a second displacement vector according to the three cursor position coordinates, wherein the first displacement vector is a displacement vector from the first cursor position coordinate to the second cursor position coordinate among the three cursor position coordinates, and the second displacement vector is a displacement vector from the second cursor position coordinate to the third cursor position coordinate among the three cursor position coordinates; Subtracting the first displacement vector from the second displacement vector to obtain an acceleration increment vector; The second displacement vector is added to the acceleration increment vector to obtain a third displacement vector, and the elements of the third displacement vector are the first displacement amounts.

9. The method according to claim 7, characterized in that The controlling the movement of the cursor according to the second displacement comprises: Predicting a first duration, where the first duration is a predicted interval duration for controlling the movement of the cursor according to the image captured by the camera; Divide the first duration by m to obtain a second duration, where m is the sum of the preset number of interpolated frames and 1; The cursor movement is controlled according to the second displacement at every second time period.

10. A cursor control device, characterized in that: The device comprises: A first acquisition module is used to acquire a target image captured by a camera; A second acquisition module is used to acquire the position coordinates of the operator in the target image; A first determination module is used to determine the target position coordinates according to the position coordinates of the operator in the target image, wherein the target position coordinates are the average position coordinates of the operator in the target image and the continuous n-1 frames of images collected by the camera before the target image, where n is an integer greater than or equal to 2; A second determination module is used to determine the displacement of the first operator according to the target position coordinates and the historical position coordinates, wherein the historical position coordinates are the average position coordinates of the operator in the continuous n frames of images collected by the camera before the target image; A control module is used to control the movement of a cursor on the screen according to the displacement of the first operating hand.

11. A computer device, characterized in that: The computer device comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the method according to any one of claims 1 to 9 when executed by the processor.

12. A computer program product, characterized in that When the computer program product is executed on a computer device, the computer device is caused to execute the method according to any one of claims 1 to 9.

Images