A target control method and system based on motion recognition

Abstract

The present invention discloses a target control method and system based on motion recognition, the method comprising: obtaining a first movement of a finger relative to the wrist based on an optical sensor worn on the lower part of the wrist; obtaining a second movement of the wrist relative to a target position on the user's body based on a collection device worn on the wrist; superimposing the first movement of the finger relative to the wrist and the second movement of the wrist relative to the target position on the user's body, and controlling the target according to the superimposed movement result. The present invention controls the target after superimposing the movement of the finger relative to the wrist and the movement of the wrist relative to the target position on the user's body, because the user's hand has a larger range of motion, so that a larger amplitude and more sophisticated and complex control can be achieved, avoiding excessive amplification of finger movements to cause overly obvious shaking, and improving the operating experience.

Description

Technical Field

[0001] The present invention relates to the technical field of human-computer interaction, and in particular to a target control method and system based on motion recognition. Background Art

[0002] It is a development trend to realize the output of commands by recognizing gestures, so as to achieve human-computer interaction, which can bring convenience to users to operate related equipment.

[0003] Existing technology can be used to control the target device by acquiring the movement of the fingers relative to the wrist. However, the movement range of the fingers relative to the wrist is relatively limited. For example, the limit range of the index finger's left and right movement relative to the wrist is usually about 20 centimeters (the comfortable and flexible movement range is usually no more than 10 centimeters), and the limit range of forward and backward movement is usually about 15 centimeters (the comfortable and flexible movement range is usually no more than 7 centimeters). If you want to perform large-scale and precise target manipulation, such as controlling a cursor on a larger screen with a width of 2 meters, then the amplitude of the finger's left and right movement must be magnified by at least 10 times (if the comfortable movement range is considered, it needs to be magnified by 20 times) to fully cover the entire target screen. This will cause two problems:

[0004] (1) Finger movement is overly magnified, which makes it difficult for the finger to perform delicate and complex operations on the target screen. When icons are arranged closely, it is easy to cause accidental touches.

[0005] (2) When optically identifying finger movement, the key points of the identified finger will fluctuate within a small range. When the finger movement is over-magnified, small fluctuations will also cause the cursor representing the finger on the target screen to shake violently, which will greatly affect the accuracy and user experience.

[0006] The above situation limits the size of the target screen to be controlled. That is, when the target screen size is too large, this solution is not applicable to control it.

[0007] In the prior art, a positioning device can be worn on the wrist (or a handheld positioning device) to measure the spatial movement of the wrist. The positioning device can be an inertial positioning device (such as an accelerometer, a gyroscope), a magnetometer, an optical sensor, etc. However, this method of using the movement of the wrist alone to control the elements on the screen of the target device has low operation efficiency and poor flexibility because the movement of the wrist is far less flexible than that of the fingers.

[0008] Therefore, the prior art still needs to be improved and developed. Summary of the invention

[0009] The main purpose of the present invention is to provide a target control method and system based on motion recognition, aiming to solve the problem in the prior art that it is inconvenient to perform fine control operations on the target screen.

[0010] To achieve the above object, the present invention provides a target control method based on motion recognition, the target control method based on motion recognition comprising the following steps:

[0011] acquiring a first movement of the finger relative to the wrist based on an optical sensor worn on the lower part of the wrist;

[0012] Based on the acquisition device worn on the wrist, a second movement of the wrist relative to a target position on the user's body is acquired;

[0013] A first movement of the finger relative to the wrist and a second movement of the wrist relative to a target position on the user's body are superimposed, and the target is controlled according to the superimposed movement result.

[0014] Optionally, in the target control method based on motion recognition, the acquisition device includes at least one accessory worn on the human body, and the accessory is used to assist the wearable device body in measuring its own spatial position and / or spatial movement relative to the accessory.

[0015] Optionally, in the target control method based on motion recognition, the accessory is a pendant hung around the neck, a sticker affixed to the body or clothes, a component fixed to clothes, and a component placed in a pocket.

[0016] Optionally, in the target control method based on motion recognition, the second movement of the wrist relative to the target position on the body is directly calculated based on information obtained by a collection device worn on the wrist, without setting any wearable accessories on the human body.

[0017] Optionally, in the target control method based on motion recognition, a calibration action is added so that the acquisition device can accurately identify the target position on the body; the method of the calibration action is: setting part of the body information acquired by the acquisition device as the target position.

[0018] Optionally, the target control method based on motion recognition, wherein the first movement of the finger relative to the wrist and the second movement of the wrist relative to the target position on the user's body are superimposed, and the target is controlled according to the superimposed movement result, specifically includes:

[0019] The first movement M of the finger relative to the wrist and the second movement N of the wrist relative to the target position on the user's body are superimposed to obtain the movement result P:

[0020] Movement result P=first movement M of the finger relative to the wrist+second movement N of the wrist relative to the target position on the user's body.

[0021] Optionally, in the target control method based on motion recognition, superimposing the first motion M of the finger relative to the wrist and the second motion N of the wrist relative to the target position on the user's body to obtain the motion result P specifically includes:

[0022] Decompose the first motion M into Mx, My, and Mz along three mutually perpendicular directions;

[0023] Decompose the second motion N into Nx, Ny, and Nz along three mutually perpendicular directions;

[0024] Decompose the motion result P into Px, Py, and Pz along three mutually perpendicular directions;

[0025] Then the motion result P is expressed as:

[0026] Px=kx1*Mx+kx2*Nx;

[0027] Py=ky1*My+ky2*Ny;

[0028] Pz=kz1*Mz+kz2*Nz;

[0029] Among them, kx1, kx2, ky1, ky2, kz1, kz2 are all coefficients;

[0030] Control the target device according to Px, Py and Pz in the motion result P;

[0031] The decomposition of the first motion M and the second motion N are in the same coordinate system or in different coordinate systems.

[0032] Optionally, in the target control method based on motion recognition, the acquisition device for acquiring the second movement of the wrist relative to the target position of the body and the sensor for acquiring the first movement of the finger relative to the wrist are the same sensor.

[0033] Optionally, in the target control method based on motion recognition, the target position is the chest or chest of the user.

[0034] In addition, to achieve the above-mentioned purpose, the present invention also provides a target control system based on motion recognition, wherein the target control system based on motion recognition comprises: a target, an optical sensor and a collection device;

[0035] An optical sensor worn on the lower part of the wrist is used to obtain a first movement of the finger relative to the wrist;

[0036] The acquisition device worn on the wrist is used to obtain a second movement of the wrist relative to a target position on the user's body;

[0037] A first movement of the finger relative to the wrist and a second movement of the wrist relative to a target position on the user's body are superimposed, and the target is controlled according to the superimposed movement result.

[0038] In the present invention, based on the optical sensor worn on the lower part of the wrist, the first movement of the finger relative to the wrist is obtained; based on the acquisition device worn on the wrist, the second movement of the wrist relative to the target position on the user's body is obtained; the first movement of the finger relative to the wrist and the second movement of the wrist relative to the target position on the user's body are superimposed, and the target is controlled according to the superimposed movement result. The present invention controls the target after superimposing the movement of the finger relative to the wrist and the movement of the wrist relative to the target position on the user's body. Because the user's hand has a larger range of motion, this can achieve a larger amplitude and more sophisticated and complex control, avoid excessive amplification of finger movements to cause overly obvious shaking, and improve the operating experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a flow chart of a preferred embodiment of the target control method based on action recognition of the present invention. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solution and advantages of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0041] The target control method based on action recognition described in the preferred embodiment of the present invention is as follows: Figure 1 As shown, the target control method based on action recognition includes the following steps:

[0042] Step S10: Acquire a first movement of the finger relative to the wrist based on the optical sensor worn on the lower part of the wrist.

[0043] Specifically, an optical sensor (preferably a 3D optical sensor) worn on the user's wrist senses light from the finger; the optical sensor acquires the real-time three-dimensional coordinates of at least one feature point of the finger (preferably the fingertip) based on the light to obtain the first movement of the finger relative to the wrist. The optical sensor is worn on the side of the user's wrist or forearm facing the palm.

[0044] Step S20: Based on the acquisition device worn on the wrist, a second movement of the wrist relative to a target position on the user's body is acquired.

[0045] Specifically, at least one accessory is worn on the user's body, and the accessory is used to assist the wearable device body in measuring its own spatial position and / or spatial movement relative to the accessory.

[0046] Specifically, the accessories can be divided into hanging objects, fixed objects, adhesive objects, and containing objects according to the fixing method. Hanging objects include pendants, fixed objects include pins, adhesive objects include stickers, and containing objects include sensors placed in pockets.

[0047] The accessories can be divided into electromagnetic wave positioning, magnetic positioning, sound wave positioning, etc. according to the working principle. Electromagnetic wave positioning such as beacons, magnetic positioning such as permanent magnets, and sound wave positioning such as ultrasonic transmitters.

[0048] The accessories can be divided into active type, passive type, etc. according to whether they actively transmit signals. Active type includes ultrasonic transmitters, and passive type includes QR code stickers.

[0049] The acquisition device worn on the wrist acquires the second movement of the wrist relative to the target position on the body by acquiring information from the accessory.

[0050] Specifically, the second movement of the wrist relative to the target position on the body can be calculated by using the information obtained by the acquisition device worn on the wrist, without the need for accessories worn on the human body. For example, the acquisition device is a 3D sensor that can measure the three-dimensional contour of the body and identify the target position, thereby calculating the second movement of the wrist relative to the target position on the body.

[0051] Specifically, a calibration action can be added so that the acquisition device can more accurately identify the target position on the body. The method of the calibration action is: set part of the body information collected by the acquisition device as the target position. For example, if the acquisition device is a 3D sensor, the calibration mode is turned on and then the information is collected at the target position on the body, the device will set the information collected at this time as the target position; for another example, the information containing the target position is first collected through the 3D sensor, and then the target position is marked in the collected information.

[0052] The optical sensor and the acquisition device may both be arranged on a smart wearable device, and the smart wearable device is worn on the wrist of the user to acquire the movement of the fingers relative to the wrist and the movement of the wrist relative to the target position on the user's body.

[0053] It should be noted that step S10 (obtaining the first movement of the finger relative to the wrist) and step S20 (obtaining the second movement of the wrist relative to the target position on the user's body) can be performed simultaneously, regardless of order.

[0054] Step S30: superimpose the first movement of the finger relative to the wrist and the second movement of the wrist relative to the target position on the user's body, and control the target according to the superimposed movement result.

[0055] Specifically, the first movement M of the finger relative to the wrist and the second movement N of the wrist relative to the target position on the user's body are superimposed to obtain a movement result P (P is used to control the target device), and the movement result P is:

[0056] Movement result P = first movement M of fingers relative to wrist + second movement N of wrist relative to target position on user's body (ie movement of user's fingers relative to chest = movement of fingers relative to wrist + movement of wrist relative to chest).

[0057] Among them, the three movements of M, N, and P can be decomposed into several sub-movements in a certain coordinate system. The movement here is the concept of displacement in physics, including distance and direction. Take the rectangular coordinate system as an example:

[0058] Decompose the first motion M into Mx, My, and Mz along three mutually perpendicular directions;

[0059] Decompose the second motion N into Nx, Ny, and Nz along three mutually perpendicular directions;

[0060] Decompose the motion result P into Px, Py, and Pz along three mutually perpendicular directions;

[0061] Then the motion result P can be expressed as:

[0062] Px=kx1*Mx+kx2*Nx;

[0063] Py=ky1*My+ky2*Ny;

[0064] Pz=kz1*Mz+kz2*Nz;

[0065] Among them, kx1, kx2, ky1, ky2, kz1, kz2 are all coefficients;

[0066] Two or more of the above six coefficients may be the same, or all of them may be different. These six coefficients are determined by the designer based on experience, and can be replaced with another set of different coefficients when controlling different target devices. They can also be replaced when used by different users. The significance of these coefficients lies in the contribution (amplification or reduction) made to the control target for the two movements M and N.

[0067] The decomposition of the first motion M and the second motion N can be decomposed in the same coordinate system or in different coordinate systems.

[0068] The above example decomposes the motion in a rectangular coordinate system, but the motion can also be decomposed in a polar coordinate system or other coordinate systems.

[0069] Finally, the target is controlled according to Px, Py and Pz in the motion result P.

[0070] Specifically, the acquisition device for acquiring the second movement of the wrist relative to the target position of the body can be the same sensor as the sensor for acquiring the first movement of the finger relative to the wrist. For example, a 3D sensor is used to simultaneously acquire the first movement and the second movement, or the first movement and the second movement are acquired alternately at a certain interval.

[0071] Since the wrist has a larger range of motion relative to the body, for example, the left and right movement range of the wrist relative to the body can easily reach 80 centimeters, and the left and right movement range of the fingers relative to the wrist can reach 1 meter. In this case, the left and right movement only needs to be magnified by 2 times to complete the control of the content on a screen with a width of 2 meters, that is, fine control of the larger screen can be achieved.

[0072] Furthermore, the control target can be used to achieve human-computer interaction between people and other devices, such as realizing device switching operation, device screen switching, sound adjustment, text input, game control, etc. It can also be used as an object for signal or command output to control some functions on the target device, such as device switching operation, opening or closing applications in the device, switching of screens in applications, adjustment of sounds in applications, etc.

[0073] Beneficial effects:

[0074] (1) The magnification is small, so fingers can perform fine and complex manipulation of the content;

[0075] (2) The magnification is small, so the slight shaking of the finger or wrist will not be magnified to an overly obvious level, which improves the user experience;

[0076] (3) Can be used for equipment that requires greater control.

[0077] Furthermore, based on the above-mentioned target control method based on motion recognition, the present invention also provides a target control system based on motion recognition, wherein the target control system based on motion recognition includes: a target, an optical sensor and a collection device; the optical sensor worn on the lower part of the wrist is used to obtain a first movement of the finger relative to the wrist; the collection device worn on the wrist is used to obtain a second movement of the wrist relative to the target position on the user's body; the first movement of the finger relative to the wrist and the second movement of the wrist relative to the target position on the user's body are superimposed, and the target is controlled according to the superimposed movement result.

[0078] In summary, the present invention provides a target control method and system based on motion recognition, the method comprising: obtaining a first movement of a finger relative to a wrist based on an optical sensor worn on the lower part of the wrist; obtaining a second movement of the wrist relative to a target position on the user's body based on a collection device worn on the wrist; superimposing the first movement of the finger relative to the wrist and the second movement of the wrist relative to the target position on the user's body, and controlling the target according to the superimposed movement result. In the prior art, the target is controlled only by detecting the first movement of the finger relative to the wrist. When a large-scale control is required, the movement of the finger needs to be amplified, so that the slight shaking of the finger will be amplified to an excessively obvious degree, affecting the user experience. The present invention controls the target after superimposing the movement of the finger relative to the wrist and the movement of the wrist relative to the target position on the user's body, so that the user's hand has a larger range of movement, can achieve a larger range of control, and the small magnification can perform fine and complex control on the target, and the slight shaking of the finger or wrist will not be amplified to an excessively obvious degree, improving the operating experience.

[0079] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0080] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware (such as a processor, a controller, etc.) through a computer program, and the program can be stored in a computer-readable storage medium that can be read by a computer, and the program can include the processes of the above-mentioned method embodiments when executed. The computer-readable storage medium can be a memory, a disk, an optical disk, etc.

[0081] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A target control method based on motion recognition, characterized in that: The target control method based on action recognition includes: acquiring a first movement of the finger relative to the wrist based on an optical sensor worn on the lower part of the wrist; An optical sensor worn on a wrist of a user senses light from a finger, and the optical sensor acquires a real-time three-dimensional coordinate of at least one feature point of the finger based on the light to obtain a first movement of the finger relative to the wrist; Based on the acquisition device worn on the wrist, a second movement of the wrist relative to a target position on the user's body is acquired; The acquisition device includes at least one accessory worn on the human body, and the accessory is used to assist the wearable device body in measuring its own spatial position and / or spatial movement relative to the accessory; Based on the information obtained by the acquisition device worn on the wrist, the second movement of the wrist relative to the target position on the body is directly calculated without setting any accessories worn on the human body; Add a calibration action so that the acquisition device can accurately identify the target position on the body; the calibration action method is: set part of the body information collected by the acquisition device as the target position; superimposing a first movement of the finger relative to the wrist and a second movement of the wrist relative to a target position on the user's body, and controlling the target according to the superimposed movement result; The step of superimposing a first movement of the finger relative to the wrist and a second movement of the wrist relative to a target position on the user's body, and controlling the target according to the superimposed movement result specifically includes: The first movement M of the finger relative to the wrist and the second movement N of the wrist relative to the target position on the user's body are superimposed to obtain the movement result P: Movement result P = first movement M of the finger relative to the wrist + second movement N of the wrist relative to the target position on the user's body; The superposition of the first movement M of the finger relative to the wrist and the second movement N of the wrist relative to the target position on the user's body to obtain the movement result P specifically includes: Decompose the first motion M into Mx, My, and Mz along three mutually perpendicular directions; Decompose the second motion N into Nx, Ny, and Nz along three mutually perpendicular directions; Decompose the motion result P into Px, Py, and Pz along three mutually perpendicular directions; Then the motion result P is expressed as: Px=kx1*Mx+kx2*Nx; Py=ky1*My+ky2*Ny; Pz=kz1*Mz+kz2*Nz; Among them, kx1, kx2, ky1, ky2, kz1, kz2 are all coefficients; Control the target device according to Px, Py and Pz in the motion result P; The decomposition of the first motion M and the second motion N are in the same coordinate system or in different coordinate systems.

2. The target control method based on motion recognition according to claim 1, characterized in that: The accessories are pendants hung around the neck, stickers pasted on the body or clothes, parts fixed on clothes, and parts placed in pockets.

3. The target control method based on motion recognition according to claim 1, characterized in that: The acquisition device for acquiring the second movement of the wrist relative to the target position of the body and the sensor for acquiring the first movement of the finger relative to the wrist are the same sensor.

4. The target control method based on motion recognition according to any one of claims 1 to 3, characterized in that: The target position is the chest or the chest of the user.

5. A target control system based on motion recognition, characterized in that: The target control system based on motion recognition includes: a target, an optical sensor and a collection device; An optical sensor worn on the lower part of the wrist is used to obtain a first movement of the finger relative to the wrist; An optical sensor worn on a wrist of a user senses light from a finger, and the optical sensor acquires a real-time three-dimensional coordinate of at least one feature point of the finger based on the light to obtain a first movement of the finger relative to the wrist; The acquisition device worn on the wrist is used to obtain a second movement of the wrist relative to a target position on the user's body; The acquisition device includes at least one accessory worn on the human body, and the accessory is used to assist the wearable device body in measuring its own spatial position and / or spatial movement relative to the accessory; Based on the information obtained by the acquisition device worn on the wrist, the second movement of the wrist relative to the target position on the body is directly calculated without setting any accessories worn on the human body; Add a calibration action so that the acquisition device can accurately identify the target position on the body; the calibration action method is: set part of the body information collected by the acquisition device as the target position; superimposing a first movement of the finger relative to the wrist and a second movement of the wrist relative to a target position on the user's body, and controlling the target according to the superimposed movement result; The step of superimposing a first movement of the finger relative to the wrist and a second movement of the wrist relative to a target position on the user's body, and controlling the target according to the superimposed movement result specifically includes: The first movement M of the finger relative to the wrist and the second movement N of the wrist relative to the target position on the user's body are superimposed to obtain the movement result P: Movement result P = first movement M of the finger relative to the wrist + second movement N of the wrist relative to the target position on the user's body; The superposition of the first movement M of the finger relative to the wrist and the second movement N of the wrist relative to the target position on the user's body to obtain the movement result P specifically includes: Decompose the first motion M into Mx, My, and Mz along three mutually perpendicular directions; Decompose the second motion N into Nx, Ny, and Nz along three mutually perpendicular directions; Decompose the motion result P into Px, Py, and Pz along three mutually perpendicular directions; Then the motion result P is expressed as: Px=kx1*Mx+kx2*Nx; Py=ky1*My+ky2*Ny; Pz=kz1*Mz+kz2*Nz; Among them, kx1, kx2, ky1, ky2, kz1, kz2 are all coefficients; Control the target device according to Px, Py and Pz in the motion result P; The decomposition of the first motion M and the second motion N are in the same coordinate system or in different coordinate systems.

Images