Finger pointing coordinate correction method and device

Abstract

The invention discloses a finger pointing coordinate correction method and device. The finger pointing coordinate correction method comprises the steps that posture coordinates of a head under a current visual angle and spherical coordinates Q (r, theta, phi) of current finger pointing under a sight line reference coordinate system corresponding to the current visual angle are obtained; obtaining a deviation value of theta and phi in a spherical coordinate Q (r, theta, phi) pointed by the current finger by utilizing a pre-trained deep learning model corresponding to the posture coordinate according to the obtained posture coordinate, so as to obtain an actual coordinate pointed by the current finger; the pre-trained deep learning model is trained on the basis of actual coordinates (theta, phi) of a positioning point selected in a visual angle visible picture shot based on different attitude coordinates in a sight line reference coordinate system and a finger ball coordinate point q (r, theta, phi) obtained by pointing a finger to a corresponding positioning point under the corresponding attitude coordinates; and the output module is used for outputting deviation values delta and zeta between theta and phi in the obtained finger ball coordinate points and the actual coordinates (theta and phi) of the corresponding positioning points.

Description

Technical Field

[0001] The present invention relates to the field of space positioning technology, and more specifically to a method and device for correcting finger pointing coordinates. Background Art

[0002] Currently, in industrial and smart home application scenarios, the operator's current position is usually located by a device with a camera function, and then the operator's finger pointing can be located to know the target object the operator wants to operate, so as to perform the corresponding operation according to the operation instruction. For example, in the field of smart home, in a modeled room, the target member can be identified and located, and the curtains of the correct room can be opened by the finger pointing of the target member and the instruction of "open the curtains". Alternatively, the member can point his finger at a distance and give the instruction of "turn on this light", and the home system can turn on the light pointed by the member.

[0003] However, due to the offset installation position of the device with camera function, human body characteristics (left and right eyes, fingers, etc.), etc., the user's finger pointing may not be recognized by the system as the correct intended position, which is not conducive to the execution of subsequent operations. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a method and device for correcting the coordinates of finger pointing, so as to achieve the positioning correction of finger pointing, thereby correctly identifying the correct intended position of the finger pointing.

[0005] In a first aspect, an embodiment of the present invention provides a method for correcting finger pointing coordinates, which includes: obtaining the posture coordinates of the head at a current viewing angle and the spherical coordinates Q (r, θ, φ) of the current finger pointing in a sight line reference coordinate system corresponding to the current viewing angle; wherein the sight line reference coordinate system is a coordinate system constructed with the center of the two eyes of the human body as the origin, the direction of the sight line center as the x-axis, the direction perpendicular to the line connecting the two eyes as the y-axis, and the direction parallel to the line connecting the two eyes as the z-axis, and r is the distance from the finger to the origin; according to the acquired posture coordinates, using a pre-trained deep learning model corresponding to the posture coordinates to obtain the deviation values ​​of θ and φ in the spherical coordinates Q (r, θ, φ) of the current finger pointing, thereby obtaining the actual coordinates of the current finger pointing; wherein the pre-trained deep learning model is trained based on the actual coordinates (θ, φ) of the positioning point selected in the picture visible from the viewing angle shot with different posture coordinates in the sight line reference coordinate system and the finger spherical coordinate point q (r, θ, φ) obtained when the finger points to the corresponding positioning point at the corresponding posture coordinates, and is used to output the deviation values ​​δ, ζ of θ and φ in the obtained finger spherical coordinate point and the actual coordinates (θ, φ) of the corresponding positioning point.

[0006] In a second aspect, an embodiment of the present invention further provides a finger pointing coordinate correction device, comprising a unit for executing the above method.

[0007] Compared with the prior art, the present invention constructs a sight reference coordinate system based on the current viewing angle of the user, that is, the sight reference coordinate system is constructed with the center of the two eyes of the human body as the origin, the direction of the sight center as the x-axis, the direction perpendicular to the line connecting the two eyes as the y-axis, and the direction parallel to the line connecting the two eyes as the z-axis, to obtain the posture coordinates of the head at the current viewing angle and the spherical coordinates Q (r, θ, φ) of the current finger pointing in the sight reference coordinate system corresponding to the current viewing angle, and obtain the deviation values ​​of θ and φ in the spherical coordinates Q (r, θ, φ) of the current finger pointing according to the acquired posture coordinates using the posture coordinates corresponding to the actual coordinates (θ, φ) of the positioning point in the picture visible based on the shooting perspective and the finger spherical coordinate point q (r, θ, φ) obtained by the finger pointing positioning point, as correction parameters, thereby obtaining the actual coordinates of the current finger pointing, realizing positioning correction of the finger pointing, so as to correctly identify the intended position of the finger pointing, and solving the problem that the user's finger pointing coordinates cannot be correctly identified due to the installation position offset of the device with a camera function or human body characteristics. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 A schematic diagram of a flow chart of a finger pointing coordinate correction method provided by an embodiment of the present invention; Figure 2 A schematic diagram of a sub-process of a finger pointing coordinate correction method provided by an embodiment of the present invention; Figure 3 A schematic block diagram of a device for correcting sorted finger pointing coordinates provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0009] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0010] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0011] It should also be understood that the term "and / or" used in the present description and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0012] See also Figure 1 , Figure 1 FIG. 1 is a flow chart of a method for correcting finger pointing coordinates provided by an embodiment of the present invention. As shown in the figure, the method includes the following steps S110-120.

[0013] S110, obtaining the posture coordinates of the head at the current viewing angle and the spherical coordinates Q (r, θ, φ) of the current finger pointing in the sight reference coordinate system corresponding to the current viewing angle.

[0014] In the present invention, the sight reference coordinate system is a coordinate system constructed with the center of the two eyes of the human body as the origin, the direction of the sight center as the x-axis, the direction perpendicular to the line connecting the two eyes as the y-axis, and the direction parallel to the line connecting the two eyes as the z-axis, and r is the distance from the finger to the origin. It can be understood that the sight reference coordinate system is a dynamic coordinate system established based on the user's current viewing angle, and its origin and coordinate axis direction will change with the movement of the head, that is, with the different posture coordinates of the head. r represents the distance from the finger to the origin, θ is the angle between the z-axis and the projection to the xz plane, φ is the angle between the x-axis and the projection to the xy plane, and θ and φ jointly determine the direction of the finger relative to the sight. Preferably, in this embodiment, r can be defined as the distance from the tip of the middle finger to the origin.

[0015] In this step, the posture coordinates of the head at the current viewing angle and the spherical coordinates corresponding to the current finger pointing at the current viewing angle can be obtained through a camera device (such as a head-mounted camera device or a camera device fixed in the scene, etc.), and the posture coordinates include three Euler angles (pitch angle, yaw angle and roll angle) measured according to the current line of sight direction.

[0016] S120. According to the acquired posture coordinates, a pre-trained deep learning model corresponding to the posture coordinates is used to obtain the deviation values ​​of θ and φ in the spherical coordinates Q(r, θ, φ) pointed to by the current finger, so as to obtain the actual coordinates pointed to by the current finger; wherein the pre-trained deep learning model is trained based on the actual coordinates (θ, φ) of the positioning point selected in the visible picture taken from different perspectives of posture coordinates in the line of sight reference coordinate system and the finger spherical coordinate point q(r, θ, φ) obtained when the finger points to the corresponding positioning point under the corresponding posture coordinates, and is used to output the deviation values ​​δ, ζ of θ and φ in the obtained finger spherical coordinate point and the actual coordinates (θ, φ) of the corresponding positioning point.

[0017] like Figure 2 As shown, this step specifically includes: S121, searching the database for a deep learning model pre-trained with the same posture coordinates as the acquired posture coordinates according to the acquired posture coordinates, if not, executing steps S122-S124; if yes, executing step S125; S122. Select multiple pre-trained deep learning models according to a preset threshold of the posture coordinates, and finally select multiple deep learning models whose distances from the spherical coordinates Q (r, θ, φ) pointed to by the current finger from the multiple pre-trained deep learning models.

[0018] In this step, multiple deep learning models trained based on the perspective of posture coordinates close to the acquired posture coordinates are selected according to the posture coordinate preset threshold, and then according to the θ and φ of the spherical coordinates pointed by the finger, multiple pre-trained deep learning models whose training coordinates are closest to the spherical coordinates are found, that is, multiple deep learning models whose distances (the sum of the squares of θ and φ) between the training coordinates and the spherical coordinates are within the preset distance threshold are found. It can be understood that the posture coordinate preset threshold and the preset distance threshold can be set according to actual needs; S123, inputting the spherical coordinates Q (r, θ, φ) pointed by the current finger into a plurality of finally selected deep learning models to calculate and obtain corresponding deviation values ​​δ and ζ of θ and φ; S124. Comprehensively calculate the deviation values ​​δ and ζ obtained by calculating the multiple deep learning models finally selected according to the distance, and finally obtain the deviation values ​​of θ and φ in the spherical coordinates Q(r, θ, φ).

[0019] Specifically, in this step, weights are added to each of the finally selected deep learning models according to the distance, and the deviation values ​​of θ and φ in the spherical coordinates Q(r, θ, φ) are calculated using a weighted algorithm according to the weights and deviation values ​​δ and ζ of each deep learning model. That is, the weighting coefficient of each finally selected deep learning model is calculated according to the distance. The closer the distance, the larger the weighting coefficient. The weighting coefficient of each deep learning model is then multiplied by the δ and ζ calculated by the corresponding deep learning model, and then added to obtain the angular deviation value of θ and φ in the spherical coordinates Q(r, θ, φ).

[0020] S125. Use the searched pre-trained deep learning model to obtain the deviation values ​​of θ and φ in the spherical coordinates Q(r, θ, φ) pointed to by the current finger.

[0021] In some embodiments, the deep learning model can be trained using the following steps: (1) Obtain a picture visible from the perspective corresponding to the current posture coordinates captured by the camera device, and select multiple positioning points from the picture, and obtain the coordinates (θ, φ) of the multiple positioning points in the current line of sight reference coordinate system, to form an actual coordinate training set and input into the deep learning model to be trained. Specifically, when the camera device is shooting, the user keeps the body below the neck still, turns his head to look at the target to be measured, so as to capture the picture visible from the current perspective, and the selecting multiple positioning points from the picture and obtaining the coordinates (θ, φ) of the multiple positioning points in the current line of sight reference coordinate system includes: selecting multiple positioning points from the picture, and selecting multiple corresponding environmental markers (such as line corners, intersections, etc. in the modeled environmental space) within a preset range of each positioning point, obtaining the coordinates (θ, φ) of the center of the environmental markers corresponding to all the positioning points in the current line of sight reference coordinate system, so as to obtain the coordinates (θ, φ) of the multiple positioning points in the current line of sight reference coordinate system, to form an actual coordinate training set. Preferably, the number of the selected positioning points may be five or more, and the selected multiple positioning points may be evenly distributed around and in the center of the picture; for example, the number of positioning points may be 9, and the 9 positioning points constitute a 3*3 dot matrix. In the sight reference coordinate system, the [θ, φ] values ​​of these 9 points from left to right and from top to bottom in front of the screen may be [3π / 8, π / 8], [π / 2, π / 8], [5π / 8, π / 8], [3π / 8, 0], [π / 2, 0], [5π / 8, 0], [3π / 8, -π / 8], [π / 2, -π / 8], [5π / 8, -π / 8], that is, the 5th point is in the center of the field of vision, and the other points are offset by an angle of π / 8 up, down, left, and right. It can be understood that the coordinates (i.e., spherical coordinates) of the center of the environmental marker in the current sight reference coordinate system can be converted from the world coordinates of the center of the environmental marker in the modeled environmental space; (2) Obtain the finger ball coordinate points q (r, θ, φ) corresponding to the different distances r between the finger tip and the origin of the current sight reference coordinate system when the finger points to each positioning point under the current viewing angle, form a pointing coordinate training set and input it into the deep learning model to be trained. In this step, the user can use the middle finger or other fingers of the dominant hand to point to the environmental marker corresponding to the positioning point, with the arm and fingers straight and the eyes open when pointing, or can use the main habitual eye for positioning (left eye or right eye), and keep them consistent during subsequent calibration; and when obtaining the finger ball coordinate points q (r, θ, φ) corresponding to the different distances r between the finger tip and the origin under the current viewing angle, the user slowly bends his arm while pointing to the environmental marker corresponding to the positioning point, gradually moves the finger closer to the head, and stops when it is less than 15 cm away from the head. During the process, keep the finger unbendable and always point to the environmental marker, so that the camera device can obtain the coordinates θ and φ when the finger tip is at different distances from the origin. It can be understood that in the above process, the user should not move other body parts (including the head) except the arm and eyes, and should not move the head when the eyes look at the environmental marker; (3) The deep learning model calculates the deviation values ​​δ and ζ of the coordinate points θ and φ of each finger ball at different distances r and the coordinates (θ, φ) of the corresponding positioning point in the current line of sight reference coordinate system, and fits the functional relationship between r and δ and ζ in each positioning point according to r and the calculated δ and ζ. In the present invention, the functional relationship between r and δ and ζ in each positioning point can be fitted using a polynomial regression algorithm according to r and the calculated δ and ζ in each positioning point. Specifically, in this embodiment, the formula can be used according to r and the calculated δ in each positioning point. Fit the functional relationship between r and δ at the positioning point, where , a is the regression coefficient; according to r and ζ calculated at each positioning point, the formula is used Fit the functional relationship between r and ζ at the positioning point, where , b is the regression coefficient. That is, the training sample set can be composed of r pointing to the environmental markers corresponding to multiple positioning points and the obtained δ and ζ, and the functional relationship between r and δ and ζ in the positioning points can be fitted by using the polynomial regression algorithm. Preferably, the least squares method can also be used to solve and obtain the regression coefficients a and b. It can be understood that any deep learning model that can learn the mapping relationship between input data and angle deviation can be used as the initial model of the present invention, that is, the deep learning model to be trained; (4) Re-input all the finger ball coordinate points q (r, θ, φ) of the multiple positioning points and the obtained δ, ζ into the above-mentioned trained deep learning model for training. The above steps can obtain the correction parameters δ, ζ when the finger points to the environmental markers corresponding to all the positioning points. In this training step, all the finger ball coordinate points q (r, θ, φ) and the obtained δ, ζ are re-input into the above-mentioned trained deep learning model for training, and a deep learning model for calculating the θ, φ deviation δ, ζ of the actual pointing coordinates of the finger under any r, θ, φ can be obtained.

[0022] Furthermore, in some embodiments, all posture coordinates and the corresponding spherical coordinates Q(r, θ, φ) of all current finger pointings and the corresponding deviation values ​​are input into a pre-trained deep learning model for training. Understandably, the spherical coordinates Q(r, θ, φ) of the current finger pointing obtained by the deep learning model corresponding to all posture coordinates and the deviation values ​​of θ and φ in the spherical coordinates Q(r, θ, φ) can be trained together to obtain a unified deep learning model, the input of which considers the influence of finger pointing and head posture at the same time, and directly outputs the angle correction parameters δ and ζ of θ and φ. Thereafter, the newly added correction data will be directly used for the incremental training of this unified model to continuously optimize the model performance.

[0023] In summary, the present invention constructs a sight reference coordinate system based on the user's current viewing angle, obtains the posture coordinates of the head at the current viewing angle and the spherical coordinates Q (r, θ, φ) of the current finger pointing in the sight reference coordinate system corresponding to the current viewing angle, and uses the posture coordinates corresponding to the actual coordinates (θ, φ) of the positioning point in the picture visible based on the shooting perspective in the sight reference coordinate system and the finger spherical coordinate point q (r, θ, φ) obtained by the finger pointing positioning point. The pre-trained deep learning model obtains the deviation values ​​of θ and φ in the spherical coordinates Q (r, θ, φ) of the current finger pointing as correction parameters, thereby obtaining the actual coordinates of the current finger pointing, realizing the positioning correction of the finger pointing, so as to correctly identify the intended position of the finger pointing, solving the problem that the user's finger pointing coordinates cannot be correctly identified due to the offset of the installation position of the device with a camera function or human body characteristics, and can eliminate the deviation influence caused by some equipment and user characteristics (left and right eyes, fingers, etc.), and in the subsequent use process, the user can continue to complete more finger pointing positioning corrections, so that the model is more accurate and more in line with the user's physical characteristics.

[0024] Figure 3 FIG. 3 is a schematic block diagram of a finger pointing coordinate correction device 300 provided in an embodiment of the present invention. Figure 3As shown, corresponding to the finger pointing coordinate correction method of the above embodiment, the present invention also provides a finger pointing coordinate correction device 300. The finger pointing coordinate correction device 300 can be configured in a server. Figure 3 The finger pointing coordinate correction device 300 includes a training information acquisition unit 301, a model training generation unit 302, a correction information acquisition unit 303 and a correction information calculation unit 304.

[0025] The training information acquisition unit 301 is used to obtain the posture coordinates of the head under different viewing angles, and to obtain the pictures visible at the corresponding viewing angles under the different posture coordinates captured by the camera device. It is also used to obtain the finger ball coordinate points q (r, θ, φ) corresponding to the different distances r between the fingertip and the origin of the current line of sight reference coordinate system when the finger points to each positioning point selected in the picture under each viewing angle, to form a pointing coordinate training set; wherein the line of sight reference coordinate system is a coordinate system constructed with the center of the two eyes of the human body as the origin, the direction of the line of sight as the x-axis, the direction perpendicular to the line connecting the two eyes as the y-axis, and the direction parallel to the line connecting the two eyes as the z-axis, and r is the distance from the finger to the origin.

[0026] The model training generation unit 302 is used to select multiple positioning points from each acquired picture, and obtain the coordinates (θ, φ) of the multiple positioning points in the current line of sight reference coordinate system to form an actual coordinate training set. Based on the corresponding actual coordinate training sets and corresponding pointing coordinate training sets under multiple perspectives, multiple deep learning models are trained to output the deviation values ​​δ and ζ of θ and φ of each finger ball coordinate point at different distances r and the coordinates (θ, φ) of the corresponding positioning point in the current line of sight reference coordinate system. The deep learning model can fit the functional relationship between r and δ and ζ in each positioning point according to r and the calculated δ and ζ; and each deep learning model is also trained based on all finger ball coordinate points q (r, θ, φ) of multiple positioning points in its corresponding perspective and the obtained δ and ζ. Preferably, in this embodiment, the step of selecting multiple positioning points from each acquired picture and obtaining the coordinates (θ, φ) of the multiple positioning points in the current sight reference coordinate system to form an actual coordinate training set includes: selecting multiple positioning points from the picture, and selecting multiple corresponding environmental markers within a preset range of each positioning point, obtaining the coordinates (θ, φ) of the center of the environmental markers corresponding to all positioning points in the current sight reference coordinate system, so as to obtain the coordinates (θ, φ) of the multiple positioning points in the current sight reference coordinate system to form an actual coordinate training set. In this embodiment, the functional relationship between r and δ, ζ in each positioning point can be fitted using a polynomial regression algorithm according to r and δ, ζ obtained by calculation in each positioning point.

[0027] The correction information acquisition unit 303 is used to acquire the posture coordinates of the head at the current viewing angle and the spherical coordinates Q (r, θ, φ) of the current finger pointing in the sight reference coordinate system corresponding to the current viewing angle.

[0028] The correction information calculation unit 304 is used to obtain the deviation values ​​of θ and φ in the spherical coordinates Q(r, θ, φ) pointed by the current finger according to the acquired posture coordinates using the deep learning model corresponding to the posture coordinates trained and generated in the model training generation unit 302, as correction parameters, so as to obtain the actual coordinates pointed by the current finger. Specifically, in this embodiment, it is possible to search the database based on the acquired posture coordinates to see whether there is a deep learning model pre-trained with the same posture coordinates as the acquired posture coordinates; if so, the pre-trained deep learning model searched is used to obtain the deviation values ​​of θ and φ in the spherical coordinates Q(r, θ, φ) pointed by the current finger; if not, multiple pre-trained deep learning models are selected based on the preset threshold of the posture coordinates, and finally multiple deep learning models whose distance from the spherical coordinates Q(r, θ, φ) pointed by the current finger are selected from the multiple pre-trained deep learning models based on the spherical coordinates Q(r, θ, φ) pointed by the current finger. The method comprises the following steps: inputting the spherical coordinate Q(r, θ, φ) pointed by the current finger into a plurality of finally selected deep learning models respectively, and calculating the corresponding deviation values ​​δ and ζ of θ and φ; performing comprehensive calculation on the deviation values ​​δ and ζ calculated by the plurality of finally selected deep learning models according to the distance, and finally obtaining the deviation values ​​of θ and φ in the spherical coordinate Q(r, θ, φ), that is, adding weights to each finally selected deep learning model according to the distance, and calculating the angular deviation values ​​of θ and φ in the spherical coordinate Q(r, θ, φ) by using a weighted algorithm according to the weights and deviation values ​​δ and ζ of each deep learning model.

[0029] Furthermore, the model training generation unit 302 can also be used to uniformly train the pre-trained deep learning model based on all posture coordinates obtained by the correction information acquisition unit 303 and the corresponding spherical coordinates Q (r, θ, φ) of all current finger pointings and the deviation values ​​calculated by the correction information calculation unit 304 to obtain a unified model compatible with head posture and finger pointing.

[0030] From the above, it can be seen that the finger pointing coordinate correction device 300 of the present invention can train a deep learning model based on the actual coordinates (θ, φ) of the positioning point in the picture visible from the shooting perspective in the line of sight reference coordinate system and the finger ball coordinate point q (r, θ, φ) obtained by the finger pointing positioning point, and obtain a model for outputting angle deviation values ​​δ and ζ according to any (r, θ, φ), which can be used to correct the coordinates of any finger pointing position, realize positioning correction of the finger pointing, and correctly identify the intended position of the finger pointing.

[0031] It should be noted that those skilled in the art can clearly understand that the specific implementation process of the above-mentioned finger pointing coordinate correction device 300 and each unit can refer to the corresponding description in the aforementioned method embodiment, and for the convenience and brevity of description, it will not be repeated here.

[0032] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of each unit is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0033] The steps in the method of the embodiment of the present invention can be adjusted in order, combined and deleted according to actual needs. The units in the device of the embodiment of the present invention can be combined, divided and deleted according to actual needs. In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0034] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for correcting finger pointing coordinates, characterized in that: The finger pointing coordinate correction method comprises: Obtain the posture coordinates of the head at the current viewing angle and the spherical coordinates Q(r, θ, φ) of the current finger pointing in the sight reference coordinate system corresponding to the current viewing angle; wherein the sight reference coordinate system is a coordinate system constructed with the center of the two eyes of the human body as the origin, the direction of the sight center as the x-axis, the direction perpendicular to the line connecting the two eyes as the y-axis, and the direction parallel to the line connecting the two eyes as the z-axis, and r is the distance from the finger to the origin; According to the acquired posture coordinates, the pre-trained deep learning model corresponding to the posture coordinates is used to obtain the deviation values ​​of θ and φ in the spherical coordinates Q(r, θ, φ) pointed to by the current finger, so as to obtain the actual coordinates pointed to by the current finger; wherein the pre-trained deep learning model is trained based on the actual coordinates (θ, φ) of the positioning point selected in the visible picture from the perspective shot with different posture coordinates in the line of sight reference coordinate system and the finger spherical coordinate point q(r, θ, φ) obtained when the finger points to the corresponding positioning point under the corresponding posture coordinates, and is used to output the deviation values ​​δ, ζ of θ and φ in the obtained finger spherical coordinate point and the actual coordinates (θ, φ) of the corresponding positioning point.

2. The finger pointing coordinate correction method according to claim 1, characterized in that: The method of obtaining the deviation values ​​of θ and φ in the spherical coordinates Q(r, θ, φ) pointed by the current finger according to the acquired posture coordinates by using a pre-trained deep learning model corresponding to the posture coordinates specifically includes: Searching a database according to the acquired posture coordinates to determine whether there is a deep learning model pre-trained using the same posture coordinates as the acquired posture coordinates; If not, multiple pre-trained deep learning models are selected according to a preset threshold of the posture coordinates, and multiple deep learning models whose distances to the spherical coordinates Q(r, θ, φ) pointed to by the current finger are finally selected from the multiple pre-trained deep learning models. Input the spherical coordinates Q(r, θ, φ) pointed by the current finger into multiple finally selected deep learning models to calculate and obtain the corresponding deviation values ​​δ and ζ of θ and φ; The deviation values ​​δ and ζ calculated by the multiple finally selected deep learning models are comprehensively calculated according to the distance, and finally the deviation values ​​of θ and φ in the spherical coordinates Q(r, θ, φ) are obtained.

3. The finger pointing coordinate correction method according to claim 2, characterized in that: The deviation values ​​δ and ζ calculated by the multiple finally selected deep learning models are calculated comprehensively according to the distance, and the deviation values ​​of θ and φ in the spherical coordinates Q(r, θ, φ) are finally obtained, which specifically includes: Weights are added to each finally selected deep learning model according to the distance, and the deviation values ​​of θ and φ in the spherical coordinates Q(r, θ, φ) are calculated using a weighted algorithm based on the weights and deviation values ​​δ and ζ of each deep learning model.

4. The finger pointing coordinate correction method according to claim 2, characterized in that: In the step of searching the database according to the acquired posture coordinates to see whether there is a deep learning model pre-trained with the same posture coordinates as the acquired posture coordinates, the method further includes: If so, the searched pre-trained deep learning model is used to obtain the deviation values ​​of θ and φ in the spherical coordinates Q(r, θ, φ) pointed to by the current finger.

5. The finger pointing coordinate correction method according to claim 1, characterized in that: The deep learning model is trained using the following steps: Obtain the visible pictures corresponding to the different posture coordinates taken by the camera device, select multiple positioning points from the pictures, and obtain the coordinates (θ, φ) of the multiple positioning points in the current line of sight reference coordinate system to form an actual coordinate training set and input it into the deep learning model to be trained; Obtain the finger ball coordinate points q (r, θ, φ) corresponding to different distances r between the fingertip and the origin of the current sight reference coordinate system when the finger points to each positioning point under the current viewing angle, form a pointing coordinate training set and input it into the deep learning model to be trained; The deep learning model calculates the deviation values ​​δ and ζ of the coordinates (θ, φ) of each finger ball coordinate point at different distances r and the corresponding positioning point in the current line of sight reference coordinate system, and fits the functional relationship between r and δ and ζ at each positioning point according to r and the calculated δ and ζ; All the finger ball coordinate points q (r, θ, φ) of the multiple positioning points and the obtained δ and ζ are re-input into the above-trained deep learning model for training.

6. The finger pointing coordinate correction method according to claim 5, characterized in that: The selecting a plurality of positioning points from the picture and obtaining the coordinates (θ, φ) of the plurality of positioning points in the current sight line reference coordinate system specifically includes: Select multiple positioning points from the picture, and select multiple corresponding environmental markers within a preset range of each positioning point, obtain the coordinates (θ, φ) of the centers of the environmental markers corresponding to all positioning points in the current line of sight reference coordinate system, and obtain the coordinates (θ, φ) of the multiple positioning points in the current line of sight reference coordinate system to form an actual coordinate training set.

7. The finger pointing coordinate correction method according to claim 5, characterized in that: The step of re-inputting all the finger ball coordinate points q (r, θ, φ) of the plurality of positioning points and the obtained δ and ζ into the above-mentioned trained deep learning model for training further includes: All posture coordinates and the corresponding spherical coordinates Q(r, θ, φ) of all current finger pointings and deviation values ​​are input into the pre-trained deep learning model for training.

8. The finger pointing coordinate correction method according to claim 5, characterized in that: The method of fitting the functional relationship between r and δ and ζ at each positioning point according to r and the calculated δ and ζ at each positioning point specifically includes: According to r and the calculated δ and ζ at each positioning point, the functional relationship between r and δ and ζ at the positioning point is fitted using a polynomial regression algorithm.

9. The finger pointing coordinate correction method according to claim 8, characterized in that: The method of fitting the functional relationship between r and δ and ζ at each positioning point using a polynomial regression algorithm according to r at each positioning point and the calculated δ and ζ specifically includes: According to the r and δ calculated at each positioning point, the formula Fit the functional relationship between r and δ at the positioning point, where , a is the regression coefficient; According to the r and ζ calculated at each positioning point, the formula is used Fit the functional relationship between r and ζ at the positioning point, where , b is the regression coefficient.

10. A finger pointing coordinate correction device, characterized in that: The finger pointing coordinate correction device comprises a unit for executing the method according to any one of claims 1-9.

Images