Three-dimensional coordinate construction method, device and equipment of hand key point, and storage medium

Abstract

The application relates to the field of hand recognition, in particular to a three-dimensional coordinate construction method and device of hand key points, computer equipment and a storage medium, the method comprising the following steps: acquiring two-dimensional coordinates of hand key points on a two-dimensional image; acquiring an internal parameter matrix and an external parameter matrix of a camera, and calculating a projection matrix according to the internal parameter matrix and the external parameter matrix; the camera is a shooting camera of the two-dimensional image; establishing a projection relationship equation of the two-dimensional coordinates and three-dimensional coordinates of the hand key points according to the projection matrix; eliminating the projection relationship equation to obtain a homogeneous linear equation group of the two-dimensional coordinates and the three-dimensional coordinates; iteratively calculating the homogeneous linear equation group according to a preset inverse iteration algorithm, acquiring an equation solution when an iteration result meets preset requirements, and determining three-dimensional coordinates of the hand key points according to the equation solution. The application can improve the accuracy of conversion of two-dimensional coordinates of hand key points into three-dimensional coordinates.

Description

Technical Field

[0001] This application relates to the field of hand recognition, and in particular to a method, apparatus, computer device, and storage medium for constructing three-dimensional coordinates of key points of the hand. Background Technology

[0002] Currently, most 3D coordinate recognition of gestures uses depth cameras or infrared cameras. Although these can provide depth information, the hardware is expensive. Using 2D images for 3D coordinate recognition is problematic because the number of pixels occupied by key hand points in 2D images is small, the features are not obvious, and they are easily occluded. Especially in scenarios where 2D key point detection is implemented on high-speed mobile devices, it is difficult to avoid a large deviation from the actual position of key points in the 2D image. As a result, the accuracy of current 3D coordinate recognition based on pixel coordinates of 2D images for 3D reconstruction of the hand is low. Summary of the Invention

[0003] The main purpose of this application is to provide a method, apparatus, computer device and storage medium for constructing three-dimensional coordinates of key points of the hand, in order to solve the problem of low accuracy in the reconstruction of three-dimensional coordinates of key points of the hand.

[0004] To achieve the aforementioned objectives, this application proposes a method for constructing the three-dimensional coordinates of key hand points, comprising:

[0005] Obtain the 2D coordinates of key hand points on a 2D image;

[0006] Obtain the intrinsic and extrinsic parameter matrices of the camera, and calculate the projection matrix based on the intrinsic and extrinsic parameter matrices; the camera is the camera that captures the two-dimensional image;

[0007] Based on the projection matrix, establish the projection relationship equation between the two-dimensional coordinates and the three-dimensional coordinates of the key points of the hand;

[0008] Eliminating variables from the projection relationship equations yields a homogeneous system of linear equations for the two-dimensional and three-dimensional coordinates.

[0009] The homogeneous linear equation system is iteratively calculated according to a preset inverse iterative algorithm to obtain the equation solution when the iteration result meets the preset requirements, and the three-dimensional coordinates of the key points of the hand are determined according to the equation solution.

[0010] This application also provides a three-dimensional coordinate construction device for key points of the hand, including:

[0011] The 2D coordinate module is used to obtain the 2D coordinates of key hand points on a 2D image;

[0012] The camera parameter module is used to obtain the intrinsic and extrinsic parameter matrices of the camera and calculate the projection matrix based on the intrinsic and extrinsic parameter matrices; the camera is the camera that captures the two-dimensional image.

[0013] The coordinate projection module is used to establish the projection relationship equation between the two-dimensional coordinates and the three-dimensional coordinates of the key points of the hand based on the projection matrix;

[0014] The alignment elimination module is used to eliminate variables in the projection relationship equation to obtain the homogeneous linear equation system of the two-dimensional coordinates and the three-dimensional coordinates.

[0015] The iterative calculation module is used to perform iterative calculations on the homogeneous linear equation system according to a preset inverse iterative algorithm, obtain the equation solution when the iterative result meets the preset requirements, and determine the three-dimensional coordinates of the key points of the hand based on the equation solution.

[0016] This application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the three-dimensional coordinate construction method for key hand points described above.

[0017] This application also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the three-dimensional coordinate construction method for key hand points described above.

[0018] This application provides a method for converting the two-dimensional coordinates of hand key points in a two-dimensional image into three-dimensional coordinates in three-dimensional space. First, the two-dimensional coordinates of the hand key points in the two-dimensional image are obtained. The intrinsic and extrinsic parameter matrices of the camera are also obtained. A projection matrix is ​​calculated based on these matrices. The camera is the one capturing the two-dimensional image. An equation relating the two-dimensional and three-dimensional coordinates of the hand key points is established based on the projection matrix. Elimination is performed on this equation to obtain a homogeneous linear equation system of the two-dimensional and three-dimensional coordinates. The homogeneous linear equation system is iteratively calculated using a preset inverse iterative algorithm to obtain the solution that satisfies preset requirements. The three-dimensional coordinates of the hand key points are determined based on the solution. By establishing the relationship between the two-dimensional and three-dimensional coordinates through the projection matrix and then performing elimination followed by iterative calculation, the iterative calculation can quickly converge to the estimated value of the three-dimensional coordinates of the hand key points with the smallest error, thereby improving the accuracy of the three-dimensional coordinates of the hand key points. Attached Figure Description

[0019] Figure 1 This is a schematic flowchart of an embodiment of the method for constructing the three-dimensional coordinates of key hand points according to this application;

[0020] Figure 2This is a schematic diagram of an embodiment of the key hand points of this application;

[0021] Figure 3 A schematic diagram of an embodiment of the three-dimensional coordinate construction device for key hand points according to this application;

[0022] Figure 4 This is a schematic block diagram of an embodiment of the computer device of this application.

[0023] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0025] Reference Figure 1 This application provides a method for constructing the three-dimensional coordinates of key points of the hand. The method includes steps S10-S50, and the detailed description of each step of the method is as follows.

[0026] S10. Obtain the two-dimensional coordinates of key hand points on the two-dimensional image.

[0027] This embodiment is applied to the recognition of key hand points. It obtains the two-dimensional coordinates of key hand points in a two-dimensional image, and then converts these coordinates into three-dimensional coordinates using a preset mapping relationship, thus obtaining the coordinates of the key hand points in three-dimensional space. First, the two-dimensional coordinates of the key hand points in the two-dimensional image are obtained. Specifically, several key points on the hand are designated to represent the entire hand, such as... Figure 2 As shown, a normal and complete hand is defined by 21 hand key points. The position coordinates of the entire hand are described based on these 21 key points, where k0 represents the wrist-metacarpal joint, and k1, k5, k9, k... 13 k 17 These represent the metacarpophalangeal joints of the five fingers from the thumb to the little finger: k2, k6, k... 10 k 14 k 18 These represent the proximal interphalangeal joints of the five fingers mentioned above: k3, k7, k... 11 k 15 k 19 These represent the distal interphalangeal joints of the five fingers mentioned above: k4, k8, k... 12 k 16 k 20These represent the five fingertips. The two-dimensional coordinates of each key point on the hand are shown below. Where the subscript i represents the number of the hand keypoint, and the superscript j represents the j-th camera (meaning that for each hand keypoint, different cameras capture different 2D images, and therefore the obtained 2D coordinates are also different). Since the subsequent calculations are performed independently for each 2D coordinate, the 2D coordinates of the hand keypoint are abbreviated as [u i v i ]

[0028] S20. Obtain the intrinsic parameter matrix and extrinsic parameter matrix of the camera, and calculate the projection matrix based on the intrinsic parameter matrix and extrinsic parameter matrix; the camera is the camera that captures the two-dimensional image.

[0029] In this embodiment, after obtaining the two-dimensional coordinates of the key hand points on the two-dimensional image, the intrinsic and extrinsic parameter matrices of the camera are obtained. In one implementation, the camera calibration algorithm is first obtained, and then the intrinsic and extrinsic parameter matrices of the camera are calculated according to the calibration algorithm. Furthermore, the camera is the camera that captures the two-dimensional image, and the number of cameras is two, which can be two monocular grayscale cameras. Then, the projection matrix is ​​calculated according to the intrinsic and extrinsic parameter matrices.

[0030] S30. Establish the projection relationship equation between the two-dimensional coordinates and the three-dimensional coordinates of the key points of the hand based on the projection matrix.

[0031] In this embodiment, after obtaining the two-dimensional coordinates of the hand key points on the two-dimensional image and obtaining the camera's intrinsic and extrinsic parameter matrices, and calculating the projection matrix based on the intrinsic and extrinsic parameter matrices, the projection relationship equation between the two-dimensional and three-dimensional coordinates of the hand key points is established based on the projection matrix. That is, the projection relationship between the two-dimensional and three-dimensional coordinates of the hand key points is written out. In one implementation, the projection relationship is the relationship where light emitted from a point light source passes through the two-dimensional hand key points and is mapped to the three-dimensional coordinate system. For example, if the three-dimensional coordinates of each hand key point to be solved are denoted as X, the projection relationship equation between the two-dimensional and three-dimensional coordinates of the hand key points is:

[0032]

[0033] Where, d i P represents the unknown scaling factor. i This represents the projection matrix; each camera has a fixed projection matrix. P represents the projection matrix corresponding to that point. i The kth row.

[0034] S40. Eliminate variables in the projection relationship equation to obtain the homogeneous linear equation system of the two-dimensional coordinates and the three-dimensional coordinates.

[0035] In this embodiment, after establishing the projection relationship equation between the two-dimensional and three-dimensional coordinates of the key hand points based on the projection matrix, it is necessary to eliminate variables from the projection relationship equation, that is, to expand the projection relationship equation and then transform it into u. i With v i Respectively with X and P i The relationship is determined by eliminating variables in the projection equation to obtain the homogeneous linear equations of the two-dimensional and three-dimensional coordinates. Specifically, one implementation method for eliminating variables in the projection equation is as follows:

[0036]

[0037]

[0038] Stacking the two-dimensional coordinate data of all key hand points according to the above two equations yields a homogeneous linear system of equations in the form AX = 0, where A is the coordinates of u. i v i P i The function.

[0039] S50. Perform iterative calculations on the homogeneous linear equation system according to the preset inverse iterative algorithm, obtain the equation solution when the iteration result meets the preset requirements, and determine the three-dimensional coordinates of the key points of the hand according to the equation solution.

[0040] In this embodiment, after establishing the projection relationship equation between the two-dimensional and three-dimensional coordinates of the key hand points based on the projection matrix, and eliminating variables in the projection relationship equation to obtain a homogeneous linear equation system of the two-dimensional and three-dimensional coordinates, the homogeneous linear equation system is iteratively calculated according to a preset inverse iteration algorithm. That is, the non-zero solution of the homogeneous linear equation system is solved by iterative calculation. Specifically, the number of iterations is first set, then the convergence condition is set, and then the homogeneous linear equation system is iteratively calculated. When the iteration result satisfies the convergence condition, that is, the iteration result satisfies the preset requirements, the equation solution at this time is obtained, and then the three-dimensional coordinates of the key hand points are determined according to the equation solution. Specifically, since the obtained equation solution X is a homogeneous coordinate equation solution, the last term of the homogeneous coordinate equation solution is normalized to 1, thereby obtaining the three-dimensional coordinates of the key hand points.

[0041] This embodiment provides a method for converting the two-dimensional coordinates of hand key points in a two-dimensional image into three-dimensional coordinates in three-dimensional space. First, the two-dimensional coordinates of the hand key points in the two-dimensional image are obtained. The intrinsic and extrinsic parameter matrices of the camera are also obtained. A projection matrix is ​​calculated based on these matrices. The camera is the one capturing the two-dimensional image. An equation relating the two-dimensional and three-dimensional coordinates of the hand key points is established based on the projection matrix. Elimination is performed on this equation to obtain a homogeneous linear equation system of the two-dimensional and three-dimensional coordinates. The homogeneous linear equation system is iteratively calculated using a preset inverse iterative algorithm to obtain the solution that satisfies preset requirements. The three-dimensional coordinates of the hand key points are determined based on the solution. By establishing the relationship between the two-dimensional and three-dimensional coordinates through the projection matrix and then performing elimination followed by iterative calculation, the iterative calculation can quickly converge to the estimated value of the three-dimensional coordinates of the hand key points with the smallest error, thereby improving the accuracy of the three-dimensional coordinates of the hand key points.

[0042] In one embodiment, the iterative calculation of the homogeneous linear equation system according to a preset inverse iterative algorithm includes:

[0043] Let a displacement scalar approaching zero be denoted as σ;

[0044] Set the initial value X of the solution to the equation. (0) And make the initial value X of the solution to the equation (0) Satisfy ||X (0) || = 1;

[0045] Set the number of iterations to start from k=0;

[0046] If the iterative calculation fails to converge, execute:

[0047] Y (k+1) =(A T A+σI) -1 X (k) ;

[0048] X (k+1) =Y (k+1) / ||X (k+1) ||;

[0049] μ (k+1) =(X (k+1) , (A T A+σI)X (k+1) );

[0050] k = k + 1;

[0051] When the iterative calculation converges, the output X is reached. (k) μ (k+1) .

[0052] In this embodiment, when iteratively calculating the homogeneous linear equation system according to the preset inverse iteration algorithm, this embodiment adopts a fast inverse iteration algorithm that can handle noise at hand key points in two-dimensional images. Specifically, firstly, a displacement scalar approaching zero, denoted as σ, is set, and then the initial value X of the equation solution is set. (0) And make the initial value X of the solution to the equation (0) Satisfy ||X (0) If || = 1, then set the iteration count to start from k = 0; if the iterative calculation does not converge, execute {

[0053] Y (k+1) =(A T A+σI) -1 X (k) ;X (k+1) =Y (k+1) / ||X (k+1) ||;

[0054] μ (k+1) =(X (k+1) , (A T A+σI)X (k+1) ); k = k + 1;

[0055] When the iterative calculation converges, the output X is... (k) μ (k+1) Where A is a function of the two-dimensional coordinates and the projection matrix, and μ (k+1) =(X (k+1) , (A T A+σI)X (k+1) The formula corresponding to the symbol (A, B) indicates the calculation of the dot product of vectors, that is, the calculation of the inner product of vectors A and B. This allows for iterative calculation of the homogeneous linear equation system, outputting the converged three-dimensional coordinates.

[0056] In one embodiment, the step of iteratively calculating the homogeneous linear equation system according to a preset inverse iterative algorithm to obtain the equation solution when the iteration result satisfies a preset requirement includes:

[0057] The homogeneous linear equation system is iteratively calculated according to a preset inverse iterative algorithm;

[0058] When the number of iterations in the iterative calculation reaches a preset maximum value, the result of the last iteration calculation is obtained as the solution to the homogeneous linear equation system.

[0059] In this embodiment, during the process of iteratively calculating the homogeneous linear equation system according to a preset inverse iterative algorithm and obtaining the equation solution when the iteration result meets the preset requirements, different convergence conditions can be configured as the preset requirements according to different application needs. Specifically, the homogeneous linear equation system is first iteratively calculated according to the preset inverse iterative algorithm, and then the maximum value of the number of iterations is set. When the number of iterations reaches the preset maximum value, the result of the last iteration is obtained as the equation solution of the homogeneous linear equation system, thereby determining the convergence conditions under different scenarios and improving the accuracy of convergence judgment.

[0060] In one embodiment, the step of iteratively calculating the homogeneous linear equation system according to a preset inverse iterative algorithm to obtain the equation solution when the iteration result satisfies a preset requirement includes:

[0061] The homogeneous linear equation system is iteratively calculated according to a preset inverse iterative algorithm;

[0062] Obtain the first result of the current iteration calculation;

[0063] Obtain the second result of the previous iteration calculation;

[0064] If the difference between the first result and the second result is less than a preset value, the first result is taken as the solution to the homogeneous linear equation system.

[0065] In this embodiment, during the process of iteratively calculating the homogeneous linear equation system according to a preset inverse iterative algorithm to obtain the equation solution when the iteration result meets the preset requirements, different convergence conditions can be configured as the preset requirements according to different application needs. Specifically, the homogeneous linear equation system is iteratively calculated according to the preset inverse iterative algorithm; the first result of the current iteration calculation is obtained; the second result of the previous iteration calculation is obtained; if the difference between the first result and the second result is less than a preset value, the first result is taken as the equation solution of the homogeneous linear equation system, that is, the result obtained from each iteration calculation is compared with the result obtained from the previous iteration calculation. When the difference between the first result and the second result is less than the preset value, iterative convergence is determined, and then the first result is taken as the equation solution of the homogeneous linear equation system, thereby determining the convergence conditions under different scenarios and improving the accuracy of convergence judgment.

[0066] In one embodiment, the number of cameras is at least two, and the cameras are monocular grayscale cameras; the step of obtaining the intrinsic and extrinsic parameter matrices of the cameras, and calculating the projection matrix based on the intrinsic and extrinsic parameter matrices, includes:

[0067] Obtain the intrinsic and extrinsic parameter matrices for each monocular grayscale camera;

[0068] The projection matrix of the corresponding grayscale monocular camera is calculated based on the intrinsic and extrinsic parameter matrices.

[0069] In this embodiment, the number of cameras is at least two, that is, two or more cameras, and the cameras are monocular grayscale cameras. When obtaining the intrinsic and extrinsic parameter matrices of the cameras, since they are different cameras, their parameters are different. That is, the intrinsic and extrinsic parameter matrices of each monocular grayscale camera are obtained separately. In the process of calculating the projection matrix, the projection matrix of the corresponding camera is calculated based on the different intrinsic and extrinsic parameter matrices. That is, the projection matrix of the corresponding grayscale monocular camera is calculated according to the intrinsic and extrinsic parameter matrices, thereby accurately determining the projection matrix of each camera and improving the accuracy of 3D coordinate transformation.

[0070] In one embodiment, obtaining the intrinsic and extrinsic parameter matrices for each monocular grayscale camera includes:

[0071] Obtain the camera calibration algorithm for each monocular grayscale camera;

[0072] The intrinsic and extrinsic parameter matrices of the corresponding monocular grayscale camera are calculated based on the camera calibration algorithm.

[0073] In this embodiment, during the process of obtaining the intrinsic and extrinsic parameter matrices of each monocular grayscale camera, since the intrinsic and extrinsic parameter matrices of the camera need to be calculated, specifically, the camera calibration algorithm of each monocular grayscale camera is obtained, and then the intrinsic and extrinsic parameter matrices of the corresponding monocular grayscale camera are calculated according to the camera calibration algorithm, thereby accurately determining the intrinsic and extrinsic parameter matrices of different cameras, and thus accurately determining the projection matrix of each camera, so as to improve the accuracy of three-dimensional coordinate transformation.

[0074] In one embodiment, the step of iteratively calculating the homogeneous linear equation system according to a preset inverse iterative algorithm to obtain the equation solution when the iteration result satisfies a preset requirement includes:

[0075] The homogeneous linear equation system is input to the graphics processor, and the graphics processor performs iterative calculations on the homogeneous linear equation system according to a preset inverse iterative algorithm to obtain the equation solution when the iteration result meets the preset requirements.

[0076] In this embodiment, during the iterative calculation of the homogeneous linear equation system according to the preset inverse iterative algorithm to obtain the equation solution when the iterative result meets the preset requirements, the iterative calculation process in this embodiment only includes matrix and vector multiplication and division and vector normalization calculation, and does not require matrix inversion. For example, for the above Y... (k+1) =(A T A+σI) -1 X (k)The calculation is transformed into a calculation of (A) T A+σI)Y (k+1) =X (k) To avoid matrix inversion calculations, the homogeneous linear equations are input to a graphics processor. The graphics processor then iteratively calculates the homogeneous linear equations according to a preset inverse iteration algorithm. By leveraging the computing power of the graphics processor to iteratively calculate the homogeneous linear equations, the solution to the equations that meets the preset requirements is obtained. This eliminates the need to calculate all eigenvalues, accelerates the convergence speed, and thus improves the efficiency of iterative calculations.

[0077] Reference Figure 3 This application also provides a three-dimensional coordinate construction device for key points of the hand, comprising:

[0078] Two-dimensional coordinate module 10 is used to obtain the two-dimensional coordinates of key hand points on a two-dimensional image;

[0079] The camera parameter module 20 is used to obtain the intrinsic parameter matrix and extrinsic parameter matrix of the camera, and calculate the projection matrix based on the intrinsic parameter matrix and extrinsic parameter matrix; the camera is the camera for capturing the two-dimensional image;

[0080] The coordinate projection module 30 is used to establish the projection relationship equation between the two-dimensional coordinates and the three-dimensional coordinates of the key points of the hand based on the projection matrix;

[0081] Alignment elimination module 40 is used to eliminate variables in the projection relationship equation to obtain the homogeneous linear equation system of the two-dimensional coordinates and the three-dimensional coordinates.

[0082] The iterative calculation module 50 is used to perform iterative calculations on the homogeneous linear equation system according to a preset inverse iterative algorithm, obtain the equation solution when the iterative result meets the preset requirements, and determine the three-dimensional coordinates of the key points of the hand according to the equation solution.

[0083] As described above, it can be understood that each component of the three-dimensional coordinate construction device for hand key points proposed in this application can realize the function of any of the three-dimensional coordinate construction methods for hand key points as described above.

[0084] In one embodiment, the iterative calculation of the homogeneous linear equation system according to a preset inverse iterative algorithm includes:

[0085] Let a displacement scalar approaching zero be denoted as σ;

[0086] Set the initial value X of the solution to the equation. (0) And make the initial value X of the solution to the equation (0) Satisfy ||X (0) || = 1;

[0087] Set the number of iterations to start from k=0;

[0088] If the iterative calculation fails to converge, execute:

[0089] Y (k+1) =(A T A+σI) -1 X (k) ;

[0090] X (k+1) =Y (k+1) / ||X (k+1) ||;

[0091] μ (k+1) =(X (k+1) , (A T A+σI)X (k+1) );

[0092] k = k + 1; where A is a function of the two-dimensional coordinates and the projection matrix;

[0093] When the iterative calculation converges, the output X is reached. (k) μ (k+1) .

[0094] In one embodiment, the step of iteratively calculating the homogeneous linear equation system according to a preset inverse iterative algorithm to obtain the equation solution when the iteration result satisfies a preset requirement includes:

[0095] The homogeneous linear equation system is iteratively calculated according to a preset inverse iterative algorithm;

[0096] When the number of iterations in the iterative calculation reaches a preset maximum value, the result of the last iteration is obtained as the solution to the homogeneous linear equation system.

[0097] In one embodiment, the step of iteratively calculating the homogeneous linear equation system according to a preset inverse iterative algorithm to obtain the equation solution when the iteration result satisfies a preset requirement includes:

[0098] The homogeneous linear equation system is iteratively calculated according to a preset inverse iterative algorithm;

[0099] Obtain the first result of the current iteration calculation;

[0100] Obtain the second result of the previous iteration calculation;

[0101] If the difference between the first result and the second result is less than a preset value, the first result is taken as the solution to the homogeneous linear equation system.

[0102] In one embodiment, the number of cameras is at least two, and the cameras are monocular grayscale cameras; the step of obtaining the intrinsic and extrinsic parameter matrices of the cameras, and calculating the projection matrix based on the intrinsic and extrinsic parameter matrices, includes:

[0103] Obtain the intrinsic and extrinsic parameter matrices for each monocular grayscale camera;

[0104] The projection matrix of the corresponding grayscale monocular camera is calculated based on the intrinsic and extrinsic parameter matrices.

[0105] In one embodiment, obtaining the intrinsic and extrinsic parameter matrices for each monocular grayscale camera includes:

[0106] Obtain the camera calibration algorithm for each monocular grayscale camera;

[0107] The intrinsic and extrinsic parameter matrices of the corresponding monocular grayscale camera are calculated based on the camera calibration algorithm.

[0108] In one embodiment, the step of iteratively calculating the homogeneous linear equation system according to a preset inverse iterative algorithm to obtain the equation solution when the iteration result satisfies a preset requirement includes:

[0109] The homogeneous linear equation system is input to the graphics processor, and the graphics processor performs iterative calculations on the homogeneous linear equation system according to a preset inverse iterative algorithm to obtain the equation solution when the iteration result meets the preset requirements.

[0110] Reference Figure 4 This application also provides a computer device, which can be a mobile terminal, and its internal structure can be as follows: Figure 4 As shown, the computer device includes a processor, memory, network interface, display device, and input device connected via a system bus. The network interface is used for communication with external terminals via a network connection. The display device is used to display offline applications. The input device is used to receive user input in offline applications. The processor provides computational and control capabilities. The memory includes non-volatile storage media. This non-volatile storage media stores the operating system, computer programs, and a database. The database stores raw data. When executed by the processor, the computer program implements a method for constructing three-dimensional coordinates of key hand points.

[0111] The processor described above executes the method for constructing the three-dimensional coordinates of the hand key points, including: acquiring the two-dimensional coordinates of the hand key points on a two-dimensional image; acquiring the intrinsic and extrinsic parameter matrices of a camera, and calculating a projection matrix based on the intrinsic and extrinsic parameter matrices; the camera being the camera that captures the two-dimensional image; establishing a projection relationship equation between the two-dimensional and three-dimensional coordinates of the hand key points based on the projection matrix; eliminating variables in the projection relationship equation to obtain a homogeneous linear equation system of the two-dimensional and three-dimensional coordinates; iteratively calculating the homogeneous linear equation system according to a preset inverse iteration algorithm, obtaining the equation solution when the iteration result meets preset requirements, and determining the three-dimensional coordinates of the hand key points based on the equation solution.

[0112] The computer device provides a method for converting the two-dimensional coordinates of hand key points in a two-dimensional image into three-dimensional coordinates in three-dimensional space. First, the two-dimensional coordinates of the hand key points in the two-dimensional image are obtained. The intrinsic and extrinsic parameter matrices of the camera are also obtained. A projection matrix is ​​calculated based on these matrices. The camera is the one capturing the two-dimensional image. An equation relating the two-dimensional and three-dimensional coordinates of the hand key points is established based on the projection matrix. Elimination is performed on this equation to obtain a homogeneous linear equation system of the two-dimensional and three-dimensional coordinates. The homogeneous linear equation system is iteratively calculated using a preset inverse iterative algorithm to obtain the solution that satisfies preset requirements. The three-dimensional coordinates of the hand key points are determined based on the solution. By establishing the relationship between the two-dimensional and three-dimensional coordinates through the projection matrix and then performing elimination followed by iterative calculation, the iterative calculation can quickly converge to the estimated value of the three-dimensional coordinates of the hand key points with the smallest error, thereby improving the accuracy of converting the two-dimensional coordinates into the three-dimensional coordinates of the hand key points.

[0113] Embodiments of this application also provide a computer-readable storage medium storing a computer program thereon. When executed by the processor, the computer program implements a method for constructing three-dimensional coordinates of hand key points, including the following steps: obtaining two-dimensional coordinates of hand key points on a two-dimensional image; obtaining the intrinsic and extrinsic parameter matrices of a camera, and calculating a projection matrix based on the intrinsic and extrinsic parameter matrices; the camera being a camera capturing the two-dimensional image; establishing a projection relationship equation between the two-dimensional and three-dimensional coordinates of the hand key points based on the projection matrix; eliminating variables in the projection relationship equation to obtain a homogeneous linear equation system of the two-dimensional and three-dimensional coordinates; iteratively calculating the homogeneous linear equation system according to a preset inverse iterative algorithm, obtaining the equation solution when the iteration result meets preset requirements, and determining the three-dimensional coordinates of the hand key points based on the equation solution.

[0114] The computer-readable storage medium provides a method for converting the two-dimensional coordinates of hand key points in a two-dimensional image into three-dimensional coordinates in three-dimensional space. First, the two-dimensional coordinates of the hand key points in the two-dimensional image are obtained. The intrinsic and extrinsic parameter matrices of the camera are also obtained. A projection matrix is ​​calculated based on these matrices. The camera is the one capturing the two-dimensional image. An equation relating the two-dimensional and three-dimensional coordinates of the hand key points is established based on the projection matrix. Elimination is performed on this equation to obtain a homogeneous linear equation system of the two-dimensional and three-dimensional coordinates. The homogeneous linear equation system is iteratively calculated using a preset inverse iterative algorithm to obtain the solution that satisfies preset requirements. The three-dimensional coordinates of the hand key points are determined based on the solution. By establishing the relationship between the two-dimensional and three-dimensional coordinates through the projection matrix and then performing elimination followed by iterative calculation, the iterative calculation can quickly converge to the estimated value of the three-dimensional coordinates of the hand key points with the smallest error, thereby improving the accuracy of the three-dimensional coordinates of the hand key points.

[0115] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media provided in this application and in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual-speed SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0116] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0117] The above description is only a preferred embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural changes made based on the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for constructing the three-dimensional coordinates of key points of a hand, characterized in that, include: Obtain the 2D coordinates of key hand points on a 2D image; Obtain the intrinsic and extrinsic parameter matrices of the camera, and calculate the projection matrix based on the intrinsic and extrinsic parameter matrices; the camera is the camera that captures the two-dimensional image; Based on the projection matrix, establish the projection relationship equation between the two-dimensional coordinates and the three-dimensional coordinates of the key points of the hand; Eliminating variables from the projection relationship equations yields a homogeneous system of linear equations for the two-dimensional and three-dimensional coordinates. The homogeneous linear equation system is iteratively calculated according to a preset inverse iterative algorithm to obtain the equation solution when the iteration result meets the preset requirements, and the three-dimensional coordinates of the key points of the hand are determined according to the equation solution. The step of iteratively calculating the homogeneous linear equation system according to a preset inverse iterative algorithm includes: Let a displacement scalar approaching zero be denoted as ; Set the initial values ​​for the solution of the equation. And make the initial value of the solution to the equation satisfy ; Set the number of iterations from start; If the iterative calculation fails to converge, execute: ； ； ； in, For the purposes of this article The function; When the iterative calculation converges, the output is... , .

2. The method for constructing three-dimensional coordinates of key hand points according to claim 1, characterized in that, The step of iteratively calculating the homogeneous linear equation system according to a preset inverse iterative algorithm to obtain the equation solution when the iteration result satisfies preset requirements includes: The homogeneous linear equation system is iteratively calculated according to a preset inverse iterative algorithm; When the number of iterations in the iterative calculation reaches a preset maximum value, the result of the last iteration is obtained as the solution to the homogeneous linear equation system.

3. The method for constructing three-dimensional coordinates of key hand points according to claim 1, characterized in that, The step of iteratively calculating the homogeneous linear equation system according to a preset inverse iterative algorithm to obtain the equation solution when the iteration result satisfies preset requirements includes: The homogeneous linear equation system is iteratively calculated according to a preset inverse iterative algorithm; Obtain the first result of the current iteration calculation; Obtain the second result of the previous iteration calculation; If the difference between the first result and the second result is less than a preset value, the first result is taken as the solution to the homogeneous linear equation system.

4. The method for constructing three-dimensional coordinates of key hand points according to claim 1, characterized in that, The number of cameras is at least two, and the cameras are monocular grayscale cameras; The process of obtaining the camera's intrinsic and extrinsic parameter matrices and calculating the projection matrix based on these matrices includes: Obtain the intrinsic and extrinsic parameter matrices for each monocular grayscale camera; The projection matrix of the corresponding grayscale monocular camera is calculated based on the intrinsic and extrinsic parameter matrices.

5. The method for constructing the three-dimensional coordinates of key hand points according to claim 4, characterized in that, The process of obtaining the intrinsic and extrinsic parameter matrices for each monocular grayscale camera includes: Obtain the camera calibration algorithm for each monocular grayscale camera; The intrinsic and extrinsic parameter matrices of the corresponding monocular grayscale camera are calculated based on the camera calibration algorithm.

6. The method for constructing three-dimensional coordinates of key hand points according to claim 1, characterized in that, The step of iteratively calculating the homogeneous linear equation system according to a preset inverse iterative algorithm to obtain the equation solution when the iteration result satisfies preset requirements includes: The homogeneous linear equation system is input to the graphics processor, and the graphics processor performs iterative calculations on the homogeneous linear equation system according to a preset inverse iterative algorithm to obtain the equation solution when the iteration result meets the preset requirements.

7. A device for constructing three-dimensional coordinates of key points of a hand, characterized in that, include: The 2D coordinate module is used to obtain the 2D coordinates of key hand points on a 2D image; The camera parameter module is used to obtain the intrinsic and extrinsic parameter matrices of the camera and calculate the projection matrix based on the intrinsic and extrinsic parameter matrices; the camera is the camera that captures the two-dimensional image. The coordinate projection module is used to establish the projection relationship equation between the two-dimensional coordinates and the three-dimensional coordinates of the key points of the hand based on the projection matrix; The alignment elimination module is used to eliminate variables in the projection relationship equation to obtain the homogeneous linear equation system of the two-dimensional coordinates and the three-dimensional coordinates. The iterative calculation module is used to perform iterative calculations on the homogeneous linear equation system according to a preset inverse iterative algorithm, obtain the equation solution when the iterative result meets the preset requirements, and determine the three-dimensional coordinates of the key points of the hand according to the equation solution; The step of iteratively calculating the homogeneous linear equation system according to a preset inverse iterative algorithm includes: Let a displacement scalar approach zero be defined, denoted as ; Set the initial values ​​for the solution of the equation. And make the initial value of the solution to the equation satisfy ; Set the number of iterations from start; If the iterative calculation fails to converge, execute: ； ； ； in, For the purposes of this article The function; When the iterative calculation converges, the output is... , .

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the three-dimensional coordinate construction method for key hand points according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the three-dimensional coordinate construction method for key hand points as described in any one of claims 1 to 6.

Images