Method, device, computer equipment and storage medium for determining joint point coordinates

Abstract

The present application relates to a method, device, computer equipment and storage medium for determining joint point coordinates. The method comprises: obtaining the driver's biometric information and the driver's driving images at different acquisition angles; for each driving image at each acquisition angle, inputting the biometric information and the driving image into a preset neural network model to obtain the initial coordinates of the driver's target part joint points corresponding to each acquisition angle; and obtaining the target coordinates of the target part joint points according to the initial coordinates of the target part joint points at each acquisition angle. The method can improve the accuracy of the determined joint point coordinates.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device, computer equipment and storage medium for determining joint point coordinates. Background Art

[0002] Posture recognition technology refers to the technology of automatically detecting and identifying human posture through computer vision technology and pattern recognition technology. Driver posture recognition is a type of posture recognition. Driver posture recognition can provide warnings and interventions for the driver's illegal driving behavior during driving to avoid potential driving accidents. Among them, the determination of joint point coordinates is an indispensable step in posture recognition technology. Therefore, when performing driver posture recognition, it is necessary to first obtain the driver's joint point coordinates.

[0003] In conventional technology, the coordinates of the joint points of the driver are mainly obtained by identifying the driver's image. However, the coordinates of the joint points obtained in conventional technology have a problem of low determination accuracy. Summary of the invention

[0004] Based on this, it is necessary to provide a method, device, computer equipment and storage medium for determining joint point coordinates that can improve the accuracy of the determined joint point coordinates in response to the above technical problems.

[0005] In a first aspect, the present application provides a method for determining joint point coordinates. The method comprises:

[0006] Acquiring biometric information of a driver and driving images of the driver acquired at different acquisition angles;

[0007] For each driving image acquired at each acquisition angle, the biometric information and the driving image are input into a preset neural network model to obtain the initial coordinates of the target part joint points of the driver corresponding to each acquisition angle;

[0008] According to the initial coordinates of the target part joint points at each acquisition angle, the target coordinates of the target part joint points are obtained.

[0009] In one embodiment, the neural network model includes a first feature extraction network, a second feature extraction network and a fusion network; the step of inputting the biometric information and the driving image into a preset neural network model to obtain the initial coordinates of the target part joint points of the driver corresponding to each of the acquisition angles includes:

[0010] Extracting a first feature from the biometric information through the first feature extraction network to obtain first feature information;

[0011] Performing second feature extraction on the driving image through the second feature extraction network to obtain second feature information; wherein the tensor dimension of the first feature information is the same as the tensor dimension of the second feature information;

[0012] The first feature information and the second feature information are processed by the fusion network to obtain initial coordinates of the target part joint points of the driver corresponding to each of the acquisition angles.

[0013] In one embodiment, the first feature extraction network includes a convolutional layer and a fully connected layer, and the first feature extraction of the biometric information by the first feature extraction network to obtain the first feature information includes:

[0014] Performing convolution processing on the biometric information through the convolution layer to obtain processed biometric information;

[0015] The dimension of the processed biometric feature information is processed by the fully connected layer to obtain the first feature information.

[0016] In one embodiment, acquiring the target coordinates of the target part joint point according to the initial coordinates of the target part joint point at each acquisition angle includes:

[0017] Converting the initial coordinates of the target part joint points at each acquisition angle into coordinates in the same coordinate system to obtain converted coordinates of the target part joint points at each acquisition angle;

[0018] The target coordinates of the target part joint point are obtained according to the transformed coordinates of the target part joint point at each acquisition angle.

[0019] In one embodiment, obtaining the target coordinates of the target part joint point according to the transformed coordinates of the target part joint point at each acquisition angle includes:

[0020] Taking the converted coordinates of the target part joint points at the target acquisition angle as a reference, the converted coordinates of the target part joint points at each acquisition angle except the target acquisition angle are used to correct the converted coordinates of the target part joint points at the target acquisition angle to obtain the target coordinates of the target part joint points; the target acquisition angle is any acquisition angle among the acquisition angles.

[0021] In one embodiment, the method further comprises:

[0022] Acquiring a characteristic image of the driver;

[0023] The feature image is input into a preset feature extraction model to obtain the biometric feature information.

[0024] In one embodiment, the second feature extraction network is a Visual Geometry Group (VGG) network.

[0025] In a second aspect, the present application also provides a device for determining joint point coordinates. The device comprises:

[0026] A first acquisition module is used to acquire the driver's biometric information and the driver's driving images acquired at different acquisition angles;

[0027] A second acquisition module is used to input the biometric information and the driving image into a preset neural network model for each driving image acquired at each acquisition angle, and acquire the initial coordinates of the target part joint points of the driver corresponding to each acquisition angle;

[0028] The correction module is used to obtain the target coordinates of the target part joint point according to the initial coordinates of the target part joint point at each acquisition angle.

[0029] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0030] Acquiring biometric information of a driver and driving images of the driver acquired at different acquisition angles;

[0031] For each driving image acquired at each acquisition angle, the biometric information and the driving image are input into a preset neural network model to obtain the initial coordinates of the target part joint points of the driver corresponding to each acquisition angle;

[0032] According to the initial coordinates of the target part joint point at each of the acquisition angles, the target coordinates of the target part joint point are acquired.

[0033] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0034] Acquiring biometric information of a driver and driving images of the driver acquired at different acquisition angles;

[0035] For each driving image acquired at each acquisition angle, the biometric information and the driving image are input into a preset neural network model to obtain the initial coordinates of the target part joint points of the driver corresponding to each acquisition angle;

[0036] According to the initial coordinates of the target part joint point at each of the acquisition angles, the target coordinates of the target part joint point are acquired.

[0037] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0038] Acquiring biometric information of a driver and driving images of the driver acquired at different acquisition angles;

[0039] For each driving image acquired at each acquisition angle, the biometric information and the driving image are input into a preset neural network model to obtain the initial coordinates of the target part joint points of the driver corresponding to each acquisition angle;

[0040] According to the initial coordinates of the target part joint point at each of the acquisition angles, the target coordinates of the target part joint point are acquired.

[0041] The above-mentioned method, device, computer equipment and storage medium for determining the coordinates of the joint points obtain the driver's biometric information and the driver's driving images collected at different collection angles, and then for each driving image collected at a collection angle, the driver's biometric information and driving image are input into a preset neural network model to obtain the initial coordinates of the driver's target part joint points corresponding to each collection angle. When obtaining the initial coordinates of the driver's target part joint points, the driver's driving images collected at multiple collection angles are used, which can avoid the problem of large errors caused by interference factors such as occlusion or poor light when using a driving image of a single collection angle. In addition, compared with the prior art that only uses driving images to obtain the coordinates of the target part joint points, the present application also combines the driver's biometric information, so that the driver's target part joint point coordinates are obtained through the driver's biometric information and the driver's driving image, which improves the accuracy of locating the position of the target part joint point, thereby improving the accuracy of the determined joint point coordinates. Further, the target coordinates of the target part joint point are obtained based on the initial coordinates of the target part joint point. Since the accuracy of the initial coordinates of the target part joint point is improved, the accuracy of the obtained target coordinates is also improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 FIG. 1 is an application environment diagram of a method for determining joint point coordinates in an embodiment;

[0043] Figure 2 Schematic diagram of a flow chart of a method for determining joint point coordinates in one embodiment;

[0044] Figure 3A schematic diagram of the location of each camera in an embodiment;

[0045] Figure 4 is a flow chart of a method for determining joint point coordinates in another embodiment;

[0046] Figure 5 is a schematic diagram of a neural network model in one embodiment;

[0047] Figure 6 is a flow chart of a method for determining joint point coordinates in another embodiment;

[0048] Figure 7 is a flow chart of a method for determining joint point coordinates in another embodiment;

[0049] Figure 8 is a flow chart of a method for determining joint point coordinates in another embodiment;

[0050] Fig. 9 is a structural block diagram of a device for determining joint point coordinates in an embodiment;

[0051] Fig.10 is a structural block diagram of a device for determining joint point coordinates in another embodiment;

[0052] Fig.11 is a structural block diagram of a device for determining joint point coordinates in another embodiment;

[0053] Fig.12 It is a structural block diagram of a device for determining joint point coordinates in another embodiment. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0055] The method for determining the joint point coordinates provided in the embodiment of the present application can be applied to Figure 1 The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 1As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a method for determining the coordinates of a joint point is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a button, a trackball or a touch pad set on the computer device housing, or an external keyboard, touch pad or mouse, etc.

[0056] In one embodiment, Figure 2 As shown, a method for determining the coordinates of joint points is provided, and this method is applied to Figure 1 The terminal in is used as an example to illustrate, including the following steps:

[0057] S201, acquiring biometric information of the driver and driving images of the driver collected at different angles.

[0058] Among them, the driver's biometric information includes the driver's age, height, weight, gender and other information.

[0059] In this embodiment, for example, it is assumed that Figure 3 A schematic diagram showing the interior space of a vehicle is shown. As an optional implementation, the interior of the vehicle may be configured as follows: Figure 3 The first camera, the second camera, the third camera and the fourth camera are respectively set at the four different positions shown in the figure, that is, the first camera is set on the side of the connecting column connecting the roof and the front cabin on the left front, the second camera is set above or below the rearview mirror of the vehicle, the third camera is set on the side of the connecting column connecting the roof and the front cabin on the right front, and the fourth camera is set on the top plate of the cockpit directly above the driver. The driver's driving image is collected by these four cameras with different collection angles. It is understandable that since the arrangement position of the camera will affect the image quality of the collected driving image, and the improper location of the camera may block the driver's line of sight, block light, etc., the following method is adopted. Figure 3 The arrangement of the cameras shown can minimize the impact of the camera settings on the driver. Figure 3The camera arrangement position in the figure is an example of a camera arrangement position, and the camera can also be arranged at other positions in the vehicle that do not affect the driver's line of sight and light.

[0060] Optionally, the driver can input biometric information such as age, height, weight, and gender into the terminal so that the terminal can obtain the driver's biometric information. Alternatively, the terminal can output an acquisition instruction to instruct the driver to input his or her own biometric information.

[0061] Optionally, the method for determining the joint point coordinates provided in the embodiment of the present application can be used in a sport utility vehicle (SUV), a small passenger car or other types of vehicles, and the method is not limited here. Optionally, the camera used to collect driving images can be an RGB color camera, or any other camera that meets the collection requirements, and the method is not limited here.

[0062] S202, for each driving image acquired at each acquisition angle, inputting the biometric information and the driving image into a preset neural network model to obtain the initial coordinates of the target part joint points of the driver corresponding to each acquisition angle.

[0063] Among them, the target parts of the driver may include the driver's head, torso and upper limbs, etc. The head joints may include the left eye, right eye, head center of mass, mouth and other joints, the torso joints may include the neck, chest, abdomen and other joints, and the upper limb joints may include the shoulder, elbow, palm and other joints.

[0064] In this embodiment, the driving images of multiple drivers collected by cameras with different collection angles are combined with the drivers' biometric information as inputs of a preset neural network model, and the multiple groups of inputs are respectively input into the preset neural network model to obtain the initial coordinates of the target joint points of the driver corresponding to each collection angle. Exemplarily, the image collected by the first camera and the driver's biometric information are input into the preset neural network model to obtain the first initial coordinates of the head joint points, trunk joint points, and left upper limb joint points corresponding to the first camera, wherein the first initial coordinates corresponding to the head joint points are represented as C1(x, y, z), the first initial coordinates corresponding to the trunk joint points are represented as H1(x, y, z), and the first initial coordinates corresponding to the left upper limb joint points are represented as U 1-l(x, y, z); the image captured by the second camera and the driver's biometric information are input into the preset neural network model, and the second initial coordinates of the head joints, trunk joints, and upper limb joints corresponding to the second camera are obtained, wherein the first initial coordinates corresponding to the head joints are expressed as C2(x, y, z), the first initial coordinates corresponding to the trunk joints are expressed as H2(x, y, z), and the first initial coordinates corresponding to the upper limb joints are expressed as U2(x, y, z); the image captured by the third camera and the driver's biometric information are input into the preset neural network model, and the third initial coordinates of the head joints, trunk joints, and right upper limb joints corresponding to the third camera are obtained, wherein the first initial coordinates corresponding to the head joints are expressed as C3(x, y, z), the first initial coordinates corresponding to the trunk joints are expressed as H3(x, y, z), and the first initial coordinates corresponding to the left upper limb joints are expressed as U 3-r (x, y, z); the image captured by the fourth camera and the driver's biometric information are input into a preset neural network model to obtain the fourth initial coordinates of the head joint points and the upper limb joint points corresponding to the fourth camera, wherein the first initial coordinates corresponding to the head joint points are expressed as C4 (x, y, z), and the first initial coordinates corresponding to the upper limb joint points are expressed as U4 (x, y, z).

[0065] Optionally, the preset neural network model can be a convolutional neural network (CNN), a recurrent neural network (RNN), a self-attention network (transformer), etc.

[0066] S203, acquiring target coordinates of the target part joint points according to the initial coordinates of the target part joint points at each acquisition angle.

[0067] Optionally, calculations are performed based on the initial coordinates of the target part joint points at each acquisition angle to obtain the target coordinates of the target part joint points. For example, the average value of the initial coordinates of the target part joint points at each acquisition angle can be used as the target coordinates of the target part joint points. Alternatively, weighted calculations can be performed on the initial coordinates of the target part joint points at each acquisition angle based on preset weight values ​​of each acquisition angle to obtain the target coordinates of the target part joint points. Alternatively, the initial coordinates of the target part joint points at each acquisition angle can be input into a preset coordinate model, and each initial coordinate is processed by the preset coordinate model to obtain the target coordinates of the target part joint points. Alternatively, a correspondence between each acquisition angle and each lighting condition corresponding to each acquisition angle can be established in advance, and the initial coordinates of the target part joint points corresponding to the acquisition angle with better lighting conditions can be used as the target coordinates of the target part joint points.

[0068] In the above-mentioned method for determining the coordinates of the joint points, the driver's biometric information and the driver's driving images collected at different collection angles are obtained, and then for each driving image collected at a collection angle, the driver's biometric information and the driving image are input into a preset neural network model to obtain the initial coordinates of the driver's target part joint points corresponding to each collection angle. When obtaining the initial coordinates of the driver's target part joint points, the driver's driving images collected at multiple collection angles are used, which can avoid the problem of large errors caused by interference factors such as occlusion or poor light when using a driving image of a single collection angle. In addition, compared with the prior art in which only the driving image is used to obtain the coordinates of the target part joint points, the present application also combines the driver's biometric information, so that the driver's target part joint point coordinates are obtained through the driver's biometric information and the driver's driving image, which improves the accuracy of locating the position of the target part joint point, thereby improving the accuracy of the determined joint point coordinates. Further, the target coordinates of the target part joint point are obtained based on the initial coordinates of the target part joint point. Since the accuracy of the initial coordinates of the target part joint point is improved, the accuracy of the obtained target coordinates is also improved.

[0069] In the above scenario where the biometric information and the driving image are input into a preset neural network model for each collected angle driving image to obtain the initial coordinates of the target part joint points of the driver corresponding to each collected angle, the neural network model can be used to extract features from the driving image and the biometric information of the driver at each collected angle, thereby obtaining the initial coordinates of the target part joint points of the driver corresponding to each collected angle based on the extracted feature information. In one embodiment, the neural network model includes a first feature extraction network, a second feature extraction network, and a fusion network, such as Figure 4 As shown, the above S202 includes:

[0070] S301, extracting a first feature from biometric information through a first feature extraction network to obtain first feature information.

[0071] The first feature information is feature information obtained by extracting the driver's biometric feature information through a first feature extraction network.

[0072] In this embodiment, if Figure 5 As shown, the preset neural network model includes a first feature extraction network model, a second feature extraction network model and a fusion network. Optionally, the first feature extraction network may include a fully connected layer, a convolutional layer, etc. For example, the first feature extraction network may include Figure 5The fully connected layer, 1×1 convolutional layer and 3×3 convolutional layer shown in the figure, in this embodiment, the fully connected layer and convolutional layer in the first feature extraction network can be used to perform feature dimension upgrading and feature information extraction on the above-mentioned biometric information, so as to obtain the first feature information corresponding to the above-mentioned biometric information.

[0073] S302, performing second feature extraction on the driving image through a second feature extraction network to obtain second feature information; wherein the tensor dimension of the first feature information is the same as the tensor dimension of the second feature information.

[0074] The second characteristic information is characteristic information of the driver obtained through driving images.

[0075] Optionally, the second feature extraction network can be a Visual Geometry Group (VGG) network, or a Residual Network (ResNet) or the like. In this embodiment, the driving image can be feature extracted by the VGG network to obtain image information after feature extraction, and then the image information after feature extraction is convolved by a convolution layer to obtain second feature information with the same tensor dimension as the first feature information. It should be noted that it has been verified in advance through experiments that the use of the VGG network as the second extraction network can achieve the best feature extraction effect. Therefore, in this embodiment, when the second feature extraction network is a VGG network, it is a better implementation scheme. Furthermore, in order to ensure the effect of feature extraction, as a preferred implementation, the second extraction network can specifically be a VGG-19 network in the VGG network.

[0076] S303, processing the first feature information and the second feature information through a fusion network to obtain initial coordinates of the target part joint points of the driver corresponding to each acquisition angle.

[0077] Optionally, in this embodiment, the fusion network can be a transformer model. It should be noted that the transformer model is a relatively lightweight model that does not require convolution or loop calculations. The transformer model can realize the meeting of sequences through the self-attention mechanism, so that the sequence can be processed efficiently, and can be calculated in parallel, thereby reducing the time complexity of training and reasoning.

[0078] Optionally, as shown in 3, the fusion network includes a transformer encoder and a transformer decoder. The fusion network obtains the initial coordinates of the target part joint points of the driver corresponding to each acquisition angle by encoding and decoding the first feature information and the second feature information based on the attention mechanism.

[0079] In this embodiment, the neural network model includes a first feature extraction network, a second feature extraction network and a fusion network. The first feature extraction network and the second feature extraction network are used to extract features of the driving image and the biometric information respectively, so that the acquired first feature information and the second feature information have the same tensor dimension and higher accuracy. Then, the fusion network is used to process the first feature information and the second feature information, which has faster calculation speed and reduces the complexity of neural network model reasoning.

[0080] This embodiment will illustrate the detailed process of extracting the first feature of the biometric information through the first feature extraction network to obtain the first feature information. In one embodiment, the first feature extraction network includes a convolutional layer and a fully connected layer, such as Figure 6 As shown, the above S301 includes:

[0081] S401, performing convolution processing on the biometric information through a convolution layer to obtain processed biometric information.

[0082] Among them, the convolution layer is composed of several convolution units, and the parameters of each convolution unit are optimized through the back propagation algorithm.

[0083] In this embodiment, if Figure 5 As shown, a fully connected layer may be included before the convolution layer in the first feature extraction network. The convolution layer in this embodiment may include: Figure 5 The 1×1 convolutional layer and the 3×3 convolutional layer shown in the figure, the first feature extraction network can perform feature dimension upscaling processing on the biometric information through the above-mentioned fully connected layer to obtain the biometric information after feature dimension upscaling, and use the above-mentioned 1×1 convolutional layer and the 3×3 convolutional layer to perform convolution processing on the biometric information after feature dimension upscaling to obtain the processed biometric information.

[0084] S402, processing the dimension of the processed biometric feature information through a fully connected layer to obtain first feature information.

[0085] In this embodiment, if Figure 5As shown, a fully connected layer may be included after the convolution layer, and the fully connected layer is used to process the dimension of the processed biometric information so that the tensor dimension of the biometric information is the same as the tensor dimension of the second feature information, and the biometric information after the dimension processing is determined as the first feature information. Exemplarily, the tensor dimension of the second feature information is related to the parameters of the camera corresponding to each acquisition angle and the image quality of the acquired driving image. If the tensor dimension of the processed biometric information is lower than the tensor dimension of the second feature information, the fully connected layer after the convolution layer can perform feature dimension increase processing on the processed biometric information; if the tensor dimension of the processed biometric information is higher than the tensor dimension of the second feature information, the processed biometric information is subjected to feature dimension reduction processing to obtain the above-mentioned first feature information.

[0086] In this embodiment, the convolution layer in the first feature extraction network performs convolution processing on the biometric information, and the fully connected layer processes the dimension of the processed biometric information, so that the tensor dimension of the obtained first feature information is the same as the tensor dimension of the second feature information. In this way, when the fusion network performs fusion processing on the first feature information and the second feature information, the tensor dimensions of the two are the same, thereby reducing the processing complexity of the fusion network and improving the efficiency of obtaining the initial coordinates of the joint points of the target part.

[0087] In the above scenario where the target coordinates of the target part joint points are obtained based on the initial coordinates of the target part joint points at each acquisition angle, the initial coordinates of the target part joint points at each acquisition angle can be converted into coordinates in the same coordinate system, and then the target coordinates of the target part joint points can be further obtained. Figure 7 As shown, the above S203 includes:

[0088] S501, converting the initial coordinates of the target part joint points at each acquisition angle into coordinates in the same coordinate system to obtain the converted coordinates of the target part joint points at each acquisition angle.

[0089] In this embodiment, the initial coordinates of the target part joint points of each acquisition angle are in different coordinate systems, and one of the coordinate systems can be selected as the target coordinate system, and the initial coordinates in the remaining coordinate systems are transferred to the target coordinate system through coordinate transformation, so as to obtain the transformed coordinates of the target part joint points of each acquisition angle. Exemplarily, the coordinate system corresponding to the first initial coordinates can be represented as the first coordinate system, the coordinate system corresponding to the second initial coordinates can be represented as the second coordinate system, the coordinate system corresponding to the third initial coordinates can be represented as the third coordinate system, and the coordinate system corresponding to the fourth initial coordinates can be represented as the fourth coordinate system. Further, the initial coordinates of the target part joint points of each acquisition angle can be converted into coordinates in the coordinate system of the second acquisition angle, wherein the coordinate transformation formula can be expressed as (Formula 1)-(Formula 3).

[0090] C x ′=R·C x +T (Formula 1)

[0091] U x ′=R·U x +T (Formula 2)

[0092] H x ′=R·H x +T (Formula 3)

[0093] Among them, C x represents the set of coordinates corresponding to the head joint points before coordinate transformation, C x ′ represents the set of coordinates corresponding to the head joint points after coordinate transformation; U x Represents the set of coordinates corresponding to the upper limb joints before coordinate transformation, U x ′ represents the set of coordinates corresponding to the upper limb joints after coordinate transformation; H x represents the set of coordinates corresponding to the trunk joints before coordinate transformation, H x ′ represents the set of coordinates corresponding to the trunk joints after the coordinate transformation; R represents the rotation matrix, and T represents the translation matrix. It should be noted that the various parameters in the coordinate transformation are obtained by calibration during the camera setting process, and tools such as a chessboard may be required during the calibration process.

[0094] It should be noted that the initial coordinates corresponding to the target coordinate system are the same before and after the transformation. For example, if the second coordinate system is used as the target coordinate system, the transformed coordinates of the target part joint point at the second acquisition angle corresponding to the second coordinate system are the second initial coordinates.

[0095] S502, obtaining target coordinates of the target part joint points according to the transformed coordinates of the target part joint points at each acquisition angle.

[0096] Optionally, if there are no interference factors and errors in the process of collecting and extracting features of the driver's driving images, the converted coordinates of the target part joint points of each collection angle transferred to the same coordinate system are the same, and the converted coordinates of the target part joint points of any collection angle can be determined as the target coordinates of the target part joint points.

[0097] Optionally, in actual applications, there are certain errors in feature extraction for driving images with different acquisition angles, and it is necessary to further calculate the transformed coordinates of the target part joint points at each acquisition angle to obtain the target coordinates of the target part joint points. For example, the distribution map method can be used to remove the coordinates with large errors in the transformed coordinates of the target part joint points at each acquisition angle, and the average value of the remaining coordinates is used as the target coordinates of the target part joint points.

[0098] In this embodiment, the initial left side of the target part joint point of each acquisition angle is converted into coordinates under the same coordinate system. The coordinates under the same coordinate system are more convenient for unified calculation or processing. Compared with processing the initial coordinates of the target part joint points of each acquisition angle under different coordinate systems to obtain the target coordinates of the target part joint points, the target coordinates of the target part joint points obtained by the converted coordinates of the target part joint points of each acquisition angle in this embodiment have higher accuracy.

[0099] In this embodiment, the detailed process of obtaining the target coordinates of the target part joint points at each acquisition angle is described. In one embodiment, the above S502 includes: taking the converted coordinates of the target part joint points at the target acquisition angle as a reference, using the converted coordinates of the target part joint points at each acquisition angle except the target acquisition angle, correcting the converted coordinates of the target part joint points at the target acquisition angle to obtain the target coordinates of the target part joint points; the target acquisition angle is any acquisition angle among the acquisition angles.

[0100] In this embodiment, any one of the above-mentioned acquisition angles is used as the target acquisition angle, and the converted coordinates of the target part joint point of the target acquisition angle are corrected to obtain the target coordinates of the target part joint point. Exemplarily, in this embodiment, the acquisition angle of the second camera can be used as the target acquisition angle, and the converted coordinates of the target part joint point of the second acquisition angle can be used as a reference. According to the converted coordinates of the target part joint point of the first acquisition angle, the third acquisition angle, and the fourth acquisition angle, the converted coordinates of the target part joint point of the second acquisition angle are corrected to obtain the target coordinates of the target part joint point. Optionally, the correction processing of the converted coordinates of the target part joint point of the second acquisition angle can be performed according to the following (Formula 4)-(Formula 6).

[0101]

[0102]

[0103]

[0104] Among them, C′ is the target coordinate of the trunk joint, C 2-x is the x coordinate of the initial coordinates of the trunk joint point at the second acquisition angle, C′ i-x is the x-coordinate of the initial coordinates of the trunk joint point at the i-th acquisition angle, C 2-y is the y coordinate of the initial coordinates of the trunk joint point at the second acquisition angle, C′ i-y is the y coordinate of the initial coordinates of the trunk joint point at the i-th acquisition angle, C 2-z is the z coordinate of the initial coordinates of the trunk joint point at the second acquisition angle, C′ i-z is the z coordinate of the initial coordinates of the trunk joint at the i-th acquisition angle; U′ is the target coordinate of the upper limb joint, U 2-x is the x coordinate of the initial coordinates of the upper limb joint point at the second acquisition angle, U′ i-x is the x coordinate of the initial coordinates of the upper limb joint point at the i-th acquisition angle, U 2-y is the y coordinate of the initial coordinates of the upper limb joint point at the second acquisition angle, U′ i-y is the y coordinate of the initial coordinates of the upper limb joint point at the i-th acquisition angle, U 2-z is the z coordinate of the initial coordinates of the upper limb joint point at the second acquisition angle, U′ i-z is the z coordinate of the initial coordinates of the upper limb joint point at the i-th acquisition angle; H′ is the target coordinate of the head joint point, H 2-x is the x coordinate of the initial coordinates of the head joint point at the second acquisition angle, H′ i-x is the x coordinate of the initial coordinates of the head joint point at the i-th acquisition angle, H 2-y is the y coordinate of the initial coordinates of the head joint point at the second acquisition angle, H′ i-y is the y coordinate of the initial coordinates of the head joint point at the i-th acquisition angle, H 2-z is the z coordinate of the initial coordinates of the head joint point at the second acquisition angle, H i ' -z is the z coordinate of the initial coordinates of the head joint point at the i-th acquisition angle.

[0105] It should be noted that in the above calibration process, the following method is used: Figure 3 The arrangement of the camera positions shown in FIG. 1 is used as an example to illustrate the driver image captured by the camera position arrangement. Figure 3It can be seen that due to the acquisition angles of the first camera and the third camera, the coordinates of the right upper limb joint points determined based on the driver image captured by the first camera are less reliable, and the coordinates of the left upper limb joint points determined based on the driver image captured by the third camera are less reliable. Therefore, in order to ensure the accuracy of the target coordinates of the target part joint points acquired during the correction process, the coordinates of the right upper limb joint points determined based on the driver image captured by the first camera and the coordinates of the left upper limb joint points determined based on the driver image captured by the third camera may not be used.

[0106] In this embodiment, the converted coordinates of the target part joint points in each acquisition angle except the target acquisition angle are used to correct the converted coordinates of the target part joint points in the target acquisition angle, and the corrected coordinates are determined as the target coordinates of the target part joint points, so that the accuracy of the determined target coordinates of the target part joint points is higher.

[0107] When obtaining the driver's biometric information, the driver's feature image may be obtained first, and then the driver's biometric information may be obtained based on the driver's feature image and a preset feature extraction model. Figure 8 As shown, the above method also includes:

[0108] S601, obtaining a characteristic image of the driver.

[0109] Optionally, the driving image of the driver captured by the camera at any of the above-mentioned acquisition angles can be used as the driver's characteristic image; or, the driver's image captured in advance can be used as the driver's characteristic image; or, the image input by the driver to the terminal can be used as the driver's characteristic image.

[0110] S602, input the feature image into a preset feature extraction model to obtain biometric feature information.

[0111] In this embodiment, the trained feature extraction model is pre-stored in the terminal, and the feature image is input into the preset feature extraction model, and the feature extraction model performs image pre-processing, image feature extraction, image recognition and other processing on the feature model, thereby obtaining the driver's biometric information. Optionally, the feature extraction model can be constructed based on a neural network method; or, the feature extraction model can be constructed based on a wavelet moment method; or, the feature extraction model can be constructed based on classification features.

[0112] In this embodiment, the acquired feature image of the driver is processed by a preset feature extraction model to avoid the problem of data tampering when the driver inputs biometric information, making the acquired biometric information of the driver more accurate and reliable.

[0113] An embodiment of the present disclosure is described below in conjunction with a specific joint point coordinate determination scenario. The method includes the following steps:

[0114] S1, acquiring the driver's biometric information and the driver's driving images collected at different angles;

[0115] S2, for each driving image acquired at each acquisition angle, input the biometric information and the driving image into a preset neural network model, perform first feature extraction on the biometric information through a first feature extraction network of the neural network model to obtain first feature information, perform second feature extraction on the driving image through a second feature extraction network of the neural network model to obtain second feature information, process the first feature information and the second feature information through a fusion network of the neural network model to obtain initial coordinates of the joint points of the target part of the driver corresponding to each acquisition angle; wherein the tensor dimension of the first feature information is the same as the tensor dimension of the second feature information; the first feature extraction network includes a convolution layer and a fully connected layer, and the first feature extraction on the biometric information through the first feature extraction network to obtain the first feature information includes: performing convolution processing on the biometric information through the convolution layer to obtain the processed biometric information; processing the dimension of the processed biometric information through the fully connected layer to obtain the first feature information.

[0116] S3, converting the initial coordinates of the joint points of the target parts at each acquisition angle into coordinates in the same coordinate system to obtain the converted coordinates of the joint points of the target parts at each acquisition angle;

[0117] S4, taking the transformed coordinates of the target part joint points at the target acquisition angle as a reference, using the transformed coordinates of the target part joint points at each acquisition angle except the target acquisition angle, correcting the transformed coordinates of the target part joint points at the target acquisition angle to obtain the target coordinates of the target part joint points.

[0118] In the above-mentioned method for determining the coordinates of the joint points, the driver's biometric information and the driver's driving images collected at different collection angles are obtained, and then for each driving image collected at a collection angle, the driver's biometric information and the driving image are input into a preset neural network model to obtain the initial coordinates of the driver's target part joint points corresponding to each collection angle. When obtaining the initial coordinates of the driver's target part joint points, the driver's driving images collected at multiple collection angles are used, which can avoid the problem of large errors caused by interference factors such as occlusion or poor light when using a driving image of a single collection angle. In addition, compared with the prior art in which only the driving image is used to obtain the coordinates of the target part joint points, the present application also combines the driver's biometric information, so that the driver's target part joint point coordinates are obtained through the driver's biometric information and the driver's driving image, which improves the accuracy of locating the position of the target part joint point, thereby improving the accuracy of the determined joint point coordinates. Further, the target coordinates of the target part joint point are obtained based on the initial coordinates of the target part joint point. Since the accuracy of the initial coordinates of the target part joint point is improved, the accuracy of the obtained target coordinates is also improved.

[0119] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0120] Based on the same inventive concept, the embodiment of the present application also provides a device for determining joint point coordinates for implementing the method for determining joint point coordinates involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the embodiments of the device for determining one or more joint point coordinates provided below can refer to the limitations of the method for determining joint point coordinates above, and will not be repeated here.

[0121] In one embodiment, Fig. 9 As shown, a device for determining joint point coordinates is provided, comprising: a first acquisition module 10, a second acquisition module 11 and a third acquisition module 12, wherein:

[0122] The first acquisition module 10 is used to acquire the driver's biometric information and the driver's driving images acquired at different acquisition angles.

[0123] The second acquisition module 11 is used to input the biometric information and the driving image into a preset neural network model for each driving image acquired at each acquisition angle, and acquire the initial coordinates of the joint points of the target part of the driver corresponding to each acquisition angle.

[0124] The third acquisition module 12 is used to acquire the target coordinates of the target part joint points according to the initial coordinates of the target part joint points at each acquisition angle.

[0125] The device for determining the joint point coordinates provided in this embodiment can execute the above method embodiments, and its implementation principles and technical effects are similar, which will not be repeated here.

[0126] In one embodiment, Fig.10 As shown, the second acquisition module 11 includes: a first extraction unit 111, a second extraction unit 112 and a processing unit 113, wherein:

[0127] The first extraction unit 111 is used to extract the first feature of the biometric information through a first feature extraction network to obtain first feature information.

[0128] The second extraction unit 112 is used to perform second feature extraction on the driving image through a second feature extraction network to obtain second feature information; wherein the tensor dimension of the first feature information is the same as the tensor dimension of the second feature information.

[0129] The processing unit 113 is used to process the first feature information and the second feature information through a fusion network to obtain the initial coordinates of the target part joint points of the driver corresponding to each acquisition angle.

[0130] The device for determining the joint point coordinates provided in this embodiment can execute the above method embodiments, and its implementation principles and technical effects are similar, which will not be repeated here.

[0131] In one embodiment, the first extraction unit 111 is configured to perform convolution processing on the biometric information through a convolution layer to obtain processed biometric information; and process the dimension of the processed biometric information through a fully connected layer to obtain first feature information.

[0132] The device for determining the joint point coordinates provided in this embodiment can execute the above method embodiments, and its implementation principles and technical effects are similar, which will not be repeated here.

[0133] In one embodiment, Fig.11 As shown, the third acquisition module 12 includes a conversion unit 121 and an acquisition unit 122, wherein:

[0134] The conversion unit 121 is used to convert the initial coordinates of the target part joint points at each acquisition angle into coordinates in the same coordinate system to obtain the converted coordinates of the target part joint points at each acquisition angle.

[0135] The acquisition unit 122 is used to obtain the target coordinates of the target part joint point according to the transformed coordinates of the target part joint point at each acquisition angle.

[0136] The device for determining the joint point coordinates provided in this embodiment can execute the above method embodiments, and its implementation principles and technical effects are similar, which will not be repeated here.

[0137] In one embodiment, the above-mentioned conversion unit 121 is used to take the converted coordinates of the target part joint points of the target acquisition angle as a reference, and use the converted coordinates of the target part joint points in each acquisition angle except the target acquisition angle to correct the converted coordinates of the target part joint points at the target acquisition angle to obtain the target coordinates of the target part joint points; the target acquisition angle is any acquisition angle among the acquisition angles.

[0138] The device for determining the joint point coordinates provided in this embodiment can execute the above method embodiments, and its implementation principles and technical effects are similar, which will not be repeated here.

[0139] In one embodiment, Fig.12 As shown, the above device further includes: a fourth acquisition module 13 and a fifth acquisition module 14, wherein:

[0140] The fourth acquisition module 13 is used to acquire a characteristic image of the driver.

[0141] The fifth acquisition module 14 is used to input the feature image into a preset feature extraction model to obtain biometric feature information.

[0142] The device for determining the joint point coordinates provided in this embodiment can execute the above method embodiments, and its implementation principles and technical effects are similar, which will not be repeated here.

[0143] Each module in the above-mentioned joint point coordinate determination device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0144] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0145] Acquire the driver's biometric information and the driver's driving images collected from different angles;

[0146] For each driving image collected at each angle, the biometric information and the driving image are input into a preset neural network model to obtain the initial coordinates of the target joint points of the driver corresponding to each collection angle;

[0147] According to the initial coordinates of the target part joint points at each acquisition angle, the target coordinates of the target part joint points are obtained.

[0148] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0149] Extracting a first feature from the biometric information through a first feature extraction network to obtain first feature information;

[0150] Performing a second feature extraction on the driving image through a second feature extraction network to obtain second feature information; wherein the tensor dimension of the first feature information is the same as the tensor dimension of the second feature information;

[0151] The first feature information and the second feature information are processed through a fusion network to obtain initial coordinates of the target part joint points of the driver corresponding to each acquisition angle.

[0152] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0153] The biometric information is convolved through a convolution layer to obtain processed biometric information;

[0154] The dimension of the processed biometric information is processed through a fully connected layer to obtain the first feature information.

[0155] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0156] The initial coordinates of the joint points of the target parts at each acquisition angle are converted into coordinates in the same coordinate system to obtain the converted coordinates of the joint points of the target parts at each acquisition angle;

[0157] According to the transformed coordinates of the target part joint points at each acquisition angle, the target coordinates of the target part joint points are obtained.

[0158] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0159] Taking the transformed coordinates of the target part joint points of the target acquisition angle as a reference, the transformed coordinates of the target part joint points in each acquisition angle except the target acquisition angle are used to correct the transformed coordinates of the target part joint points at the target acquisition angle to obtain the target coordinates of the target part joint points; the target acquisition angle is any acquisition angle among the acquisition angles.

[0160] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0161] Acquire a characteristic image of the driver;

[0162] Input the feature image into the preset feature extraction model to obtain biometric information.

[0163] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0164] The second feature extraction network is the Visual Geometry Group (VGG) network.

[0165] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0166] Acquire the driver's biometric information and the driver's driving images collected from different angles;

[0167] For each driving image collected at each angle, the biometric information and the driving image are input into a preset neural network model to obtain the initial coordinates of the target joint points of the driver corresponding to each collection angle;

[0168] According to the initial coordinates of the target part joint points at each acquisition angle, the target coordinates of the target part joint points are obtained.

[0169] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0170] Extracting a first feature from the biometric information through a first feature extraction network to obtain first feature information;

[0171] Performing a second feature extraction on the driving image through a second feature extraction network to obtain second feature information; wherein the tensor dimension of the first feature information is the same as the tensor dimension of the second feature information;

[0172] The first feature information and the second feature information are processed through a fusion network to obtain initial coordinates of the target part joint points of the driver corresponding to each acquisition angle.

[0173] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0174] The biometric information is convolved through a convolution layer to obtain processed biometric information;

[0175] The dimension of the processed biometric information is processed through a fully connected layer to obtain the first feature information.

[0176] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0177] The initial coordinates of the joint points of the target parts at each acquisition angle are converted into coordinates in the same coordinate system to obtain the converted coordinates of the joint points of the target parts at each acquisition angle;

[0178] According to the transformed coordinates of the target part joint points at each acquisition angle, the target coordinates of the target part joint points are obtained.

[0179] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0180] Taking the transformed coordinates of the target part joint points of the target acquisition angle as a reference, the transformed coordinates of the target part joint points in each acquisition angle except the target acquisition angle are used to correct the transformed coordinates of the target part joint points at the target acquisition angle to obtain the target coordinates of the target part joint points; the target acquisition angle is any acquisition angle among the acquisition angles.

[0181] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0182] Acquire a characteristic image of the driver;

[0183] Input the feature image into the preset feature extraction model to obtain biometric information.

[0184] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0185] The second feature extraction network is the Visual Geometry Group (VGG) network.

[0186] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0187] Acquire the driver's biometric information and the driver's driving images collected from different angles;

[0188] For each driving image collected at each angle, the biometric information and the driving image are input into a preset neural network model to obtain the initial coordinates of the target joint points of the driver corresponding to each collection angle;

[0189] According to the initial coordinates of the target part joint points at each acquisition angle, the target coordinates of the target part joint points are obtained.

[0190] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0191] Extracting a first feature from the biometric information through a first feature extraction network to obtain first feature information;

[0192] Performing a second feature extraction on the driving image through a second feature extraction network to obtain second feature information; wherein the tensor dimension of the first feature information is the same as the tensor dimension of the second feature information;

[0193] The first feature information and the second feature information are processed through a fusion network to obtain initial coordinates of the target part joint points of the driver corresponding to each acquisition angle.

[0194] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0195] The biometric information is convolved through a convolution layer to obtain processed biometric information;

[0196] The dimension of the processed biometric information is processed through a fully connected layer to obtain the first feature information.

[0197] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0198] The initial coordinates of the joint points of the target parts at each acquisition angle are converted into coordinates in the same coordinate system to obtain the converted coordinates of the joint points of the target parts at each acquisition angle;

[0199] According to the transformed coordinates of the target part joint points at each acquisition angle, the target coordinates of the target part joint points are obtained.

[0200] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0201] Taking the transformed coordinates of the target part joint points of the target acquisition angle as a reference, the transformed coordinates of the target part joint points in each acquisition angle except the target acquisition angle are used to correct the transformed coordinates of the target part joint points at the target acquisition angle to obtain the target coordinates of the target part joint points; the target acquisition angle is any acquisition angle among the acquisition angles.

[0202] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0203] Acquire a characteristic image of the driver;

[0204] Input the feature image into the preset feature extraction model to obtain biometric information.

[0205] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0206] The second feature extraction network is the Visual Geometry Group (VGG) network.

[0207] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0208] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0209] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0210] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for determining the coordinates of a driver's joint points by integrating biometric information, characterized in that: The method comprises: Acquiring biometric information of a driver and driving images of the driver acquired at different acquisition angles; For each driving image acquired at each acquisition angle, the biometric information and the driving image are input into a preset neural network model to obtain the initial coordinates of the target part joint points of the driver corresponding to each acquisition angle; Acquire the target coordinates of the target part joint points according to the initial coordinates of the target part joint points at each acquisition angle; The neural network model includes a first feature extraction network, a second feature extraction network and a fusion network; the biometric information and the driving image are input into a preset neural network model to obtain the initial coordinates of the target part joint points of the driver corresponding to each acquisition angle, including: The first feature extraction network is used to extract the first feature of the biometric information to obtain the first feature information; the second feature extraction network is used to extract the second feature of the driving image to obtain the second feature information; wherein the tensor dimension of the first feature information is the same as the tensor dimension of the second feature information; the first feature information and the second feature information are processed by the fusion network to obtain the initial coordinates of the joint points of the target part of the driver corresponding to each of the acquisition angles.

2. The method according to claim 1, characterized in that The first feature extraction network includes a convolutional layer and a fully connected layer, and the first feature extraction of the biometric feature information through the first feature extraction network to obtain the first feature information includes: Performing convolution processing on the biometric information through the convolution layer to obtain processed biometric information; The dimension of the processed biometric feature information is processed by the fully connected layer to obtain the first feature information.

3. The method according to claim 1 or 2, characterized in that: The step of acquiring the target coordinates of the target part joint points according to the initial coordinates of the target part joint points at each acquisition angle comprises: Converting the initial coordinates of the target part joint points at each acquisition angle into coordinates in the same coordinate system to obtain converted coordinates of the target part joint points at each acquisition angle; The target coordinates of the target part joint point are obtained according to the transformed coordinates of the target part joint point at each acquisition angle.

4. The method according to claim 3, characterized in that The step of obtaining the target coordinates of the target part joint point according to the transformed coordinates of the target part joint point at each acquisition angle comprises: Taking the converted coordinates of the target part joint points at the target acquisition angle as a reference, the converted coordinates of the target part joint points at each acquisition angle except the target acquisition angle are used to correct the converted coordinates of the target part joint points at the target acquisition angle to obtain the target coordinates of the target part joint points; the target acquisition angle is any acquisition angle among the acquisition angles.

5. The method according to claim 1 or 2, characterized in that: The method further comprises: Acquiring a characteristic image of the driver; The feature image is input into a preset feature extraction model to obtain the biometric feature information.

6. The method according to claim 1, characterized in that The second feature extraction network is a visual geometry group (VGG) network.

7. A device for determining the coordinates of a driver's joint points by integrating biometric information, characterized in that: The device comprises: A first acquisition module is used to acquire the driver's biometric information and the driver's driving images acquired at different acquisition angles; A second acquisition module is used to input the biometric information and the driving image into a preset neural network model for each driving image acquired at each acquisition angle, and acquire the initial coordinates of the target part joint points of the driver corresponding to each acquisition angle; The neural network model includes a first feature extraction network, a second feature extraction network and a fusion network; the biometric information and the driving image are input into a preset neural network model to obtain the initial coordinates of the target part joint points of the driver corresponding to each acquisition angle, including: Performing a first feature extraction on the biometric information through the first feature extraction network to obtain first feature information; performing a second feature extraction on the driving image through the second feature extraction network to obtain second feature information; wherein the tensor dimension of the first feature information is the same as the tensor dimension of the second feature information; processing the first feature information and the second feature information through the fusion network to obtain initial coordinates of the target part joint points of the driver corresponding to each of the acquisition angles; The third acquisition module is used to acquire the target coordinates of the target part joint point according to the initial coordinates of the target part joint point at each acquisition angle.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images