A detection method for face alignment

Abstract

The present invention discloses a detection method for face alignment, comprising: shooting a video image and converting it into a picture, eliminating non-face and blurred pictures, splitting to obtain a training set and a test set, and marking the training set; using a face detection model to detect the pictures of the training set and the test set respectively, and obtaining a face detection frame; inputting the marked training set into a constructed FAN convolutional neural network, using a FAN loss function to perform end-to-end optimization on the FAN convolutional neural network, and obtaining an optimal network model; importing the pictures of the test set into the optimal network model, generating the key point positions of the face, the probability of the position and the visibility probability; performing affine transformation on the pictures corresponding to the key point positions of the face and the collected video images, and obtaining an aligned face image. The present invention can solve the problem that the live recognition rate is not high due to the large inclination angle of the face image. It can also solve the problem that the L2 loss function has a large error in initial positioning.

Description

Technical Field

[0001] The present invention relates to the technical field of face recognition, and in particular to a detection method for face alignment. Background Art

[0002] In the face liveness detection system, face alignment is a very critical preprocessing step. At present, the face alignment methods in the prior art are mainly based on active shape model (ASM), active appearance model (AAM) and deep learning methods. Among them, the face alignment method based on mainstream deep learning has a good effect, which includes direct regression and heat map-based methods. The direct regression method uses a stacked network or a densely connected U-shaped network, which is initially used to estimate the two-dimensional posture of the human body and then used to align face images. In addition, the tree-based regression method can align faces faster, and the cascade regression method can align faces more accurately. The heat map-based method generates a real heat map based on a Gaussian distribution with a fixed variance, and then uses the distance between each predicted heat map and the real heat map for training, and the maximum value of each predicted heat map is used to estimate the key points of the face. Afterwards, many scholars improved the face alignment method by changing the variance of the pixels of the heat map. However, the heat map-based method has many disadvantages. Since the training goal is to simulate a heat map with a fixed Gaussian distribution, it is not suitable for key point probability estimation. In addition, the maximum value of the heat map is only determined by the nearest pixel, and the error in generating key points is large. In order to locate the key points of the face, the maximum value of the heat map is replaced by the spatial average value.

[0003] Face alignment detection methods have achieved great success in the field of computer vision. However, many methods still have the following problems:

[0004] First, before face liveness detection, the existing technology simply removes noise and non-face images, and then inputs the cleaned images into the convolutional neural network for training. These face images may be tilted or tilted at too large an angle to see the front face, resulting in a low liveness recognition rate;

[0005] Second, face key point detection mainly uses the L2 loss function to calculate the loss in the network. L2 can handle smaller losses, but when the initial positioning error is large, the L2 loss function cannot recover quickly from these large errors;

[0006] Third, face alignment methods (5 key points of the face) perform well in most cases, but these methods only predict the locations of the key points without estimating the probability of their locations or predicting whether the key points are occluded.

[0007] Traditionally, there are two probabilistic methods for estimating key point locations. The first one uses a non-parametric method to estimate the probability of face alignment, convolving the heat map with a Gaussian kernel to obtain a kernel density network. The other method uses a direct regression method (without a heat map) to calculate the probability of key points, directly predicting the mean and precision matrix of the Gaussian distribution. The key point locations will change as different people mark them, and this change will also change on different images. For example, for images with occlusion and poor lighting, the key point locations change greatly. In real scenes, each key point in a face image has only one true marked location, so it is difficult to accurately estimate the probability distribution of key point location changes.

[0008] Currently, there are two methods for key point probability estimation: sampling and non-sampling. Sampling-based methods include Bayesian neural networks and Monte Carlo, which estimate the probability of key points based on multiple inputs. In addition, sampling-based methods are suitable for one-dimensional regression and not for high-dimensional regression. Non-sampling methods have two outputs, one for estimating the value of the key point and the other for evaluating the probability of the key point and optimizing the Gaussian log-likelihood function without calculating the loss value of classification and regression.

[0009] In addition, in the Chinese invention patent with patent application number "202010166807.8" and patent name "Video face recognition method based on dynamic interval loss function and probability features", it uses the learned uncertainty as the importance of the feature to aggregate the input video feature set to identify the face image in the video; this technology uses the pre-trained RetinaFace face detection model to output the face bounding box and the position of the face key points, which has the following problems:

[0010] First, this technique only predicts the location of key points without estimating the probability of their location. In occluded and poorly lit face images, the location of key points will change significantly, resulting in poor face alignment results.

[0011] Second, the model of this technology does not determine whether the face is occluded, resulting in the inability to accurately locate the key points of the face when the face is occluded, thereby reducing the generalization ability of face and key point detection.

[0012] Therefore, there is an urgent need to build an efficient and accurate face alignment detection model to detect the position, position probability and visibility probability of key points in real time and accurately, so as to correct the face image. Summary of the invention

[0013] In view of the above problems, the object of the present invention is to provide a method for detecting face alignment. The technical solution adopted by the present invention is as follows:

[0014] A method for detecting face alignment comprises the following steps:

[0015] Shoot video images and convert them into pictures, remove non-face and blurred pictures, split them into training and test sets, and mark the training set;

[0016] Use the face detection model to detect the images of the training set and the test set respectively, and obtain the face detection frame;

[0017] Inputting the labeled training set into the constructed FAN convolutional neural network, and performing end-to-end optimization on the FAN convolutional neural network using the FAN loss function to obtain the optimal network model;

[0018] Import the test set images into the optimal network model to generate the key point positions, position probabilities, and visibility probabilities of the faces;

[0019] Affine transformation is performed on the picture corresponding to the key point position of the face and the collected video image, and an aligned face image is obtained.

[0020] Furthermore, the optimization process of the FAN convolutional neural network includes the following steps:

[0021] Add the mean estimator, visibility estimator network and location estimator network to the bottom layer of any U-Net network to obtain the constructed FAN convolutional neural network;

[0022] The labeled training set is input into the constructed FAN convolutional neural network to obtain the weighted spatial average of the heat map and use it as the position p of any key point j ;

[0023] The position estimator network is used to find the probability of the position of any key point, and the visibility estimator network is used to estimate the probability u of the visibility of any key point in any image. j ;

[0024] The probability of key point positions and key point visibility is used in combination with the FAN loss function to optimize and obtain the optimal network model.

[0025] Furthermore, obtaining the weighted spatial average value of the heat map includes the following steps:

[0026] The value H of the heat map pixel (x, y) of the jth key point of the i-th U-Net network ij (x,y);

[0027] Use RELU function to analyze the heat map H ij The pixels at (x, y) are processed to obtain the weighted spatial average of the heat map (i.e. the position u of the key point ij ), whose expression is:

[0028]

[0029] Among them, σ(H i,j (x,y)) represents the output value after the heat map is processed using the RELU function (σ).

[0030] Furthermore, the position estimator network is used to obtain the probability of the position of any key point, and the visibility estimator network is used to estimate the probability u of the visibility of any key point in any picture. j , including the following steps:

[0031] The Bernoulli distribution is used to estimate the joint distribution of visibility and position of the jth key point of the i-th U-Net network, and its expression is:

[0032]

[0033] in, represents the predicted probability of visibility, q u (u) represents the probability of Bernoulli distribution, q z (z|u) represents the probability of estimating the visibility of the key point at point z, q(u,z) represents the probability of the joint distribution of visibility u and position z, u=1 means that the key point position is visible, u=0 means that the key point position is not visible;

[0034] q z (z|u=1)=P(p j |u ij ,∑ ij );

[0035]

[0036] Among them, P(p j |u ij ,∑ ij ) represents the estimated position u of the key point at point z ij and the probability of occurrence ∑ij, Indicates that the key point does not exist.

[0037] Furthermore, the FAN loss function is used to perform end-to-end optimization on the FAN convolutional neural network, and its expression is:

[0038]

[0039] Among them, u j represents the visibility of the jth key point, represents the predicted probability of visibility, P(p j |u ij ,∑ ij) represents the estimated position u of the key point at point z ij and the probability of occurrence ∑ij,p j Indicates the location of key points.

[0040] Furthermore, the distribution of the positions of the key points is obtained using a two-dimensional Gaussian likelihood model, and its expression is:

[0041]

[0042] Furthermore, the expression of the FAN loss function is:

[0043]

[0044] Among them, u j represents the visibility of the jth key point, represents the predicted probability of visibility, and ∑ij represents the occurrence probability of key points.

[0045] Furthermore, the distribution of the positions of the key points is obtained using the two-dimensional Laplace likelihood, which is expressed as follows:

[0046]

[0047] in, Represents the true position probability z and predicted position probability u of the i-th U-Net network key point ij The reciprocal of the sum of the differences.

[0048] Furthermore, the expression of the FAN loss function is:

[0049]

[0050] Among them, u j represents the visibility of the jth key point, represents the predicted probability of visibility, ∑ij represents the probability of occurrence of key points, Represents the true position probability p of the i-th U-Net network key point j and the predicted position probability u ij The reciprocal of the sum of the differences.

[0051] Furthermore, the total loss L of the image is the weighted sum of the losses of any U-Net network, which is expressed as follows:

[0052]

[0053]

[0054] Among them, N p Indicates the number of key points.

[0055] Compared with the prior art, the present invention has the following beneficial effects:

[0056] (1) The present invention ingeniously introduces a FAN network structure to align facial images, aligning tilted facial images to the front, thereby correcting the facial images, solving the problem in the prior art that the tilt angle of facial images is too large, resulting in a low liveness recognition rate;

[0057] (2) The present invention proposes a joint loss function (FAN) to estimate the facial key point positions, key point position probabilities and visibility probabilities, which can not only detect the key point positioning results after face alignment, but also obtain the corresponding probabilities. In addition, the present invention also solves the problem of large errors in the initial positioning of the L2 loss function.

[0058] (3) The present invention adds a network branch (PEN) at the key point position to estimate the covariance matrix of the multivariate Gaussian or Laplace probability distribution, obtains a parameter model of the position probability distribution, and solves the problem of inaccurate key point detection. In addition, the present invention estimates the model parameters of each key point of the face image by training the neural network structure to estimate the probability of the key point position of the face image.

[0059] (4) The present invention adds a visibility estimator network structure (SEN) to predict whether each key point is blocked according to the position of the key point, thereby obtaining the visibility of each key point.

[0060] (5) The present invention cleverly uses a preprocessing method to remove the data set of non-face and blurred images in the face image, which is beneficial to improving the accuracy of liveness detection and also increases the calculation speed of the network model, thereby improving the performance of the overall algorithm.

[0061] (6) The face alignment algorithm of the present invention can accurately locate on multiple data sets, generate position probability and visibility probability, and can be applied in real time in real scenes.

[0062] In summary, the present invention has the advantages of real-time accuracy, precise positioning, reliable recognition, etc., and has high practical value and promotion value in the field of face recognition technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope of protection. For those skilled in the art, other related drawings can be obtained based on these drawings without creative work.

[0064] Figure 1 It is a flow chart of the present invention.

[0065] Figure 2 It is a structural schematic diagram of the face alignment network of the present invention.

[0066] Figure 3 The figure shows five key points of the FAN network of the present invention.

[0067] Figure 4 It is a schematic diagram of the face alignment effect of the present invention.

[0068] In the above drawings, the component names corresponding to the reference numerals are as follows: DETAILED DESCRIPTION

[0069] In order to make the purpose, technical scheme and advantages of this application clearer, the present invention is further described below in conjunction with the accompanying drawings and embodiments, and the embodiments of the present invention include but are not limited to the following embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0070] Example

[0071] like Figures 1 to 4 As shown, this embodiment provides a method and system for detecting face alignment, wherein:

[0072] In the data preprocessing module S101, the camera captures the face video image, converts the video into an image, then removes the non-face and blurred images to obtain the training and test sets, and manually marks the training set images.

[0073] Face detection module S102: The pre-processed face image is detected by a face detection model to obtain a face detection frame.

[0074] The network structure training module S103 inputs the labeled training set into the constructed FAN convolutional neural network for training, adjusts the parameters of the network structure to obtain the optimal network model, and then inputs the test set into the optimal network model to generate the key point positions of the face, the probability of the positions and the visibility probability.

[0075] The face alignment module S104 performs affine transformation on the original image through the positions of key points to obtain an aligned face image.

[0076] A method for detecting face alignment is described in detail below. The detailed steps of the FAN network structure of this embodiment are as follows:

[0077] The first step is to shoot video images and convert them into pictures, remove non-face and blurred pictures, split them into training and test sets, and mark the training set;

[0078] In the second step, the face detection model is used to detect the images of the training set and the test set respectively, and the face detection frame is obtained.

[0079] The third step is to input the labeled training set into the constructed FAN convolutional neural network to obtain the optimal network model. In this embodiment, the FAN loss function L is used. ij Realize end-to-end optimization of the entire network structure and obtain the position estimate u of the i-th U-net network and the j-th key point ij , location probability Σij and visibility estimation Specifically:

[0080] The input RGB face image is passed through the U-Net network structure, where three components are added from each U-Net branch. Since the location of the maximum value of the heat map is used as the key point of the face, the location of the generated key point is inaccurate. Therefore, the first component (mean estimator) is added to the bottom layer of U-net to calculate the weighted spatial average of the heat map as the location of each key point p j The second new component is the Seeing Estimator Network (SEN) and the third component is the Position Estimator Network (PEN). The weights of PEN and SEN are shared among all U-Net networks. PEN estimates the probability (covariance matrix coefficients) of each keypoint location. This probability uses a parametric metric that directly estimates the parameters of the multivariate Laplace or Gaussian distribution. SEN estimates the probability u of visibility of each keypoint in the image. j ∈{0, 1}, 1 means visible, 0 means invisible.

[0081] This embodiment considers three different functions σ, ReLU function (the simplest and best estimation method), softmax function (makes the average value estimate the maximum value of the heat map) and temperature-controlled softmax function (depending on the temperature setting, providing a continuous softmax function from the maximum value to a uniform distribution). For the value H of the heat map pixel (x, y) of the jth key point of the i-th U-Net network ij (x,y), the estimated value u of the key point position ij =[u ijx ,u ijy ] T , use the ReLU function to transform the heat map H ij The pixels at (x, y) are processed to obtain the weighted spatial average of the heat map (i.e. the position u of the key point ij ), whose expression is:

[0082]

[0083] Among them, σ(H i,j (x,y)) represents the output value after the heat map is processed using the RELU function (σ).

[0084] In order to explicitly measure the visibility of key points, this embodiment uses a mixed random variable to model the probability distribution of key point locations. For each key point j in the image, the key point is represented by a binary variable u j ∈{0, 1} represents the visibility of the marker, where 1 represents the position of the key point (p j ) is visible. If the keypoint is not visible (u j =0), then the key point does not exist

[0085] This embodiment estimates the joint distribution of visibility and position of the jth key point of the i-th U-Net network. The Bernoulli distribution is used to estimate the joint distribution of visibility u and position z of the jth key point of the i-th U-Net network. The expression is:

[0086]

[0087] in, represents the predicted probability of visibility, q u (u) represents the probability of Bernoulli distribution, q z (z|u) represents the probability of estimating the visibility of the key point at point z, q(u,z) represents the probability of the joint distribution of visibility u and position z, u=1 means that the key point position is visible, u=0 means that the key point position is not visible;

[0088] q z (z|u=1)=P(p j |u ij ,∑ ij );

[0089]

[0090] Among them, P(p j |u ij ,∑ ij ) represents the estimated position u of the key point at point z ij and the probability of occurrence ∑ij, Indicates that the key point does not exist.

[0091] In this embodiment, the FAN loss is the negative log-likelihood of q(u,z), so the minimum loss is equivalent to the maximum likelihood estimate. This embodiment uses the FAN loss function to perform end-to-end optimization on the FAN convolutional neural network, and its expression is:

[0092]

[0093] Among them, this embodiment predicts three values ​​of each key point (position, probability of position and visibility probability), the first term is the regression loss of position estimation, and the last two terms are the classification loss of visibility. In the formula, u j represents the visibility of the jth key point, represents the predicted probability of visibility, P(p j |u ij ,∑ ij ) represents the estimated position u of the key point at point z ij and the probability of occurrence ∑ij,p j Indicates the location of key points.

[0094] The overall FAN loss (minimum of negative log-likelihood) is equal to the minimum of KL divergence.

[0095]

[0096] Among them, q(u j ,p j ) and p(u j ,p j ) represent the visibility u of the jth key point j and position p j The probability of the joint distribution of .

[0097] This embodiment uses two different models (Gaussian likelihood and Laplace model) to obtain the position distribution of key points; wherein, a two-dimensional Gaussian likelihood model is used to obtain, and its expression is:

[0098]

[0099] The expression of the optimized FAN loss function is:

[0100]

[0101] Among them, u j represents the visibility of the jth key point, represents the predicted probability of visibility, and ∑ij represents the occurrence probability of key points.

[0102] Second, the two-dimensional Laplace likelihood is used to obtain the expression:

[0103]

[0104] in, Represents the true position probability z and predicted position probability u of the i-th U-Net network key point ij The reciprocal of the sum of the differences.

[0105] The expression of the optimized FAN loss function is:

[0106]

[0107] Among them, u j represents the visibility of the jth key point, represents the predicted probability of visibility, ∑ij represents the probability of occurrence of key points, Represents the true position probability p of the i-th U-Net network key point j and the predicted position probability u ij The reciprocal of the sum of the differences.

[0108] In this embodiment, if ∑ij is the identity matrix, all key points are visible. The FAN loss value obtained by the two-dimensional Gaussian likelihood is reduced to l 2 The square of the distance, the FAN loss value obtained by the two-dimensional Laplace likelihood is reduced to l 2 distance.

[0109] This embodiment uses a 2×2 covariance matrix To represent the probability of each key point position, this probability is a symmetrical diagonal element. Therefore, this embodiment outputs the value through the RELU activation function, and adds a constant to the function to ensure that the output is always a diagonal element. In addition, the PEN decomposition obtains the covariance matrix In order to obtain the triangular matrix L ij Each U-net network adds a position estimator network (PEN), which is a fully connected linear layer with an input 2048-dimensional vector (128×4×4=2048 dimensions) and an output N p ×3-dimensional vector, N p is the number of key points.

[0110] This embodiment adds a fully connected linear layer to estimate the visibility of key points. Its input is a 2048-dimensional vector (128×4×4=2048 dimensions) and its output is N p dimensional vector. This is passed through the sigmoid activation function, so the predicted value of visibility is Between 0 and 1. The loss value of a single U-net is the loss of all key points (j=1,...,N p ) average value L ij , the total loss L for each input image is the weighted sum of all K U-net network losses, and its expression is as follows:

[0111]

[0112]

[0113] Among them, N p Indicates the number of key points.

[0114] Step 4: This example uses the 300W, Menpo, and COFW68 datasets. 300W contains 8,000 training images and 1,300 test images. Menpo contains 7,000 test images and COFW68 contains 1,000 test images. The experiment uses the standard indicator mean error (NME box ) and AUC box To verify the accuracy of the model. Standardized mean error (NME box ) is:

[0115]

[0116] Among them, u j 、p j and u kj Represent the visibility, true position and predicted position of the jth key point respectively. box Set d to the average of the width and height of the face detection box In order to evaluate the accuracy of the algorithm, we first calculate the mean error NME of the test set. box (%), and then calculated as the area under the curve AUC, the larger the area, the more accurate the algorithm.

[0117] Table 1 Comparison of average error and AUC of different data sets

[0118]

[0119] First, it is trained on 300W, and then tested on 300W, Menpo, and COFW68 datasets. Table 1 shows the results of the experiment on the Menpo and COFW datasets. In the three datasets, the average error of FAN is lower than that of the other four methods. In addition, the AUC area of ​​FAN is larger and more accurate than that of the other four methods. This is because the other four algorithms use L2 or other loss functions, which produce larger errors.

[0120] The fifth step is to use the FAN network to detect face images on the 3 million dataset. The results are as follows: Figure 3 As shown, there are five key points detected by the FAN network (two eyes, nose and two mouth corners).

[0121] Step 6: The result of face image alignment using FAN network is as follows Figure 4 As shown, Figure (a) shows the original image, and Figure (b) shows the aligned face image.

[0122] In this application, all actions to obtain signals, information or data are carried out in compliance with the relevant data protection laws and policies of the country where the user is located and with the authorization of the corresponding device owner.

[0123] The above embodiments are only preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any changes that adopt the design principles of the present invention and are made through non-creative labor on this basis should fall within the protection scope of the present invention.

Claims

1. A face alignment detection method, characterized in that: The following steps are involved: Shoot video images and convert them into pictures, remove non-face and blurred pictures, split them into training and test sets, and mark the training set; Use the face detection model to detect the images of the training set and the test set respectively, and obtain the face detection frame; Inputting the labeled training set into the constructed FAN convolutional neural network, and performing end-to-end optimization on the FAN convolutional neural network using the FAN loss function to obtain the optimal network model; Import the test set images into the optimal network model to generate the key point positions, position probabilities, and visibility probabilities of the faces; Perform affine transformation on the image corresponding to the key point position of the face and the collected video image, and obtain an aligned face image; The optimization process of the FAN convolutional neural network includes the following steps: Add the mean estimator, visibility estimator network and location estimator network to the bottom layer of any U-Net network to obtain the constructed FAN convolutional neural network; The labeled training set is input into the constructed FAN convolutional neural network to obtain the weighted spatial average of the heat map and use it as the position p of any key point j ; The position estimator network is used to find the probability of the position of any key point, and the visibility estimator network is used to estimate the probability u of the visibility of any key point in any image. j ; The probability of key point position and key point visibility is used in combination with the FAN loss function to optimize and obtain the optimal network model after optimization; The position estimator network is used to obtain the probability of the position of any key point, and the visibility estimator network is used to estimate the probability u of the visibility of any key point in any picture. j , including the following steps: Bernoulli distribution is used to estimate the joint distribution of visibility and position of the jth key point of the i-th U-Net network, and its expression is: in, represents the predicted probability of visibility, q u (u) represents the probability of Bernoulli distribution, q z (z|u) represents the probability of estimating the visibility of the key point at point z, q(u,z) represents the probability of the joint distribution of visibility u and position z, u=1 means that the key point position is visible, u=0 means that the key point position is not visible; q z (z|u=1)=P(p j |u ij ,Σ ij ); Among them, P(p j |u ij ,∑ ij ) represents the estimated position u of the key point at point z ij and the probability of occurrence ∑ij, Indicates that the key point does not exist; Whether each key point is blocked is predicted based on its position, thereby obtaining the visibility of each key point.

2. A face alignment detection method according to claim 1, characterized in that: The weighted spatial average value of the heat map is obtained by the following steps: The value H of the heat map pixel (x, y) of the jth key point of the i-th U-Net network ij (x,y); Use RELU function to analyze the heat map H ij The pixels at (x, y) are processed to obtain the weighted spatial average of the heat map, which is expressed as: Among them, σ(H i,j (x,y)) represents the output value after the heat map is processed using the RELU function (σ).

3. The method for detecting face alignment according to claim 1, characterized in that: The FAN loss function is used to perform end-to-end optimization on the FAN convolutional neural network, and its expression is: Among them, u j represents the visibility of the jth key point, represents the predicted probability of visibility, P(p j |u ij ,∑ ij ) represents the estimated position u of the key point at point z ij and the probability of occurrence ∑ij,p j Indicates the location of key points.

4. The method for detecting face alignment according to claim 3, characterized in that: The distribution of the positions of the key points is obtained using a two-dimensional Gaussian likelihood model, and its expression is:

5. A face alignment detection method according to claim 4, characterized in that: The expression of the FAN loss function is: Among them, u j represents the visibility of the jth key point, represents the predicted probability of visibility, and ∑ij represents the occurrence probability of key points.

6. The method for detecting face alignment according to claim 3, characterized in that: The distribution of the positions of the key points is obtained using the two-dimensional Laplace likelihood, which is expressed as: in, Represents the true position probability z and predicted position probability u of the i-th U-Net network key point ij The reciprocal of the sum of the differences.

7. The method for detecting face alignment according to claim 6, characterized in that: The expression of the FAN loss function is: Among them, u j represents the visibility of the jth key point, represents the predicted probability of visibility, ∑ij represents the probability of occurrence of key points, Represents the true position probability p of the i-th U-Net network key point j and the predicted position probability u ij The reciprocal of the sum of the differences.

8. The method for detecting face alignment according to claim 3, characterized in that: The total loss L of the image is the weighted sum of the losses of any U-Net network, and its expression is as follows: Among them, N p Indicates the number of key points.

Images