Method for complex environment face key point detection of real and synthetic data

Abstract

The application provides a method for complex environment face key point detection by combining real and synthetic data, comprising: establishing a synthetic face data set and a real face data set; constructing a cycle generation network, using an optimized generator and discriminator to train the optimized cycle generation network; using the synthetic face data set and the real face data set as inputs, respectively optimizing a model structure and a loss function to improve a face key point detection model; using the cycle generation network to convert real face test data into generated face data with both synthetic face and real face feature information, inputting the generated face data into the improved face key point detection model to detect position information of face key points, and then mapping the position information to the real face test data. The application can be used for privacy data protection and small sample data intelligent tasks in a complex environment, and has high accuracy, strong generalization ability and good real-time performance.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of face information recognition, and particularly relates to a method for complex environment face key point detection by combining real and synthetic data. BACKGROUND

[0002] Artificial intelligence development with the help of deep learning technology has attracted comprehensive attention and impetus, and all parties have reached a consensus that it has become a development hotspot. In order to obtain a more accurate intelligent model, it is often hoped that the training data can be more diversified and rich. However, it is more and more difficult to rely on manual collection of training data and annotation of big data. For example, due to privacy reasons (protection of privacy of minors, protection of privacy of the medical industry, protection of consumer data by enterprises) or commercial value, data sets are not publicly available. Moreover, the environment in real-world scenarios is often complex and variable, and some scene samples are difficult to obtain. Therefore, the data in actual application is often composed of a small amount of strong labeled data and a large amount of weak labeled data. Although transfer learning can appropriately reduce the amount of training samples or can double the amount of data through data augmentation. However, these methods are prone to overfitting in the face of few-sample intelligent tasks and cannot better improve the test precision effect.

[0003] In the past decade, the face key point detection task has made substantial progress. Many methods have achieved high accuracy in detecting key points in front-facing and near-front-facing face images. Common methods can be roughly divided into three categories: coordinate regression methods, heat map regression methods, and three-dimensional model fitting methods. The coordinate regression method takes the landmark coordinates as the regression target through a neural network, with the input being a face image and the output being a landmark coordinate vector. The output of the heat map regression method is a likelihood response map for each key point. The three-dimensional model fitting method is based on the obvious relationship between 2D face landmarks and 3D face shapes. In short, the 3D model fitting method reconstructs the 3D face shape from the 2D image, and then projects it onto the image plane to obtain the 2D landmarks. The disadvantage of coordinate regression is that it causes a large amount of spatial information to be lost, and the three-dimensional model fitting method has high time complexity due to the complexity of the model. Therefore, the current research is mostly based on heat map regression, but the face dataset in complex scenarios faces the challenges of difficult annotation tasks, data imbalance, and blurred or even invisible key points, which makes it difficult for existing key point detection tasks to accurately locate the plane at large angles and to train a unified model.

[0004] In recent years, researchers use synthetic data to enhance training data, thinking that synthetic data sets can fill all potential use cases and edge cases, save data acquisition costs, and meet privacy protection requirements. However, existing synthetic techniques face challenges such as training instability, gradient disappearance, and mode collapse. Especially when faced with small sample data, they cannot learn all the features of the modeling image, overemphasize a few image features, and are prone to drift and artifacts. In addition, synthetic images lack authenticity and have poor performance when generalized to actual test scenarios, and have weak migration ability. SUMMARY

[0005] The synthetic images generated by the traditional generative network model lack authenticity and have poor performance when generalized to actual test scenarios. Existing face key point detection models in complex scenarios also face challenges such as difficult data labeling tasks, data imbalance, and inaccurate key point positioning. The present application provides a method for detecting face key points in complex environments using real and synthetic data. The present application can be applied to intelligent tasks with small sample data, has high accuracy, strong generalization ability, and good real-time performance.

[0006] The technical solution of the present application is:

[0007] A method for detecting face key points in complex environments using real and synthetic data, comprising:

[0008] Synthetic face data x and real face data y are used to establish synthetic face data set X and real face data set Y;

[0009] A cycle generation network is constructed, the generator and discriminator of the cycle generation network are optimized using least squares, the cycle consistency function is used to ensure that the generated image data generated by the generator can be converted back to the original target domain, the ontology implicit mapping function is used to ensure the color distribution similarity of the input image and the output image of the generator, and the perceptual loss function is used to ensure the consistency of the feature information between the input image and the output image of the generator; The optimized generator and discriminator are used to train the optimized cycle generation network;

[0010] The synthetic face data set and the real face data set are used as input, the residual block in the Hourglass model is optimized using the attention mechanism, and the loss function is optimized using the face facial landmark heat map information, and the improved face key point detection model is trained;

[0011] The real face test data is converted into generated face data with both synthetic face features and real face features using the cycle generation network, the generated face data is input into the improved face key point detection model, the position information of the face key points is detected, and the face key point position information is mapped to the real face test data.

[0012] Further, the synthetic human face data x in the synthetic human face dataset X is generated by FaceGen Modeller, and the human face data has a unique identity.

[0013] Further, the generator and discriminator specifically include two generators G x and G y , and two discriminators D x and D y .

[0014] The generator and discriminator of the network are generated by using a least square optimization cycle, and specifically include: establishing a least square optimization function L LSGAN .

[0015] L LSGAN = E x～Px [(Dx(x)) 2 ]+ E x～Py [(1- Dx(G x (x))) 2 ] (1)

[0016] Wherein, x represents synthetic human face data, E x～Px represents the probability that x comes from the synthetic data distribution P x , Dx(x) represents that the closer to 1, the better the discrimination of the synthetic sample, E x～Py represents the probability that x comes from the real data set distribution P y , and G x (x) represents the data set generated by using the generator G x .

[0017] Further, the cycle consistency function is used to ensure that the image data generated by the generator can be converted back to the original target domain, and specifically includes: establishing a cycle consistency function L Cyc .

[0018] L Cyc = E x～Px [||G y (G x (x))- x||1]+ E y～Py [||G x (G y (y))-y||1] (2)

[0019] Wherein, E x～Px represents the probability that x comes from the synthetic human face dataset P x , G y (G x (x)) represents the reconstructed synthetic human face dataset, y represents real human face data, and E y～PyP represents the distribution of real face data. y Probability, G x (G y (y) represents the reconstructed real face dataset.

[0020] Furthermore, the step of using an ontology mapping function to ensure the color distribution similarity between the generator's input and output images specifically includes: establishing an ontology mapping function L. Ide ,

[0021] L Ide =E x～Px [||x- G y (x)||1]+ E y～Py [||y- G x (y)||1] (3)

[0022] Among them, E x～Px P represents the distribution of synthetic face data from which x originates. x Probability, G y (x) represents inputting the synthetic face dataset x into G. y Reconstruction is performed in the generator, E y～Py P represents the distribution of real face data. y Probability, G x (y) represents inputting the real face data y into G. x Reconstruction is performed in the generator.

[0023] Furthermore, the step of using a perceptual loss function to ensure the consistency of feature information between the input and output images specifically includes: establishing a perceptual loss function L... per ,

[0024]

[0025] Where, φ j These are high-level features extracted by a pre-trained ResNet50 network at a specific j-th level, where each level generates a (C) j *H j *W j The feature map is denoted by , where C represents the number of channels in the feature map, H represents the height of the feature map, W represents the width of the feature map, and j is a natural number.

[0026] Furthermore, based on the attention mechanism, the residual blocks in the Hourglass model are optimized, and the loss function is optimized using facial landmark heatmap information to train and improve the facial landmark detection model, specifically including:

[0027] H(x n )=F(x n ,ωn )+x n (5)

[0028] wherein, x n represents the original input into the nth residual block, F(x n ,ω n ) represents the feature map obtained by processing the input x n through a mechanism of learning different channel weights ω n ; H(x n ) represents the output of the residual block;

[0029] The loss function is optimized as follows:

[0030]

[0031] wherein p(i,j) and g(i,j) represent the predicted heat map and the ground truth heat map at pixel position (i,j) respectively, N is the number of key points, and i,j are natural numbers.

[0032] Compared with the prior art, the present application has the following advantages:

[0033] 1. The present application introduces a synthetic dataset to make up for the lack of real data in order to solve a series of problems such as real data being mostly small-sample weak-labeled data in a complex environment, and annotated data being difficult to obtain. Three-dimensional data information of a synthetic human face under self-defined settings is obtained by using FaceGen Modeller to reconstruct, so as to obtain a two-dimensional synthetic image library under different angles and different occlusions.

[0034] 2. The present application considers that synthetic images lack realness, and the model trained by using a synthetic dataset has poor effect when generalized to an actual test scene. An improved generation network model is proposed in the algorithm model, so that the synthetic data converted by the finally trained R→S model has the characteristic information of real images.

[0035] 3. The present application proposes that only a small amount of real dataset and a large amount of synthetic dataset are needed to train a human face key point detection model, and SE attention is added in the training model to store important information about images. In the evaluation stage, the real test data is converted by the R→S model and then input into the trained key point detection model for prediction, and the predicted value is fed back to the real test data, so that the synthetic data can solve the problem of low detection accuracy of small-sample data in a complex scene. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 is a schematic diagram of the training and test stages of the present application;

[0037] Figure 2is a synthetic image database graph generated by FaceGen Modeller of the present application;

[0038] Figure 3 is an example graph of real and synthetic data sets after some preprocessing of the present application;

[0039] Figure 4 is a conversion model block diagram based on an improved cycle generation network of the present application;

[0040] Figure 5 is a face key point detection graph based on an improved HourGlass model of the present application. DETAILED DESCRIPTION

[0041] The present application proposes a method for complex environment face key point detection by combining real and synthetic data. It includes:

[0042] Synthetic face data x is constructed by using face generator FaceGen Modeller and real face data y is collected to establish synthetic face data set X and real face data set Y.

[0043] A cycle generation network is constructed, the generator and discriminator of the cycle generation network are optimized by least square, the cycle consistency function is used to ensure that the image data generated by the generator can be converted back to the original target domain, the ontological implicit function is used to ensure the color distribution similarity of the input image and the output image of the generator, the perceptual loss function is used to ensure the feature information consistency between the input image and the output image of the generator; the optimized generator and discriminator are used to train the optimized cycle generation network.

[0044] The synthetic face data set and the real face data set are used as input, the residual block in the Hourglass model is optimized by using attention mechanism, and the loss function is optimized by using face facial landmark heat map information to train the improved face key point detection model.

[0045] The real face test data is converted into generated face data with both synthetic face and real face features by using the cycle generation network, the generated face data is input into the improved face key point detection model, the position information of the face key point is detected, and then the face key point position information is mapped to the real face test data.

[0046] The application discloses a face key point detection method in a complex environment by using real and synthetic data in combination. The method firstly acquires initial three-dimensional data files of different face attribute settings by self-definition under a FaceGen Modeller, and obtains a two-dimensional synthetic image database under different angles and different occlusions by means of three-dimensional reconstruction technology. Secondly, a new network structure design is proposed on the basis of further improvement of a traditional CycleGAN generation network, which can generate a synthetic database that simultaneously retains the feature information of synthetic faces and real faces, so as to further improve the reality of the synthetic data and reduce the domain gap between the synthetic data set and the real data set. Thirdly, a large amount of synthetic data set and a small amount of real data set are used as a training set to train a complex environment face key point detection model, and a squeeze-and-excitation network (SE) is introduced into the detection model, so that the model can not only learn the feature of excitation information, but also accelerate the whole network to reduce the parameter and calculation workload. Finally, the face key point detection network is used to predict the coordinate points of the test image after transformation (R to S model) in the evaluation process, and finally the predicted coordinate point position information is fed back to the real test image. The method reduces the dependence of the model on the real world data set, explores the complementarity and availability of the synthetic data set, and provides a new research idea for data protection and small sample intelligent tasks in a complex environment.

[0047] In combination Figure 1 The implementation process of the overall technical solution is described.

[0048] The application firstly generates a large amount of synthetic face data set by means of a FaceGen Modeller generator, and optimizes the traditional CycleGAN generation network structure model, so that the generated image retains the feature information of the real and synthetic data set. A large amount of synthetic data set and a small amount of real data set are used to train a face key point detection model in a complex environment, and finally in the evaluation stage, the real test data set is converted by the generation model in advance, the converted image is input into the face key point detection model, and the position coordinate information predicted by the model is finally fed back to the real test face image, so as to further improve the face key point detection precision in a complex scene. The specific steps of the method are as follows:

[0049] Step (1), the initial three-dimensional data file is obtained by using FaceGen Modeller, which can automatically generate a face with unique identity, and allows users to randomize, switch and standardize the face, and supports setting various parameters such as emotions, nationalities, races and genders on the face. Then, three-dimensional reconstruction is performed using three-dimensional data information to obtain a synthetic face two-dimensional database under different angles and different occlusions in complex scenes. At the same time, in the preprocessing stage, data augmentation is performed on the synthetic and real data sets to increase the diversity of the data;

[0050] Step (2), the synthetic data obtained in step (1) under different scene settings and the data under real scene are uniformly cropped to 256*256 size as the input of the improved generation network model. In the network structure design, an optimized Resnet block is used to quickly filter the information of interest and important information from a large amount of information. In the optimization objective function, the cross-entropy loss function is replaced by the least squares-based optimization. Since the cross-entropy uses a logarithmic function when calculating the probability, it will cause numerical overflow or underflow problems, resulting in a larger gradient for the wrong samples. However, the least squares calculation value is relatively stable, and it can better reflect the difference between the generated samples and the real samples in the generation network, and can balance the fitting of all samples. Then, the pre-trained Resnet50 network model is used to extract the high-level features of the image (such as image texture combination and structural semantic information) to calculate the perceptual loss between the images. The low-level features provide basic information about the image, such as color and edge, but usually lack high-level semantic information. High-level features are a higher level of abstraction and representation based on low-level features (such as object shape, structure, texture combination, pose, etc.). The purpose of adding perceptual calculation is to improve the quality and realism of the generated images. Traditional generation networks may have problems such as image blur, distortion, and missing details in generated images. By adding it, the details and structure of the generated images can be better learned.

[0051] Step (3), a face key point detection model is trained by combining a large amount of synthetic data sets and a small amount of real data sets under complex environments. On the basis of the traditional Hourglass model, SE attention optimized residual blocks are added to reduce the dimension of image data to store important information about the image. Then, multiple Hourglasses are directly connected to make the model better utilize the complementarity between features. This step is to train a key point model for faces in complex scenes. The accuracy of the model is tested, but the test image cannot be directly input into the key point model, but needs to be converted by the R→S model trained in step (2) before being input into the key point model.

[0052] Step (4), in the test evaluation stage, the trained generative network model in step (2) is defined as a R→S model, and the purpose of the model is to convert real test data into synthetic data with real and synthetic human face features. After conversion, the real test data is input into the face key point detection model obtained in step (3) to obtain key point coordinate information, and then the coordinate information is fed back to the real test data.

[0053] The present application provides a kind of human face key point detection method under complex environment by real and synthetic data joint, and the main steps are as follows:

[0054] Synthetic human face image acquisition and preprocessing

[0055] First, the three-dimensional texture structure information file of the relevant synthetic human face is obtained by using FaceGen Modeller self-defined face attribute setting (such as age, nationality, hairstyle, expression, etc.), and then a two-dimensional synthetic image database under different angles and occlusion conditions is constructed by means of three-dimensional reconstruction technology, as shown in Figure 2 .

[0056] At the same time, the real human face data set faces the problem of different sizes and small sample size, so image augmentation preprocessing is a very important link. As shown in Figure 3 , the sample figures of real and synthetic data sets after some preprocessing are given.

[0057] Conversion model based on cycle generative network

[0058] Based on the structure of the cycle-consistent generative network model, two generators G x :S→G and G y :R→S are used, as well as two adversarial discriminators D x and D y , which are used to distinguish whether the image has been translated from another domain. As shown in Figure 4As shown, the first generator encoding process consists of three convolutional layers, and the activation function ReLU is used for each layer. Second, in the conversion process, nine optimized residual blocks are constructed. Finally, the decoding process consists of two decoding layers and a convolutional layer. The structure of different discriminators can use different patch sizes, such as from 1x1 PixelGANs to 256x256 ImageGANs that segment the entire image. PixelGANs do not help spatial clarity, but can relatively improve the effect of color. Using 16x16 PatchGANs can further improve the clarity of the output, but some unnatural textures will appear. Using 256x256 can integrate the context information of the entire spatial image, but the effect of detailed texture information is not good. The discriminator structure in this application is selected as 70x70 PatchGANs. The PatchGAN discriminator network is a deep CNN model composed of multiple convolutional layers and fully connected layers. It predicts whether each 70x70 block is part of a real image or a generated image by locally discriminating each 70x70 block. By combining the discrimination results of these small blocks, the discrimination result of the entire image can be obtained. Finally, the output is a probability between 0 and 1, indicating the likelihood that each small block belongs to a real image. In addition to the input layer and the output layer, instance normalization is used for all layers of the generator and discriminator network.

[0059] The present application also balances the loss between the generation model and the discrimination model by changing the cost function to obtain a better optimization goal. Specifically as follows:

[0060] (1) In the training process of the generation model in step (2), the cross-entropy loss function is replaced by the least squares-based optimization L LSGAN Attempts are made to use different distance metrics to build a more stable and faster converging network. The goal is to learn a mapping function so that the distribution of images generated from G x (x) is indistinguishable from Y real images. This loss can ensure that the samples generated by the generator have the same distribution as the real samples.

[0061] L LSGAN = E x～Px [(Dx(x)) 2 ]+ E x～Py [(1- Dx(G x (x))) 2 ] (1)

[0062] Where x represents the synthetic data set generated by FaceGen Modeller, E x～Px represents the P x probability that x comes from the synthetic data distribution, Dx(x) represents that the closer to 1, the better the discrimination of the synthetic sample, and E x～PyP y (x) represents the probability that x comes from the real dataset distribution x Probability, G x (x) represents the generated dataset.

[0063] (2) To ensure that the generated dataset has the same distribution as the real-world samples, the cycle consistency function L Cyc is used to ensure that the image can be converted back to the original target domain.

[0064] L Cyc = E x～Px [||G y (G x (x))- x||1]+ E y～Py [||G x (G y (y))-y||1] (2)

[0065] where G y (G x (x)) represents the reconstructed synthetic face dataset, y represents the real face data, E y～Py represents the probability that y comes from the real face data distribution y Probability, G x (G y (y)) represents the reconstructed real face dataset.

[0066] (3) When optimizing the generator, the ontology mapping function L Ide is used to ensure that the color distribution of the input and output images is similar.

[0067] L Ide = E x～Px [||x- G y (x)||1]+ E y～Py [||y- G x (y)||1] (3)

[0068] where G y (x) represents inputting the synthetic face dataset x into the G y generator for reconstruction, and G x (y) represents inputting the real face data y into the G x generator for reconstruction.

[0069] (4) To encourage the transfer, keep the consistency of feature information between input and output. Add perceptual loss to force the generator to approach the information of the input. Perceptual loss is to use a fixed network (here using pre-trained Resnet50) to get the corresponding output features by taking the real image and the network generated result as the input respectively, and then use the feature information to construct the loss to approximate the deep information between the real image and the network generated result. Compared with L2 loss, it can enhance the detailed information of the output features.

[0070]

[0071] where φ j is the high-level feature extracted by the pre-trained Resnet50 network at a specific j level, where each level generates a feature map of size (C j *H j *W j ), C represents the channel number of the feature map, H represents the height of the feature map, W represents the width of the feature map, and j is a natural number.

[0072] Face key point detection model based on Hourglass

[0073] The training set of the application is a synthetic data set generated by FaceGen Modeller combined with a small amount of real face data set, and the test set is real face data in complex scenes. The face key point detection model estimates the face landmark heat map information based on the structure of the improved HourGlass network. The essence of heat map regression is to output a Gaussian distribution centered on each ground truth landmark.

[0074] As shown in Figure 5 , HourGlass is an asymmetric top-down and bottom-up fully convolutional network. The feature extraction part uses an optimized residual block, which adds attention channels in the connection of the traditional Resnet residual block, assigns different weights to different parts of the given input, so that the model only pays attention to the local salient features of the target related image, can extract more discriminative features, and enhances the expression ability of the model. It can be represented as follows:

[0075] H(x n )=F(x n ,ω n )+x n (5)

[0076] where x n represents the original input to the nth residual block, F(x n ,ω n ) represents the mechanism of learning different channel weights ω n for input x nThe processed feature map. The model can adaptively adjust the importance of each channel, improve the representation ability and discrimination of the feature, H(x n ) represents the output of the residual block, that is, the result of adding the result from the weight layer and the result of the original input.

[0077] In the training process, the following loss function is optimized:

[0078]

[0079] Where p(i,j) and g(i,j) represent the predicted heat map and the ground truth heat map at pixel position (i,j) respectively, N is the number of key points, i,j are natural numbers.

[0080] The test of the test evaluation module is mainly for faces in complex scenes, such as large angles, exaggerated expressions, etc. As shown in the figure, Figure 1 The converted image is input into the face key point detection model, and the 68 key point coordinate prediction values of the face are predicted, and the obtained coordinate information is fed back to the real test image, so as to obtain the key point detection result of the real test image.

Claims

1. A method for real and synthetic data joint complex environment face key point detection, characterized in that, It comprises: Using synthetic face data x and real face data y, a synthetic face data set X and a real face data set Y are established; A recurrent generative network is constructed, and least square optimization is used to optimize the generator and discriminator of the recurrent generative network, wherein the generator and the discriminator include two generators G x and G y , and two discriminators D x and D y ​ The generator and the discriminator of the generator network are optimized by the least square optimization cycle, and the optimization function L of the least square optimization cycle comprises: LSGAN , L LSGAN = E x～Px [(Dx(x)) 2 ] + E x～Py [(1-Dx(G x (x))) 2 ] (1) Where x represents synthetic face data, E x～Px P represents the distribution of the synthetic data from which x originates. x The probability, Dx(x), represents the probability that the value is as close to 1 as possible when judging synthetic samples, and E... x～Py P represents the distribution of x from the real dataset. y Probability, G x (x) represents the use of generator G x The generated dataset; The cyclic consistency function is used to ensure that the image data generated by the generator can be converted back to the original target domain, and the specific method comprises the following steps: Cyc , L Cyc = E x～Px [||G y (G x (x))-x||1]+E y～Py [||G x (G y (y))-y||1] (2) where E x～Px represents the probability that x comes from the synthetic face dataset P x , G y (G x (x)) represents the reconstructed synthetic face dataset, y represents the real face data, E y～Py represents the probability that y comes from the real face data distribution P y , G x (G y (y)) represents the reconstructed real face dataset; The use of the ontology isomorphism function ensures the similarity of the color distribution of the generator input image and the output image, and the specific method comprises the following steps: Ide , L Ide = E x～Px [||x-G y (x)||1]+E y～Py [||y-G x (y)||1] (3) where E x～Px denotes the probability that x comes from the distribution of synthetic human face data, P x G y (x) denotes the reconstruction of inputting the synthetic human face data set x into the G y generator, E y～Py denotes the probability that y comes from the distribution of real human face data, P y G x (y) denotes the reconstruction of inputting the real human face data y into the G x generator; The perceptual loss function is used to ensure that the generator ensures the consistency of feature information between the input image and the output image; the specific method comprises the following steps: establishing a perceptual loss function L per , wherein φ j is a high-level feature extracted by a pre-trained Resnet50 network at a specific j level, wherein each level generates a feature map of size (C j *H j *W j ), C represents the number of channels of the feature map, H represents the height of the feature map, W represents the width of the feature map, and j is a natural number; Using the optimized generator and discriminator, the optimization cycle generation network is trained; Taking the synthetic face data set and the real face data set as input, the residual block in the Hourglass model is optimized by using the attention mechanism, and the loss function is optimized by using the face landmark heat map information, and the improved face key point detection model is trained; Using the cycle generation network, the real face test data is converted into generated face data with both synthetic face features and real face features, the generated face data is input into the improved face key point detection model, the position information of the face key points is detected, and then the face key point position information is mapped to the real face test data.

2. The method of claim 1, wherein, The synthetic face data x in the synthetic face data set X is generated by FaceGen Modeller, and the face data has a unique identity.

3. The method of claim 1, wherein, Based on the attention mechanism to optimize the residual block in the Hourglass model and the face landmark heat map information to optimize the loss function, the improved face key point detection model is trained, which specifically comprises: H(x n ) = F(x n , ω n ) + x n (5) wherein x n represents the original input into the nth residual block, F(x n , ω n ) represents a feature map obtained by processing the input x n through a mechanism of learning different channel weights ω n ; and H(x n ) represents the output of the residual block. Optimize the following loss function: Where p(i,j) and g(i,j) represent the predicted heat map and the ground truth heat map at pixel position (i,j) respectively, N is the number of key points, and i,j is a natural number.

Images