A mask matching method and system based on face recognition

Abstract

The present invention belongs to the field of computer vision, and provides a mask matching method and system based on face recognition. A human face image is input into the system, and a three-dimensional human face image is reconstructed based on an intelligent face recognition algorithm based on deep learning, and feature information of the three-dimensional human face image is extracted; based on the feature information of the three-dimensional human face image and a mask database, the most suitable mask for the face is calculated; based on the three-dimensional human face image and the most suitable mask for the face, a corresponding mask image is generated on the three-dimensional human face image and displayed on a user display, so that the user can intuitively see the degree of fit between the mask and the face and the wearing effect. The defects of the existing mask matching method, i.e., the manual trial wearing method, are improved, the comfort and accuracy of mask matching are improved, and accurate and rapid personalized matching of the mask and the human face is achieved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to a mask matching method and system based on face shape recognition. Background Art

[0002] As people's awareness of safety protection gradually increases, more and more people choose to wear masks when going out. Therefore, the issue of choosing suitable masks and enhancing protection has gradually received widespread attention.

[0003] When people choose masks that suit their face shape, they usually need to try on masks of different brands, models and sizes manually, which consumes a lot of time, manpower and resources. In addition, the manual wearing method will be affected by personal subjective feelings, and it is impossible to guarantee that the right mask can be directly selected. Using a dedicated fit test instrument to help people choose a suitable mask requires expensive and complex equipment, and the personnel using the instrument need to be trained before they can operate the instrument, which is difficult to meet the diversity of different scenarios and needs. The above methods all require people to choose the model and size of the mask according to their personal subjective feelings, and then use manual trial wearing or special testing instruments to test the fit of the face and the selected mask. The testing process is long, resulting in a lot of cost, time and manpower required for mask matching.

[0004] Therefore, we need to develop a mask matching method and system based on face recognition, which can simply, quickly and accurately select suitable masks without spending a lot of time on manual trial wearing or using special testing instruments, while also controlling costs and ensuring the universality of the solution. Summary of the invention

[0005] The purpose of the present invention is to provide a mask matching method and system based on face shape recognition to solve the problem that the existing mask matching method mentioned in the above background technology cannot accurately and quickly achieve personalized matching between the mask and the face.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] According to one aspect of the present invention, a mask matching method based on face shape recognition is provided, and the specific steps of the method are as follows:

[0008] Step S1: inputting a human face image into the system, reconstructing a three-dimensional human face image based on a deep learning intelligent face recognition algorithm, and extracting feature information of the three-dimensional human face image;

[0009] Step S2: Calculating the most suitable mask for the face based on the three-dimensional face image feature information and the mask database;

[0010] Step S3: Based on the three-dimensional human face facial image and the mask that best suits the face, a corresponding mask image is generated on the three-dimensional human face facial image and displayed on a user display.

[0011] Furthermore, the intelligent face recognition algorithm in step S1 includes:

[0012] Step S101, face image extraction: detecting a face area from the input face image, and cropping and normalizing it to obtain a standardized face image;

[0013] Step S102, 3D model construction: performing 3D reconstruction on the standardized face image, using deep learning or other technologies to generate a 3D face image, including surface mesh and texture information;

[0014] Step S103, facial feature extraction: performing multi-scale feature extraction on the three-dimensional facial image, and calculating multiple groups of feature vectors to represent the shape and size of the face according to the geometric structure or appearance information of the face;

[0015] Furthermore, the multi-scale feature extraction is a multi-scale feature extraction algorithm based on the curvature of the local surface of the appearance. The algorithm uses a curvature-based feature detector, which can detect significant feature points of the three-dimensional model at different scales, and can detect and extract features such as the contour line, curvature, and concavity of the face; the algorithm uses a feature descriptor based on shape index, which can describe the local geometric shape around the feature point;

[0016] Step S104, feature prediction and fusion: using algorithms such as deep learning to predict and fuse the global feature vectors of the multiple groups of faces, to obtain a group of global feature vectors of the face;

[0017] Step S105, face shape matching judgment: use a classification algorithm to classify or match a set of global feature vectors of the face, and judge which type of face shape the face belongs to and the various sizes of the face according to predefined standards or databases.

[0018] Furthermore, the mask database described in step S2 contains mask data of various types and specifications, which can be updated and expanded according to different scenarios and needs. Each mask data contains information such as the size, shape, material, filtering effect, breathability, etc. of the mask, and is classified according to different scenarios and needs;

[0019] Furthermore, the calculation of the most suitable mask for the face described in step S2 is to compare and calculate the facial features extracted by the intelligent face recognition algorithm with the mask data in the mask database, specifically focusing on facial contours and facial curves, based on facial feature parameters such as morphological face length, face width, width between the two tragus, width between the two lower collar corners, nose height, nose width and arc length under the collar between the two tragus points, and performing comparison calculations of different sizes with masks in the mask database. The comparison parameters include mask length, width, curvature, expansion ratio and other parameters, which can be adjusted according to actual needs to obtain the recommendation result of the mask that best suits the face in the mask database.

[0020] According to one aspect of the present invention, a mask matching system based on face shape recognition is provided, the system comprising: a feature information extraction module, a best matching calculation module and a mask virtual wearing module, wherein:

[0021] A feature information extraction module, based on a deep learning intelligent face recognition algorithm, reconstructs a three-dimensional face image from the face image input into the system, and extracts feature information of the three-dimensional face image;

[0022] The best matching calculation module calculates the most suitable mask for the face based on the facial image feature information of the three-dimensional face and the mask database;

[0023] The mask virtual wearing module generates a corresponding mask image on the three-dimensional face image according to the mask recommendation result of the best matching calculation module, and displays it on the user's screen.

[0024] Furthermore, the feature information extraction module includes:

[0025] The face image extraction unit detects the face area from the input face image, and crops and normalizes the face area to obtain a standardized face image;

[0026] A three-dimensional model building unit: performs three-dimensional reconstruction on the standardized face image, and generates a three-dimensional face image of the face, including surface mesh and texture information, by using deep learning or other technologies;

[0027] A face feature extraction unit: performs multi-scale feature extraction on the three-dimensional face image of the face, and calculates multiple groups of feature vectors of different scales according to the geometric structure or appearance information of the face to represent the shape and size of the face;

[0028] Feature prediction and fusion unit: for the plurality of feature vectors, using algorithms such as deep learning, the feature vectors are predicted and fused to obtain a set of global feature vectors;

[0029] Face shape matching judgment unit: classifies or matches the set of global feature vectors, and judges the type of face shape and the size of the face according to predefined standards or databases.

[0030] It can be seen from the above technical solutions that the present invention has at least the following advantages and positive effects compared with the prior art:

[0031] (1) The present invention uses an intelligent recognition algorithm to extract facial features, and combined with the mask database information, can directly match the most suitable mask for the face, and display it on the user display interface, so that the user can intuitively see the fit between the mask and the face and the wearing effect, saving a lot of resources. It is a new intelligent testing method, which is more convenient and simple than manually trying on masks and using special instruments to select suitable masks, and is intelligent and efficient.

[0032] (2) The present invention adopts a multi-scale feature extraction method when extracting facial features using an intelligent recognition algorithm. By extracting facial feature vector groups of different scales and predicting and fusing the vector groups, a set of complete and accurate facial feature vectors is obtained, so that the matching between the face and the mask is more accurate, and a more appropriate mask recommendation result can be obtained in the matching calculation;

[0033] (3) The present invention proposes an update and expansion of the mask database, and the mask data can be adjusted according to actual needs. When calculating the matching and comparison of facial features and masks, the calculation of various sizes is considered, and it can also be adjusted according to actual needs. The design is relatively flexible, which helps to improve the accuracy of mask matching tests. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for the specific embodiments or the description of the prior art. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn according to the actual scale.

[0035] Figure 1 A flowchart of a mask matching method based on intelligent face recognition according to an embodiment of the present invention is shown;

[0036] Figure 2 A schematic diagram showing a flow chart of an intelligent face recognition algorithm according to an embodiment of the present invention is shown;

[0037] Figure 3 A structural diagram of a mask matching system based on intelligent face recognition according to an embodiment of the present invention is shown;

[0038] Figure 4A flowchart of multi-scale facial feature extraction of a human face according to an embodiment of the present invention is shown;

[0039] Figure 5 A schematic diagram showing multi-scale extraction of a group of global feature vectors of a human face according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0040] In order to more clearly explain the purpose, technical solutions and advantages of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. The example implementation methods can be implemented in various forms and should not be understood as being limited to the examples described herein. On the contrary, these implementation methods are provided to make the present invention more comprehensive and complete, and to fully convey the concepts of the example implementation methods to those skilled in the art.

[0041] In addition, the described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present invention. However, those skilled in the art will appreciate that the technical solution of the present invention can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring various aspects of the present invention.

[0042] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0043] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0044] The present invention is described in detail below in conjunction with specific embodiments:

[0045] Example 1

[0046] As attached Figure 1 As shown, an embodiment of the present invention provides a mask matching method based on intelligent face recognition, and the specific steps of the method are as follows:

[0047] Step S1, inputting a human face image into the system, and extracting feature information of the human face image based on an intelligent face recognition algorithm based on deep learning;

[0048] It should be noted that, as attached Figure 2 As shown, the intelligent face recognition algorithm described in step S1, the specific steps of the method are as follows:

[0049] Step S101, face image extraction: detecting the face area from the input face image, cropping and normalizing it to obtain a standardized face image;

[0050] Specifically, in this embodiment, the face detection algorithm and Haar Cascade classifier in the OpenCV library of Python are used to identify the face area in the input image and crop the face image, and then the face image is normalized and scaled using image processing technology so that the image has a fixed size and direction, and images of different scales are comparable, which is convenient for subsequent processing;

[0051] Step S102, 3D model construction: perform 3D reconstruction on the standardized face image, and use deep learning or other technologies to generate a 3D mesh model of the face, that is, a 3D face image, including surface mesh and texture information;

[0052] Specifically, the SFM algorithm or tracking algorithm in Python's OpenCV library can be used to process standardized face images to obtain face point cloud data, then the point cloud data can be spliced ​​into a three-dimensional grid, and finally texture mapping can be performed to obtain a three-dimensional face model; deep learning technology, such as 3D Face Reconstruction Network, can also be used to generate three-dimensional facial images from two-dimensional images;

[0053] Step S103, face feature extraction: performing multi-scale feature extraction on the three-dimensional face image, and calculating a set of global feature vectors to represent the shape and size of the face according to the geometric structure or appearance information of the face;

[0054] Specifically, as attached Figure 4As shown, the multi-scale feature extraction of the three-dimensional face image in step S103 is a multi-scale feature extraction algorithm based on the curvature of the local surface of the appearance. The algorithm uses a curvature-based feature detector, which can detect the significant feature points of the three-dimensional model at different scales, and can detect and extract the contour line, curvature, concavity and other features of the face; the algorithm uses a shape-indexed feature descriptor, which can describe the local geometric shape around the feature point; by using the multi-scale feature extraction algorithm, representative facial geometric feature points on the three-dimensional face image can be found, and the accuracy and precision of face recognition can be improved, including the following steps:

[0055] Step S1031, establishing a three-dimensional image Gaussian pyramid: defining multiple different scale space levels, and defining different filter sizes for each level; in the three-dimensional face image, using a three-dimensional discrete Gaussian kernel to perform Gaussian convolution processing on the three-dimensional face image in each scale space in the three dimensions of x, y, and z, and obtaining a smooth three-dimensional face image in each scale space, and using discrete Gaussian kernels of different sizes and different standard deviations to eliminate scale differences and generate three-dimensional face images with different smoothing effects; the three-dimensional Gaussian kernel is a cubic matrix that defines a three-dimensional Gaussian distribution, in which the main parameter is the standard deviation; the obtained smooth three-dimensional face image is doubled as the first layer of the first group of the Gaussian pyramid, and the image of the first layer of the first group is used as the second layer of the first group of the pyramid after Gaussian convolution, and the Gaussian convolution function is:

[0056]

[0057] Among them, x, y, and z represent the three dimensions of the image respectively. The parameter σ0 represents the smoothing factor used for the first Gaussian convolution to smooth the image, which takes a fixed value of 1.6. Multiply σ0 by a proportional coefficient k to get a new smoothing factor σ=k*σ0, which is used to smooth the first group of second-layer images. The smoothed image is used as the first group of third layers of the pyramid. Repeat this process to obtain the first group of L-layer images of the pyramid. In the same group, the size of each layer of the image is the same, but the smoothing factor is different. The corresponding smoothing factors for each layer are: 0, σ0, kσ0, k 2 σ0,k 3 σ0……k L-2 σ0;

[0058] The third-to-last layer image of the first group of the pyramid is downsampled by a scale factor of 2, and the obtained image is used as the first layer of the second group. Then, the first layer image of the second group is Gaussian smoothed with a smoothing factor of σ0 to obtain the second layer of the second group. The second group is smoothed in the same way as the Gaussian convolution operation of the first group to obtain the L-layer image of the second group. In terms of size, the second group is half of the first group of images. Repeat the previous steps to obtain a total of O groups, each with L layers, and a total of O*L images, which together constitute a Gaussian pyramid of a three-dimensional facial image. In the Gaussian pyramid, the image sizes of different layers in the same group are the same, and the Gaussian smoothing factor σ of the image of the next layer is k times the smoothing factor of the image of the previous layer. In different groups, the first image of the next group is downsampled by half of the third-to-last layer image of the previous group, and the image size is half of the previous group. Images of different scales have different resolutions. The higher the level in the pyramid, the smaller the image and the lower the resolution, and the feature maps of all levels have strong semantic information.

[0059] Step S1032, extracting local features: using the curvature calculation method of Mean Curvature, the average curvature of a point in the surface geometry can be calculated, and curvature-based feature extraction and Reeb graph construction are performed on each group and each layer of the image in the Gaussian pyramid; the Reeb graph is segmented to divide the image into several regions, each region representing a local geometric feature of a face, especially the part that needs to wear a mask, such as the nose, mouth, chin, cheek, cheekbone, ear, etc.; using a spherical spectrum analysis algorithm, such as Spherical Harmonic Transform, spherical spectrum analysis is performed on each sub-part of each group and each layer in the Gaussian pyramid to extract the local geometric features of the corresponding face;

[0060] Step S1033, filtering multi-scale local features: based on Gaussian feature descriptors and normal vector feature descriptors, the extracted local geometric features of the face in different scale spaces are analyzed and calculated, and similar feature descriptors at different scales are matched by calculating indicators such as the distance and similarity between similar features. The matching process can adopt a matching algorithm based on image features to match and screen representative facial geometric features in different scale spaces, so as to find representative facial geometric feature points. The matching of facial features can improve the robustness and accuracy of the algorithm.

[0061] Specifically, the Gaussian feature descriptor can represent the texture information of the image surface, and the normal vector feature descriptor can represent the geometric shape information of the image surface, which can be combined to better describe the facial image features of the human face; based on the extracted local geometric features, the Gaussian descriptor and the normal vector descriptor are used to respectively analyze and calculate the morphological face length, face width, width between the two tragus, width between the two lower collar corners, nose height, nose width and arc length under the collar between the two tragus points of the three-dimensional facial image, and combine them to represent a global feature vector; through the calculation of the feature descriptor, it is possible to filter and match the facial geometric features in space and scale, and filter out some unstable and redundant features. In this embodiment, the facial features that will not be used to contact the mask when the mask is worn are filtered out for each group and each layer of the image in the Gaussian pyramid, and the facial features in the mask wearing area are retained to obtain multiple groups of global feature vectors of the face; the descriptor is a method of converting each feature point into a vector, and a vector with a certain regularity is generated by calculating the pixel points around the feature point to describe the feature of the feature point, thereby reducing the redundancy of the feature point and improving the matching accuracy;

[0062] Step S104, feature prediction and fusion: using algorithms such as deep learning to predict and fuse the global feature vectors of the multiple groups of faces, to obtain a group of global feature vectors of the face;

[0063] Specifically, the global feature vectors of multiple groups of faces are extracted based on the multi-scale feature extraction algorithm, and feature prediction is performed using a convolutional neural network (CNN) model or other deep learning models. In the present embodiment, the feature prediction adopts a convolutional neural network (CNN) model, and a CNN prediction model is defined, wherein the CNN model is composed of multiple convolutional layers and pooling layers, wherein the convolutional layer is responsible for extracting the features of the input data, and the pooling layer is responsible for reducing the dimension and data volume of the feature map. The process of alternating between the convolutional layer and the pooling layer is called the feature extraction process, wherein the convolutional layer extracts the feature vector of each scale, and the pooling layer is mainly used to reduce the size of the feature map to make it easier to process and calculate. During feature prediction, the CNN model accepts the overall feature vector as input, and then passes through several layers of convolution and pooling. CNN passes multi-scale features to multiple fully connected or convolutional layers to generate prediction results, which can be converted into prediction labels by classifiers such as softmax or SVM. The prediction model is optimized and trained using the stochastic gradient descent algorithm SGD. At the end of each Epoch, the loss function and the prediction accuracy on the training set are calculated, and the best prediction model is recorded. In order to prevent overfitting, some common techniques are used in the CNN network training process, such as dropout. In addition, the training model needs to be applied to the validation set for verification to see if the model has overfitting problems. Some regularization methods, such as L1 regularization and L2 regularization, can also be used to further optimize the prediction results.

[0064] Furthermore, since the three-dimensional face image may have problems such as scale change and rotation, it is necessary to use feature fusion to comprehensively consider the gradual cracks and robustness of different features of the input data. Feature fusion can adopt different methods, one of which is to perform weighted averaging of feature vectors of multiple scales according to certain weights; by taking the feature vectors after feature fusion as input, a set of global feature vectors is obtained to represent the global feature vectors of the face, so as to perform the shape recognition task of the three-dimensional model;

[0065] Step S105, face shape matching judgment: use a classification algorithm to classify or match a group of global feature vectors of the face, and use a measurement method such as Euclidean distance to compare the extracted global feature vector of the face with the face shape in the database according to a predefined standard or database, calculate the shape similarity between different models, and judge the type of face shape of the face, such as round, square, long, etc., and the various sizes of the face, such as width, height, length, etc., based on the global feature vector of the face.

[0066] Step S2, calculating the most suitable mask for the face based on the three-dimensional face image feature information and the mask database;

[0067] It should be noted that the mask database described in step S2 contains mask data of various types and specifications, which can be updated and expanded according to different scenarios and needs. Each mask data contains information such as the size, shape, material, filtering effect, air permeability, etc. of the mask, and is classified according to different scenarios and needs. For example, due to different working environments, the standards required for masks are different, and masks can be classified according to the working environment requirements;

[0068] Specifically, when there is a demand for new mask models and sizes, new mask data can be collected by purchasing, drawing on other databases, collecting on the Internet, etc., and preprocessing the mask data to ensure that the mask image is consistent with the existing data in format and features. Then, according to the scene and needs, the new mask data is classified and saved in the mask database. Specifically, the technical standards, functional types, sizes, shapes, materials, colors and other information of the masks need to be saved for subsequent analysis and recommendation. After updating and expanding the mask database, the performance of the database needs to be tested to ensure that the newly added mask data can be correctly retrieved and participate in the comparison calculation. Specifically, regular performance tests can be performed to monitor the database's accuracy, recall rate, response speed and other indicators. When higher requirements are required, the database can also be trained and optimized.

[0069] It should be noted that the calculation of the most suitable mask for the face described in step S2 is based on the face shape calculation matching result, and the facial information extracted by the intelligent face recognition algorithm and the mask data of the face shape suitable for facial features in the mask database are compared and calculated in various sizes, specifically based on facial contour and facial curve, based on facial feature parameters such as morphological face length, face width, width between the two tragus, width between the two lower collar corners, nose height, nose width and arc length under the collar between the two tragus points, and masks of different sizes in the mask library are compared and calculated. The comparison parameters include mask length, width, curvature, expansion ratio and other parameters, which can be adjusted according to actual needs to obtain the recommendation result of the mask most suitable for the face in the mask database.

[0070] Step S3, based on the three-dimensional human face image and the mask that best suits the face, a corresponding mask image is generated on the three-dimensional human face image and displayed on a user display, so that the mask wearing effect, that is, the fit between the mask and the human face can be intuitively displayed, and the user can intuitively see the effect, which is convenient for the user to screen the selection of masks, and greatly saves people's time in selecting masks that suit their faces;

[0071] Furthermore, the three-dimensional facial image of the human face can also be adjusted according to actual needs, such as making corresponding modifications to the expression, facial structure, etc. of the three-dimensional facial image according to the needs of occupations or different occasions, which can ensure that the mask and the face of the test face still have a good fit when doing some daily actions while wearing a mask, and can better protect the health of the mask wearer; specifically, for example, people usually perform 7 actions after wearing a mask: normal breathing in a normal standing posture, deep breathing during heavy physical labor, turning the head left and right when breathing normally in a normal standing posture, looking up and lowering the head when breathing normally in a normal standing posture, talking in a normal standing posture so that others can clearly hear the text content, smiling or frowning to create a certain gap between the mask and the face, and bending over when breathing normally in a standing posture; the three-dimensional facial images of the human face in these different actions can be used for mask matching calculations, the wearing effect can be displayed in the user interface, and the fit between the face and the mask under different actions can be observed, so that the mask matching results are more accurate and practical, and have a wider applicability.

[0072] In this embodiment, by using an intelligent algorithm to implement the mask matching effect test, the mask that best suits the face can be directly matched and displayed on the user display interface, so that the user can intuitively see the degree of fit between the mask and the face and the wearing effect, saving a lot of resources. It is a new type of intelligent mask matching method, which is more convenient and simple than manually trying on masks and using special instruments to select suitable masks, and is intelligent and efficient.

[0073] Example 2

[0074] As attached Figure 3 As shown, Embodiment 2 of the present invention provides a mask matching system based on face shape recognition, comprising:

[0075] S201, inputting a human face image into the system, and extracting feature information of the human face image based on an intelligent face recognition algorithm based on deep learning;

[0076] S202, a mask database module is used to provide mask data of various types and specifications, and can also update and expand the mask data according to different scenarios and needs. Each mask data includes information such as the size, shape, and material of the mask;

[0077] S203, a mask matching recommendation module, which is used to match and calculate the most suitable mask type and specification for the face according to the facial information extracted by the intelligent face recognition algorithm module and the mask data in the mask database module, and give corresponding mask recommendation results;

[0078] S204, a mask virtual wearing module, is used to generate a corresponding mask image on the input face image according to the recommendation result of the mask matching recommendation module, and display it on the user's screen, so that the user can intuitively see the fit between the mask and the face and the wearing effect.

[0079] It should be noted that the intelligent face recognition algorithm module includes the following units:

[0080] S2011, face image extraction unit: used to detect the face area from the input face image, and crop and normalize it to obtain a standardized face image;

[0081] S2012, a three-dimensional model construction unit: configured to perform three-dimensional reconstruction on the standardized face image, and generate a three-dimensional model of the face, including surface mesh and texture information, by using deep learning or other technologies;

[0082] S2013, a face feature extraction unit: used to extract features from the three-dimensional model of the face, and calculate a set of feature vectors to represent the shape and size of the face according to the geometric structure or appearance information of the face;

[0083] S2014, face shape matching judgment unit: used to classify or match the feature vector of the face, and judge the type of face shape and the size of the face according to a predefined standard or database.

[0084] It should be noted that the above modules correspond to the steps in Example 1, and the examples and application scenarios implemented by the above modules and the corresponding steps are the same, but are not limited to the contents disclosed in the above Example 1. It should be noted that the above modules, as part of the system, can be executed in a computer system such as a set of computer executable instructions.

[0085] In further embodiments, as shown in the attached Figure 5 As shown, it is assumed that each layer of image in the figure is a three-dimensional face image;

[0086] Specifically, the facial feature extraction in the intelligent face recognition algorithm is to first downsample the lowest-level three-dimensional face image into multiple groups of multi-layer gradually reduced images, that is, images that are reduced toward the upper end of the pyramid in the figure, to obtain a Gaussian pyramid;

[0087] Perform local feature extraction and filtering on each group and layer of the images in the Gaussian pyramid to obtain the global feature vectors of multiple groups of faces shown in the second step of the figure;

[0088] For the global feature vectors of multiple groups of faces, a convolutional neural network (CNN) is used to predict the feature vectors and extract the feature vectors of each scale, i.e., the feature vectors of different scales shown in the third step of the figure;

[0089] Based on the feature vectors of different scales obtained by feature prediction, feature fusion is performed to obtain a set of global feature vectors to represent the features of the face.

[0090] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. This application is intended to cover any modification, use or adaptation of the present invention, which follows the general principles of the present invention and includes common knowledge or customary techniques in the art that are not disclosed by the present invention. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present invention are indicated by the claims. It should be understood that the present invention is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A mask matching method based on face shape recognition, characterized in that: include: Input a human face image into the system, reconstruct a three-dimensional human face image based on a deep learning intelligent face recognition algorithm, and extract feature information of the three-dimensional human face image; wherein the intelligent face recognition algorithm includes: Processing the input human face image to obtain a standardized human face image; Performing three-dimensional reconstruction on the standardized face image to generate a three-dimensional face image, and performing multi-scale face feature extraction; Determining the face shape and face size of a person based on the extracted multi-scale face features and a predefined standard or database; The multi-scale face feature extraction is to use deep learning technology to generate a three-dimensional mesh model of the face, including surface mesh and texture information, perform multi-scale feature extraction on the three-dimensional model of the face according to the geometric structure or appearance information of the face, and calculate multiple groups of feature vectors at different scales to represent the shape and size of the face; the multi-scale feature extraction is a multi-scale feature extraction algorithm based on local surface curvature, using a curvature-based feature detector, which can detect significant feature points of three-dimensional facial images at different scales, and uses a shape-indexed feature descriptor, which can describe the local geometric shape around the feature points, including the following steps: Define multiple different scale space levels, and define different filter sizes for each level; use a three-dimensional discrete Gaussian kernel to perform Gaussian convolution processing on the three-dimensional face image in each scale space in the three dimensions of x, y, and z, and obtain a smooth three-dimensional face image in each scale space; The obtained smooth three-dimensional facial image is doubled and used as the first layer of the first group of the Gaussian pyramid. The image of the first layer of the first group is used as the second layer of the first group of the pyramid after Gaussian convolution. The Gaussian convolution function is: Among them, x, y, and z represent the three dimensions of the image respectively, and the parameter σ0 represents the smoothing factor used for the first Gaussian convolution to smooth the image, which takes a fixed value of 1.6; σ0 is multiplied by a proportional coefficient k to obtain a new smoothing factor σ=k*σ0, which is used to smooth the second layer image of the first group, and the smoothed image is used as the third layer of the first group of the pyramid; this is repeated to obtain the L layer image of the first group of the pyramid; The third-to-last layer image of the first group of the pyramid is downsampled with a scale factor of 2, and the obtained image is used as the first layer of the second group; the first layer image of the second group is Gaussian smoothed with a smoothing factor of σ0 to obtain the second layer of the second group, and the previous steps are repeated to obtain a total of O groups, each with L layers, and a total of O*L images, to form a Gaussian pyramid of a three-dimensional facial image; Performing curvature-based feature extraction and Reeb graph construction on each group of each layer of images in the Gaussian pyramid, and segmenting the Reeb graph to divide the image into a plurality of regions, each region representing a local geometric feature of a face; Using the spherical spectrum analysis algorithm, a spherical spectrum analysis is performed on each sub-part of each group and each layer in the Gaussian pyramid to extract the local geometric features of the corresponding face; Based on Gaussian feature descriptors and normal vector feature descriptors, local geometric features of the face are analyzed and calculated, and representative facial geometric features in different scale spaces are matched and screened to obtain multiple groups of global feature vectors of the face; For the global feature vectors of the multiple groups of faces, a deep learning algorithm is used to predict and fuse the feature vectors to obtain a group of global feature vectors of the faces; Using a classification algorithm to classify or match the global feature vectors of the group of faces, using a Euclidean distance measurement method to compare the extracted global feature vectors of the face with the face shapes in the database according to a predefined standard or database, calculating the shape similarity between different models, and judging the face shape and face size based on the global feature vectors of the face; Calculate the most suitable mask for the face based on the three-dimensional facial image feature information and the mask database; Based on the three-dimensional human face facial image and the mask that best suits the face, a corresponding mask image is generated on the three-dimensional human face facial image and displayed on a user display.

2. A mask matching method based on face shape recognition according to claim 1, characterized in that: The method for calculating the most suitable mask for the face is to compare and calculate the facial features with the mask data in the mask database to obtain a recommendation result of the most suitable mask for the face.

3. A mask matching method based on face shape recognition according to claim 1, characterized in that: The mask database contains mask data of various types and specifications, which can be updated and expanded according to different scenarios and needs. Each type of mask data contains the size, shape, material, filtering effect, and breathability information of the mask, and is classified according to different scenarios and needs.

4. A mask matching method based on face recognition according to claim 2, characterized in that: The facial features are compared and calculated with the mask data in the mask database, specifically focusing on facial contours and facial curves, based on facial feature parameters such as face length, face width, width between two tragus, width between two lower collar corners, nose height, nose width and arc length under the collar between two tragus points, and comparison calculations of different sizes are performed with masks in the mask database, and the comparison parameters include mask length, width, curvature, and expansion ratio.

5. A mask matching method based on face shape recognition according to claim 1, characterized in that: The processing of the input human face image is to detect the human face area from the input human face image, and to crop and normalize it to obtain a standardized human face image.

6. A mask matching system based on face recognition, characterized in that: include: The feature information extraction module reconstructs a three-dimensional face image based on the face image input into the system by using an intelligent face recognition algorithm based on deep learning, and extracts feature information of the three-dimensional face image; wherein the feature information extraction module comprises: The face image extraction unit detects the face area from the input face image, and crops and normalizes the face area to obtain a standardized face image; A three-dimensional model building unit: performs three-dimensional reconstruction on the standardized face image, and generates a three-dimensional face image of the face, including surface mesh and texture information, by using deep learning technology; The face feature extraction unit performs multi-scale feature extraction on the three-dimensional face image of the face, and calculates multiple groups of feature vectors of different scales according to the geometric structure or appearance information of the face to represent the shape and size of the face; the multi-scale feature extraction is a multi-scale feature extraction algorithm based on the local surface curvature, using a curvature-based feature detector, which can detect significant feature points of the three-dimensional face image at different scales, and uses a shape-indexed feature descriptor, which can describe the local geometric shape around the feature point, including the following steps: Define multiple different scale space levels, and define different filter sizes for each level; use a three-dimensional discrete Gaussian kernel to perform Gaussian convolution processing on the three-dimensional face image in each scale space in the three dimensions of x, y, and z, and obtain a smooth three-dimensional face image in each scale space; The obtained smooth three-dimensional facial image is doubled and used as the first layer of the first group of the Gaussian pyramid. The image of the first layer of the first group is used as the second layer of the first group of the pyramid after Gaussian convolution. The Gaussian convolution function is: Among them, x, y, and z represent the three dimensions of the image respectively, and the parameter σ0 represents the smoothing factor used for the first Gaussian convolution to smooth the image, which takes a fixed value of 1.6; σ0 is multiplied by a proportional coefficient k to obtain a new smoothing factor σ=k*σ0, which is used to smooth the second layer image of the first group, and the smoothed image is used as the third layer of the first group of the pyramid; this is repeated to obtain the L layer image of the first group of the pyramid; The third-to-last layer image of the first group of the pyramid is downsampled with a scale factor of 2, and the obtained image is used as the first layer of the second group; the first layer image of the second group is Gaussian smoothed with a smoothing factor of σ0 to obtain the second layer of the second group, and the previous steps are repeated to obtain a total of O groups, each with L layers, and a total of O*L images, to form a Gaussian pyramid of a three-dimensional facial image; Performing curvature-based feature extraction and Reeb graph construction on each group of each layer of images in the Gaussian pyramid, and segmenting the Reeb graph to divide the image into a plurality of regions, each region representing a local geometric feature of a face; Using the spherical spectrum analysis algorithm, a spherical spectrum analysis is performed on each sub-part of each group and each layer in the Gaussian pyramid to extract the local geometric features of the corresponding face; Based on Gaussian feature descriptors and normal vector feature descriptors, local geometric features of the face are analyzed and calculated, and representative facial geometric features in different scale spaces are matched and screened to obtain multiple groups of global feature vectors of the face; Feature prediction and fusion unit: for the global feature vectors of the multiple groups of faces, a deep learning algorithm is used to predict and fuse the feature vectors to obtain a set of global feature vectors; A face shape matching judgment unit: classifies or matches the set of global feature vectors, and judges the type of face shape and the size of the face according to a predefined standard or database; The best matching calculation module calculates the most suitable mask for the face based on the facial image feature information of the three-dimensional face and the mask database; The mask virtual wearing module generates a corresponding mask image on the three-dimensional face image according to the mask recommendation result of the best matching calculation module, and displays it on the user's screen.

Images