Face key point enhancement method and device, electronic equipment and computer readable medium
By combining the multi-angle shooting of facial images and key point models, the key points of the face are amplified, and the problem of insufficient regional positioning accuracy in the medical beauty industry is solved, achieving higher treatment refinement and reliability.
Patent Information
- Application Number
- CN202510317565.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-18
- Publication Date
- 2025-06-13
AI Technical Summary
In the medical beauty industry, the positioning of the treatment area depends on the experience and skills of the operator, which makes it difficult to achieve high-precision positioning under complex angles or postures, affecting the accuracy of analysis and treatment.
By acquiring multiple face images taken from multiple angles, the original face key points are extracted using the disclosed key point model, and amplifying the preset number of candidate face key points is obtained based on these key points, and key points amplification is performed according to the position matching amplification pattern of candidate key points.
Provide more accurate facial information, achieve more comprehensive and flexible detection and labeling of key points on the face, and significantly improve the refinement and reliability of the treatment process.
Smart Images
Figure CN120148090A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of facial key point detection, and particularly to a facial key point enhancement method, a facial key point enhancement device, an electronic device, and a computer-readable medium. Background Art
[0002] The technology of facial key point detection has been widely used in many scenarios (such as face recognition, artificial intelligence-assisted diagnosis and analysis, etc.). However, in the actual operation of the medical aesthetics industry, the positioning of the treatment area often depends on the experience and skills of the operators. There are inevitably differences among different personnel, and it is difficult to achieve high-precision positioning under complex angles or postures, resulting in the difficulty of stably ensuring the accuracy of analysis and treatment. Summary of the Invention
[0003] In view of the above problems, the present invention is proposed to provide a facial key point enhancement method and a corresponding facial key point enhancement device, an electronic device, and a computer-readable medium that overcome the above problems or at least partially solve the above problems.
[0004] The present invention discloses a facial key point enhancement method, and the method includes:
[0005] Obtain multiple facial images taken from multiple angles;
[0006] Extract the original facial key points from the multiple facial images by using a publicly available key point model, and based on the original facial key points, obtain a preset number of candidate facial key points by selection and amplification;
[0007] Match the amplification mode according to the positions of the newly added facial key points in the candidate facial key points, and perform key point amplification according to the matched amplification mode.
[0008] Optionally, matching the amplification mode according to the positions of the newly added facial key points in the candidate facial key points, and performing key point amplification according to the matched amplification mode includes:
[0009] If the newly added facial key point is located between two points, then taking the two points as a benchmark, calculate a new facial key point by taking a preset fixed coefficient in linear interpolation;
[0010] If the newly added facial key point is not between two points or the accuracy of the reference point is lower than the first preset accuracy threshold, then taking a single point as a benchmark, calculate a new facial key point in combination with the head tilt angle and a preset fixed pixel value;
[0011] If there are no fiducial points with a precision higher than the second preset precision threshold around the newly added facial key points, a fiducial point training dataset is created, and a new facial key point is obtained by fitting the model using the fiducial point training dataset; the second preset precision threshold is smaller than the first preset precision threshold.
[0012] Optionally, the original facial key points are extracted from the multiple facial images using a publicly available key point model, and based on the original facial key points, a preset number of candidate facial key points are obtained by selection and amplification, including:
[0013] The original facial key points are extracted from the multiple facial images using a publicly available key point model, facial key points are selected from the original facial key points, and at the same time, the original facial key points are amplified by fitting to obtain a preset number of candidate facial key points.
[0014] Optionally, if the newly added facial key point is not between two points or the precision of the fiducial point is lower than the first preset precision threshold, a new facial key point is calculated based on a single point, in combination with the head tilt angle and a preset fixed pixel value, including:
[0015] If the newly added facial key point is not between two points or the precision of the fiducial point is lower than the first preset precision threshold, the positions of specific key points in the candidate facial key points are analyzed, the head tilt angle is calculated, and a new facial key point is calculated based on a single point, in combination with the head tilt angle and a preset fixed pixel value.
[0016] Optionally, the positions of specific key points in the candidate facial key points are analyzed, and the head tilt angle is calculated, including:
[0017] Four points at the corners of the eyes are selected from the candidate facial key points and fitted into a straight line using the least squares method, and the tilt angle is converted using the arctangent function and the slope of the straight line to obtain the head tilt angle.
[0018] Optionally, a fiducial point training dataset is created, and a new facial key point is obtained by fitting the model using the fiducial point training dataset, including:
[0019] A circle of fiducial points with a precision not higher than the second preset precision threshold is selected around the newly added facial key point as the training dataset, and using the newly added facial key point as the label, a regression-based machine learning model is used for fitting to obtain a new facial key point; the regression-based machine learning model includes a linear regression model, a support vector machine, and a random forest model.
[0020] Optionally, the multiple angles include a front 0° view, a left 45° view, and a right 45° view.
[0021] The present invention also discloses a facial key point enhancement device, and the device includes:
[0022] A face image acquisition module for acquiring multiple face images captured from multiple angles;
[0023] A candidate key point selection and amplification module for extracting original face key points from the multiple face images using a publicly available key point model, and obtaining a preset number of candidate face key points through selection and amplification based on the original face key points;
[0024] A new key point amplification module for matching an amplification pattern according to the positions of the newly added face key points among the candidate face key points, and performing key point amplification according to the matched amplification pattern.
[0025] Optionally, the new key point amplification module includes:
[0026] A first key point amplification sub-module for, if a newly added face key point is located between two points, taking the two points as a reference and calculating a new face key point by taking a preset fixed coefficient in linear interpolation;
[0027] A second key point amplification sub-module for, if a newly added face key point is not between two points or the accuracy of the reference point is lower than a first preset accuracy threshold, taking a single point as a reference and calculating a new face key point in combination with the head tilt angle and a preset fixed pixel value;
[0028] A third key point amplification sub-module for, if there is no reference point with an accuracy higher than a second preset accuracy threshold around a newly added face key point, making a reference point training data set and obtaining a new face key point by fitting the reference point training data set with a model; the second preset accuracy threshold is less than the first preset accuracy threshold.
[0029] Optionally, the candidate key point selection and amplification module includes:
[0030] A candidate key point selection and amplification sub-module for extracting original face key points from the multiple face images using a publicly available key point model, selecting face key points from the original face key points, and simultaneously amplifying the original face key points by fitting to obtain a preset number of candidate face key points.
[0031] Optionally, the second key point amplification sub-module includes:
[0032] A tilt angle weighting unit for, if a newly added face key point is not between two points or the accuracy of the reference point is lower than a first preset accuracy threshold, analyzing the positions of specific key points among the candidate face key points, calculating the head tilt angle, and taking a single point as a reference and calculating a new face key point in combination with the head tilt angle and a preset fixed pixel value.
[0033] Optionally, the key point amplification unit includes:
[0034] A head tilt angle calculation unit is configured to select four points at the eye corners from the candidate facial key points, fit them into a straight line by using the least squares method, convert them into a tilt angle by using the arctangent function and the slope of the straight line, and obtain the head tilt angle.
[0035] Optionally, the third key point amplification module includes:
[0036] A reference point training set fitting unit is configured to select a circle of reference points with an accuracy not higher than a second preset accuracy threshold around the newly added facial key points as a training data set, use the newly added facial key points as labels, and perform fitting by using a regression-based machine learning model to obtain new facial key points; the regression-based machine learning model includes a linear regression model, a support vector machine, and a random forest model.
[0037] Optionally, the multi-angles include a front 0° view, a left 45° view, and a right 45° view.
[0038] The present invention also discloses an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory complete communication with each other through the communication bus;
[0039] The memory is used for storing a computer program;
[0040] When the processor is configured to execute the program stored on the memory, it implements the facial key point enhancement method as described in the present invention.
[0041] The present invention also discloses one or more computer-readable media, on which instructions are stored, and when executed by one or more processors, cause the processors to execute the facial key point enhancement method as described in the present invention.
[0042] The present invention has the following advantages:
[0043] In the facial key point enhancement method of the present invention, multiple facial images taken from multiple angles are obtained, original facial key points are extracted from the multiple facial images by using a publicly available key point model, and based on the original facial key points, a preset number of candidate facial key points are obtained by selection and amplification. An amplification mode is matched according to the positions of the newly added facial key points in the candidate facial key points, and key point amplification is performed according to the matched amplification mode. Through the facial key point enhancement method of the present invention, more accurate facial information can be provided, more comprehensive and flexible detection and annotation of facial key points can be realized, and thus the refinement and reliability of the treatment process can be significantly improved. Description of the Drawings
[0044] Figure 1 is a flowchart of the steps of a facial key point enhancement method provided by an embodiment of the present invention;
[0045] Figure 2 It is a computer vision processing framework diagram corresponding to the facial key point enhancement method of the present invention;
[0046] Figure 3 It is a structural block diagram of a facial key point enhancement device provided by an embodiment of the present invention;
[0047] Figure 4 It is a block diagram of an electronic device provided by an embodiment of the present invention;
[0048] Figure 5 It is a schematic diagram of a computer-readable medium provided by an embodiment of the present invention. Specific embodiments
[0049] To make the above objects, features, and advantages of the present invention more obvious and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0050] Refer to Figure 1 , which shows a step flow chart of a facial key point enhancement method provided by an embodiment of the present invention, and specifically may include the following steps:
[0051] Step 101, obtain multiple facial images taken from multiple angles;
[0052] Step 102, extract original facial key points from the multiple facial images using a publicly available key point model, and based on the original facial key points, obtain a preset number of candidate facial key points by selection and amplification;
[0053] Step 103, match the amplification pattern according to the positions of the newly added facial key points in the candidate facial key points, and perform key point amplification according to the matched amplification pattern.
[0054] In the present invention, first, multiple facial images taken from multiple angles can be obtained. For example, the facial images can include facial data taken from three angles, namely, the front face at 0°, the left face at -45°, and the right face at 45°. Then, two publicly available key point models, MediaPipe and SPIGA, can be used for these three facial photos to extract 478 and 68 facial key points respectively. Since the position accuracy of the key points obtained from the middle view is insufficient, the key points of the side face will be selected from the left and right views. For example, a total of 114 facial key points can be selected and amplified from these three pictures.
[0055] The present invention sets three basic amplification modes for the possible positions of newly added facial key points. These three basic amplification modes will be combined in actual operation. Specifically, first, determine the positions of each newly added facial key point among the candidate facial key points, then match the amplification mode based on the positions of each newly added facial key point, and use the matched amplification mode to perform key point amplification. Further, first add non-existent key points in the amplification mode, and then amplify multiple times based on the newly added key points to finally obtain accurate key point data customized with the personal data of the person being photographed.
[0056] It can be seen that through the facial key point enhancement method of the present invention, more accurate facial information can be provided, and more comprehensive and flexible detection and annotation of facial key points can be realized, thereby significantly improving the refinement and reliability of the treatment process.
[0057] In an alternative embodiment of the present invention, the original facial key points are extracted from the multiple facial images using a publicly available key point model, and based on the original facial key points, a preset number of candidate facial key points are obtained through selection and amplification, including:
[0058] The original facial key points are extracted from the multiple facial images using a publicly available key point model, facial key points are selected from the original facial key points, and at the same time, the original facial key points are amplified by fitting to obtain a preset number of candidate facial key points.
[0059] In this embodiment, the MediaPipe key point module and SPIGA have 478 and 68 facial key points respectively. The required target points can be selected from these key points as candidate facial key points, such as the key points corresponding to the corners of the mouth and the corners of the eyes.
[0060] On the premise that the MediaPipe and SPIGA key points cannot meet the requirements, the amplification method is to amplify the existing MediaPipe and SPIGA key points by fitting.
[0061] It can be seen that through this alternative embodiment, the original facial key points can be extracted from the publicly available key point model, and the key points can be selected and amplified according to specific requirements, so as to obtain a preset number of candidate facial key points. This method can not only make full use of the existing key point model, but also make up for the shortage of the existing key point quantity through the fitting amplification method, providing the required key point basis for further amplifying facial key points, so as to improve the accuracy and applicability of facial key point detection.
[0062] In an alternative embodiment of the present invention, the amplification mode is matched according to the positions of each newly added facial key point among the candidate facial key points, and key point amplification is performed according to the matched amplification mode, including:
[0063] If the newly added facial key point is located between two points, then using the two points as a reference, a preset fixed coefficient is taken in linear interpolation to calculate the new facial key point;
[0064] If the newly added facial key point is not between two points or the accuracy of the reference point is lower than the first preset accuracy threshold, then using a single point as a reference, combining the head tilt angle and a preset fixed pixel value to calculate the new facial key point;
[0065] If there is no reference point around the newly added facial key point with an accuracy higher than the second preset accuracy threshold, then a reference point training data set is made, and the model is fitted using the reference point training data set to obtain the new facial key point; the second preset accuracy threshold is less than the first preset accuracy threshold.
[0066] In this embodiment, there are three basic amplification modes. First, if the newly added target point is between two points, then using the two points as a reference, a fixed coefficient is taken in linear interpolation to calculate the newly added point. Second, if the newly added target point is not between two points or the accuracy of the reference point is not high, then using a single point as a reference, combining the tilt angle and the fixed pixel value to add a key point. Third, if there is no clear reference point around the newly added target point, then a reference point training data set is made, and the model is fitted using the reference point training data set to add a key point.
[0067] In the first amplification mode, different coefficients are determined according to the requirements of different newly added points. For example, A = 0.3*B + 0.7*C, D = 0.6*E + 0.4*F, which are empirical values.
[0068] In the second amplification mode, by introducing the calculation of the head tilt degree and combining the weighted average strategy, the key points are amplified, so as to improve the adaptability and positioning accuracy for different head angles. Specifically, a single high-precision reference point around the newly added target point can be selected, for example, it is required to meet a confidence level higher than a preset threshold. Then, the facial geometric transformation matrix is calculated through the head tilt angle, and the local coordinate system of the reference point is adjusted according to the tilt angle (such as rotation or translation). In the transformed coordinate system, a fixed pixel value is offset along a preset direction (such as horizontal or vertical) to generate a new key point. As for the pixel value, different fixed pixel values can be determined according to the requirements of different newly added points, which are empirical values.
[0069] Refer to Figure 2, the facial key point enhancement method of the present invention mainly includes Google's computer vision processing framework MediaPipe key point module, open-source key point algorithm SPIGA key point module, head tilt module, and weighting module. Among them, the MediaPipe key point module and SPIGA key point are used to detect the key point data of the picture; the head tilt module is used to calculate the tilt angle; the weighting module is used to amplify the number of key points, and there are the above three basic amplification modes in the weighting module.
[0070] In an alternative embodiment of the present invention, if the newly added facial key point is not between two points or the accuracy of the reference point is lower than the first preset accuracy threshold, then taking a single point as the reference, combining the head tilt angle and the preset fixed pixel value to calculate the new facial key point, including:
[0071] If the newly added facial key point is not between two points or the accuracy of the reference point is lower than the first preset accuracy threshold, then analyze the position of specific key points in the candidate facial key points, calculate the head tilt angle, and taking a single point as the reference, combine the head tilt angle and the preset fixed pixel value to calculate the new facial key point.
[0072] Different from the existing method that only relies on the results output by the key point network, the present invention not only based on the original key points detected by the network, but also calculates the tilt degree of the human face by analyzing the position of specific key points, and then combines the tilt angle to weight and amplify the new key points. This method can still maintain a high detection accuracy under complex head postures, thereby improving the accuracy and stability of face positioning and analysis.
[0073] In an alternative embodiment of the present invention, analyze the position of specific key points in the candidate facial key points, calculate the head tilt angle, including:
[0074] Select four points at the corners of the eyes from the candidate facial key points and fit them into a straight line by the least square method, and use the arctangent function and the slope of the straight line to convert it into a tilt angle to obtain the head tilt angle.
[0075] In this embodiment, the head tilt module can select four points at the corners of the eyes from the candidate facial key point data, fit them into a straight line by the least square method, and then use the arctangent function and the slope of the straight line to convert it into a tilt angle.
[0076] In an alternative embodiment of the present invention, make a reference point training data set, and use the reference point training data set to fit the model to obtain new facial key points, including:
[0077] Select a circle of reference points with a precision not higher than the second preset precision threshold around the newly added facial key points as the training data set, and use the newly added facial key points as labels, and use a regression-based machine learning model to fit to obtain new facial key points; the regression-based machine learning model includes a linear regression model, a support vector machine, and a random forest model.
[0078] If there are no clear reference points around the newly added target point, then select a circle of fuzzy reference points around the target point as the training set, such as about 4-8 points, use the target point as the label, and then use a regression-based machine learning model, such as algorithms like linear regression, support vector machine, and random forest, to fit the model.
[0079] In an alternative embodiment of the present invention, the data of specific facial parts can be calculated through key points, such as the length and width of the eyes, the distance from the upper lip to the tip of the nose, etc. Replace the fixed pixel values with this data.
[0080] It should be noted that for the method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the embodiments of the present invention are not limited by the described action sequence, because according to the embodiments of the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.
[0081] Refer to Figure 3 , which shows a structural block diagram of a facial key point enhancement device provided in an embodiment of the present invention, and specifically may include the following modules:
[0082] A facial image acquisition module 301, configured to acquire a plurality of facial images captured from multiple angles;
[0083] A candidate key point selection and amplification module 302, configured to extract original facial key points from the plurality of facial images by using a publicly available key point model, and obtain a preset number of candidate facial key points through selection and amplification based on the original facial key points;
[0084] A new key point amplification module 303, configured to match an amplification pattern according to the positions of the newly added facial key points in the candidate facial key points, and perform key point amplification according to the matched amplification pattern.
[0085] Optionally, the new key point amplification module includes:
[0086] A first key point amplification sub-module, configured to, if the newly added facial key point is located between two points, use the two points as a reference, and calculate a new facial key point by taking a preset fixed coefficient in linear interpolation;
[0087] The second key point amplifying sub-module is used to calculate new facial key points based on a single point, in combination with the head tilt angle and a preset fixed pixel value, if the newly added facial key point is not between two points or the accuracy of the reference point is lower than the first preset accuracy threshold.
[0088] The third key point amplifying sub-module is used to create a reference point training data set and obtain new facial key points by fitting the model with the reference point training data set if there is no reference point with an accuracy higher than the second preset accuracy threshold around the newly added facial key point; the second preset accuracy threshold is less than the first preset accuracy threshold.
[0089] Optionally, the candidate key point selection and amplification module includes:
[0090] The candidate key point selection and amplifying sub-module is used to extract the original facial key points from the multiple facial images using a publicly available key point model, select facial key points from the original facial key points, and at the same time amplify the original facial key points by fitting to obtain a preset number of candidate facial key points.
[0091] Optionally, the second key point amplifying sub-module includes:
[0092] The tilt angle weighting unit is used to analyze the positions of specific key points in the candidate facial key points, calculate the head tilt angle, and calculate new facial key points based on a single point, in combination with the head tilt angle and a preset fixed pixel value, if the newly added facial key point is not between two points or the accuracy of the reference point is lower than the first preset accuracy threshold.
[0093] Optionally, the key point amplifying unit includes:
[0094] The head tilt angle calculation unit is used to select four points at the corners of the eyes from the candidate facial key points and fit them into a straight line using the least squares method, and convert them into a tilt angle using the arctangent function and the slope of the straight line to obtain the head tilt angle.
[0095] Optionally, the third key point amplifying sub-module includes:
[0096] The reference point training set fitting unit is used to select a circle of reference points with an accuracy not higher than the second preset accuracy threshold around the newly added facial key point as the training data set, and use the newly added facial key point as the label to fit with a regression-based machine learning model to obtain new facial key points; the regression-based machine learning model includes a linear regression model, a support vector machine, and a random forest model.
[0097] Optionally, the multiple angles include a front 0° view, a left 45° view, and a right 45° view.
[0098] For the device embodiments, since they are basically similar to the method embodiments, the description is relatively simple. For related parts, please refer to the description of the method embodiments.
[0099] In addition, an embodiment of the present invention also provides an electronic device, as Figure 4 shown, including a processor 401, a communication interface 402, a memory 403, and a communication bus 404. Among them, the processor 401, the communication interface 402, and the memory 403 complete communication with each other through the communication bus 404.
[0100] The memory 403 is used to store computer programs.
[0101] When the processor 401 executes the programs stored on the memory 403, it implements the face key point enhancement method described in the above embodiments.
[0102] The communication bus mentioned in the above terminal may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience of representation, only a thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.
[0103] The communication interface is used for communication between the above terminal and other devices.
[0104] The memory may include a Random Access Memory (RAM), and may also include a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located far from the aforementioned processor.
[0105] The above-mentioned processor may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
[0106] As Figure 5 shown, in another embodiment provided by the present invention, there is also provided a computer-readable storage medium 501. Instructions are stored in the computer-readable storage medium. When it runs on a computer, the computer is made to execute the facial key point enhancement method described in the above embodiment.
[0107] In another embodiment provided by the present invention, there is also provided a computer program product containing instructions. When it runs on a computer, the computer is made to execute the facial key point enhancement method described in the above embodiment.
[0108] In the above embodiment, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present invention are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from a website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that the computer can access, or a data storage device such as a server or data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid-state disk (SSD)).
[0109] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the existence of additional identical elements in the process, method, article or device comprising the element.
[0110] Each embodiment in this specification is described in a related manner. For the same or similar parts among the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the apparatus embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and for the relevant parts, reference can be made to the corresponding descriptions in the method embodiments.
[0111] The above are only the preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention are all included in the protection scope of the present invention.
Claims
1. A facial key point enhancement method, characterized in that: The method comprises: Acquire multiple facial images taken from multiple angles; Extracting original facial key points from the multiple facial images using a public key point model, and obtaining a preset number of candidate facial key points by selecting and amplifying the original facial key points; An expansion pattern is matched according to the position of each newly added facial key point in the candidate facial key points, and key point expansion is performed according to the matched expansion pattern.
2. The method according to claim 1, characterized in that Matching an expansion pattern according to the position of each newly added facial key point in the candidate facial key points, and performing key point expansion according to the matched expansion pattern, including: If the newly added facial key point is located between two points, the new facial key point is calculated by taking the preset fixed coefficient in linear interpolation based on the two points; If the newly added facial key point is not between the two points or the accuracy of the reference point is lower than the first preset accuracy threshold, a new facial key point is calculated based on the single point and the head tilt angle and the preset fixed pixel value; If there are no reference points with an accuracy higher than a second preset accuracy threshold around the newly added facial key points, a reference point training data set is created, and the reference point training data set is used to fit the model to obtain new facial key points; the second preset accuracy threshold is less than the first preset accuracy threshold.
3. The method according to claim 1, characterized in that Extracting original facial key points from the multiple facial images using a public key point model, and obtaining a preset number of candidate facial key points by selecting and amplifying the original facial key points, including: Original facial key points are extracted from the multiple facial images using a public key point model, facial key points are selected from the original facial key points, and the original facial key points are amplified by fitting to obtain a preset number of candidate facial key points.
4. The method according to claim 2, characterized in that: If the newly added facial key point is not between the two points or the accuracy of the reference point is lower than the first preset accuracy threshold, a new facial key point is calculated based on the single point and the head tilt angle and the preset fixed pixel value, including: If the newly added facial key point is not between two points or the accuracy of the reference point is lower than the first preset accuracy threshold, the position of the specific key point in the candidate facial key points is analyzed, the head tilt angle is calculated, and the new facial key point is calculated based on the single point as the reference, combined with the head tilt angle and the preset fixed pixel value.
5. The method according to claim 4, characterized in that Analyzing the position of specific key points in the candidate facial key points and calculating the head tilt angle includes: Four points at the corners of the eyes are selected from the candidate facial key points and fitted into a straight line using the least square method. The inverse tangent function and the slope of the straight line are used to convert the points into an inclination angle to obtain the head inclination angle.
6. The method according to claim 2, characterized in that Create a benchmark training dataset and use it to fit the model to obtain new facial key points, including: A circle of reference points whose accuracy is not higher than a second preset accuracy threshold is selected around the newly added facial key points as a training data set, and the newly added facial key points are used as labels to obtain new facial key points by fitting using a regression machine learning model; the regression machine learning model includes a linear regression model, a support vector machine, and a random forest model.
7. The method according to claim 1, characterized in that The multiple angles include a 0° front viewing angle, a 45° left viewing angle, and a 45° right viewing angle.
8. A facial key point enhancement device, characterized in that: The device comprises: A facial image acquisition module, used to acquire multiple facial images shot from multiple angles; A candidate key point selection and expansion module, configured to extract original facial key points from the plurality of facial images using a public key point model, and obtain a preset number of candidate facial key points by selection and expansion based on the original facial key points; The new key point expansion module is used to match the expansion pattern according to the position of each newly added facial key point in the candidate facial key points, and to expand the key points according to the matched expansion pattern.
9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; The memory is used to store computer programs; The processor is used to implement the facial key point enhancement method according to any one of claims 1 to 7 when executing the program stored in the memory.
10. One or more computer-readable media having instructions stored thereon, which, when executed by one or more processors, enable the processors to perform the facial key point enhancement method according to any one of claims 1 to 7.