Face image automatic registration system based on five-degree-of-freedom movable camera group

Abstract

The invention designs a face image automatic registration system based on a five-degree-of-freedom movable camera group. The face image automatic registration system comprises the camera group, a micro manipulator, an image acquisition module, a face positioning recognition module and a registration module. The system relates to the technical field of computer vision and artificial intelligence. The method can be used for the skin measuring instrument, the angle and the position of the camera can be automatically adjusted according to the placing position of the face of a user so as to achieve the best front face and side face image collection quality, and subsequent skin analysis and calculation precision can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and artificial intelligence. Background Art

[0002] Nowadays, more and more people are paying attention to the condition of their facial skin, which has increased the popularity of the skin beauty industry. Whether it is to evaluate or improve skin conditions, the first step is to conduct a quantitative test on the skin.

[0003] Most existing skin analyzers have fixed camera positions, and users are required to place their faces in a standard position. Users put their faces on the chin rest, and the forehead fixation device fixes the face. The staff will then prompt the user to adjust the position according to the situation on the screen. The whole process is cumbersome and has a strong subjective factor, making it difficult to truly achieve standardization and consistency of placement.

[0004] There are also a small number of skin analyzers on the market that use rotating cameras, but their degrees of freedom are still very limited and cannot be adjusted in the up and down directions. The standard placement is very important for the subsequent skin measurement calculations. Different placement positions will lead to deviations in the shooting angle, which will further lead to errors in calculation accuracy. And sometimes if the operator is not careful, the position deviation will be relatively large, which will have a greater impact on the skin analysis calculation and the comparison of multiple skin tests. Summary of the invention

[0005] In view of the above defects of traditional skin analyzers, the present invention designs a facial image automatic registration system based on a five-degree-of-freedom movable camera group, which includes a camera group, a micro-manipulator, an image acquisition module, a facial positioning and recognition module, and a registration module. The system can be used in a skin analyzer, which can automatically adjust the angle and position of the camera according to the placement of the user's face to achieve the best frontal and side face image acquisition quality, and also help improve the accuracy of subsequent skin analysis calculations.

[0006] When using it, the user places his face on the chin rest and fixes the posture, and the operator starts the automatic calibration program of the camera position on the skin tester.

[0007] There are three cameras in the skin analyzer, which can shoot the front, left and right sides of the face respectively. All three cameras can automatically zoom. The micro-manipulator can move the camera group through program control, allowing the camera to move up and down, left and right, and rotate with three degrees of freedom, a total of five degrees of freedom.

[0008] The three cameras continuously shoot the face in low-resolution mode, and send each frame of the facial image to the image recognition module; the image recognition module detects the key points of the face. The image recognition module sends the recognized key points of the face to the registration module, which determines the position; based on the calculated difference between the current position and the target position, the registration module sends a position fine-tuning instruction to the micro-manipulator; the micro-manipulator adjusts the position of the camera after receiving the instruction; when the registration module determines that the position of the photographed face matches the standard template, the position calibration program automatically stops. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 Schematic diagram of the skin tester of the present invention.

[0010] Figure 2 It is an enlarged schematic diagram of the frontal camera and the micro-manipulator of the present invention.

[0011] Figure 3 This is a schematic diagram of the distribution of key points on the front face.

[0012] Figure 4 This is a schematic diagram of the key point distribution on the left side of the face.

[0013] Figure 5 This is a schematic diagram of the key point distribution on the right side of the face.

[0014] Description of the accompanying drawings: 1: Skin tester; 2: Frontal face camera; 3: Left side face camera; 4: Right side face camera; 5: Chin rest; 6: Micro manipulator. DETAILED DESCRIPTION

[0015] The following is a more detailed description of the technical solutions in the embodiments of the present invention in conjunction with the drawings of the specification. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention belong to the scope of protection of the present invention.

[0016] Figure 1 It is a schematic diagram of a skin tester according to an embodiment of the present invention. Figure 2 It is an enlarged schematic diagram of the frontal face camera and the micromanipulator of the present invention, and the structures of the cameras and micromanipulators for the left and right faces are consistent therewith.

[0017] Here we use our own trained model to identify key points of the face. For the front face, we get 87 key points, which are numbered from 0 to 86; for the left and right faces, we get 51 key points each, which are numbered from 0 to 50.

[0018] Figure 3This is a schematic diagram of the distribution of key points of the front face, where points 0 to 16 are the facial contour points under the eyes, points 17 to 25 are the forehead contour points, points 26 to 33 are the right eyebrow contour points, points 34 to 41 are the left eyebrow contour points, points 42 to 49 are the right eye contour points, points 50 to 57 are the left eye contour points, points 58 to 60 are the bridge of the nose, point 61 is the tip of the nose, points 62 to 74 are the lower edge of the nose and the wings of the nose, and points 75 to 86 are the mouth contour points.

[0019] Figure 4 This is a schematic diagram of the key point distribution of the left face, where points 0 to 7 are the left face contour points below the eyebrows, points 8 to 16 are the forehead contour points, points 17 to 24 are the left eyebrow contour points, points 25 to 26 are the two upper contour points of the right eyebrow, points 27 to 34 are the left eye contour points, points 35 to 37 are the bridge of the nose, point 38 is the tip of the nose, points 39 to 43 are the lower edge of the nose and the left wing of the nose, and points 44 to 50 are the contour points of the left half of the mouth.

[0020] Figure 5 This is a schematic diagram of the key point distribution of the right face, where points 0 to 7 are the right face contour points below the eyebrows, points 8 to 16 are the forehead contour points, points 17 to 24 are the right eyebrow contour points, points 25 to 26 are the two upper contour points of the left eyebrow, points 27 to 34 are the right eye contour points, points 35 to 37 are the bridge of the nose, point 38 is the tip of the nose, points 39 to 43 are the lower edge of the nose and the right nose wing, and points 44 to 50 are the contour points of the right half of the mouth.

[0021] A standard template for the frontal face is also defined here. Specifically, point 59 is located in the center of the entire image; points 58 to 61 (nose bridge and nose tip) are located in a straight line in the vertical direction, with an error of no more than 5 pixels; the distance between point 58 and point 54 is the same as the distance between point 58 and point 42, with an error of no more than 10 pixels; point 59 is located in the vertical center of the minimum circumscribed rectangle of the outline enclosed by points 0 to 25, with an error of no more than 10 pixels; the absolute value of the difference between the x-coordinate values ​​of point 0 and point 16 (face width) accounts for about 75% of the image width; the absolute value of the difference between the y-coordinate values ​​of point 8 and point 21 (face height) accounts for about 90% of the image height.

[0022] When in use, the user places the face on the chin rest 5 and fixes the posture to form an initial posture, and the operator starts the automatic calibration program of the camera position on the skin tester. The three cameras in the skin tester will have an initial position, and the micro-manipulator 6 will record its initial position and then automatically calibrate it.

[0023] Specifically, the front camera 2 of the image acquisition module continuously captures images at a low resolution. For each frame of the captured image, the face positioning and recognition module identifies 87 key points of the face. The registration module sends the same movement instructions to the micro-manipulators of the three cameras at the same time through the following logic. The following logic is judged based on the front camera 2, and the other two cameras synchronously follow each step of the movement of the front camera 2.

[0024] Perform straight line fitting on the four points 58 to 61, and calculate the angle between the fitted straight line and the vertical direction; if the angle is greater than 5 degrees, perform roll angle rotation on camera 2 until the angle is less than 5 degrees.

[0025] Calculate the distance between point 58 and point 54, recorded as R1; calculate the distance between point 58 and point 42, recorded as L1; calculate the distance between point 78 and point 81, recorded as R2; calculate the distance between point 78 and point 75, recorded as L2.

[0026] Get the minimum enclosing rectangle of the figure enclosed by the facial contour points (points 0 to 16) and the forehead contour points (points 17 to 25), calculate the absolute value of the difference between the horizontal coordinate of the right boundary of the minimum enclosing rectangle and the horizontal coordinate of point 58, recorded as R3; calculate the absolute value of the difference between the horizontal coordinate of the left boundary of the minimum enclosing rectangle and the horizontal coordinate of point 58, recorded as L3; calculate the absolute value of the difference between the vertical coordinate of the upper boundary of the minimum enclosing rectangle and the vertical coordinate of point 59, recorded as u; calculate the absolute value of the difference between the vertical coordinate of the lower boundary of the minimum enclosing rectangle and the vertical coordinate of point 59, recorded as d.

[0027] Let R_ave = (R1 + R2 + R3) / 3; let L_ave = (L1 + L2 + L3) / 3.

[0028] If R_ave > L_ave and R_ave - L_ave > 10, rotate the camera to the right with a yaw angle. When the face is about to leave the picture before the appropriate ratio is reached (point 0 is less than 10 pixels away from the image boundary or the face cannot be detected), translate to the left until the face is detected and point 0 is more than 10 pixels away from the image edge; then continue to rotate to the right with a yaw angle, and repeat this operation. When |R_ave - L_ave| < 10, stop the yaw rotation and translate the camera left and right until the horizontal coordinate of point 59 is at the center of the image width.

[0029] If L_ave > R_ave and L_ave - R_ave > 10, rotate the camera to the left with a yaw angle. When the face is about to leave the picture before the appropriate ratio is reached (point 16 is less than 10 pixels away from the image boundary or the face cannot be detected), translate to the right until the face is detected and point 16 is more than 10 pixels away from the image edge; then continue to rotate to the left with a yaw angle, and repeat this operation. When |L_ave - R_ave| < 10, stop the yaw rotation and translate the camera left and right until the horizontal coordinate of point 59 is at the center of the image width.

[0030] If u > d and u –d > 10, the camera is rotated upward by a pitch angle. When the face is about to leave the picture before the appropriate ratio is reached (point 8 is less than 10 pixels away from the image boundary or the face cannot be detected), it is translated downward until the face is detected and point 8 is more than 10 pixels away from the image edge; then the upward pitch angle rotation is continued, and the operation is repeated. When |u –d| < 10 is met, the pitch angle rotation is stopped and the camera is translated up and down until the vertical coordinate of point 59 is at the center of the image height.

[0031] If d > u and d –u > 10, the camera is rotated downward by a pitch angle. When the face is about to leave the picture before reaching the appropriate ratio (point 21 is less than 10 pixels away from the image boundary or the face cannot be detected), it is translated upward until a face is detected and point 21 is more than 10 pixels away from the image edge; then the pitch angle rotation is continued, and the operation is repeated. When |d –u| < 10 is met, the pitch angle rotation is stopped and the camera is translated up and down until the vertical coordinate of point 59 is at the center of the image height.

[0032] Adjust the focal length of camera 2, calculate the ratio of the absolute value of the difference between the x-coordinate values ​​of points 0 and 16 to the image width, recorded as ratio_w, and calculate the ratio of the absolute value of the difference between the y-coordinate values ​​of points 8 and 21 on the face to the image height, recorded as ratio_h; stop zooming when ratio_w = 75% and ratio_h < 90% or when ratio_h = 90% and ratio_w < 75%.

[0033] The frontal camera 2 is registered, and the remaining two cameras are adjusted next.

[0034] The left face camera 3 rotates 45 degrees to the left with itself as the rotation axis, identifies 51 facial key points on the left face through the face positioning and recognition module, adjusts the focal length of the left face camera 3, calculates the ratio of the absolute value of the difference between the x-coordinate values ​​of point 0 and point 16 to the image width, recorded as ratio_w_L, and calculates the ratio of the absolute value of the difference between the y-coordinate values ​​of point 6 and point 11 to the image height, recorded as ratio_h_L; when ratio_w_L = 75% and ratio_h_L < 90% or when ratio_h_L = 90% and ratio_w_L < 75%, stops zooming.

[0035] The right face camera 4 rotates 45 degrees to the right with itself as the rotation axis. The 51 facial key points of the right face are identified by the face positioning and recognition module, the focal length of the right face camera 4 is adjusted, the absolute value of the difference between the x-coordinate values ​​of point 0 and point 16 and the ratio of the image width are calculated, recorded as ratio_w_R, and the absolute value of the difference between the y-coordinate values ​​of point 6 and point 11 on the face and the ratio of the image height are calculated, recorded as ratio_h_R; when ratio_w_R = 75% and ratio_h_R < 90% or when ratio_h_R = 90% and ratio_w_R < 75%, the zoom is stopped.

[0036] At this point, the three cameras have been registered, and the registration system has automatically adjusted the three cameras to the best shooting position, so that subsequent skin detection and analysis can be performed. Through such an automatic facial image registration system, the faces taken at three angles during each skin test are standardized.

[0037] It is understandable that the term "three" should be understood as "at least three" or "three or more", that is, in one embodiment, the number of an element can be three, while in another embodiment, the number of the elements can be more than one, and the term "three" should not be understood as a limitation on the quantity.

[0038] Although the terms such as camera, chin rest, micromanipulator are used more frequently in this article, the possibility of using other terms is not excluded. These terms are used only to more conveniently describe and explain the essence of the present invention; interpreting them as any additional restrictions is contrary to the spirit of the present invention.

[0039] The present invention is not limited to the above-mentioned optimal implementation mode. Anyone can derive other various forms of products under the inspiration of the present invention. However, no matter what changes are made in the shape or structure, all technical solutions that are the same or similar to those of the present application fall within the protection scope of the present invention.

Claims

1. A facial image automatic registration system based on a five-degree-of-freedom movable camera group, characterized in that: It includes a camera group, a micro-manipulator, an image acquisition module, a face positioning and recognition module, and a registration module.

2. The camera assembly according to claim 1, wherein: The camera group includes three cameras, which respectively shoot the user's front face, left face, and right face; each camera can automatically zoom; each camera can shoot in different resolution modes; and the position of each camera can be moved with five degrees of freedom.

3. The micromanipulator according to claim 1, characterized in that: The movement of the camera group can be executed through program control, allowing the camera to move up and down, left and right, and rotate with three degrees of freedom, a total of five degrees of freedom.

4. The image acquisition module according to claim 1, characterized in that: Continuous low-resolution acquisitions are possible in registration mode.

5. The face positioning and recognition module according to claim 1, characterized in that: Computer vision and artificial intelligence technologies are used to identify key points of the face from the front and side.

6. The registration module according to claim 1, characterized in that: The position deviation of each camera is calculated based on the recognized facial key points, and the corresponding position adjustment instructions are sent to the micro-manipulator.

Images