A fully automatic front-face photographing device and method

Abstract

The present application discloses a full-automatic frontal face photographing device and method, relating to the field of automatic photographing. The device includes: a display screen, a depth camera, a linear lifting platform, a rotary cloud platform, a photographing camera and a processor; the depth camera and the linear lifting platform are both fixed on the display screen; the rotary cloud platform is installed on the linear lifting platform; the photographing camera is installed on the rotary cloud platform; the depth camera is used to collect RGBD images and take the face located in the three-dimensional photographing area as the target face area; the processor determines the three-dimensional coordinate set and the two-dimensional coordinates of the face border of the target face area according to the RGBD images, controls the moving distance of the linear lifting platform and the rotation angle of the rotary cloud platform, and makes the imaging surface of the photographing camera parallel to the face, so as to collect a frontal face photograph through the photographing camera. The present application can be adapted to people of different heights, and can also keep the photographing camera parallel to the face when the head deflects left and right, and the photographing camera can be independently used for photographing, improving the efficiency of frontal face photographing.

Description

Technical Field

[0001] The present application relates to the field of automatic photographing, and particularly to a fully automatic front-face photographing device and method. Background Art

[0002] Nowadays, most methods for automatically taking photos install the photographing camera on a rotating cloud platform. By tracking the image of a wide-angle tracking camera, the face is detected and the cloud platform is rotated. Then, the face photo is obtained by the photographing camera on the cloud platform, and the face frame is detected through an algorithm model. Or the photographing camera is installed on a two-dimensional linear platform. By tracking the image of a wide-angle tracking camera, the face is detected and the platform is moved for tracking. Then, the face photo is obtained by the photographing camera on the platform, and the face frame is detected through an algorithm model. However, since people cannot ensure that their head postures are correct when sitting or standing in front of the camera, and different heights also cause the fixed photographing camera to be unable to accurately capture the face. Therefore, the related technologies have the following defects.

[0003] 1. The rotating cloud platform or two-dimensional linear platform based on the wide-angle tracking camera can track the face, but cannot make the imaging plane of the photographing camera parallel to the face, that is, a front-face photo of the face cannot be obtained.

[0004] 2. Due to a certain spatial distance between the installation positions of the wide-angle tracking camera and the photographing camera, and the existence of lens distortion, there is a deviation between the center of the image obtained by the photographing camera and the center of the face, and the size of the face frame cannot be obtained. Therefore, it is necessary to use the image of the photographing camera to detect the face frame through an algorithm model, that is, the photographing camera cannot be independently used for photographing, and it is necessary to cooperate with the tracking camera to obtain the face frame.

[0005] 3. When the surrounding crowd is dense, it is necessary to manually select the person to be photographed, and full-automatic photographing cannot be achieved. Moreover, the installation and deployment of the two-dimensional linear platform occupy a large space, are bulky, and are not beautiful. Summary of the Invention

[0006] The purpose of the present application is to provide a fully automatic front-face photographing device and method, which can accurately capture front-face photos, and the photographing camera can be independently used for photographing, improving the efficiency of front-face photographing.

[0007] To achieve the above purpose, the present application provides the following solutions:

[0008] In a first aspect, the present application provides a fully automatic front-face photographing device, including: a display screen, a depth camera, a linear lifting platform, a rotating cloud platform, a photographing camera, and a processor;

[0009] The depth camera and the linear lifting platform are both fixed on the display screen; the rotating cloud platform is installed on the linear lifting platform; the photographing camera is installed on the rotating cloud platform;

[0010] The depth camera is used to collect the RGBD image of the face, and automatically detect whether there is a face in the set three-dimensional photographing area according to the RGBD image. If so, the face located in the three-dimensional photographing area is used as the target face area; otherwise, the RGBD image is collected again.

[0011] The processor is respectively connected to the depth camera, the linear lifting platform and the rotary pan-tilt head. The processor is used to determine the three-dimensional coordinate set and the two-dimensional coordinate of the face frame of the target face area according to the RGBD image, and control the moving distance of the linear lifting platform and the rotation angle of the rotary pan-tilt head according to the three-dimensional coordinate set, so that the imaging plane of the photographing camera is parallel to the face, and a frontal face photo is collected through the photographing camera.

[0012] Further, the depth camera is fixed on the top of the display screen; the linear lifting platform is fixed on the side of the display screen.

[0013] Further, the processor includes:

[0014] A three-dimensional coordinate determination module, connected to the depth camera, and used to determine the three-dimensional coordinate set of the target face area in the depth camera coordinate system according to the RGBD image by using the Mediapipe_Face model;

[0015] A data storage module, used to store the internal parameter matrix of the photographing camera, the transformation matrix between the linear lifting platform coordinate system and the depth camera coordinate system, and the transformation matrix between the linear lifting platform coordinate system and the photographing camera coordinate system;

[0016] A face frame determination module, respectively connected to the three-dimensional coordinate determination module and the data storage module, and used to determine the two-dimensional coordinate of the face frame in the photographing camera coordinate system according to the three-dimensional coordinate of the border of the target face area in the depth camera coordinate system, the internal parameter matrix of the photographing camera, the transformation matrix between the linear lifting platform coordinate system and the depth camera coordinate system, and the transformation matrix between the linear lifting platform coordinate system and the photographing camera coordinate system;

[0017] A lifting platform control module, respectively connected to the three-dimensional coordinate determination module, the data storage module and the linear lifting platform, and used to determine the nose coordinate in the depth camera coordinate system according to the three-dimensional coordinate set of the target face area in the depth camera coordinate system, convert the nose coordinate in the depth camera coordinate system to the linear lifting platform coordinate system according to the transformation matrix between the linear lifting platform coordinate system and the depth camera coordinate system, and according to the y axial coordinate value of the nose coordinate in the linear lifting platform coordinate system, control the moving distance of the linear lifting platform;

[0018] The rotation pan-tilt control module is respectively connected to the three-dimensional coordinate determination module, the data storage module and the rotation pan-tilt, and is used to determine the left face point set and the right face point set in the depth camera coordinate system according to the three-dimensional coordinate set of the target face area in the depth camera coordinate system, determine the facial pose vector according to the left face point set and the right face point set, and control the rotation angle of the rotation pan-tilt according to the facial pose vector.

[0019] Further, the transformation matrix between the linear lifting platform coordinate system and the depth camera coordinate system is:

[0020] Tmd = [[1, 0, 0, dx], [0, 1, 0, dy], [0, 0, 1, dz], [0, 0, 0, 1]];

[0021] where, Tmd is the transformation matrix between the linear lifting platform coordinate system and the depth camera coordinate system, dx is the x axis distance between the origin of the linear lifting platform coordinate system and the origin of the depth camera coordinate system, dy is the y axis distance between the origin of the linear lifting platform coordinate system and the origin of the depth camera coordinate system, dz is the z axis distance between the origin of the linear lifting platform coordinate system and the origin of the depth camera coordinate system.

[0022] Further, the transformation matrix between the linear lifting platform coordinate system and the photographing camera coordinate system is:

[0023] Tam = [[cos(rot), 0, sin(rot), 0], [0, 1, 0, mov], [-sin(rot), 0, cos(rot), 0], [0, 0, 0, 1]];

[0024] where, Tam is the transformation matrix between the linear lifting platform coordinate system and the photographing camera coordinate system, mov is the moving distance of the linear lifting platform, rot is the rotation angle of the rotation pan-tilt when the linear lifting platform moves the mov distance, cos is the cosine function, and sin is the sine function.

[0025] Further, the face border determination module uses the following formula to determine the two-dimensional coordinates of the face border in the photographing camera coordinate system:

[0026] Ia = K × Tam × Tmd × Pd;

[0027] where, Ia is the two-dimensional coordinates of the face border in the photographing camera coordinate system, K is the internal parameter matrix of the photographing camera, Tam is the transformation matrix between the linear lifting platform coordinate system and the photographing camera coordinate system, Tmd is the transformation matrix between the linear lifting platform coordinate system and the depth camera coordinate system, and Pd is the three-dimensional coordinates of the border of the target face area in the depth camera coordinate system.

[0028] Further, the lifting platform control module uses the following formula to convert the nose coordinates in the depth camera coordinate system to the linear lifting platform coordinate system:

[0029] Pnose_linearplatform = Tmd × Pnose_cam;

[0030] where Pnose_linearplatform is the nose coordinate in the linear lifting platform coordinate system, the moving distance of the linear lifting platform is the y axis coordinate value of Pnose_linearplatform, Tmd is the transformation matrix between the linear lifting platform coordinate system and the depth camera coordinate system, and Pnose_cam is the nose coordinate in the depth camera coordinate system.

[0031] Further, the rotating pan-tilt control module uses the following formula to determine the facial pose vector:

[0032] Pd_vec = mean(normalize(Pd_right - Pd_left)) = [Pd_vec_x, Pd_vec_y, Pd_vec_z];

[0033] where Pd_vec is the facial pose vector, Pd_right is the right face point set, Pd_left is the left face point set, normalize is the vector normalization function, mean is the mean function, Pd_vec_x is the x axis coordinate of the facial pose vector, Pd_vec_y is the y axis coordinate of the facial pose vector, and Pd_vec_z is the z axis coordinate of the facial pose vector.

[0034] Further, the rotating pan-tilt control module uses the following formula to determine the rotation angle of the rotating pan-tilt:

[0035] theta = arctan(Pd_vec_z / Pd_vec_x);

[0036] where theta is the rotation angle of the rotating pan-tilt, Pd_vec_x is the x axis coordinate of the facial pose vector, Pd_vec_z is the z axis coordinate of the facial pose vector, and arctan is the arctangent function.

[0037] In a second aspect, the present application provides a fully automatic frontal face photographing method, including:

[0038] Collect the RGBD image of the human face through a depth camera, and automatically detect whether there is a human face within a set three-dimensional photographing area according to the RGBD image. If so, use the human face within the three-dimensional photographing area as the target human face area; otherwise, collect the RGBD image again.

[0039] The processor determines the three-dimensional coordinate set and the two-dimensional coordinates of the human face border of the target human face area according to the RGBD image, and controls the moving distance of the linear lifting platform and the rotating angle of the rotating cloud platform according to the three-dimensional coordinate set, so that the imaging surface of the photographing camera is parallel to the human face.

[0040] Collect a frontal face photo through the photographing camera.

[0041] According to the specific embodiments provided by the present application, the present application has the following technical effects:

[0042] The present application provides a fully automatic frontal face photographing device and method. The depth camera is used to track the human face, and at the same time, the linear lifting platform and the rotating cloud platform are combined to adapt to people of different heights, and when the head deflects left and right, it can also keep the photographing camera parallel to the human face and obtain a frontal face photo. And only the depth camera is needed to accurately obtain the human face border, so that the photographing camera can be independently used for photographing. At the same time, the depth camera automatically detects the human face in the photographing area, improving the efficiency of frontal face photographing. Description of the Drawings

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0044] Figure 1 It is the front view of a fully automatic frontal face photographing device provided by an embodiment of the present application.

[0045] Figure 2 It is the side view of a fully automatic frontal face photographing device provided by an embodiment of the present application.

[0046] Figure 3 It is the schematic flow chart of a fully automatic frontal face photographing system provided by an embodiment of the present application.

[0047] Reference numerals: 101 - display screen, 102 - depth camera, 103 - linear lifting platform, 104 - rotating cloud platform, 105 - photographing camera. Detailed Description of the Embodiments

[0048] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without making creative efforts belong to the scope of protection of the present application.

[0049] To make the above objects, features, and advantages of the present application more obvious and understandable, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0050] In an exemplary embodiment, as Figure 1 and Figure 2 shown, a full-automatic frontal face photographing device is provided, which includes a display screen 101, a depth camera 102, a linear lifting platform 103, a rotary pan-tilt 104, a photographing camera 105, and a processor.

[0051] When a person stands in the set area directly in front of the display screen 101, the full-automatic frontal face photographing device provided by the present application will automatically track and take a frontal face photo with the photographing camera 105.

[0052] The depth camera 102 and the linear lifting platform 103 are both fixed on the display screen 101. The rotary pan-tilt 104 is installed on the linear lifting platform 103. The photographing camera 105 is installed on the rotary pan-tilt 104. Specifically, the center of the photographing camera 105 is located at the position of the rotation axis directly above the rotary pan-tilt 104, and the rotary pan-tilt 104 drives the photographing camera 105 to rotate left and right at the yaw angle.

[0053] In a specific application example, the depth camera 102 is fixed at the top of the display screen 101, specifically at the center position of the top. The linear lifting platform 103 is fixed on the side of the display screen 101, specifically on the left side.

[0054] The depth camera 102 is used to collect the RGBD image of the human face, and automatically detect whether there is a human face in the set three-dimensional photographing area according to the RGBD image. If so, the human face located in the three-dimensional photographing area is used as the target human face area, otherwise, the RGBD image is collected again.

[0055] The processor is respectively connected to the depth camera 102, the linear lifting platform 103, and the rotary pan-tilt 104. The processor is used to determine the three-dimensional coordinate set and the two-dimensional coordinate of the human face border of the target human face area according to the RGBD image, and control the moving distance of the linear lifting platform 103 and the rotation angle of the rotary pan-tilt 104 according to the three-dimensional coordinate set, so that the imaging plane of the photographing camera 105 is parallel to the human face, in order to collect a frontal face photo through the photographing camera 105.

[0056] In this application, the height is adjusted by the linear lifting platform 103 to adapt to people of different heights, and the yaw angle of the camera 105 for taking pictures is adjusted by the rotating cloud platform 104, so that the imaging plane of the camera 105 for taking pictures is parallel to the human face, and a frontal face photo is obtained.

[0057] In a specific application example, the processor includes: a three-dimensional coordinate determination module, a data storage module, a face frame determination module, a lifting platform control module, and a rotating cloud platform control module. The functions and principles of each module are introduced separately below.

[0058] (1) The three-dimensional coordinate determination module is connected to the depth camera 102. The three-dimensional coordinate determination module is used to determine the three-dimensional coordinate set of the target face area in the depth camera coordinate system according to the RGBD image by using the Mediapipe_Face model.

[0059] (2) The data storage module is used to store the internal parameter matrix of the camera 105 for taking pictures, the transformation matrix between the linear lifting platform coordinate system and the depth camera coordinate system, and the transformation matrix between the linear lifting platform coordinate system and the camera coordinate system for taking pictures.

[0060] Specifically, the transformation matrix between the linear lifting platform coordinate system and the depth camera coordinate system is: Tmd = [[1, 0, 0, dx], [0, 1, 0, dy], [0, 0, 1, dz], [0, 0, 0, 1]]; where Tmd is the transformation matrix between the linear lifting platform coordinate system and the depth camera coordinate system, and dx is the x axial distance between the origin of the linear lifting platform coordinate system and the origin of the depth camera coordinate system, dy is the y axial distance between the origin of the linear lifting platform coordinate system and the origin of the depth camera coordinate system, and dz is the z axial distance between the origin of the linear lifting platform coordinate system and the origin of the depth camera coordinate system.

[0061] In this application, when the linear lifting platform 103 moves a distance of mov, the rotating cloud platform 104 also rotates an angle of rot at the same time. Then the transformation matrix between the linear lifting platform coordinate system and the camera coordinate system for taking pictures is: Tam = [[cos(rot), 0, sin(rot), 0], [0, 1, 0, mov], [-sin(rot), 0, cos(rot), 0], [0, 0, 0, 1]]; where Tam is the transformation matrix between the linear lifting platform coordinate system and the camera coordinate system for taking pictures, mov is the moving distance of the linear lifting platform 103, rot is the rotation angle of the rotating cloud platform 104 when the linear lifting platform 103 moves a distance of mov, cos is the cosine function, and sin is the sine function.

[0062] (3) The face border determination module is respectively connected to the three-dimensional coordinate determination module and the data storage module. The face border determination module is used to determine the face border coordinates in the photographing camera coordinate system according to the three-dimensional border coordinates of the target face area in the depth camera coordinate system, the internal parameter matrix of the photographing camera 105, the transformation matrix between the linear lifting platform coordinate system and the depth camera coordinate system, and the transformation matrix between the linear lifting platform coordinate system and the photographing camera coordinate system.

[0063] Specifically, the face border determination module uses the following formula to determine the two-dimensional face border coordinates in the photographing camera coordinate system: Ia = K × Pa = K × Tam × Tmd × Pd; where Ia is the two-dimensional face border coordinates in the photographing camera coordinate system, K is the internal parameter matrix of the photographing camera 105, Pd is the three-dimensional border coordinates of the target face area in the depth camera coordinate system, including the three-dimensional coordinates of the upper left and lower right points of the target face area, and Pa is the three-dimensional coordinates of the border of the target face area in the depth camera coordinate system in the three-dimensional coordinates of the photographing camera coordinate system.

[0064] (4) The lifting platform control module is respectively connected to the three-dimensional coordinate determination module, the data storage module, and the linear lifting platform 103. The lifting platform control module is used to determine the nose coordinates in the depth camera coordinate system according to the three-dimensional coordinate set of the target face area in the depth camera 102 coordinate system, convert the nose coordinates in the depth camera coordinate system to the linear lifting platform coordinate system according to the transformation matrix between the linear lifting platform coordinate system and the depth camera coordinate system, and according to the y x-axis coordinate value of the nose coordinates in the linear lifting platform coordinate system, control the moving distance of the linear lifting platform 103.

[0065] Specifically, the three-dimensional coordinate point of the nose is selected as the center point of the face. The lifting platform control module uses the following formula to convert the nose coordinates in the depth camera coordinate system to the linear lifting platform coordinate system: Pnose_linearplatform = Tmd × Pnose_cam; where Pnose_linearplatform is the nose coordinates in the linear lifting platform coordinate system, the moving distance of the linear lifting platform is the y x-axis coordinate value of Pnose_linearplatform, the y x-axis coordinate value of Pnose_linearplatform is the moving distance of the linear lifting platform 103, and Pnose_cam is the nose coordinates in the depth camera coordinate system.

[0066] (5) The pan-tilt control module is respectively connected to the three-dimensional coordinate determination module, the data storage module, and the pan-tilt 104. The pan-tilt control module is configured to determine a left face point set and a right face point set in the depth camera coordinate system according to the three-dimensional coordinate set of the target face area in the depth camera coordinate system, determine a facial pose vector according to the left face point set and the right face point set, and control the rotation angle of the pan-tilt 104 according to the facial pose vector.

[0067] Specifically, the pan-tilt control module determines the facial pose vector using the following formula: Pd_vec = mean(normalize(Pd_right - Pd_left)) = [Pd_vec_x, Pd_vec_y, Pd_vec_z]; where Pd_vec is the facial pose vector, Pd_right is the right face point set, Pd_left is the left face point set, normalize is the vector normalization function, mean is the mean function, Pd_vec_x is the x x-axis coordinate of the facial pose vector, Pd_vec_y is the y y-axis coordinate of the facial pose vector, and Pd_vec_z is the z z-axis coordinate of the facial pose vector.

[0068] The pan-tilt control module determines the rotation angle theta of the pan-tilt 104 using the following formula: theta = arctan(Pd_vec_z / Pd_vec_x); where arctan is the arctangent function.

[0069] In this application, while the linear lifting platform 103 is moving, the pan-tilt 104 also rotates by the angle theta accordingly, so that the face is parallel to the imaging surface of the camera 105 and a frontal face photo can be obtained.

[0070] In this application, after detecting Pnose_linearplatform based on the image collected by the depth camera, it is detected whether the three-dimensional coordinate point of the nose is located within the three-dimensional photographing area in front of the set screen (such as a defined circle). If a person stands or sits within this area, automatic photographing starts. That is, when the depth camera detects multiple face nose coordinates Pnose_linearplatform in front of the screen, the face located within the set three-dimensional photographing area in front of the screen is selected as the target face area, and then the linear lifting platform and the pan-tilt are controlled to track and photograph the target face. Thus, when the crowd is dense, there is no need to manually select the person to be photographed, improving the photographing efficiency.

[0071] Based on the fact that when people take pictures, the yaw angle of the head is large while the pitch angle is small, this application combines the advantages of the linear lifting platform 103 and the rotating cloud platform 104, and uses the depth camera 102 to track the human face, so that when the head deflects left and right, the taking camera 105 can still be parallel to the human face and obtain a frontal face photo. Moreover, only the depth camera 102 can accurately obtain the face frame, enabling the taking camera 105 to be independently used for taking pictures, and automatically adapting to people of different heights without manual selection. When a person stands or sits at the designated position, the picture is automatically taken. The overall structure is simple, beautiful and reliable.

[0072] Based on the same inventive concept, the embodiment of this application also provides a full-automatic frontal face photographing method applying the above-mentioned full-automatic frontal face photographing device. The implementation solutions for solving problems provided by this method are similar to the implementation solutions recorded in the above method. Therefore, the specific limitations in one or more embodiments of the following full-automatic frontal face photographing method can refer to the limitations on the full-automatic frontal face photographing device in the above text, and will not be elaborated here.

[0073] In an exemplary embodiment, as Figure 3 shown, a full-automatic frontal face photographing method is provided, including the following steps 201 to step 203.

[0074] Step 201, collect the RGBD image of the human face through the depth camera, and automatically detect whether there is a human face in the set three-dimensional photographing area according to the RGBD image. If so, use the human face located in the three-dimensional photographing area as the target face area, otherwise collect the RGBD image again.

[0075] Step 202, determine the three-dimensional coordinate set and the face frame coordinates of the target face area according to the RGBD image through the processor, and control the moving distance of the linear lifting platform and the rotating angle of the rotating cloud platform according to the three-dimensional coordinate set, so that the imaging surface of the taking camera is parallel to the human face.

[0076] Step 203, collect the frontal face photo through the taking camera.

[0077] This application has been verified in actual applications. In actual applications, a linear lifting platform with a length of 1 meter is used, which can automatically adapt to different heights from 1.2 meters to 2.1 meters, and only the depth camera can obtain an accurate face frame (precision error < 8 pixels), enabling the taking camera to be independently used for taking pictures, and a frontal face photo can also be obtained when the human head deflects left and right. By setting the tracking area as a circle with a radius of 30 cm centered 1 meter in front of the display screen, when a person stands into the set area, the picture is automatically taken, greatly improving the effect and efficiency of frontal face photographing.

[0078] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with relevant regulations.

[0079] In this application, all actions of obtaining signals, information, or data are carried out on the premise of complying with the corresponding data protection regulations and policies of the country where the location is located and obtaining authorization from the owner of the corresponding device.

[0080] The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logics, data processing logics based on quantum computing, etc., and are not limited thereto.

[0081] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

[0082] Specific examples are used in this article to elaborate on the principles and implementation manners of this application. The descriptions of the above embodiments are only used to help understand the method and its core idea of this application; at the same time, for those of ordinary skill in the art, based on the idea of this application, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation to this application.

Claims

1. A fully automatic face photography device, characterized in that: The fully automatic face-photographing device comprises: a display screen, a depth camera, a linear lifting platform, a rotating gimbal, a camera and a processor; The depth camera and the linear lifting platform are both fixed on the display screen; the rotating platform is installed on the linear lifting platform; the camera is installed on the rotating platform; The depth camera is used to collect an RGBD image of a human face, and automatically detect whether there is a human face in a set three-dimensional photographing area according to the RGBD image. If so, the human face in the three-dimensional photographing area is taken as the target human face area, otherwise, the RGBD image is re-collected; The processor is connected to the depth camera, the linear lifting platform and the rotating pan-tilt platform respectively, and is used to determine the three-dimensional coordinate set of the target face area and the two-dimensional coordinates of the face frame according to the RGBD image, and control the moving distance of the linear lifting platform and the rotation angle of the rotating pan-tilt platform according to the three-dimensional coordinate set, so that the imaging surface of the camera is parallel to the face, so as to collect a frontal face photo through the camera; The processor comprises: A three-dimensional coordinate determination module, connected to the depth camera, for determining a three-dimensional coordinate set of a target face region in a depth camera coordinate system using a Mediapipe_Face model according to the RGBD image; A data storage module, used to store the intrinsic parameter matrix of the camera, the transformation matrix between the linear lifting platform coordinate system and the depth camera coordinate system, and the transformation matrix between the linear lifting platform coordinate system and the camera coordinate system; A face frame determination module is connected to the three-dimensional coordinate determination module and the data storage module respectively, and is used to determine the two-dimensional coordinates of the face frame in the camera coordinate system according to the three-dimensional coordinates of the frame of the target face area in the depth camera coordinate system, the intrinsic parameter matrix of the camera, the transformation matrix between the linear lifting platform coordinate system and the depth camera coordinate system, and the transformation matrix between the linear lifting platform coordinate system and the camera coordinate system; The lifting platform control module is respectively connected to the three-dimensional coordinate determination module, the data storage module and the linear lifting platform, and is used to determine the nose coordinates in the depth camera coordinate system according to the three-dimensional coordinate set of the target face area in the depth camera coordinate system, and convert the nose coordinates in the depth camera coordinate system to the linear lifting platform coordinate system according to the transformation matrix between the linear lifting platform coordinate system and the depth camera coordinate system, and convert the nose coordinates in the depth camera coordinate system to the linear lifting platform coordinate system according to the transformation matrix between the linear lifting platform coordinate system and the depth camera coordinate system. y Axis coordinate values, controlling the moving distance of the linear lifting platform; The rotating gimbal control module is respectively connected to the three-dimensional coordinate determination module, the data storage module and the rotating gimbal, and is used to determine the left face point set and the right face point set in the depth camera coordinate system according to the three-dimensional coordinate set of the target face area in the depth camera coordinate system, determine the facial posture vector according to the left face point set and the right face point set, and control the rotation angle of the rotating gimbal according to the facial posture vector.

2. The fully automatic frontal face photography device according to claim 1, characterized in that: The depth camera is fixed on the top of the display screen; and the linear lifting platform is fixed on the side of the display screen.

3. The fully automatic frontal face photography device according to claim 1, characterized in that: The transformation matrix between the linear lift platform coordinate system and the depth camera coordinate system is: Tmd=[[1,0,0,dx],[0,1,0,dy],[0,0,1,dz],[0,0,0,1]]; Among them, Tmd is the transformation matrix between the linear lift platform coordinate system and the depth camera coordinate system, dx is the transformation matrix between the origin of the linear lift platform coordinate system and the origin of the depth camera coordinate system. x Axis distance, dy is the distance between the origin of the linear lift platform coordinate system and the origin of the depth camera coordinate system y Axis distance, dz is the distance between the origin of the linear lift platform coordinate system and the origin of the depth camera coordinate system z Axle distance.

4. The fully automatic face photography device according to claim 1, characterized in that: The transformation matrix between the linear lifting platform coordinate system and the camera coordinate system is: Tam=[[cos(rot),0,sin(rot),0],[0,1,0,mov],[-sin(rot),0,cos(rot),0],[0,0,0,1]]; Among them, Tam is the transformation matrix between the linear lifting platform coordinate system and the camera coordinate system, mov is the moving distance of the linear lifting platform, rot is the rotation angle of the rotary head when the linear lifting platform moves the mov distance, cos is the cosine function, and sin is the sine function.

5. The fully automatic frontal face photography device according to claim 1, characterized in that: The face frame determination module uses the following formula to determine the two-dimensional coordinates of the face frame in the camera coordinate system: Ia=K×Tam×Tmd×Pd; Among them, Ia is the two-dimensional coordinate of the face bounding box in the camera coordinate system, K is the intrinsic parameter matrix of the camera, Tam is the transformation matrix between the linear lifting platform coordinate system and the camera coordinate system, Tmd is the transformation matrix between the linear lifting platform coordinate system and the depth camera coordinate system, and Pd is the three-dimensional coordinate of the bounding box of the target face area in the depth camera coordinate system.

6. The fully automatic frontal face photography device according to claim 1, characterized in that: The lifting platform control module uses the following formula to convert the nose coordinates in the depth camera coordinate system to the linear lifting platform coordinate system: Pnose_linearplatform=Tmd×Pnose_cam; Among them, Pnose_linearplatform is the nose coordinate in the linear lifting platform coordinate system, and the moving distance of the linear lifting platform is Pnose_linearplatform. y Axis coordinate value, Tmd is the transformation matrix between the linear lifting platform coordinate system and the depth camera coordinate system, and Pnose_cam is the nose coordinate in the depth camera coordinate system.

7. The fully automatic frontal face photography device according to claim 1, characterized in that: The pan / tilt control module uses the following formula to determine the facial posture vector: Pd_vec=mean(normalize(Pd_right-Pd_left))=[Pd_vec_x,Pd_vec_y,Pd_vec_z]; Among them, Pd_vec is the facial posture vector, Pd_right is the right face point set, Pd_left is the left face point set, normalize is the vector normalization function, mean is the mean function, and Pd_vec_x is the facial posture vector x Axis coordinates, Pd_vec_y is the facial posture vector y Axis coordinates, Pd_vec_z is the facial posture vector z Axis coordinates.

8. The fully automatic frontal face photography device according to claim 1, characterized in that: The pan-tilt control module uses the following formula to determine the rotation angle of the pan-tilt: theta=arctan(Pd_vec_z / Pd_vec_x); Among them, theta is the rotation angle of the pan / tilt, Pd_vec_x is the facial posture vector x Axis coordinates, Pd_vec_z is the facial posture vector z Axis coordinates, arctan is the inverse tangent function.

9. A fully automatic frontal face photography method, using the fully automatic frontal face photography device according to any one of claims 1 to 8, characterized in that: The fully automatic frontal face photography method comprises: The depth camera is used to collect an RGBD image of a human face, and the RGBD image is used to automatically detect whether there is a human face in the set 3D photographing area. If so, the human face in the 3D photographing area is used as the target human face area, otherwise, the RGBD image is collected again; Determine the three-dimensional coordinate set of the target face area and the two-dimensional coordinates of the face frame through the processor according to the RGBD image, and control the moving distance of the linear lifting platform and the rotation angle of the rotating pan / tilt according to the three-dimensional coordinate set, so that the imaging surface of the camera is parallel to the face; The frontal face photo is collected by the camera.

Images