Full-automatic front face photographing device and method

Abstract

The invention discloses a full-automatic front face photographing device and method, and relates to the field of automatic photographing, and the device comprises a display screen, a depth camera, a linear lifting platform, a rotary holder, a photographing camera, and a processor. The depth camera and the linear lifting platform are fixed on the display screen; the rotary holder is mounted on the linear lifting platform; the photographing camera is mounted on the rotary holder; the depth camera is used for collecting an RGBD image, and taking a human face located in the three-dimensional photographing area as a target human face area; and the processor determines a three-dimensional coordinate set of a target face area and two-dimensional coordinates of a face frame according to the RGBD image, and controls the moving distance of the linear lifting platform and the rotating angle of the rotating holder, so that the imaging surface of the photographing camera is parallel to the face, and a front face picture is collected through the photographing camera. The camera can adapt to people of different heights, the camera can be kept parallel to the human face when the head deflects left and right, the camera can be independently used for photographing, and the front face photographing efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of automatic photography, and in particular to a fully automatic face photography device and method. Background Art

[0002] Nowadays, most methods of automatically taking photos are to install a camera on a rotating gimbal, detect the face through the image of the wide-angle tracking camera and rotate the gimbal, then obtain the face photo through the camera on the gimbal, and detect the face frame through the algorithm model, or install the camera on a two-dimensional linear platform, detect the face through the image of the wide-angle tracking camera and move the platform to track, then obtain the face photo through the camera on the platform, and detect the face frame through the algorithm model. However, since people cannot keep their head posture correct when sitting or standing in front of the camera to take pictures, and different heights also make it impossible for the fixed camera to accurately capture the face. Therefore, the relevant technology has the following defects.

[0003] 1. A rotating gimbal or a two-dimensional linear platform based on a wide-angle tracking camera can track a human face, but it is impossible to make the imaging surface of the camera parallel to the human face, that is, it is impossible to obtain a frontal photo of the human face.

[0004] 2. Since the wide-angle tracking camera and the camera are installed at a certain spatial distance, and there is lens distortion, there is a deviation between the center of the image obtained by the camera and the center of the face, and the size of the face border cannot be obtained. Therefore, the image of the camera is required to detect the face border through the algorithm model. That is, the camera cannot be used independently for taking pictures, and it must be used in conjunction with the tracking camera to obtain the face border.

[0005] 3. When there are dense crowds around, you need to manually select the people you want to take pictures of, and it is impossible to take pictures automatically. In addition, the installation and deployment of the two-dimensional linear platform takes up a large space, is bulky, and is not beautiful. Summary of the invention

[0006] The purpose of the present application is to provide a fully automatic frontal face photography device and method, which can accurately take frontal face photos, and the camera can be used independently for taking photos, thereby improving the efficiency of frontal face photography.

[0007] To achieve the above objectives, this application provides the following solutions: In a first aspect, the present application provides a fully automatic frontal face photography device, comprising: a display screen, a depth camera, a linear lifting platform, a rotating gimbal, a camera and a processor; The depth camera and the linear lifting platform are both fixed on the display screen; the rotating platform is installed on the linear lifting platform; the camera is installed on the rotating platform; The depth camera is used to collect an RGBD image of a human face, and automatically detect whether there is a human face in a set three-dimensional photographing area according to the RGBD image. If so, the human face in the three-dimensional photographing area is taken as the target human face area, otherwise, the RGBD image is re-collected; The processor is connected to the depth camera, the linear lifting platform and the rotating gimbal respectively. The processor is used to determine the three-dimensional coordinate set of the target face area and the two-dimensional coordinates of the face frame according to the RGBD image, and control the moving distance of the linear lifting platform and the rotation angle of the rotating gimbal according to the three-dimensional coordinate set, so that the imaging surface of the camera is parallel to the face, so as to collect a frontal face photo through the camera.

[0008] Furthermore, the depth camera is fixed on the top of the display screen; and the linear lifting platform is fixed on the side of the display screen.

[0009] Furthermore, the processor comprises: A three-dimensional coordinate determination module, connected to the depth camera, for determining a three-dimensional coordinate set of a target face region in a depth camera coordinate system using a Mediapipe_Face model according to the RGBD image; A data storage module, used to store the intrinsic parameter matrix of the camera, the transformation matrix between the linear lifting platform coordinate system and the depth camera coordinate system, and the transformation matrix between the linear lifting platform coordinate system and the camera coordinate system; A face frame determination module is connected to the three-dimensional coordinate determination module and the data storage module respectively, and is used to determine the two-dimensional coordinates of the face frame in the camera coordinate system according to the three-dimensional coordinates of the frame of the target face area in the depth camera coordinate system, the intrinsic parameter matrix of the camera, the transformation matrix between the linear lifting platform coordinate system and the depth camera coordinate system, and the transformation matrix between the linear lifting platform coordinate system and the camera coordinate system; The lifting platform control module is respectively connected to the three-dimensional coordinate determination module, the data storage module and the linear lifting platform, and is used to determine the nose coordinates in the depth camera coordinate system according to the three-dimensional coordinate set of the target face area in the depth camera coordinate system, and convert the nose coordinates in the depth camera coordinate system to the linear lifting platform coordinate system according to the transformation matrix between the linear lifting platform coordinate system and the depth camera coordinate system, and convert the nose coordinates in the depth camera coordinate system to the linear lifting platform coordinate system according to the transformation matrix between the linear lifting platform coordinate system and the depth camera coordinate system. y Axis coordinate values, controlling the moving distance of the linear lifting platform; The rotating gimbal control module is respectively connected to the three-dimensional coordinate determination module, the data storage module and the rotating gimbal, and is used to determine the left face point set and the right face point set in the depth camera coordinate system according to the three-dimensional coordinate set of the target face area in the depth camera coordinate system, determine the facial posture vector according to the left face point set and the right face point set, and control the rotation angle of the rotating gimbal according to the facial posture vector.

[0010] Furthermore, the transformation matrix between the linear lifting platform coordinate system and the depth camera coordinate system is: Tmd=[[1,0,0,dx],[0,1,0,dy],[0,0,1,dz],[0,0,0,1]]; Among them, Tmd is the transformation matrix between the linear lift platform coordinate system and the depth camera coordinate system, dx is the transformation matrix between the origin of the linear lift platform coordinate system and the origin of the depth camera coordinate system. x Axis distance, dy is the distance between the origin of the linear lift platform coordinate system and the origin of the depth camera coordinate system y Axis distance, dz is the distance between the origin of the linear lift platform coordinate system and the origin of the depth camera coordinate system z Axle distance.

[0011] Furthermore, the transformation matrix between the linear lifting platform coordinate system and the camera coordinate system is: Tam=[[cos(rot),0,sin(rot),0],[0,1,0,mov],[-sin(rot),0,cos(rot),0],[0,0,0,1]]; Among them, Tam is the transformation matrix between the linear lifting platform coordinate system and the camera coordinate system, mov is the moving distance of the linear lifting platform, rot is the rotation angle of the rotary head when the linear lifting platform moves the mov distance, cos is the cosine function, and sin is the sine function.

[0012] Furthermore, the face frame determination module uses the following formula to determine the two-dimensional coordinates of the face frame in the camera coordinate system: Ia=K×Tam×Tmd×Pd; Among them, Ia is the two-dimensional coordinate of the face bounding box in the camera coordinate system, K is the intrinsic parameter matrix of the camera, Tam is the transformation matrix between the linear lifting platform coordinate system and the camera coordinate system, Tmd is the transformation matrix between the linear lifting platform coordinate system and the depth camera coordinate system, and Pd is the three-dimensional coordinate of the bounding box of the target face area in the depth camera coordinate system.

[0013] Furthermore, the lifting platform control module uses the following formula to convert the nose coordinates in the depth camera coordinate system to the linear lifting platform coordinate system: Pnose_linearplatform=Tmd×Pnose_cam; Among them, Pnose_linearplatform is the nose coordinate in the linear lifting platform coordinate system, and the moving distance of the linear lifting platform is Pnose_linearplatform. y Axis coordinate value, Tmd is the transformation matrix between the linear lifting platform coordinate system and the depth camera coordinate system, and Pnose_cam is the nose coordinate in the depth camera coordinate system.

[0014] Furthermore, the pan / tilt control module uses the following formula to determine the facial posture vector: Pd_vec=mean(normalize(Pd_right-Pd_left))=[Pd_vec_x,Pd_vec_y,Pd_vec_z]; Among them, Pd_vec is the facial posture vector, Pd_right is the right face point set, Pd_left is the left face point set, normalize is the vector normalization function, mean is the mean function, and Pd_vec_x is the facial posture vector x Axis coordinates, Pd_vec_y is the facial posture vector y Axis coordinates, Pd_vec_z is the facial posture vector z Axis coordinates.

[0015] Furthermore, the pan-tilt control module uses the following formula to determine the rotation angle of the pan-tilt: theta=arctan(Pd_vec_z / Pd_vec_x); Among them, theta is the rotation angle of the pan / tilt, Pd_vec_x is the facial posture vector x Axis coordinates, Pd_vec_z is the facial posture vector z Axis coordinates, arctan is the inverse tangent function.

[0016] In a second aspect, the present application provides a fully automatic frontal face photography method, comprising: The depth camera is used to collect an RGBD image of a human face, and the RGBD image is used to automatically detect whether there is a human face in the set 3D photographing area. If so, the human face in the 3D photographing area is used as the target human face area, otherwise, the RGBD image is collected again; Determine the three-dimensional coordinate set of the target face area and the two-dimensional coordinates of the face frame through the processor according to the RGBD image, and control the moving distance of the linear lifting platform and the rotation angle of the rotating pan / tilt according to the three-dimensional coordinate set, so that the imaging surface of the camera is parallel to the face; The frontal face photo is collected by the camera.

[0017] According to the specific embodiments provided in this application, this application has the following technical effects: The present application provides a fully automatic frontal face photography device and method, which uses a depth camera to track human faces, and is combined with a linear lifting platform and a rotating pan head to adapt to people of different heights, and can keep the camera parallel to the face and obtain frontal face photos when the head is deflected left and right. In addition, only the depth camera is needed to accurately obtain the face frame, so that the camera can be used independently for taking photos, and the depth camera automatically detects the face in the photo area, which improves the efficiency of frontal face photography. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 A front view of a fully automatic front face photography device provided in one embodiment of the present application.

[0020] Figure 2 A side view of a fully automatic frontal face photography device provided in one embodiment of the present application.

[0021] Figure 3 A schematic diagram of the process of a fully automatic face photography system provided in one embodiment of the present application.

[0022] Reference numerals: 101 - display screen, 102 - depth camera, 103 - linear lifting platform, 104 - rotating gimbal, 105 - camera. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0024] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0025] In an exemplary embodiment, Figure 1 and Figure 2 As shown, a fully automatic face-photographing device is provided, including a display screen 101, a depth camera 102, a linear lifting platform 103, a rotating gimbal 104, a camera 105 and a processor.

[0026] When a person stands in a set area directly in front of the display screen 101 , the fully automatic front face photography device provided in the present application will automatically track and take a front face photo by the camera 105 .

[0027] The depth camera 102 and the linear lifting platform 103 are both fixed on the display screen 101. The rotating platform 104 is installed on the linear lifting platform 103. The camera 105 is installed on the rotating platform 104. Specifically, the center of the camera 105 is located at the rotation axis position just above the rotating platform 104, and the rotating platform 104 drives the camera 105 to rotate left and right at a yaw angle.

[0028] In a specific application example, the depth camera 102 is fixed on the top of the display screen 101, specifically at the top center. The linear lifting platform 103 is fixed on the side of the display screen 101, specifically on the left side.

[0029] The depth camera 102 is used to collect RGBD images of human faces, and automatically detect whether there is a human face in a set three-dimensional photographing area based on the RGBD image. If so, the human face in the three-dimensional photographing area is taken as the target human face area, otherwise the RGBD image is re-collected.

[0030] The processor is connected to the depth camera 102, the linear lifting platform 103 and the rotating pan-tilt platform 104 respectively, and is used to determine the three-dimensional coordinate set of the target face area and the two-dimensional coordinates of the face frame according to the RGBD image, and control the moving distance of the linear lifting platform 103 and the rotation angle of the rotating pan-tilt platform 104 according to the three-dimensional coordinate set, so that the imaging surface of the camera 105 is parallel to the face, so as to capture a frontal face photo through the camera 105.

[0031] The present application adjusts the height through the linear lifting platform 103 to accommodate people of different heights, and uses the rotating gimbal 104 to adjust the yaw angle of the camera 105 so that the imaging surface of the camera 105 is parallel to the face of the person, thereby obtaining a frontal face photo.

[0032] In a specific application example, the processor includes: a three-dimensional coordinate determination module, a data storage module, a face frame determination module, a lifting platform control module and a rotating pan / tilt control module. The functions and principles of each module are described below.

[0033] (1) A three-dimensional coordinate determination module is connected to the depth camera 102 , and is used to determine a three-dimensional coordinate set of a target face region in a depth camera coordinate system according to the RGBD image and using a Mediapipe_Face model.

[0034] (2) The data storage module is used to store the intrinsic parameter matrix of the camera 105, the transformation matrix between the linear lift platform coordinate system and the depth camera coordinate system, and the transformation matrix between the linear lift platform coordinate system and the camera coordinate system.

[0035] Specifically, the transformation matrix between the linear lift platform coordinate system and the depth camera coordinate system is: Tmd=[[1,0,0,dx],[0,1,0,dy],[0,0,1,dz],[0,0,0,1]]; where Tmd is the transformation matrix between the linear lift platform coordinate system and the depth camera coordinate system, dx is the distance between the origin of the linear lift platform coordinate system and the origin of the depth camera coordinate system. x Axis distance, dy is the distance between the origin of the linear lift platform coordinate system and the origin of the depth camera coordinate system y Axis distance, dz is the distance between the origin of the linear lift platform coordinate system and the origin of the depth camera coordinate system z Axle distance.

[0036] In the present application, when the linear lifting platform 103 moves the mov distance, the rotating pan-tilt head 104 also rotates the rot angle at the same time. Then, the transformation matrix between the linear lifting platform coordinate system and the camera coordinate system is: Tam=[[cos(rot),0,sin(rot),0],[0,1,0,mov],[-sin(rot),0,cos(rot),0],[0,0,0,1]]; wherein Tam is the transformation matrix between the linear lifting platform coordinate system and the camera coordinate system, mov is the moving distance of the linear lifting platform 103, rot is the rotation angle of the rotating pan-tilt head 104 when the linear lifting platform 103 moves the mov distance, cos is the cosine function, and sin is the sine function.

[0037] (3) A face frame determination module is connected to the three-dimensional coordinate determination module and the data storage module respectively. The face frame determination module is used to determine the face frame coordinates in the camera coordinate system according to the three-dimensional coordinates of the frame of the target face area in the depth camera coordinate system, the intrinsic parameter matrix of the camera 105, the transformation matrix between the linear lifting platform coordinate system and the depth camera coordinate system, and the transformation matrix between the linear lifting platform coordinate system and the camera coordinate system.

[0038] Specifically, the face frame determination module uses the following formula to determine the two-dimensional coordinates of the face frame in the camera coordinate system: Ia=K×Pa=K×Tam×Tmd×Pd; wherein Ia is the two-dimensional coordinates of the face frame in the camera coordinate system, K is the internal parameter matrix of the camera 105, Pd is the three-dimensional coordinates of the frame of the target face area in the depth camera coordinate system, including the three-dimensional coordinates of the upper left and lower right points of the target face area, and Pa is the three-dimensional coordinates of the three-dimensional coordinates of the frame of the target face area in the depth camera coordinate system in the camera coordinate system.

[0039] (4) The lifting platform control module is connected to the three-dimensional coordinate determination module, the data storage module and the linear lifting platform 103 respectively. The lifting platform control module is used to determine the nose coordinates in the depth camera coordinate system according to the three-dimensional coordinate set of the target face area in the depth camera 102 coordinate system, and to convert the nose coordinates in the depth camera coordinate system to the linear lifting platform coordinate system according to the transformation matrix between the linear lifting platform coordinate system and the depth camera coordinate system. y Axis coordinate value, controls the moving distance of the linear lifting platform 103.

[0040] Specifically, the three-dimensional coordinate point of the nose is selected as the center point of the face. The lifting platform control module uses the following formula to convert the nose coordinates in the depth camera coordinate system to the linear lifting platform coordinate system: Pnose_linearplatform=Tmd×Pnose_cam; where Pnose_linearplatform is the nose coordinates in the linear lifting platform coordinate system, and the moving distance of the linear lifting platform is Pnose_linearplatform. y Axis coordinate values, Pnose_linearplatform y The axis coordinate value is the moving distance of the linear lifting platform 103, and Pnose_cam is the nose coordinate in the depth camera coordinate system.

[0041] (5) A rotating gimbal control module is connected to the three-dimensional coordinate determination module, the data storage module and the rotating gimbal 104 respectively. The rotating gimbal control module is used to determine a left face point set and a right face point set in the depth camera coordinate system according to the three-dimensional coordinate set of the target face area in the depth camera coordinate system, determine a facial posture vector according to the left face point set and the right face point set, and control the rotation angle of the rotating gimbal 104 according to the facial posture vector.

[0042] Specifically, the pan / tilt control module uses the following formula to determine the facial posture vector: Pd_vec=mean(normalize(Pd_right-Pd_left))=[Pd_vec_x,Pd_vec_y,Pd_vec_z]; wherein Pd_vec is the facial posture vector, Pd_right is the right face point set, Pd_left is the left face point set, normalize is the vector normalization function, mean is the mean function, and Pd_vec_x is the facial posture vector. x Axis coordinates, Pd_vec_y is the facial posture vector y Axis coordinates, Pd_vec_z is the facial posture vector z Axis coordinates.

[0043] The pan-tilt control module determines the rotation angle theta of the pan-tilt 104 using the following formula: theta=arctan(Pd_vec_z / Pd_vec_x); wherein arctan is an inverse tangent function.

[0044] In the present application, when the linear lifting platform 103 moves, the rotating gimbal 104 also rotates theta angle accordingly, so that the human face and the imaging surface of the camera 105 are parallel and a frontal face photo is obtained.

[0045] In this application, after detecting Pnose_linearplatform based on the image captured by the depth camera, it detects whether the three-dimensional coordinate point of the nose is located in the three-dimensional photo-taking area (such as the circle drawn) in front of the set screen. If the person is standing or sitting in this area, it starts to take pictures automatically. That is, when the depth camera detects that there are multiple face nose coordinates Pnose_linearplatform in front of the screen, the face located in the three-dimensional photo-taking area set in front of the screen is selected as the target face area, and then the linear lifting platform and the rotating pan-tilt are controlled to track and take pictures of the target face, so that when the crowd is dense, there is no need to manually select the person to be photographed, which improves the efficiency of taking pictures.

[0046] The present application starts from the fact that the yaw angle of the head posture is large and the pitch angle is small when people take pictures, combines the advantages of the linear lifting platform 103 and the rotating gimbal 104, and adopts the depth camera 102 to track the face, so that when the head deflects left and right, the camera 105 can be kept parallel to the face and a frontal face photo can be obtained, and only the depth camera 102 can be used to accurately obtain the face frame, so that the camera 105 can be used independently for taking pictures, and automatically adapt to people of different heights without manual selection. When a person stands or sits in a specified position, the photo is automatically taken. The overall structure is simple, beautiful and reliable.

[0047] Based on the same inventive concept, the embodiment of the present application also provides a fully automatic frontal face photography method using the fully automatic frontal face photography device mentioned above. The implementation scheme for solving the problem provided by the method is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more embodiments of the fully automatic frontal face photography method provided below can refer to the limitations of the fully automatic frontal face photography device above, and will not be repeated here.

[0048] In an exemplary embodiment, Figure 3 As shown, a fully automatic frontal face photography method is provided, including the following steps 201 to 203.

[0049] Step 201, collect an RGBD image of a face through a depth camera, and automatically detect whether there is a face in a set three-dimensional photographing area based on the RGBD image. If so, the face in the three-dimensional photographing area is taken as the target face area, otherwise the RGBD image is re-collected.

[0050] Step 202, the processor determines the three-dimensional coordinate set and the face frame coordinates of the target face area according to the RGBD image, and controls the moving distance of the linear lifting platform and the rotation angle of the rotating gimbal according to the three-dimensional coordinate set, so that the imaging surface of the camera is parallel to the face.

[0051] Step 203, collecting a frontal face photo by using the camera.

[0052] This application has been verified in practical applications. In actual applications, a 1-meter-long linear lifting platform is used, which can automatically adapt to different heights of 1.2 meters to 2.1 meters, and only the depth camera can be used to obtain accurate facial frames (accuracy error <8 pixels), so that the camera can be used independently for taking pictures, and frontal facial photos can be obtained when the head is tilted left or right. By setting the tracking area to a circle with a radius of 30 centimeters and a center of 1 meter in front of the display screen, the photo is automatically taken when the person stands in the set area, which greatly improves the effect and efficiency of frontal facial photography.

[0053] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0054] In this application, all actions to obtain signals, information or data are carried out in compliance with the relevant data protection laws and policies of the country where they are located and with the authorization given by the owner of the corresponding device.

[0055] The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, etc., but is not limited thereto.

[0056] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0057] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A fully automatic face photography device, characterized in that: The fully automatic face-photographing device comprises: a display screen, a depth camera, a linear lifting platform, a rotating gimbal, a camera and a processor; The depth camera and the linear lifting platform are both fixed on the display screen; the rotating platform is installed on the linear lifting platform; the camera is installed on the rotating platform; The depth camera is used to collect an RGBD image of a human face, and automatically detect whether there is a human face in a set three-dimensional photographing area according to the RGBD image. If so, the human face in the three-dimensional photographing area is taken as the target human face area, otherwise, the RGBD image is re-collected; The processor is connected to the depth camera, the linear lifting platform and the rotating gimbal respectively. The processor is used to determine the three-dimensional coordinate set of the target face area and the two-dimensional coordinates of the face frame according to the RGBD image, and control the moving distance of the linear lifting platform and the rotation angle of the rotating gimbal according to the three-dimensional coordinate set, so that the imaging surface of the camera is parallel to the face, so as to collect a frontal face photo through the camera.

2. The fully automatic frontal face photography device according to claim 1, characterized in that: The depth camera is fixed on the top of the display screen; and the linear lifting platform is fixed on the side of the display screen.

3. The fully automatic frontal face photography device according to claim 1, characterized in that: The processor comprises: A three-dimensional coordinate determination module, connected to the depth camera, for determining a three-dimensional coordinate set of a target face region in a depth camera coordinate system using a Mediapipe_Face model according to the RGBD image; A data storage module, used to store the intrinsic parameter matrix of the camera, the transformation matrix between the linear lifting platform coordinate system and the depth camera coordinate system, and the transformation matrix between the linear lifting platform coordinate system and the camera coordinate system; A face frame determination module is connected to the three-dimensional coordinate determination module and the data storage module respectively, and is used to determine the two-dimensional coordinates of the face frame in the camera coordinate system according to the three-dimensional coordinates of the frame of the target face area in the depth camera coordinate system, the intrinsic parameter matrix of the camera, the transformation matrix between the linear lifting platform coordinate system and the depth camera coordinate system, and the transformation matrix between the linear lifting platform coordinate system and the camera coordinate system; The lifting platform control module is respectively connected to the three-dimensional coordinate determination module, the data storage module and the linear lifting platform, and is used to determine the nose coordinates in the depth camera coordinate system according to the three-dimensional coordinate set of the target face area in the depth camera coordinate system, and convert the nose coordinates in the depth camera coordinate system to the linear lifting platform coordinate system according to the transformation matrix between the linear lifting platform coordinate system and the depth camera coordinate system, and convert the nose coordinates in the depth camera coordinate system to the linear lifting platform coordinate system according to the transformation matrix between the linear lifting platform coordinate system and the depth camera coordinate system. y Axis coordinate values, controlling the moving distance of the linear lifting platform; The rotating gimbal control module is respectively connected to the three-dimensional coordinate determination module, the data storage module and the rotating gimbal, and is used to determine the left face point set and the right face point set in the depth camera coordinate system according to the three-dimensional coordinate set of the target face area in the depth camera coordinate system, determine the facial posture vector according to the left face point set and the right face point set, and control the rotation angle of the rotating gimbal according to the facial posture vector.

4. The fully automatic frontal face photography device according to claim 3, characterized in that: The transformation matrix between the linear lift platform coordinate system and the depth camera coordinate system is: Tmd=[[1,0,0,dx],[0,1,0,dy],[0,0,1,dz],[0,0,0,1]]; Among them, Tmd is the transformation matrix between the linear lift platform coordinate system and the depth camera coordinate system, dx is the transformation matrix between the origin of the linear lift platform coordinate system and the origin of the depth camera coordinate system. x Axis distance, dy is the distance between the origin of the linear lift platform coordinate system and the origin of the depth camera coordinate system y Axis distance, dz is the distance between the origin of the linear lift platform coordinate system and the origin of the depth camera coordinate system z Axle distance.

5. The fully automatic frontal face photography device according to claim 3, characterized in that: The transformation matrix between the linear lifting platform coordinate system and the camera coordinate system is: Tam=[[cos(rot),0,sin(rot),0],[0,1,0,mov],[-sin(rot),0,cos(rot),0],[0,0,0,1]]; Among them, Tam is the transformation matrix between the linear lifting platform coordinate system and the camera coordinate system, mov is the moving distance of the linear lifting platform, rot is the rotation angle of the rotary head when the linear lifting platform moves the mov distance, cos is the cosine function, and sin is the sine function.

6. The fully automatic frontal face photography device according to claim 3, characterized in that: The face frame determination module uses the following formula to determine the two-dimensional coordinates of the face frame in the camera coordinate system: Ia=K×Tam×Tmd×Pd; Among them, Ia is the two-dimensional coordinate of the face bounding box in the camera coordinate system, K is the intrinsic parameter matrix of the camera, Tam is the transformation matrix between the linear lifting platform coordinate system and the camera coordinate system, Tmd is the transformation matrix between the linear lifting platform coordinate system and the depth camera coordinate system, and Pd is the three-dimensional coordinate of the bounding box of the target face area in the depth camera coordinate system.

7. The fully automatic frontal face photography device according to claim 3, characterized in that: The lifting platform control module uses the following formula to convert the nose coordinates in the depth camera coordinate system to the linear lifting platform coordinate system: Pnose_linearplatform=Tmd×Pnose_cam; Among them, Pnose_linearplatform is the nose coordinate in the linear lifting platform coordinate system, and the moving distance of the linear lifting platform is Pnose_linearplatform. y Axis coordinate value, Tmd is the transformation matrix between the linear lifting platform coordinate system and the depth camera coordinate system, and Pnose_cam is the nose coordinate in the depth camera coordinate system.

8. The fully automatic frontal face photography device according to claim 3, characterized in that: The pan / tilt control module uses the following formula to determine the facial posture vector: Pd_vec=mean(normalize(Pd_right-Pd_left))=[Pd_vec_x,Pd_vec_y,Pd_vec_z]; Among them, Pd_vec is the facial posture vector, Pd_right is the right face point set, Pd_left is the left face point set, normalize is the vector normalization function, mean is the mean function, and Pd_vec_x is the facial posture vector x Axis coordinates, Pd_vec_y is the facial posture vector y Axis coordinates, Pd_vec_z is the facial posture vector z Axis coordinates.

9. The fully automatic frontal face photography device according to claim 3, characterized in that: The pan-tilt control module uses the following formula to determine the rotation angle of the pan-tilt: theta=arctan(Pd_vec_z / Pd_vec_x); Among them, theta is the rotation angle of the pan / tilt, Pd_vec_x is the facial posture vector x Axis coordinates, Pd_vec_z is the facial posture vector z Axis coordinates, arctan is the inverse tangent function.

10. A fully automatic frontal face photography method, using the fully automatic frontal face photography device according to any one of claims 1 to 9, characterized in that: The fully automatic frontal face photography method comprises: The depth camera is used to collect an RGBD image of a human face, and the RGBD image is used to automatically detect whether there is a human face in the set 3D photographing area. If so, the human face in the 3D photographing area is used as the target human face area, otherwise, the RGBD image is collected again; Determine the three-dimensional coordinate set of the target face area and the two-dimensional coordinates of the face frame through the processor according to the RGBD image, and control the moving distance of the linear lifting platform and the rotation angle of the rotating pan / tilt according to the three-dimensional coordinate set, so that the imaging surface of the camera is parallel to the face; The frontal face photo is collected by the camera.

Images