Three-dimensional imaging eye region processing method, device, equipment and medium

Abstract

The invention discloses a three-dimensional imaging eye region processing method and device, equipment and a medium, and the method comprises the steps: obtaining a color image of a human face and an initial human face point cloud, carrying out the feature extraction of the color image of the human face, and obtaining eye socket key points; projecting the orbit key points to the initial face point cloud to obtain orbit point clouds and second face point clouds not including the orbit point clouds; constructing an eyeball three-dimensional model based on a least square method, and filtering point clouds of the eyeball three-dimensional model according to the orbit key points; and smoothing the filtered eyeball three-dimensional model, and adding the eyeball three-dimensional model to the second face point cloud to obtain a target face point cloud. According to the invention, the point cloud depression or discontinuity phenomenon of the eyeball area can be improved, and the accuracy and integrity of the depth camera in capturing and reconstructing the depth information of the eyeball area can be improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and particularly relates to a method, device, equipment and medium for processing an eye region in 3D imaging. Background Art

[0002] With the rapid development of 3D vision technology, depth cameras, as important tools for capturing 3D information of the real world, play an important role in human-computer interaction, virtual reality, augmented reality, biometrics, etc. However, in practical applications, depth cameras face a technical challenge, that is, for human body parts with special physiological structures, such as eyeballs, especially there are significant errors in the depth perception of eyeballs. As a unique organ of the human body, the surface of the eyeball is rich in moisture and has highly light-absorbing characteristics, resulting in the difficulty for existing depth sensing technologies to achieve ideal accuracy when capturing the details of the eyeball surface.

[0003] When existing depth cameras scan a human face, due to the absorption of infrared light by the eyeballs and the complexity of the surface reflection characteristics, the point cloud data in the eyeball region often shows discontinuity or depression. This phenomenon not only reduces the accuracy of the overall facial model, but also directly affects the performance of high-precision applications such as face recognition and iris recognition based on depth information. Although there are already various algorithms in the market attempting to optimize the point cloud quality through subsequent processing means, the solutions for the particularity of the eyeballs are still scarce and have not fundamentally solved this problem. Summary of the Invention

[0004] In order to overcome one or more of the above technical defects, the present invention provides a method and device for processing an eye region in 3D imaging.

[0005] To solve the above problems, in a first aspect, the present invention provides a method for processing an eye region in 3D imaging, including:

[0006] Obtain a color image of a human face and an initial human face point cloud, extract features from the color image of the human face to obtain orbital key points;

[0007] Project the orbital key points onto the initial human face point cloud to obtain an orbital point cloud and a second human face point cloud excluding the orbital point cloud;

[0008] Construct a 3D model of the eyeball based on the least squares method, and filter the point cloud of the 3D model of the eyeball according to the orbital key points;

[0009] Smooth the filtered 3D model of the eyeball and add it to the second human face point cloud to obtain a target human face point cloud.

[0010] Further, the projecting the orbital key points onto the initial human face point cloud to obtain an orbital point cloud and a second human face point cloud excluding the orbital point cloud includes:

[0011] Project the orbital key points onto the initial human face point cloud to obtain the orbital point cloud;

[0012] Construct a mask for the eye region based on the orbital key points, and project the mask of the eye region onto the initial human face point cloud based on the external reference matrix to obtain a second human face point cloud excluding the orbital point cloud.

[0013] Further, the projecting the orbital key points onto the initial human face point cloud to obtain the orbital point cloud and the second human face point cloud excluding the orbital point cloud includes:

[0014] Construct a three-dimensional eyeball model based on the least squares method and a preset radius value;

[0015] Randomly select a number of coordinate points on the spherical surface of the three-dimensional eyeball model as sampling points;

[0016] Fit a reference plane based on the orbital key points to obtain the normal vector of the reference plane and a reference point located on the reference plane;

[0017] Judge the positional relationship between the sampling points, the center of the three-dimensional eyeball model, the reference point, and the reference plane;

[0018] Filter the point cloud of the three-dimensional eyeball model according to the judgment result to obtain the spherical point cloud of the eyeball.

[0019] Further, the smoothly processing the filtered three-dimensional eyeball model and adding it to the second human face point cloud to obtain the target human face point cloud includes:

[0020] Based on the kd-tree index, obtain all the point clouds within the first radius distance and all the point clouds within the second radius distance from the spherical point cloud of the eyeball, and divide them into a first neighborhood point cloud and a second neighborhood point cloud, where the first radius is less than the second radius, the first neighborhood point cloud includes the spherical point cloud of the eyeball, and the second neighborhood point cloud includes the first neighborhood point cloud and the spherical point cloud of the eyeball;

[0021] Perform smooth processing on the second neighborhood point cloud using MLS, and extract the first neighborhood point cloud from the processing result and add it to the second human face point cloud to obtain the target human face point cloud.

[0022] In a second aspect, the present invention provides an eye region processing device for three-dimensional imaging, which is used to implement the above-mentioned eye region processing method for three-dimensional imaging, and includes:

[0023] An orbital key point extraction module, which is used to obtain the color image of the human face and the initial human face point cloud, perform feature extraction on the color image of the human face, and obtain the orbital key points;

[0024] A projection module, configured to project the orbital key points onto the initial human face point cloud to obtain an orbital point cloud and a second human face point cloud excluding the orbital point cloud;

[0025] An eyeball fitting module, configured to construct a three-dimensional eyeball model based on the least squares method and filter the point cloud of the three-dimensional eyeball model according to the orbital key points;

[0026] A smoothing module, configured to perform smoothing processing on the filtered three-dimensional eyeball model and add it to the second human face point cloud to obtain a target human face point cloud.

[0027] In a third aspect, the present invention provides a method for three-dimensional human face reconstruction, including:

[0028] Collecting human face image data, where the human face image data includes a color image of the human face and human face point cloud data;

[0029] Performing eye region processing on the color image of the human face by using the above-mentioned eye region processing method for three-dimensional imaging;

[0030] Inputting the target human face point cloud into a three-dimensional reconstruction model to obtain human face three-dimensional image data.

[0031] In a fourth aspect, the present invention provides a computer device, including a memory and a processor, where the memory stores a computer program, and when the processor executes the computer program, the steps of the above-mentioned method are implemented.

[0032] In a fifth aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned method are implemented.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] The present invention discloses a method and a device for eye region processing in three-dimensional imaging, including obtaining a color image of a human face and an initial human face point cloud, extracting features from the color image of the human face to obtain orbital key points; projecting the orbital key points onto the initial human face point cloud to obtain an orbital point cloud and a second human face point cloud excluding the orbital point cloud; constructing a three-dimensional eyeball model based on the least squares method and filtering the point cloud of the three-dimensional eyeball model according to the orbital key points; performing smoothing processing on the filtered three-dimensional eyeball model and adding it to the second human face point cloud to obtain a target human face point cloud. By using the orbital key point information of the human face color image, the point cloud information of the eye region in the human face point cloud is located, the point cloud data of the three-dimensional eyeball model is screened and smoothed and then added to the second human face point cloud, so as to improve the phenomenon of point cloud depression or discontinuity in the eyeball region, and improve the accuracy and integrity of the depth camera when capturing and reconstructing the depth information of the eyeball region. Description of the Drawings

[0035] The following further elaborates on the specific implementation manners of the present invention in conjunction with the accompanying drawings, where:

[0036] Figure 1 is a flowchart of the method for processing the eye region in 3D imaging described in Embodiment 1;

[0037] Figure 2 is a schematic structural diagram of the device for processing the eye region in 3D imaging described in Embodiment 2;

[0038] Figure 3 is a flowchart of the operation of the device for processing the eye region in 3D imaging described in Embodiment 2;

[0039] Figure 4 is a flowchart of the method for 3D face reconstruction described in Embodiment 3;

[0040] Figure 5 is a schematic structural diagram of the computer device described in Embodiment 4;

[0041] Marking description: 110, orbital key point extraction module; 120, projection module; 130, eyeball fitting module; 140, smoothing module. Specific implementation manners

[0042] The following describes the preferred embodiments of the present invention in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only for the purpose of illustrating and explaining the present invention and are not intended to limit the present invention.

[0043] Embodiment 1

[0044] This embodiment discloses a method for processing the eye region in 3D imaging, such as Figure 1 , including:

[0045] S1. Obtain the color image color_image and the initial face point cloud cloud of the face, perform feature extraction on the color image color_image of the face, and obtain the orbital key points. In this embodiment, the mediapipe open-source model is used for feature extraction to obtain 478 key points of the face. The key points with serial numbers 263, 249, 390, 373, 374, 380, 381, 382, 362, 398, 384, 385, 386, 387, 388, 466 are taken as the left eye orbital key points left_eye_outer_landmark_indices; the key points with serial numbers 246, 161, 160, 159, 158, 157, 173, 133, 155, 154, 153, 145, 144, 163, 7, 33 are taken as the right eye orbital key points right_eye_outer_landmark_indices.

[0046] S2. Project the orbital key points onto the initial facial point cloud to obtain the orbital point cloud and a second facial point cloud excluding the orbital point cloud.

[0047] Specifically, step S2 includes:

[0048] Project the left eye orbital key points left_eye_outer_landmark_indices and the right eye orbital key points right_eye_outer_landmark_indices onto the initial facial point cloud through the extrinsic matrix to obtain the left eye orbital point cloud left_eye_boundary and the right eye orbital point cloud right_eye_boundary.

[0049] Construct a mask for the eye region based on the left eye orbital key points left_eye_outer_landmark_indices and the right eye orbital key points right_eye_outer_landmark_indices, and project the mask of the eye region onto the initial facial point cloud based on the extrinsic matrix to filter the point cloud corresponding to the mask, obtaining a second facial point cloud cloud_filtered excluding the orbital point cloud.

[0050] S3. Construct a three-dimensional eyeball model based on the least squares method and filter the point cloud of the three-dimensional eyeball model according to the orbital key points.

[0051] Since there is a lot of noise around the orbital point cloud, the noise easily leads to the problem of non-convergence of the radius-center joint optimization. Therefore, in this embodiment, a preset radius value r is set, and the center position is predicted by the least squares method.

[0052] Calculate the initial guess value center_guess, add the line-of-sight direction constraint eye_ray_dir to the calculation result, and use the constrained least squares optimization to obtain a stable solution. Specifically, calculate the centroid coordinate of the point cloud center_guess = Σpoint / N to generate the initial guess value; construct the constraint conditions: use the residual term (‖point - center‖ - r) 2 as the spherical constraint, and use the Euclidean distance constraint and the direction consistency constraint as the position constraints, so that the optimization result is near the initial guess result and in the line-of-sight direction. Construct a weighted objective function: λ1 × spherical error + λ2 × position error, and use the DENSE_QR linear solver to iteratively optimize and output the final center coordinate center_optimized.

[0053] In this embodiment, step S3 includes:

[0054] Construct a three-dimensional eyeball model based on the least squares method and a preset radius value r.

[0055] Randomly select several coordinate points on the spherical surface of the three-dimensional eyeball model in the Cartesian coordinate system as sampling points sample_point. Specifically, randomly sample within [0, 2π] Randomly sample θ within [0, π], and respectively obtain the x, y, and z of a sampling point (x, y, z) on the spherical surface according to the following formula:

[0056]

[0057]

[0058] z = center.z + r * cos(θ)

[0059] Based on the key points of the eye socket, fit a reference plane to obtain the normal vector of the reference plane and a reference point on the reference plane. Specifically, taking the left eye as an example, fit a reference plane based on the left eye socket point cloud left_eye_boundary to obtain the normal vector plane_normal of the reference plane and a reference point plane_point on the reference plane.

[0060] Specifically, use the least squares method to perform plane fitting on the three-dimensional eye socket point cloud: construct a linear equation system AX = B, and solve for the plane parameters. Among them, the plane equation form is: z = ax + by + c. Construct a 3×3 covariance matrix A for accumulating the quadratic terms of each point coordinate; construct the right vector B for accumulating the linear terms of each point coordinate. Use QR decomposition to solve the linear equation system, extract the plane normal vector (a, b, -1) from the solution vector X = [a, b, c] and normalize it, and calculate a reference point on the plane (select the intersection with the coordinate axes)

[0061] By taking the global coordinate system of the input point cloud as the reference, use the least squares method to minimize the sum of the squares of the perpendicular distances from all points to the plane. Finally, the plane parameters automatically adapt to the spatial distribution of the point cloud, so that plane fitting does not require specifying a specific reference point.

[0062] Specifically, input the left eye socket point cloud, construct a linear equation system, solve for the plane parameters, and output the unit normal vector representing the plane orientation and any point on the plane used to determine the plane position.

[0063] Determine the positional relationship between the sampling point and the center of the three-dimensional eyeball model, the reference point, and the reference plane. Specifically, determine the positional relationship between the sampling point sample_point and the center of the three-dimensional eyeball model center, the reference point plane_point, and the reference plane. According to the corresponding human body structure, it can be known that the center of the eyeball is in the eye, that is, the center of the three-dimensional eyeball model is inside the reference plane. Determine whether the sampling point sample_point is located outside the reference plane:

[0064] value_1 = plane_normal.dot(sample_point - plane_point)

[0065] value_2 = plane_normal.dot(center - plane_point)

[0066] Filter the point cloud of the three-dimensional eyeball model according to the judgment result to obtain the spherical point cloud of the eyeball. Specifically, when the sampling point sample_point and the center position center are on the same side of the reference plane, value_1 * value_2 > 0, otherwise value_1 * value_2 < 0. According to common sense, the center coordinates of the eyeball are inside the reference plane. Filter the points on the spherical surface of the three-dimensional eyeball model where value_1 * value_2 > 0 to obtain the left-eye spherical point cloud left_eye_cloud. Similarly, the right-eye spherical point cloud right_eye_cloud can be obtained.

[0067] There are usually certain gaps in the spherical point cloud of the eyeball obtained according to step S3 and it cannot be directly and completely matched with the second face point cloud cloud_filtered, otherwise it will bring certain difficulties to the subsequent 3D reconstruction process. Therefore, smooth the eyeball and its edge area.

[0068] S4. Smooth the filtered three-dimensional eyeball model and add it to the second face point cloud to obtain the target face point cloud.

[0069] In this embodiment, the smoothing the filtered three-dimensional eyeball model and adding it to the second face point cloud to obtain the target face point cloud includes:

[0070] Taking the spherical point cloud of the left eye, left_eye_cloud, as an example, all point clouds within the first radius distance r1 from the spherical point cloud of the left eye, left_eye_cloud, and all point clouds within the second radius distance r2 are obtained based on the kd-tree index. They are divided into the first neighborhood point cloud region1 and the second neighborhood point cloud region2. Here, r1 < r2. The first neighborhood point cloud region1 includes the spherical point cloud of the left eye, left_eye_cloud, and the second neighborhood point cloud region2 includes the first neighborhood point cloud region1 and the spherical point cloud of the left eye, left_eye_cloud.

[0071] Use MLS to perform filtering and smoothing on the second neighborhood point cloud region2, and extract the first neighborhood point cloud region1 from the processing result and add it to the second face point cloud cloud_filtered to obtain the target face point cloud.

[0072] In the present invention, orbital key points are obtained by extracting features from the color image of the face. Based on the orbital key points, the orbital point cloud is obtained and the second face point cloud without the orbital point cloud is obtained. The three-dimensional model of the eyeball is fitted based on the orbital point cloud, and the outer eyeball point cloud is screened and obtained through the positional relationship between the sampling points and the reference plane fitted by the orbital point cloud, and added to the second face point cloud to obtain the target face point cloud, improving the discontinuous or sunken phenomenon in the eyeball area during the three-dimensional imaging process, improving the modeling effect of three-dimensional reconstruction, and improving the accuracy and integrity of the depth camera when capturing and reconstructing the depth information of the eyeball area.

[0073] Embodiment 2

[0074] This embodiment discloses an eye region processing device for three-dimensional imaging, such as Figure 2 and 3 , which is used to implement the eye region processing method for three-dimensional imaging described in Embodiment 1, including an orbital key point extraction module 110, a projection module 120, an eyeball fitting module 130, and a smoothing module 140. Specifically, the orbital key point extraction module 110 is used to obtain the color image of the face and the initial face point cloud, extract features from the color image of the face, and obtain the orbital key points; the projection module 120 is used to project the orbital key points onto the initial face point cloud to obtain the orbital point cloud and the second face point cloud without the orbital point cloud; the eyeball fitting module 130 is used to construct a three-dimensional model of the eyeball based on the least squares method and filter the point cloud of the three-dimensional model of the eyeball according to the orbital key points; the smoothing module 140 is used to perform smoothing on the filtered three-dimensional model of the eyeball and add it to the second face point cloud to obtain the target face point cloud.

[0075] For other specific implementation details, please refer to Embodiment 1 and will not be elaborated here.

[0076] Each module in the above-mentioned eye region processing device for three-dimensional imaging can be implemented in whole or in part by software, hardware, and their combination. Each of the above modules can be embedded in or independent of the processor in a computer device in the form of hardware, or stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0077] Embodiment 3

[0078] This embodiment discloses a method for three-dimensional reconstruction of a human face, such as Figure 4 , including:

[0079] Collect human face image data, where the human face image data includes a color image of the human face and human face point cloud data.

[0080] Use the eye region processing method for three-dimensional imaging described in Embodiment 1 to process the eye region of the color image of the human face to obtain a target human face point cloud.

[0081] Input the target human face point cloud into a three-dimensional reconstruction model to obtain human face three-dimensional image data.

[0082] For other specific implementation details, please refer to Embodiment 1, which will not be elaborated here.

[0083] Combined with the eye region processing method for three-dimensional imaging provided in Embodiment 1, it can improve the discontinuous or sunken phenomenon in the three-dimensional imaging of the eyeball region in the prior art, improve the modeling effect and image reconstruction effect of three-dimensional reconstruction, and improve the accuracy and integrity of the depth camera when capturing and reconstructing the depth information of the eyeball region.

[0084] Embodiment 4

[0085] This embodiment discloses a computer device, which can be a server or a terminal integrated with a scheduler, and its internal structure diagram can be as Figure 5 shown. The computer device includes a processor, a memory, and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements a method for processing the eye region of three-dimensional imaging.

[0086] Those skilled in the art can understand that Figure 5The structure shown is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have a different component arrangement.

[0087] In this embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0088] Obtain a color image of a human face and an initial human face point cloud, perform feature extraction on the color image of the human face, and obtain orbital key points;

[0089] Project the orbital key points onto the initial human face point cloud to obtain an orbital point cloud and a second human face point cloud excluding the orbital point cloud;

[0090] Construct a three-dimensional eyeball model based on the least squares method, and filter the point cloud of the three-dimensional eyeball model according to the orbital key points;

[0091] Perform smoothing processing on the filtered three-dimensional eyeball model and add it to the second human face point cloud to obtain a target human face point cloud.

[0092] In this embodiment, when the processor executes the computer program, the following steps are also implemented:

[0093] Project the orbital key points onto the initial human face point cloud to obtain an orbital point cloud;

[0094] Construct a mask for the eye region based on the orbital key points, and project the mask of the eye region onto the initial human face point cloud based on the external reference matrix to obtain a second human face point cloud excluding the orbital point cloud.

[0095] In this embodiment, when the processor executes the computer program, the following steps are also implemented:

[0096] Construct a three-dimensional eyeball model based on the least squares method and a preset radius value;

[0097] Randomly select several coordinate points on the sphere of the three-dimensional eyeball model as sampling points;

[0098] Based on the orbital key points, fit a reference plane to obtain the normal vector of the reference plane and a reference point located on the reference plane;

[0099] Judge the positional relationship between the sampling points, the center of the sphere of the three-dimensional eyeball model, the reference point, and the reference plane;

[0100] Filter the point cloud of the three-dimensional eyeball model according to the judgment result to obtain the spherical point cloud of the eyeball.

[0101] In this embodiment, when the processor executes the computer program, the following steps are further implemented:

[0102] Based on the kd-tree index, all point clouds within the first radius distance and all point clouds within the second radius distance from the spherical point cloud of the eyeball are obtained and divided into the first neighborhood point cloud and the second neighborhood point cloud;

[0103] The MLS is used to smooth the second neighborhood point cloud, and the first neighborhood point cloud is extracted from the processing result and added to the second face point cloud to obtain the target face point cloud.

[0104] Embodiment 5

[0105] This embodiment discloses a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0106] Obtain a color image of the face and an initial face point cloud, perform feature extraction on the color image of the face to obtain orbital key points;

[0107] Project the orbital key points onto the initial face point cloud to obtain an orbital point cloud and a second face point cloud excluding the orbital point cloud;

[0108] Construct a three-dimensional eyeball model based on the least squares method, and filter the point cloud of the three-dimensional eyeball model according to the orbital key points;

[0109] Smooth the filtered three-dimensional eyeball model and add it to the second face point cloud to obtain the target face point cloud.

[0110] In this embodiment, when the computer program is executed by the processor, the following steps are further implemented:

[0111] Project the orbital key points onto the initial face point cloud to obtain an orbital point cloud;

[0112] Construct a mask for the eye region based on the orbital key points, and project the mask of the eye region onto the initial face point cloud based on the external parameter matrix to obtain a second face point cloud excluding the orbital point cloud.

[0113] In this embodiment, when the computer program is executed by the processor, the following steps are further implemented:

[0114] Construct a three-dimensional eyeball model based on the least squares method and a preset radius value;

[0115] Randomly select several coordinate points on the sphere of the three-dimensional eyeball model as sampling points;

[0116] Based on the orbital key points, fit a reference plane to obtain the normal vector of the reference plane and a reference point located on the reference plane;

[0117] Determine the positional relationship between the sampling point and the center of the three-dimensional eyeball model, the reference point, and the reference plane;

[0118] Filter the point cloud of the three-dimensional eyeball model according to the judgment result to obtain the spherical point cloud of the eyeball.

[0119] In this embodiment, when the computer program is executed by the processor, the following steps are further implemented:

[0120] Based on the kd-tree index, obtain all the point clouds within the first radius distance and all the point clouds within the second radius distance from the spherical point cloud of the eyeball, and divide them into the first neighborhood point cloud and the second neighborhood point cloud;

[0121] Perform smoothing processing on the second neighborhood point cloud by using MLS, and extract the first neighborhood point cloud from the processing result and add it to the second face point cloud to obtain the target face point cloud.

[0122] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, a database, or other media used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memories can include read-only memory (ROM), magnetic tapes, floppy disks, flash memories, optical memories, high-density embedded non-volatile memories, resistive random access memories (ReRAMs), magnetoresistive random access memories (MRAMs), ferroelectric random access memories (FRAMs), phase change memories (PCMs), graphene memories, etc. Volatile memories can include random access memories (RAMs) or external cache memories, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchains, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., without limitation.

[0123] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

[0124] In the description of this specification, the descriptions referring to terms such as "in this embodiment" or "specifically" mean that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0125] As described above, it is only a preferred embodiment of the present invention, and there is no limitation in any form to the present invention. Therefore, any modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention without departing from the content of the technical solution of the present invention still fall within the scope of the technical solution of the present invention.

Claims

1. A method for processing an eye region for three-dimensional imaging, characterized in that, Including: Obtain a color image of a human face and an initial human face point cloud, perform feature extraction on the color image of the human face to obtain orbital key points; Project the orbital key points onto the initial human face point cloud to obtain an orbital point cloud and a second human face point cloud excluding the orbital point cloud; Construct a three-dimensional eyeball model based on the least squares method, and filter the point cloud of the three-dimensional eyeball model according to the orbital key points; Smooth the filtered three-dimensional eyeball model and add it to the second human face point cloud to obtain a target human face point cloud.

2. The three-dimensional imaging method for processing an eye region according to claim 1, wherein, The step of projecting the orbital key points onto the initial human face point cloud to obtain an orbital point cloud and a second human face point cloud excluding the orbital point cloud includes: Project the orbital key points onto the initial human face point cloud to obtain an orbital point cloud; Construct a mask for the eye region based on the orbital key points, and project the mask of the eye region onto the initial human face point cloud based on the external reference matrix to obtain a second human face point cloud excluding the orbital point cloud.

3. The three-dimensional imaging method for processing an eye region according to claim 1, wherein, The step of projecting the orbital key points onto the initial human face point cloud to obtain an orbital point cloud and a second human face point cloud excluding the orbital point cloud includes: Construct a three-dimensional eyeball model based on the least squares method and a preset radius value; Randomly select several coordinate points on the spherical surface of the three-dimensional eyeball model as sampling points; Based on the orbital key points, fit a reference plane to obtain the normal vector of the reference plane and a reference point located on the reference plane; Judge the positional relationship between the sampling points, the center of the three-dimensional eyeball model, the reference point, and the reference plane; Filter the point cloud of the three-dimensional eyeball model according to the judgment result to obtain an eyeball spherical surface point cloud.

4. The three-dimensional imaging method for processing an eye region according to claim 1, characterized in that The step of smoothing the filtered three-dimensional eyeball model and adding it to the second human face point cloud to obtain a target human face point cloud includes: Based on the kd-tree index, obtain all the point clouds within the first radius distance and all the point clouds within the second radius distance from the eyeball spherical surface point cloud, and divide them into a first neighborhood point cloud and a second neighborhood point cloud, where the first radius is less than the second radius, the first neighborhood point cloud includes the eyeball spherical surface point cloud, and the second neighborhood point cloud includes the first neighborhood point cloud and the eyeball spherical surface point cloud; Perform smoothing processing on the second neighborhood point cloud using MLS, and extract the first neighborhood point cloud from the processing result and add it to the second human face point cloud to obtain a target human face point cloud.

5. An eye region processing device for three-dimensional imaging, which is used to implement the three-dimensional imaging eye region processing method according to any one of claims 1-4, characterized in that, Including: An orbital key point extraction module for obtaining a color image of a human face and an initial human face point cloud, and performing feature extraction on the color image of the human face to obtain orbital key points; A projection module for projecting the orbital key points onto the initial human face point cloud to obtain an orbital point cloud and a second human face point cloud excluding the orbital point cloud; An eyeball fitting module for constructing a three-dimensional eyeball model based on the least squares method and filtering the point cloud of the three-dimensional eyeball model according to the orbital key points; A smoothing module for smoothing the filtered three-dimensional eyeball model and adding it to the second human face point cloud to obtain a target human face point cloud.

6. A three-dimensional face reconstruction method, characterized in that, Including: Collect human face image data, where the human face image data includes a color image of a human face and human face point cloud data; Perform eye region processing on the color image of the human face using the three-dimensional imaging eye region processing method according to any one of claims 1-4; Input the target human face point cloud into a three-dimensional reconstruction model to obtain human face three-dimensional image data.

7. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method described in any one of claims 1-4.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the method described in any one of claims 1-4.

Images