Human eye reconstruction method and device based on multi-view stereoscopic vision

Abstract

The invention provides a human eye reconstruction method based on multi-view stereoscopic vision, which comprises the following steps: a hardware setting step: calculating a field depth range intersection of a plurality of cameras, and selecting shooting distances corresponding to the plurality of cameras in the field depth range intersection; a camera calibration step: selecting a target size matched with the size of the human eyes, synchronously acquiring calibration images by a plurality of cameras, and calculating camera calibration parameters according to the plurality of calibration images; a feature matching step: cutting the plurality of calibration images to obtain a plurality of corresponding human eye images, extracting at least one key feature point from the plurality of human eye images, and calculating a feature matching relationship of the key feature points; and a three-dimensional reconstruction step: calculating human eye point cloud information according to the camera calibration parameters and the feature matching relationship. The invention further provides a human eye reconstruction device based on the multi-view stereoscopic vision, a storage medium and electronic equipment. Therefore, the precision of human eye three-dimensional reconstruction in the micro-distance environment can be improved, and the usability and the efficiency are improved.

Description

Technical Field

[0001] The present invention relates to the fields of computer vision and artificial intelligence technology, and in particular to a method, device, storage medium and electronic device for reconstructing human eyes based on multi-view stereo vision. Background Art

[0002] With the rapid development of computer technology, the research and application of virtual humans have made significant progress in academia and industry. Unlike traditional biology, which studies the human body through an anatomical perspective, virtual humans digitally reconstruct and simulate the human body's structure and behavior through mathematical modeling, geometry, and computer vision. This involves not only realistic reproduction of appearance but also the simulation of complex aspects such as behavior, emotions, and interactions.

[0003] Today, 3D reconstruction technology has advanced to the point where it can achieve high-precision 3D reconstruction using multi-camera systems. However, with the industry's increasing demand for precision, especially in macro environments, the requirements for reconstruction accuracy have become even more stringent. As the most detailed part of the face, the human eye presents significant challenges in capturing high-resolution images and reconstructing its geometric model with high precision.

[0004] In multi-camera systems, the core technology for 3D reconstruction is usually based on the principle of triangulation. For a given 3D point X, its 2D projection point xi at each camera's viewpoint can be expressed by the following relationship:

[0005]

[0006] Among them, P i Represents the projection matrix of the i-th camera, and N is the total number of cameras. By capturing multiple perspective images of the same scene through a multi-camera system and combining the projection matrices of each camera, the coordinates of the point X in 3D space can be restored by solving the following optimization problem:

[0007]

[0008] However, in practical applications, the existing multi-camera 3D reconstruction process has some urgent problems to be solved when reconstructing the human eye, especially in high-precision requirements and complex scenes. These problems are particularly prominent.

[0009] Specifically, the existing 3D reconstruction process for the human eye has the following main problems:

[0010] (1) The co-focusing problem of multiple cameras: The tiny structure and complex morphology of the human eye require multiple cameras to achieve precise co-focusing during the shooting process. However, due to the curved surface and complex reflective properties of the human eye, existing methods have difficulty ensuring that all cameras focus on the same focal plane, resulting in the inability to accurately align the reconstructed 3D data, affecting the final reconstruction effect.

[0011] (2) Calibration parameter accuracy: To accurately reconstruct the three-dimensional structure of the human eye, the camera's internal and external parameters must be accurately calibrated. However, existing calibration methods often struggle to provide sufficient accuracy for tiny targets like the human eye. Especially when dealing with the complex surface structure and reflected light of the human eye, calibration errors can significantly affect the accuracy of reconstruction.

[0012] (3) Feature matching accuracy: Human eye images contain a large number of repetitive texture regions, such as the sclera and iris, which pose a huge challenge to feature matching. Existing feature matching algorithms are prone to mismatching in these regions, resulting in positional deviations of key points during reconstruction, thus affecting the accuracy of the 3D model of the human eye.

[0013] (4) Complex process and low degree of automation: The entire process of 3D reconstruction of the human eye usually involves a lot of manual adjustments and complex operations, which are difficult to complete automatically by existing systems. This not only increases the difficulty of operation, but also easily introduces human errors, affecting the consistency and accuracy of the reconstruction results.

[0014] In summary, the existing technology has obvious inconveniences and defects in actual use, so it is necessary to improve it. Summary of the Invention

[0015] In response to the above-mentioned defects, the purpose of the present invention is to provide a method, device, storage medium and electronic device for human eye reconstruction based on multi-eye stereo vision, which can improve the accuracy of three-dimensional reconstruction of the human eye in a macro environment and enhance ease of use and efficiency.

[0016] In order to solve the above-mentioned technical problems, the present invention is achieved as follows:

[0017] In a first aspect, an embodiment of the present invention provides a method for reconstructing human eyes based on multi-view stereo vision, comprising:

[0018] A hardware setting step of calculating the intersection of depth of field ranges of multiple cameras, and selecting shooting distances corresponding to the multiple cameras within the intersection of the depth of field ranges;

[0019] a camera calibration step, selecting a target size that matches the size of the human eye, synchronously acquiring calibration images with the multiple cameras, and calculating camera calibration parameters based on the multiple calibration images;

[0020] a feature matching step of cropping the plurality of calibration images to obtain a plurality of corresponding human eye images, extracting at least one key feature point from the plurality of human eye images, and calculating a feature matching relationship of the key feature point;

[0021] The three-dimensional reconstruction step calculates the human eye point cloud information based on the camera calibration parameters and the feature matching relationship.

[0022] According to the method for human eye reconstruction based on multi-eye stereo vision of the present invention, the hardware setting step further includes:

[0023] Calculate the front depth of field ΔL1, the back depth of field ΔL2 and the depth of field range ΔL of the multiple cameras respectively using Formulas 1 to 3;

[0024]

[0025] Where δ represents the diameter of the permissible circle of confusion, which is the maximum blur circle allowed in the image; f represents the focal length of the lens, that is, the distance from the optical center of the lens to the imaging plane; F represents the aperture value of the lens, which is used to control the aperture size; L represents the focusing distance, that is, the distance of the subject at which the lens focuses;

[0026] Obtaining a depth of field range intersection according to the depth of field ranges of the multiple cameras;

[0027] Within the intersection range of the depth of field ranges, a shortest shooting distance that enables the multiple cameras to form clear images in the human eye area is selected.

[0028] 4. According to the method for human eye reconstruction based on multi-view stereo vision of the present invention, the step of calculating the foreground depth of field ΔL1, the back depth of field ΔL2, and the depth of field range ΔL of the multiple cameras using Formulas 1 to 3 respectively includes:

[0029] An m*n camera array is used and arranged in an array form, where m and n are natural numbers greater than or equal to 2, and m and n can be the same or different;

[0030] Adjust the aperture of a single camera to f11, ISO to 100, and shutter time to 1s;

[0031] After the step of selecting a shortest shooting distance within the intersection range of the depth of field ranges that enables the multiple cameras to clearly image the human eye area, the following steps are included:

[0032] Prevent the flash from entering the central area around the camera;

[0033] The pupil of the human eye is controlled to contract by lighting.

[0034] According to the method for human eye reconstruction based on multi-view stereo vision of the present invention, the camera calibration step further includes:

[0035] Select a target size that matches the size of the human eye and use a checkerboard as the target, where the side length of the checkerboard is 2 mm;

[0036] The multiple cameras synchronously capture the calibration image to ensure that the corner points are clearly imaged in the multiple cameras;

[0037] Performing image filtering on all collected calibration images to remove noise;

[0038] Performing corner detection on all the collected calibration images;

[0039] Each camera records the image number in which the complete checkerboard corner points are detected, and the intersection of the image numbers in which the corner points can be detected by all the cameras is taken as the calibration image set finally used;

[0040] A main camera is selected from the multiple cameras as the first camera A; a second camera B is selected with a shorter baseline than the first camera A. Given the intrinsic parameters of the first camera A and the second camera B and their corresponding key feature points, the epipolar constraint equation is used to solve the rotation matrix R and the translation vector T between the first camera A and the second camera B:

[0041]

[0042] Among them, x A and x B are the coordinates of the image points in the first camera A and the second camera B respectively, and F is the basic matrix. The basic matrix F is calculated by the camera's intrinsic parameter matrix K, rotation matrix R and translation vector T:

[0043]

[0044] After the intrinsic parameters and relative pose of the first camera pair (A, B) are known, the three-dimensional coordinates of the calibration plate are obtained by triangulation equations. For the remaining camera set {C1, C2, ..., C n}, and obtain its relative position relative to the first camera A through the PNP algorithm, that is, the rotation matrix and translation vectors

[0045]

[0046] in, is the intrinsic parameter matrix of the i-th camera, X is the three-dimensional point cloud coordinate, is the coordinate of the image point in the i-th camera;

[0047] After obtaining the initial internal and external parameters, the optimizable parameters are adjusted by optimizing the objective function to minimize the reprojection error of all image points. The optimization objective function is as follows:

[0048]

[0049] Among them, K i is the intrinsic parameter matrix of the i-th camera, R i and T i are the rotation matrix and translation vector of the i-th camera, X j is the jth three-dimensional point, x ij is the image coordinate of the jth point in the i-th camera, π(·) is the projection function, and the internal and external parameters are iteratively adjusted through the optimization algorithm to finally obtain the accurate camera calibration parameters.

[0050] According to the method for human eye reconstruction based on multi-view stereo vision of the present invention, the feature matching step further includes:

[0051] Using a Gaussian filter to remove noise from the calibration image;

[0052] Adjusting the white balance and brightness of the calibration image;

[0053] performing distortion correction on the calibration image based on the camera calibration parameters;

[0054] Cropping the plurality of calibration images to obtain the corresponding plurality of human eye images;

[0055] One image is selected from the multiple human eye images and defined as a main view. The other images are defined as reference views. The main view and the reference view are respectively subjected to rough image matching to obtain a first sparse feature matching relationship. This process is defined as the matching points of any two images I1 and I2 are K1 and K2, which are expressed as follows:

[0056] (k1,k2)=RoMa(I1,I2)

[0057] Calculate a projection matrix t using K1 and K2 to project the reference view onto the main view;

[0058] Perform image matching on the projected image to obtain a second sparse feature matching relationship;

[0059] Perform epipolar verification on the sparse feature points after matching:

[0060]

[0061] Here, F is the basic matrix, and the parameter d is specified to represent a threshold to filter out point pairs that do not satisfy the epipolar constraint. The Sampson distance of the matching points is compared with the preset threshold d = 0.01, and points greater than the threshold are filtered out to obtain the verified points:

[0062]

[0063] The second sparse feature matching relationship is interpolated to obtain a dense feature matching relationship as the feature matching relationship.

[0064] According to the human eye reconstruction method based on multi-eye stereo vision of the present invention, in the feature matching step, the segmentation model of everything is used to crop the multiple calibration images to obtain the corresponding multiple human eye images.

[0065] According to the method for human eye reconstruction based on multi-eye stereo vision of the present invention, the three-dimensional reconstruction step further includes:

[0066] Calculating a three-dimensional point cloud using a triangulation equation according to the dense feature matching relationship and the camera calibration parameters;

[0067] Set a main view I ref , the remaining views are represented as a set {I c}, respectively obtain the three-dimensional point clouds of the remaining views and the main view;

[0068] The three-dimensional point cloud coordinates are averaged to obtain the final human eye point cloud information:

[0069]

[0070] Where N is the number of views, X i The 3D point cloud coordinates for each view.

[0071] In a second aspect, an embodiment of the present invention provides a human eye reconstruction device based on multi-eye stereo vision constructed based on any of the methods described above, comprising:

[0072] A hardware setting module, configured to calculate an intersection of depth-of-field ranges of multiple cameras, and select shooting distances corresponding to the multiple cameras within the intersection of the depth-of-field ranges;

[0073] A camera calibration module is used to select a target size that matches the size of the human eye, the multiple cameras synchronously capture calibration images, and calculate camera calibration parameters based on the multiple calibration images;

[0074] a feature matching module, configured to crop the plurality of calibration images to obtain a plurality of corresponding human eye images, extract at least one key feature point from the plurality of human eye images, and calculate a feature matching relationship of the key feature point;

[0075] The three-dimensional reconstruction module is used to calculate the human eye point cloud information based on the camera calibration parameters and the feature matching relationship.

[0076] In a third aspect, an embodiment of the present invention provides a storage medium for storing a computer program for executing any one of the methods for reconstructing human eyes based on multi-view stereo vision.

[0077] In a fourth aspect, an embodiment of the present invention provides an electronic device, comprising a storage medium, a processor, and a computer program stored on the storage medium and runnable on the processor, characterized in that when the processor executes the computer program, it implements any one of the methods for human eye reconstruction based on multi-eye stereo vision.

[0078] Thus, the present invention is based on a human eye reconstruction technology based on multi-camera stereo vision, including: a hardware setting step, calculating the intersection of the depth of field ranges of multiple cameras, and selecting the shooting distances corresponding to the multiple cameras within the depth of field range intersection. It can effectively achieve common focusing of multiple cameras in a macro environment, ensuring the acquisition of clear and accurate image data, thereby achieving high-precision three-dimensional reconstruction of the human eye in a macro environment. A camera calibration step, selecting a target size that matches the size of the human eye, multiple cameras synchronously collecting calibration images, and calculating camera calibration parameters based on the multiple calibration images. By improving the camera calibration method, high-precision calibration is ensured in a macro environment, and the impact of calibration errors on the accuracy of three-dimensional reconstruction is effectively reduced. It is particularly suitable for photographing complex geometric structures of the human eye. A feature matching step, cropping multiple calibration images to obtain corresponding multiple human eye images, extracting at least one key feature point from the multiple human eye images, and calculating the feature matching relationship of the key feature points. An optimized feature matching algorithm is used for repeated texture areas (such as sclera and iris) in human eye images to solve the matching error problem commonly found in traditional methods. This algorithm can maintain high-precision feature matching in complex macro scenes, improving the overall accuracy of 3D reconstruction of the human eye and the detail restoration of the model. The 3D reconstruction step calculates the eye point cloud information based on the camera calibration parameters and the feature matching relationship. By optimizing the reconstruction process, the entire 3D reconstruction process is made more automated, reducing the need for human intervention and improving the system's usability and efficiency. This method can thereby improve the accuracy, usability, and efficiency of 3D reconstruction of the human eye in macro environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] Figure 1 1 is a flow chart of a method for reconstructing human eyes based on multi-view stereo vision provided in the first embodiment of the present invention;

[0080] Figure 21 is a flowchart illustrating the hardware configuration steps in the method for reconstructing the human eye based on multi-view stereo vision provided in the second embodiment of the present invention;

[0081] Figure 3 is a schematic diagram of a depth of field model provided in the second embodiment of the present invention;

[0082] Figure 4 is a schematic diagram of a depth of field overlap model provided in the second embodiment of the present invention;

[0083] Figure 5 1 is a flowchart of a camera calibration step in a method for reconstructing human eyes based on multi-view stereo vision provided in a third embodiment of the present invention;

[0084] Figure 6 1 is a flow chart of a feature matching step in a method for reconstructing human eyes based on multi-view stereo vision provided in a fourth embodiment of the present invention;

[0085] Figure 7 3D reconstruction is a flowchart of a three-dimensional reconstruction method based on multi-view stereo vision provided in a fifth embodiment of the present invention;

[0086] Figure 8 1 is a schematic structural diagram of a human eye reconstruction device based on multi-eye stereo vision provided in the first embodiment of the present invention;

[0087] Figure 9 It is a structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0088] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0089] It should be noted that references to "one embodiment," "an embodiment," "an example embodiment," etc., in this specification indicate that the described embodiment may include specific features, structures, or characteristics, but not every embodiment must include these specific features, structures, or characteristics. Furthermore, such references do not necessarily refer to the same embodiment. Furthermore, when specific features, structures, or characteristics are described in conjunction with an embodiment, whether or not explicitly described, it is understood that incorporating such features, structures, or characteristics into other embodiments is within the knowledge of those skilled in the art.

[0090] In addition, certain words are used in the specification and subsequent claims to refer to specific components or parts. It should be understood by those with ordinary knowledge in the relevant field that manufacturers may use different nouns or terms to refer to the same component or part. This specification and subsequent claims do not use differences in names as a way to distinguish components or parts, but rather use differences in the functions of components or parts as the criteria for distinction. The words "including" and "comprising" mentioned throughout the specification and subsequent claims are open-ended terms and should be interpreted as "including but not limited to". In addition, the word "connect" here includes any direct and indirect electrical connection means. Indirect electrical connection means include connection through other devices.

[0091] The following describes in detail the method for reconstructing the human eye based on multi-view stereo vision provided by the embodiment of the present invention through specific embodiments and application scenarios in conjunction with the accompanying drawings.

[0092] The present invention aims to solve the following key problems in the overall process of the prior art when using multiple cameras to perform 3D reconstruction of the human eye in a macro environment:

[0093] (1) The problem of setting up the human eye reconstruction system in a macro environment. In the three-dimensional reconstruction process of the human eye, due to the complex geometric structure and tiny size of the human eye surface, it is difficult for multiple cameras to achieve common focus in a macro environment. The traditional three-dimensional reconstruction process is difficult to apply under such conditions, resulting in unstable or inaccurate reconstruction results. The present invention proposes a reconstruction process designed specifically for macro scenes of the human eye, which can solve the problem of difficulty in cameras to focus together and ensure high-precision three-dimensional reconstruction of the human eye in a macro environment.

[0094] (2) The accuracy of calibration parameters. The complex structure of the human eye places extremely high demands on the accuracy of camera calibration. In a macro environment, any tiny error (such as deviation in camera position, angle, and focal length) will be significantly amplified, affecting the accuracy of reconstruction. Traditional calibration methods may not be sufficient to meet these challenges. The present invention ensures high-precision calibration of the human eye in a macro environment through an improved camera calibration method, thereby improving the accuracy of the overall three-dimensional reconstruction.

[0095] (3) Feature matching accuracy. The large number of repetitive texture areas (such as the sclera and iris) in human eye images makes feature matching extremely difficult. Traditional feature matching algorithms are prone to mismatching in these areas, which in turn affects the accuracy of 3D reconstruction. By optimizing the feature matching algorithm, the present invention can maintain high-precision matching when processing complex macro scenes of the human eye, thereby further improving the accuracy of the overall 3D reconstruction.

[0096] (4) Automation and usability of the overall system. Existing 3D reconstruction processes for the human eye typically require extensive manual adjustments, resulting in complex operations and difficulty meeting the automation requirements of practical applications. This invention automates the reconstruction process, making it simpler and more convenient to perform 3D reconstruction of the human eye, reducing manual intervention and improving the system's usability and efficiency.

[0097] Figure 1 : is a flowchart of a method for reconstructing human eyes based on multi-view stereo vision provided by the first embodiment of the present invention, the method comprising:

[0098] Step S101, hardware setting step: calculating the intersection of depth of field ranges of multiple cameras, and selecting shooting distances corresponding to the multiple cameras within the intersection of the depth of field ranges.

[0099] Through the hardware setting steps, it is possible to effectively achieve co-focusing of multiple cameras in a macro environment, ensuring the acquisition of clear and accurate image data, thereby achieving high-precision three-dimensional reconstruction of the human eye in a macro environment.

[0100] Step S102, camera calibration step: select a target size that matches the size of the human eye, synchronously collect calibration images with multiple cameras, and calculate camera calibration parameters based on the multiple calibration images.

[0101] By improving the camera calibration steps, high-precision calibration is ensured in macro environments, effectively reducing the impact of calibration errors on 3D reconstruction accuracy. It is particularly suitable for capturing the complex geometric structures of the human eye.

[0102] Step S103, feature matching step: cropping multiple calibration images to obtain corresponding multiple human eye images, extracting at least one key feature point from the multiple human eye images, and calculating the feature matching relationship of the key feature points.

[0103] Preferably, in this step, the multi-object segmentation model is used to crop the multiple calibration images to obtain corresponding multiple human eye images.

[0104] Through the feature matching step, an optimized feature matching algorithm is developed for repetitive texture areas in eye images, such as the sclera and iris, to address the matching errors common in traditional methods. This algorithm maintains high-precision feature matching in complex macro scenes, improving the overall accuracy of 3D reconstruction of the eye and the detail restoration of the model.

[0105] Step S104, 3D reconstruction step: calculating the human eye point cloud information based on the camera calibration parameters and feature matching relationship.

[0106] By optimizing the reconstruction process, the entire 3D reconstruction process is made more automated, reducing the need for human intervention and improving the system's usability and efficiency.

[0107] In this way, the present invention can improve the accuracy of three-dimensional reconstruction of the human eye, and enhance its usability and efficiency. Specifically, the present invention improves the accuracy and efficiency of the system as a whole by improving the hardware settings, camera calibration, feature matching algorithm and automated process design for three-dimensional reconstruction of the human eye in a macro environment. It achieves the joint focusing of multiple cameras to obtain clearer and more accurate image data; improves the accuracy of camera calibration, and reduces the impact of errors on three-dimensional reconstruction; optimizes the feature matching algorithm, reduces the mismatch rate, and enhances the detail restoration of the model; and through process automation, reduces human intervention, and improves the usability and processing efficiency of the system. The present invention significantly improves the accuracy, stability and automation level of three-dimensional reconstruction of the human eye in a macro environment.

[0108] Figure 2 : is a flowchart of the hardware setting steps in the human eye reconstruction method based on multi-eye stereo vision provided by the second embodiment of the present invention, and the hardware setting steps further include:

[0109] In step S201 , an m*n camera array is used and arranged in an array form, where m and n are natural numbers greater than or equal to 2, and m and n may be the same or different.

[0110] Preferably, the system uses a 2x2 camera array, with a total of 4 cameras (Canon R5), arranged in an array form to ensure that the human eye can be photographed in all directions from different angles.

[0111] Step S202: Adjust the aperture of a single camera to f11, ISO to 100, and shutter time to 1 second.

[0112] Step S203, as Figure 3 The depth of field model schematic diagram shown in the figure represents the physical process of imaging. The front depth of field ΔL1, the back depth of field ΔL2 and the depth of field range ΔL of multiple cameras are calculated respectively by formulas 1 to 3.

[0113]

[0114] Where δ represents the diameter of the permissible circle of confusion, which is the maximum allowed blur circle in the image. f represents the focal length of the lens, which is the distance from the optical center of the lens to the imaging plane. F represents the aperture value of the lens, which controls the aperture size. L represents the focus distance, which is the distance of the subject at which the lens focuses.

[0115] Step S204: Obtain an intersection of depth of field ranges according to the depth of field ranges of the multiple cameras.

[0116] The cameras surround the subject, minimizing the distance between them.

[0117] Step S205 : selecting a shortest shooting distance within the intersection range of the depth of field ranges that enables multiple cameras to form clear images in the human eye area.

[0118] like Figure 4 In the overlapping depth of field model shown, there is some depth of field loss when multiple cameras are co-focused. Based on the minimum shooting distance corresponding to the depth of field range calculated in step 2, move a little further. Select an optimal shooting distance that allows multiple cameras to clearly image the human eye area.

[0119] Step S206 , preventing the flash from entering the central area surrounded by the cameras, so that each camera receives light evenly.

[0120] Step S207: Control the pupil of the human eye to contract by using the lighting.

[0121] Preferably, the pupil contraction of the human eye is controlled by LED lights to collect data.

[0122] Figure 5 : is a flowchart of the camera calibration step in the human eye reconstruction method based on multi-view stereo vision provided by the third embodiment of the present invention, wherein the camera calibration step further includes:

[0123] Step S501: Select a target size that matches the size of the human eye. Use a high-precision checkerboard as the target, with a square length of 2 mm. Selecting a target size that matches the size of the human eye ensures that all corners of the target can be accurately captured in a macro environment.

[0124] In step S502, multiple cameras synchronously capture calibration images to ensure that corner points are clearly imaged in multiple cameras.

[0125] Step S503: performing image filtering on all collected calibration images to remove noise.

[0126] Step S504: performing corner detection on all collected calibration images.

[0127] In step S505 , each camera records the image numbers of the complete checkerboard corner points detected, and the intersection of the image numbers of the corner points detected by all cameras is taken as the final calibration image set.

[0128] Step S506: Select a main camera from the multiple cameras as the first camera A. Then select a second camera B with a shorter baseline than the first camera A. Given a certain shooting distance, a shorter baseline means a smaller angle between them. Given the intrinsic parameters of the first and second cameras A and B and their corresponding key feature points, use the epipolar constraint equation to solve the rotation matrix R and translation vector T between the first and second cameras A and B:

[0129]

[0130] Among them, x A and x B are the coordinates of the image points in the first camera A and the second camera B respectively, and F is the basic matrix. The basic matrix F is calculated by the camera's intrinsic parameter matrix K, rotation matrix R and translation vector T:

[0131]

[0132] Step S507: After the intrinsic parameters and relative pose of the first camera pair (A, B) are known, the three-dimensional coordinates of the calibration plate are obtained by triangulation equations. For the remaining camera sets {C1, C2, ..., C n}, and obtain its relative position relative to the first camera A through the PNP algorithm, that is, the rotation matrix and translation vectors

[0133]

[0134] in, is the intrinsic parameter matrix of the i-th camera, X is the three-dimensional point cloud coordinate, is the coordinate of the image point in the i-th camera.

[0135] In step S508, after obtaining the initial intrinsic and extrinsic parameters, the calibration accuracy needs to be further improved through nonlinear optimization. The optimization process adopts the method of minimizing the reprojection error, that is, adjusting the optimizable parameters by optimizing the objective function to minimize the reprojection error of all image points. The optimization objective function is as follows:

[0136]

[0137] Among them, K i is the intrinsic parameter matrix of the i-th camera, R i and T i are the rotation matrix and translation vector of the i-th camera, X j is the jth three-dimensional point, x ij is the image coordinate of the jth point in the i-th camera, π(·) is the projection function, and the internal and external parameters are iteratively adjusted through an optimization algorithm (such as the Levenberg-Marquardt algorithm) to finally obtain accurate camera calibration parameters.

[0138] Figure 6 : is a flow chart of a feature matching step in a method for reconstructing human eyes based on multi-eye stereo vision provided by a fourth embodiment of the present invention, wherein the feature matching step further includes:

[0139] Step S601: Use a Gaussian filter to remove noise from the calibration image to ensure image quality during feature extraction.

[0140] Step S602: Adjust the white balance and brightness of the calibration image to unify the visual characteristics of the multi-view images and reduce the matching error caused by different lighting conditions.

[0141] Step S603 : performing distortion correction on the calibration image based on the camera calibration parameters to eliminate the geometric distortion introduced by the camera lens.

[0142] Step S604 , cropping the multiple calibration images to obtain corresponding multiple human eye images.

[0143] Preferably, in this step, the multi-object segmentation model is used to crop the multiple calibration images to obtain the corresponding multiple human eye images. Since we only need to reconstruct the geometric shape of the human eye area, we need to crop all the images.

[0144] In step S605, one image is selected from the multiple human eye images and defined as the main view. The other images are defined as reference views. Rough image matching is performed on the main view and the reference view to obtain a first sparse feature matching relationship. This process is defined as the matching points of any two images I1 and I2 are K1 and K2, which are expressed as follows:

[0145] (k1,k2)=RoMa(I1,I2)

[0146] Step S606: Calculate a projection matrix t using the obtained K1 and K2, and project the reference view onto the main view.

[0147] Step S607: performing image matching on the projected image to obtain a second sparse feature matching relationship.

[0148] Step S608: perform epipolar verification on the matched sparse feature points:

[0149]

[0150] Here, F is the basic matrix, and the parameter d is specified to represent a threshold to filter out point pairs that do not satisfy the epipolar constraint. The Sampson distance of the matching points is compared with the preset threshold d = 0.01, and points greater than the threshold are filtered out to obtain the verified points:

[0151]

[0152] Step S409: interpolate the second sparse feature matching relationship to obtain a dense feature matching relationship as the feature matching relationship.

[0153] Figure 73D reconstruction is a flowchart of a three-dimensional reconstruction step in a method for reconstructing human eyes based on multi-eye stereo vision provided by a fifth embodiment of the present invention. The three-dimensional reconstruction step further includes:

[0154] Step S701 : Calculate a three-dimensional point cloud using a triangulation equation based on dense feature matching relationships and camera calibration parameters.

[0155] Step S702, set a main view I ref , the remaining views are represented as a set {I c}, calculate the 3D point clouds of the remaining views and the main view respectively.

[0156] Step S703: average the three-dimensional point cloud coordinates to obtain the final human eye point cloud information:

[0157]

[0158] Where N is the number of views, X i The 3D point cloud coordinates for each view.

[0159] In the human eye reconstruction method based on multi-view stereo vision provided in the sixth embodiment of the present invention, the system automates the process from image acquisition to 3D model generation. The key steps include:

[0160] S801, all cameras in Example 1 are triggered synchronously using wired signals during acquisition. Simply press the main control button to synchronize the captured data and store it in a server or SD card.

[0161] S802: Perform automatic multi-thread batch conversion on all collected images using predetermined color correction parameters, and output the processed images in PNG format.

[0162] S803, the camera calibration process in Example 2, automatically screens the available calibration images for each camera and takes the intersection of all cameras. Fully automatic calibration is performed through the set calibration steps, and the calibration results are written to the calibration folder.

[0163] S804: Since the feature matching step requires human eye cropping, this manual processing is very time-consuming. The SAM (SegmentAnything Model) model is used to segment the original image, greatly speeding up the processing speed.

[0164] The above embodiments describe in detail the specific implementation method of the present invention for three-dimensional reconstruction of the human eye in a macro environment. Through these embodiments, the technical advantages of the present invention in improving reconstruction accuracy and achieving automation are demonstrated.

[0165] The core of this invention is to provide a complete and optimized 3D reconstruction process for the human eye, specifically designed for the special needs of human eye reconstruction in macro environments. It includes key links such as camera calibration, feature matching, and multi-view data fusion, ultimately realizing a high-precision 3D reconstruction system for the human eye and improving the automation level of the overall process.

[0166] The key points of the present invention include the following aspects, and the corresponding technical effects are as follows:

[0167] (1) Hardware Setup Method for Human Eye Reconstruction in Macro Environments: This paper proposes a hardware system design method specifically for human eye reconstruction in macro scenarios, addressing the difficulty of achieving co-focusing of multiple cameras under macro conditions. This setup method effectively achieves co-focusing of multiple cameras in macro environments, ensuring the acquisition of clear and accurate image data, thereby achieving high-precision three-dimensional reconstruction of the human eye in macro environments, overcoming the limitations of traditional methods in similar scenarios.

[0168] (2) High-precision camera calibration method: This invention improves the camera calibration method to ensure high-precision calibration in macro environments, making it particularly suitable for capturing the complex geometric structures of the human eye. By improving calibration accuracy, the impact of calibration errors on 3D reconstruction accuracy is effectively reduced, thereby improving the accuracy of the final 3D model of the human eye and ensuring reliability under high-precision requirements.

[0169] (3) Optimized feature matching algorithm: This paper develops an optimized feature matching algorithm for repetitive texture regions (such as the sclera and iris) in human eye images to address the matching errors common in traditional methods. This algorithm maintains high-precision feature matching in complex macro scenes, reducing the occurrence of mismatches and thereby improving the overall accuracy of 3D reconstruction of the human eye and the detail restoration of the model.

[0170] (4) Automated reconstruction process design: By optimizing the reconstruction process, the invention makes the entire 3D reconstruction process more automated, reducing the need for human intervention. This improves the system's usability and efficiency, reduces operational complexity and the possibility of human error, and makes the human eye 3D reconstruction system more suitable for practical application scenarios, especially those that require efficient processing of large amounts of data.

[0171] It should be noted that the method for reconstructing the human eye based on multi-eye stereo vision provided in the embodiments of the present invention can be executed by an electronic device, apparatus, or a control module within the apparatus for executing the method. The embodiments of the present invention use an apparatus executing the method as an example to illustrate the apparatus for reconstructing the human eye based on multi-eye stereo vision provided in the embodiments of the present invention.

[0172] Figure 81 is a schematic diagram of the structure of a human eye reconstruction device based on multi-view stereo vision provided in the first embodiment of the present invention. The device 100 includes a hardware setting module 10, a camera calibration module 20, a feature matching module 30, and a 3D reconstruction module 40, wherein:

[0173] The hardware setting module 10 is used to calculate the intersection of the depth of field ranges of multiple cameras and select shooting distances corresponding to the multiple cameras within the intersection of the depth of field ranges.

[0174] The camera calibration module 20 is used to select a target size that matches the size of the human eye, and multiple cameras synchronously capture calibration images, and calculate camera calibration parameters based on the multiple calibration images.

[0175] The feature matching module 30 is used to crop multiple calibration images to obtain corresponding multiple human eye images, extract at least one key feature point from the multiple human eye images, and calculate the feature matching relationship of the key feature points.

[0176] The three-dimensional reconstruction module 40 is used to calculate the human eye point cloud information based on the camera calibration parameters and feature matching relationship.

[0177] Preferably, the hardware setting module 10 further performs the following actions:

[0178] The foreground depth of field ΔL1, the back depth of field ΔL2, and the depth of field range ΔL of multiple cameras are calculated using Formulas 1 to 3, respectively.

[0179]

[0180]

[0181] Where δ represents the diameter of the permissible circle of confusion, which is the maximum allowed blur circle in the image. f represents the focal length of the lens, which is the distance from the optical center of the lens to the imaging plane. F represents the aperture value of the lens, which controls the aperture size. L represents the focus distance, which is the distance of the subject at which the lens focuses.

[0182] The depth of field range intersection is obtained according to the depth of field ranges of multiple cameras.

[0183] Within the intersection of the depth of field ranges, select the shortest shooting distance that allows multiple cameras to form clear images in the human eye area.

[0184] The step of respectively calculating the front depth of field ΔL1, the back depth of field ΔL2 and the depth of field range ΔL of the plurality of cameras by using formulas 1 to 3 includes:

[0185] An m*n camera array is used and arranged in an array form, where m and n are natural numbers greater than or equal to 2, and m and n can be the same or different.

[0186] Adjust the aperture of a single camera to f11, ISO to 100, and shutter time to 1s.

[0187] After the step of selecting a shortest shooting distance within the intersection range of the depth of field ranges so that the multiple cameras can clearly image the human eye area, the following steps are performed:

[0188] Prevents the flash from entering the central area surrounding the camera.

[0189] The pupil of the human eye is controlled to contract by lighting.

[0190] Preferably, the camera calibration module 20 further performs the following actions:

[0191] A target size that matches the size of the human eye was selected, and a checkerboard pattern was used as the target, with a square edge length of 2 mm.

[0192] Multiple cameras synchronously capture calibration images to ensure that corner points are clearly imaged in multiple cameras.

[0193] All collected calibration images are filtered to remove noise.

[0194] Perform corner detection on all collected calibration images.

[0195] Each camera records the image number in which the complete checkerboard corner points are detected, and the intersection of the image numbers in which the corner points can be detected by all cameras is taken as the final calibration image set.

[0196] Select a main camera from multiple cameras as the first camera A. Then select a second camera B with a shorter baseline than the first camera A. Given the intrinsic parameters of the first and second cameras A and B and their corresponding key feature points, use the epipolar constraint equation to solve the rotation matrix R and translation vector T between the first camera A and the second camera B:

[0197]

[0198] Among them, x A and x B are the coordinates of the image points in the first camera A and the second camera B respectively, and F is the basic matrix. The basic matrix F is calculated by the camera's intrinsic parameter matrix K, rotation matrix R and translation vector T:

[0199]

[0200] After the intrinsic parameters and relative pose of the first camera A pair (A, B) are known, the three-dimensional coordinates of the calibration plate are obtained by triangulation equations. For the remaining camera sets {C1, C2, ..., C n}, and obtain its relative position relative to the first camera A through the PNP algorithm, that is, the rotation matrix and translation vectors

[0201]

[0202] in, is the intrinsic parameter matrix of the i-th camera, X is the three-dimensional point cloud coordinate, is the coordinate of the image point in the i-th camera.

[0203] After obtaining the initial internal and external parameters, the optimizable parameters are adjusted by optimizing the objective function to minimize the reprojection error of all image points. The optimization objective function is as follows:

[0204]

[0205] Among them, K i is the intrinsic parameter matrix of the i-th camera, R i and T i are the rotation matrix and translation vector of the i-th camera, X j is the jth three-dimensional point, x ij is the image coordinate of the jth point in the i-th camera, π(·) is the projection function, and the internal and external parameters are iteratively adjusted through the optimization algorithm to finally obtain the accurate camera calibration parameters.

[0206] Preferably, the feature matching module 30 further performs the following actions:

[0207] Use Gaussian filter to remove noise from the calibration image.

[0208] Adjust the white balance and brightness of the calibration image.

[0209] The calibration image is distorted based on the camera calibration parameters.

[0210] Cropping the multiple calibration images to obtain the corresponding multiple human eye images. Preferably, the multiple calibration images are cropped using the segmentation of everything model to obtain the corresponding multiple human eye images.

[0211] One image is selected from multiple human eye images and defined as the main view. The other images are defined as reference views. The main view and the reference view are roughly matched to obtain the first sparse feature matching relationship. This process is defined as the matching points of any two images I1 and I2 are K1 and K2, which are expressed as follows:

[0212] (k1,k2)=RoMa(I1,I2)

[0213] A projection matrix t is calculated using the obtained K1 and K2 to project the reference view onto the main view.

[0214] Image matching is performed on the projected image to obtain a second sparse feature matching relationship.

[0215] Perform epipolar verification on the sparse feature points after matching:

[0216]

[0217] Here, F is the basic matrix, and the parameter d is specified to represent a threshold to filter out point pairs that do not satisfy the epipolar constraint. The Sampson distance of the matching points is compared with the preset threshold d = 0.01, and points greater than the threshold are filtered out to obtain the verified points:

[0218]

[0219] The second sparse feature matching relationship is interpolated to obtain a dense feature matching relationship as the feature matching relationship.

[0220] Preferably, the 3D reconstruction module 30 further performs the following actions:

[0221] According to the dense feature matching relationship and camera calibration parameters, the three-dimensional point cloud is calculated through the triangulation equation.

[0222] Set a main view I ref , the remaining views are represented as a set {I c}, calculate the 3D point clouds of the remaining views and the main view respectively.

[0223] Average the three-dimensional point cloud coordinates to obtain the final human eye point cloud information:

[0224]

[0225] Where N is the number of views, X i The 3D point cloud coordinates for each view.

[0226] The human eye reconstruction device based on multi-eye stereo vision provided by the embodiment of the present invention can achieve Figures 1 to 7 To avoid repetition, the various processes implemented in the embodiment of the human eye reconstruction method based on multi-eye stereo vision are not described here.

[0227] The human eye reconstruction device based on multi-camera stereo vision provided by an embodiment of the present invention includes: a hardware setting module for calculating the intersection of the depth of field ranges of multiple cameras and selecting the corresponding shooting distances of the multiple cameras within the depth of field range intersection. It can effectively achieve common focusing of multiple cameras in a macro environment, ensuring the acquisition of clear and accurate image data, thereby achieving high-precision three-dimensional reconstruction of the human eye in a macro environment. A camera calibration module is used to select a target size that matches the size of the human eye, and multiple cameras synchronously capture calibration images, and calculate camera calibration parameters based on the multiple calibration images. By improving the camera calibration method, high-precision calibration is ensured in a macro environment, effectively reducing the impact of calibration errors on the accuracy of three-dimensional reconstruction, and is particularly suitable for capturing complex geometric structures of the human eye. A feature matching module is used to crop multiple calibration images to obtain corresponding multiple human eye images, extract at least one key feature point from the multiple human eye images, and calculate the feature matching relationship of the key feature points. An optimized feature matching algorithm is used for repeated texture areas (such as sclera and iris) in human eye images to solve the matching error problem commonly found in traditional methods. This algorithm can maintain high-precision feature matching in complex macro scenes, improving the overall accuracy of 3D reconstruction of the human eye and the detail restoration of the model. The 3D reconstruction module is used to calculate the eye point cloud information based on the camera calibration parameters and the feature matching relationship. By optimizing the reconstruction process, the entire 3D reconstruction process is made more automated, reducing the need for human intervention and improving the system's usability and efficiency. This method can thereby improve the accuracy, usability, and efficiency of 3D reconstruction of the human eye in macro environments.

[0228] The present invention also provides a storage medium for storing Figures 1 to 7 Any computer program of the human eye reconstruction method based on multi-eye stereo vision. For example, a computer program instruction, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer, and can achieve the same technical effect. To avoid repetition, it will not be repeated here. The program instruction for calling the method of the present invention may be stored in a fixed or removable storage medium, and / or transmitted through a data stream in a broadcast or other signal-bearing medium and / or stored in a storage medium of a computer device running according to the program instruction.

[0229] According to one embodiment of the present invention, the present invention also provides a Figure 9The electronic device 400 shown in the figure optionally includes a storage medium 200 for storing a computer program and a processor 300 for executing the computer program, wherein when the computer program is executed by the processor 300, any of the above-mentioned methods for human eye reconstruction based on multi-eye stereo vision is implemented, triggering the electronic device 400 to execute the methods and / or technical solutions based on the aforementioned multiple embodiments, and can achieve the same technical effects. To avoid repetition, they are not described here. It should be noted that the electronic devices in the embodiments of the present invention include mobile electronic devices and non-mobile electronic devices. For example, the mobile electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a wearable device, an ultra-mobile personal computer, a netbook or a personal digital assistant, etc., and the non-mobile electronic device can be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), an ATM or a self-service machine, etc., which is not specifically limited in the embodiments of the present invention.

[0230] It should be noted that the present invention can be implemented in software and / or a combination of software and hardware, for example, can be implemented using an application specific integrated circuit (ASIC), a general purpose computer or any other similar hardware device. In one embodiment, the software program of the present invention can be executed by a processor to implement the above steps or functions. Similarly, the software program of the present invention (including related data structures) can be stored in a computer-readable recording medium, for example, a RAM memory, a magnetic or optical drive or a floppy disk and similar devices. In addition, some steps or functions of the present invention can be implemented using hardware, for example, as a circuit that cooperates with a processor to perform each step or function.

[0231] The present invention can be implemented as a computer-implemented method on a computer, or in dedicated hardware, or a combination thereof. The executable code for the method according to the present invention, or portions thereof, can be stored on a computer program product. Examples of computer program products include memory devices, optical storage devices, integrated circuits, servers, online software, and the like. Optionally, the computer program product includes non-transitory program code components stored on a computer-readable medium so that when the program product is executed on a computer, the method according to the present invention is executed.

[0232] In an alternative embodiment, the computer program comprises computer program code means adapted to perform all the steps of the method according to the invention when the computer program is run on a computer.Alternatively, the computer program is embodied on a computer readable medium.

[0233] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present invention is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in the opposite order according to the functions involved. For example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0234] Of course, the present invention may have many other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art may make various corresponding changes and modifications based on the present invention, but these corresponding changes and modifications should all fall within the scope of protection of the claims attached to the present invention.

Claims

1. A method for reconstructing human eyes based on multi-eye stereo vision, characterized in that: include: A hardware setting step of calculating the intersection of depth of field ranges of multiple cameras, and selecting shooting distances corresponding to the multiple cameras within the intersection of the depth of field ranges; a camera calibration step, selecting a target size that matches the size of the human eye, synchronously acquiring calibration images with the multiple cameras, and calculating camera calibration parameters based on the multiple calibration images; a feature matching step of cropping the plurality of calibration images to obtain a plurality of corresponding human eye images, extracting at least one key feature point from the plurality of human eye images, and calculating a feature matching relationship of the key feature point; The three-dimensional reconstruction step calculates the human eye point cloud information based on the camera calibration parameters and the feature matching relationship.

2. The method for human eye reconstruction based on multi-eye stereo vision according to claim 1, characterized in that: The hardware setting step further includes: Calculate the front depth of field ΔL1, the back depth of field ΔL2 and the depth of field range ΔL of the multiple cameras respectively using Formulas 1 to 3; Where δ represents the diameter of the permissible circle of confusion, which is the maximum blur circle allowed in the image; f represents the focal length of the lens, that is, the distance from the optical center of the lens to the imaging plane; F represents the aperture value of the lens, which is used to control the aperture size; L represents the focusing distance, that is, the distance of the subject at which the lens focuses; Obtaining a depth of field range intersection according to the depth of field ranges of the multiple cameras; Within the intersection range of the depth of field ranges, a shortest shooting distance that enables the multiple cameras to form clear images in the human eye area is selected.

3. The method for human eye reconstruction based on multi-eye stereo vision according to claim 1, characterized in that: The step of respectively calculating the front depth of field ΔL1, the back depth of field ΔL2 and the depth of field range ΔL of the multiple cameras by using Formulas 1 to 3 includes: An m*n camera array is used and arranged in an array form, where m and n are natural numbers greater than or equal to 2, and m and n can be the same or different; Adjust the aperture of a single camera to f11, ISO to 100, and shutter time to 1s; After the step of selecting a shortest shooting distance within the intersection range of the depth of field ranges that enables the multiple cameras to clearly image the human eye area, the following steps are included: Prevent the flash from entering the central area around the camera; The pupil of the human eye is controlled to contract by lighting.

4. The method for human eye reconstruction based on multi-eye stereo vision according to claim 1, characterized in that: The camera calibration step further comprises: Select a target size that matches the size of the human eye and use a checkerboard as the target, where the side length of the checkerboard is 2 mm; The multiple cameras synchronously capture the calibration image to ensure that the corner points are clearly imaged in the multiple cameras; Performing image filtering on all collected calibration images to remove noise; Performing corner detection on all the collected calibration images; Each camera records the image number in which the complete checkerboard corner points are detected, and the intersection of the image numbers in which the corner points can be detected by all the cameras is taken as the calibration image set finally used; A main camera is selected from the multiple cameras as the first camera A; a second camera B is selected with a shorter baseline than the first camera A. Given the intrinsic parameters of the first camera A and the second camera B and their corresponding key feature points, the epipolar constraint equation is used to solve the rotation matrix R and the translation vector T between the first camera A and the second camera B: Among them, x A and x B are the coordinates of the image points in the first camera A and the second camera B respectively, and F is the basic matrix. The basic matrix F is calculated by the camera's intrinsic parameter matrix K, rotation matrix R and translation vector T: After the intrinsic parameters and relative pose of the first camera pair (A, B) are known, the three-dimensional coordinates of the calibration plate are obtained by triangulation equations. For the remaining camera set {C1, C2, ..., C n }, and obtain its relative position relative to the first camera A through the PNP algorithm, that is, the rotation matrix and translation vectors in, is the intrinsic parameter matrix of the i-th camera, X is the three-dimensional point cloud coordinate, is the coordinate of the image point in the i-th camera; After obtaining the initial internal and external parameters, the optimizable parameters are adjusted by optimizing the objective function to minimize the reprojection error of all image points. The optimization objective function is as follows: Among them, K i is the intrinsic parameter matrix of the i-th camera, R i and T i are the rotation matrix and translation vector of the i-th camera, X j is the jth three-dimensional point, x ij is the image coordinate of the jth point in the i-th camera, π(·) is the projection function, and the internal and external parameters are iteratively adjusted through the optimization algorithm to finally obtain the accurate camera calibration parameters.

5. The method for human eye reconstruction based on multi-eye stereo vision according to claim 1, characterized in that: The feature matching step further comprises: Using a Gaussian filter to remove noise from the calibration image; Adjusting the white balance and brightness of the calibration image; performing distortion correction on the calibration image based on the camera calibration parameters; Cropping the plurality of calibration images to obtain the corresponding plurality of human eye images; One image is selected from the multiple human eye images and defined as a main view. The other images are defined as reference views. The main view and the reference view are respectively subjected to rough image matching to obtain a first sparse feature matching relationship. This process is defined as the matching points of any two images I1 and I2 are K1 and K2, which are expressed as follows: (k1,k2)=RoMa(I1,I2) Calculate a projection matrix t using K1 and K2 to project the reference view onto the main view; Perform image matching on the projected image to obtain a second sparse feature matching relationship; Perform epipolar verification on the sparse feature points after matching: Here, F is the basic matrix, and the parameter d is specified to represent a threshold to filter out point pairs that do not satisfy the epipolar constraint. The Sampson distance of the matching points is compared with the preset threshold d = 0.01, and points greater than the threshold are filtered out to obtain the verified points: The second sparse feature matching relationship is interpolated to obtain a dense feature matching relationship as the feature matching relationship.

6. The method for human eye reconstruction based on multi-eye stereo vision according to claim 1, characterized in that: In the feature matching step, the multi-object segmentation model is used to crop the multiple calibration images to obtain the corresponding multiple human eye images.

7. The method for human eye reconstruction based on multi-eye stereo vision according to claim 5, characterized in that: The three-dimensional reconstruction step further comprises: Calculating a three-dimensional point cloud using a triangulation equation according to the dense feature matching relationship and the camera calibration parameters; Set a main view I ref , the remaining views are represented as a set {I c }, respectively obtain the three-dimensional point clouds of the remaining views and the main view; The three-dimensional point cloud coordinates are averaged to obtain the final human eye point cloud information: Where N is the number of views, X i The 3D point cloud coordinates for each view.

8. A human eye reconstruction device based on multi-eye stereo vision constructed based on the method according to any one of claims 1 to 7, characterized in that: include: A hardware setting module, configured to calculate an intersection of depth-of-field ranges of multiple cameras, and select shooting distances corresponding to the multiple cameras within the intersection of the depth-of-field ranges; A camera calibration module is used to select a target size that matches the size of the human eye, the multiple cameras synchronously capture calibration images, and calculate camera calibration parameters based on the multiple calibration images; a feature matching module, configured to crop the plurality of calibration images to obtain a plurality of corresponding human eye images, extract at least one key feature point from the plurality of human eye images, and calculate a feature matching relationship of the key feature point; The three-dimensional reconstruction module is used to calculate the human eye point cloud information based on the camera calibration parameters and the feature matching relationship.

9. A storage medium, characterized in that: Used to store a computer program for executing the human eye reconstruction method based on multi-eye stereo vision according to any one of claims 1 to 7.

10. An electronic device comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein: When the processor executes the computer program, the method for reconstructing the human eye based on multi-view stereo vision according to any one of claims 1 to 7 is implemented.

Images