Small-particle-size seed microscopic three-dimensional reconstruction device and size feature extraction method

Abstract

The invention relates to the technical field of agricultural breeding phenotype detection, in particular to a small-particle-size seed microscopic three-dimensional reconstruction device and a size feature extraction method.The small-particle-size seed microscopic three-dimensional reconstruction device comprises a rack and an image acquisition mechanism and further comprises a horizontal rotating mechanism, a light supplementing mechanism and a main control system; the image collecting mechanism, the horizontal rotating mechanism and the light supplementing mechanism are all arranged in the rack, the horizontal rotating mechanism is installed on the lower portion of the rack, to-be-detected seeds are placed on the horizontal rotating mechanism, the light supplementing mechanism and the image collecting mechanism are installed on the upper portion of the rack and conduct light supplementing and image collecting on the to-be-detected seeds, and the main control system is arranged on the side of the rack. And the image acquisition mechanism, the horizontal rotating mechanism and the light supplementing mechanism are electrically connected with the main control system. According to the method, high-quality three-dimensional reconstruction and size feature accurate extraction of the small-particle-size seeds can be realized, so that three-dimensional digital reconstruction and size feature detection of the small-particle-size seeds are completed, the detection precision and efficiency are improved, and the development of the breeding industry is promoted.

Description

Technical Field

[0001] The present invention relates to the technical field of agricultural breeding phenotype detection, and in particular to a small-diameter seed microscopic three-dimensional reconstruction device and a size feature extraction method. Background Art

[0002] Small-grained seeds (such as wheat and millet) are an important component of my country's agricultural germplasm resources. Their morphological detection and size feature extraction play a crucial role in seed breeding, facilitating the prediction of seed germination performance and growth potential, improving breeding efficiency, and selecting superior germplasm resources. Currently, the extraction of phenotypic characteristics such as seed size is primarily achieved through manual measurement, which results in low collection efficiency and is susceptible to human influence. While high-precision scanners and other precision instruments can achieve high-quality three-dimensional digital reconstruction and high-precision size feature extraction, their high cost and limitations in imaging equipment, conditions, and technical means have hindered widespread application in the field of phenotypic collection and feature extraction of small-grained seeds.

[0003] Moreover, due to the small size of seeds and the difficulty in extracting size features, existing visual methods are unable to achieve effective point cloud reconstruction of small-particle seeds, and currently there is still no high-quality reconstruction solution for small-particle seeds. Summary of the Invention

[0004] In view of the shortcomings of the existing background technology, the purpose of the present invention is to provide a high-precision, low-cost microscopic three-dimensional reconstruction device for small-particle seeds, and proposes a size feature extraction method, which can achieve high-quality three-dimensional reconstruction and precise extraction of size features of small-particle seeds, and then complete the three-dimensional digital reconstruction and size feature detection of small-particle seeds, improve detection accuracy and efficiency, and have important significance in promoting the development of the breeding industry.

[0005] To achieve the above, the technical solution of the present invention is: a small-particle seed microscopic three-dimensional reconstruction device, including a frame and an image acquisition mechanism, and also including a horizontal rotation mechanism, a fill light mechanism and a main control system. The image acquisition mechanism, horizontal rotation mechanism and fill light mechanism are all arranged in the frame, the horizontal rotation mechanism is installed at the lower part of the frame, the seeds to be detected are placed on the horizontal rotation mechanism, the fill light mechanism and the image acquisition mechanism are installed at the upper part of the frame, and fill light and image acquisition are performed on the seeds to be detected. The main control system is arranged next to the frame, and the image acquisition mechanism, horizontal rotation mechanism and fill light mechanism are all electrically connected to the main control system.

[0006] Furthermore, the frame includes a profile frame, a panel and a door panel. The panel is installed on the upper and lower sides, two sides and back of the profile frame, and the door panel is installed on the front of the profile frame. The image acquisition mechanism, fill light mechanism and horizontal rotation mechanism are located in the profile frame.

[0007] Furthermore, the horizontal rotation mechanism includes a base, a bearing, a platform, an electric motor and a speed regulator. The base is fixedly connected to the panel at the bottom of the profile frame. The base is provided with a space for accommodating the electric motor. The electric motor is fixedly connected in the base. The platform is located above the base. The output shaft of the electric motor is connected to the platform, and the electric motor drives the platform to rotate; the bearing is arranged between the base and the platform, the outer ring of the bearing is connected to the base, the inner ring of the bearing is connected to the platform and rotates with the platform, and the speed regulator is installed on the base and electrically connected to the electric motor, and is responsible for adjusting the speed of the electric motor.

[0008] Furthermore, the image acquisition mechanism includes an electric bracket, an industrial camera and a microscope magnification lens. One end of the electric bracket is fixedly connected to the top of the profile frame through a mounting plate, the industrial camera is connected to the other end of the electric bracket, and the microscope magnification lens is installed in front of the industrial camera head.

[0009] Furthermore, the electric bracket includes a deflection joint and a pitch joint, the industrial camera is installed on the pitch joint, the pitch joint is connected to the deflection joint, and the industrial camera performs deflection and pitch movements; the microscope magnification lens includes a light adjustment ring, a focus ring and a microscope magnification lens, one end of the light adjustment ring is connected to the camera head of the industrial camera, the other end of the light adjustment ring is connected to the focus ring, and the focus ring is connected to the microscope magnification lens.

[0010] Furthermore, the fill light mechanism is located above the horizontal rotation mechanism, and the fill light mechanism includes a shell, a control circuit board, a light source board, a diffuser and a driving power supply. The shell is fixedly installed on the top of the profile frame, and the control circuit board, the light source board and the diffuser are installed in the shell in sequence. The driving power supply is installed on the control circuit board, and the driving power supply is electrically connected to the light source board to provide a power source for the light source board. The light control circuit on the control circuit board controls the voltage and current of the light source board and adjusts the light intensity of the fill light mechanism. The diffuser is installed on the outside of the light source board to diffuse and disperse the light emitted by the light source board.

[0011] Furthermore, the main control system includes a computer and a power supply. The computer is connected to the image acquisition mechanism control interface, the fill light mechanism control interface and the horizontal rotation mechanism control interface. The horizontal rotation mechanism, image acquisition mechanism and fill light mechanism are automatically controlled in an integrated manner through the computer operating console, and the seed image magnified by the microscope lens in the industrial camera is observed through the computer display screen to accurately extract the characteristics of the seed sample; the power supply is connected to the horizontal rotation mechanism power interface, the image acquisition mechanism power interface and the fill light mechanism power interface to provide stable power input for the computer, horizontal rotation mechanism, image acquisition mechanism and fill light mechanism.

[0012] The present invention also relates to a method for extracting size features of small-size seeds by microscopic three-dimensional reconstruction. Based on the small-size seed microscopic three-dimensional reconstruction device as described above, the extraction method comprises:

[0013] Step 1. Camera calibration: Use the image acquisition mechanism to take multiple pictures of the black and white checkerboard calibration plate, use the coordinate relationship to calculate the camera's distortion coefficient, and obtain the camera's intrinsic parameter matrix;

[0014] Step 2. Obtaining a magnified image of the seed: Place a single or multiple seeds to be tested on the platform of the horizontal rotating mechanism, start the main control system, the fill light mechanism, and the image acquisition mechanism, and adjust the fill light mechanism and the image acquisition mechanism using the buttons or knobs on the computer console of the main control system so that the surface details of the seed are clearly visible. Use the acquisition setting module in the computer software to set the parameters such as the rotation speed, rotation angle, camera exposure time, and acquisition quantity of the horizontal rotating mechanism to perform image acquisition;

[0015] Step 3. Distortion Correction: For each pixel in the image, calculate the ideal coordinates before distortion removal based on the camera intrinsic parameter matrix and the distortion coefficient, and map the distorted coordinates to the undistorted image coordinates to obtain the corrected image.

[0016] Step 4. Point Cloud Reconstruction: Use the SIFT algorithm to detect feature points using the two rectified images, construct the camera's intrinsic matrix, and extract the relative translation vector and rotation matrix between the two images by solving the camera's intrinsic matrix and singular value decomposition. Repeat the above steps to calculate the absolute rotation matrix and displacement vector of all images in the 3D world coordinate system. Construct the relationship between the 3D coordinates of the point in the world coordinate system and its projected 2D coordinates in the image. Use the least squares method to solve the minimum value to obtain the optimal 3D point coordinates, obtain a sparse point cloud 3D model, and reconstruct a dense point cloud 3D model containing more information based on the sparse point cloud 3D model.

[0017] Step 5. Automatic point cloud segmentation: For each of the M points in the point cloud, seven types of information are established: 3D coordinates (x, y, z), color values (R, G, B), and labels. A point cloud segmentation neural network model is constructed, using a multi-layer perceptron to extract local features for each point. A maximum pooling function is then used to ensure that the model is insensitive to the order of the input points while also obtaining global features. The features of each point are merged with the global features, and segmentation is performed through an additional fully connected layer. The point cloud is segmented using this segmentation network, separating the seed 3D point cloud and saving it in the main control system, ready for the next step.

[0018] Step 6. Data centering: Assume that there are N points p in the point cloud of a seed. i =[x i ,y i , z i ] T, 1≤i≤N; centralize the point cloud data, subtract the mean of all points from the coordinates of each point, and transform the entire point cloud into a new coordinate system so that the center of mass of the point cloud is at the origin of the new coordinate system:

[0019] p′ i =p i -μ

[0020] Among them, p′ i It is p i The new coordinates after data centralization, μ is the coordinate of the mean (center of mass) of the entire point cloud in the initial coordinate system;

[0021] Step 7. Calculate the covariance matrix: Construct the covariance matrix C of the point cloud data in the new coordinate system to further describe the data distribution characteristics in each direction of the point cloud:

[0022]

[0023] Among them, P′ is composed of N p′ i An N×3 matrix composed of vectors;

[0024] Step 8. Eigenvalue decomposition: solve the eigenvalues and eigenvectors of the covariance matrix C, and use eigenvalue decomposition to find the main axis direction of the point cloud: Cv i =λ i v i , where λ i is the i-th eigenvalue of the covariance matrix C, v i is its corresponding eigenvector, where the eigenvector specifies the direction of the point cloud principal axis, and the cloud principal axis is the principal component of the point cloud;

[0025] Step 9. Principal component sorting: The eigenvalues λ1, λ2, and λ3 reflect the variance of the seed point cloud model in the direction of its corresponding eigenvector. Sort the eigenvectors according to the size of the eigenvalues to determine the directions of the three main axes that can represent the length, width, and height of the seed point cloud model: λ1 ≥ λ2 ≥ λ3. Then, select the eigenvectors (v1, v2, v3) corresponding to the eigenvalues in turn as the directions of the length, width, and height of the seed point cloud model respectively.

[0026] Step 10. Model size calculation: Calculate the boundary of the point cloud along each principal component direction, project the point cloud onto the principal component direction, find the minimum and maximum values in these directions, and the difference between them is the 3D size information of the seed point cloud model:

[0027]

[0028] l m =l max -l min , wm =w max -w min , t m =t max -t min

[0029] Among them, l m , w m , t m Represent the length, width and height values of the seed point cloud model respectively;

[0030] Step 11. Correct the magnification factor and determine the seed geometry: Correct the length, width, and height of the seed point cloud model by the magnification factor to restore the true size of the seed:

[0031]

[0032] Among them, K is the magnification of the microscope lens, l, w, and t are the seed length, width, and thickness values extracted by the microscopic three-dimensional reconstruction size feature extraction method.

[0033] Furthermore, the specific process of step 1 is as follows: prepare a calibration plate with regularly arranged black and white checkerboards, use an image acquisition mechanism to take multiple pictures of the calibration plate, and ensure that the corner points of the checkerboard are clearly visible in each image; scan the entire image, detect the corner points of the checkerboard and calculate the coordinates of all corner points; select a corner of the checkerboard as the origin, measure the actual distance between each corner point, and obtain the coordinates of the checkerboard corner point in the three-dimensional world coordinate system; use the captured internal corner point coordinates and the corresponding world coordinates to calculate the camera's intrinsic parameters and distortion coefficients, and express the camera's intrinsic parameters as a matrix F:

[0034]

[0035] Among them, f x 、f y is the focal length of the camera, c x 、c y is the principal point coordinate of the camera;

[0036] For each pixel in the image, the ideal coordinates before distortion removal are calculated based on the camera intrinsic parameter matrix F, and the distorted coordinates are mapped to the undistorted image coordinates to obtain the corrected image.

[0037] Furthermore, the specific process of step 4 is: perform feature point detection on the two images using the SIFT algorithm, generate descriptors based on the gradient information of the local area of the key points, and construct the camera's essential matrix E:

[0038]

[0039] where p i (x i ,y i , 1) and p′ i (x i ′,y i ′, 1) represent the homogeneous coordinates of the i-th pair of matching points between the two images, and then the matching points are solved by constructing a system of equations to solve E; after solving the essential matrix E, the relative translation vector t and rotation matrix R between the two images are further extracted by singular value decomposition, which is in the form of:

[0040] E=U∑V T

[0041] Where U and V are orthogonal matrices, ∑ is a singular value matrix, and the singular vector corresponding to the minimum singular value is the translation vector t, and then:

[0042] E=[t]R

[0043] Recover the rotation matrix R, where [t] is the cross product matrix of t, defined as:

[0044]

[0045] where t x , t y , t z are the values of the translation vector t in three dimensions. Select an image to establish a three-dimensional world coordinate system, and calculate the absolute rotation matrix and displacement vector of all images in the three-dimensional world coordinate system by recovering the relative rotation matrix and displacement vector between each image.

[0046] Construct the three-dimensional coordinates (X P , Y P , Z P ) and the two-dimensional coordinates (x Pi ,y Pi ) and satisfies the projection equation:

[0047] [x Pi y Pi 1] T =F[R i |t i ][X P Y P Z P 1] T

[0048] The optimal three-dimensional coordinates of point P are estimated by the least squares method to minimize the reprojection error of all projection points, thereby obtaining the optimal three-dimensional coordinate estimate of point P:

[0049]

[0050] This method is used to estimate the optimal coordinates of all three-dimensional points in sequence to obtain a sparse point cloud three-dimensional model; the depth value of each pixel is calculated using images from multiple perspectives to generate a depth map, and each pixel in the depth map is converted into a three-dimensional point. Based on the sparse point cloud three-dimensional model, a dense point cloud three-dimensional model containing more information is reconstructed.

[0051] The advantages of adopting the technical solution of the present invention are:

[0052] The present invention combines microscopic photography technology with image acquisition to achieve microscopic acquisition, solving the problem that traditional image acquisition to establish a point cloud model to extract small-size seed surface detail features is limited and the point cloud modeling quality is difficult to improve.

[0053] The present invention uses microscopically magnified images to reconstruct a seed point cloud model with sufficient local details and accuracy, and accurately converts and corrects the key dimensions of the model through point cloud data, thereby extracting the size feature information of the seeds. The present invention is highly practical and has excellent phenotype collection capabilities for point cloud reconstruction and size feature extraction of small-particle seeds. The point cloud model reconstruction quality is high, the retained detail features are rich, and the size feature extraction is accurate and effective. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only for the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0055] Figure 1 Schematic diagram of the external structure of the microscopic three-dimensional reconstruction device;

[0056] Figure 2 Schematic diagram of the internal structure of the microscopic three-dimensional reconstruction device;

[0057] Figure 3 It is a structural diagram of the horizontal rotation mechanism;

[0058] Figure 4 It is a structural diagram of the image acquisition mechanism;

[0059] Figure 5 Schematic diagram of the structure of the fill light mechanism;

[0060] Figure 6 This is a system structure diagram of the microscopic three-dimensional reconstruction device;

[0061] Figure 7 This is the workflow diagram of the microscopic three-dimensional reconstruction device;

[0062] Figure 8 Flowchart of the size feature extraction method.

[0063] The marks in the above figure are: 01, profile frame; 02, panel; 03, door panel; 04, computer display screen; 05, computer operating table; 06, power supply; 07, horizontal rotation mechanism; 08, image acquisition mechanism; 09, fill light mechanism; 11, base; 12, bearing; 13, platform; 14, electric motor; 15, speed regulator; 16, horizontal rotation mechanism power interface; 17, horizontal rotation mechanism control interface; 21, deflection joint; 22, pitch joint; 23, industrial camera; 24, dimming ring; 25, focusing ring; 26, microscope magnifying lens; 27, image acquisition mechanism control interface; 28, image acquisition mechanism power interface; 31, casing; 32, control circuit board; 33, light source board; 34, diffuser; 35, driving power supply. DETAILED DESCRIPTION

[0064] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to specific embodiments.

[0065] It should be noted that, unless otherwise defined, the technical or scientific terms used in the present invention should have the usual meanings understood by people with ordinary skills in the field to which the present invention belongs. The "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the object being described changes, the relative positional relationship may also change accordingly.

[0066] like Figures 1 to 8As shown, a small-size seed microscopic three-dimensional reconstruction device includes a frame and an image acquisition mechanism 08, and also includes a horizontal rotation mechanism 07, a fill-light mechanism 09 and a main control system. The image acquisition mechanism 08, the horizontal rotation mechanism 07 and the fill-light mechanism 09 are all arranged in the frame, the horizontal rotation mechanism 07 is installed at the bottom of the frame, the seeds to be detected are placed on the horizontal rotation mechanism 07, the fill-light mechanism 09 and the image acquisition mechanism 08 are installed at the top of the frame, and the seeds to be detected are filled with light and image acquisition. The main control system is arranged beside the frame, and the image acquisition mechanism 08, the horizontal rotation mechanism 07 and the fill-light mechanism 09 are all electrically connected to the main control system. The present invention can achieve high-quality three-dimensional reconstruction and accurate extraction of size characteristics of small-size seeds, thereby completing three-dimensional digital reconstruction and size characteristic detection of small-size seeds, improving detection accuracy and efficiency, and has important significance in promoting the development of the breeding industry.

[0067] The frame comprises a profile frame 01, panels 02, and door panels 03. Panels 02 are mounted on the top, bottom, sides, and back of the profile frame 01, while door panels 03 are mounted on the front. Image acquisition mechanism 08, fill-light mechanism 09, and horizontal rotation mechanism 07 are located within the profile frame 01. The profile frame 01 is the main body of the frame, supporting and securing the various systems within the entire microscopic 3D reconstruction device. Panels 02 are mounted on the top, bottom, sides, and back of the profile frame 01 to prevent adverse effects of external light, airflow, and other factors on the working environment within the profile frame 01, ensuring the normal operation of image acquisition mechanism 08, fill-light mechanism 09, and horizontal rotation mechanism 07. Door panels 03 open when a worker needs to place seeds to be tested on platform 13 of the horizontal rotation mechanism 07 and close during the image acquisition process, allowing for the free access of seed samples while ensuring a closed working environment.

[0068] Preferably, the profile frame is lightweight, durable, easy to process, and has good strength. The box-shaped profile frame has good structural stability and can provide stable support for the various system devices within the microscopic three-dimensional reconstruction device. Panels are installed on the bottom, top, and sides of the frame to prevent external light and other environmental factors from interfering with the seed image acquisition process and further stabilize the frame structure. The connecting parts used in the profile frame include angle brackets, nuts, bolts, hinges, and plate slots, which are used to connect and fix the profiles to each other and the profiles to the panels.

[0069] The horizontal rotation mechanism 07 includes a base 11, a bearing 12, a platform 13, an electric motor 14 and a speed regulator 15. The base 11 is fixedly connected to the panel 02 at the bottom of the profile frame 01. The base 11 is provided with a space for accommodating the electric motor 14. The electric motor 14 is fixedly connected to the base 11. The platform 13 is located above the base 11. The output shaft of the electric motor 14 is connected to the platform 13, and the electric motor 14 drives the platform 13 to rotate; the bearing 12 is arranged between the base 11 and the platform 13, the outer ring of the bearing 12 is connected to the base 11, and the inner ring of the bearing 12 is connected to the platform 13 and rotates with the platform 13. The speed regulator 15 is installed on the base and electrically connected to the electric motor 14, and is responsible for adjusting the speed of the electric motor 14.

[0070] The base 11 is the supporting component of the entire horizontal rotation mechanism 07. It can be installed on the panel 02 at the bottom of the profile frame 01 in a suitable manner to ensure that it has sufficient horizontality and stability, so that the platform 13 can rotate stably in the horizontal plane. The bearing 12 is a connecting part between the base 11 and the platform 13, which prevents the friction between the two during the relative movement and causes a reduction in the movement smoothness, further ensures the normal operation of the platform 13 rotation movement, and improves the stability of the platform's horizontal rotation movement. The platform 13 is used to place seed samples and needs to have a certain surface roughness to prevent the seeds from sliding relative to the platform 13 during the rotation movement. The electric motor 14 is used to drive the platform 13 to rotate and provide stable power for the platform rotation. The speed regulator can adjust the speed of the electric motor by adjusting the power supply frequency or voltage, so that the platform can rotate at different angular speeds.

[0071] Image acquisition mechanism 08 includes a motorized bracket, an industrial camera 23, and a microscope lens. One end of the motorized bracket is fixedly connected to the top of the profile frame 01 via a mounting plate. The industrial camera 23 is connected to the other end of the motorized bracket, and the microscope lens is mounted in front of the camera head of the industrial camera 23. The motorized bracket includes a deflection joint 21 and a pitch joint 22 to achieve deflection and pitch motion, thereby adjusting the camera position. The industrial camera 23 is mounted on the pitch joint 22, which is connected to the motorized bracket 21, allowing the industrial camera 23 to deflect and pitch. The microscope lens includes a light adjustment ring 24, a focus ring 25, and a microscope lens 26. One end of the light adjustment ring 24 is connected to the camera head of the industrial camera 23, and the other end of the light adjustment ring 24 is connected to the focus ring 25, which is connected to the microscope lens 26.

[0072] The industrial camera is used to capture images of seeds, and the quality of the captured photos directly determines the quality of point cloud reconstruction; the industrial camera 23 should have sufficient configuration and good shooting performance so that it can clearly capture the surface features of the seeds placed on the platform 13 of the horizontal rotating mechanism 07.

[0073] The fixed magnification of the microscope magnifying lens is K, which can micro-magnify the surface features of the seeds, thereby helping to restore more details during the seed point cloud reconstruction process; specifically, the light adjustment ring 24 and the focus ring 25 of the microscope magnifying lens can respectively adjust the aperture size and telescopic distance of the lens, and the surface feature details of the seeds can be appropriately magnified through the microscope magnifying lens 26, helping the industrial camera 23 to achieve higher quality feature extraction, thereby restoring more surface features of the seeds during the point cloud reconstruction process and reconstructing a more accurate point cloud three-dimensional model.

[0074] The electric bracket is connected to the base of the industrial camera. By controlling the rotation of the joint, it drives the industrial camera to perform pitch and yaw movements, thereby adjusting the camera position and obtaining a suitable field of view. In addition, the control interface on the electric bracket integrates the control circuit of the entire image acquisition mechanism, which is used to connect to the main control system. The computer uniformly controls the acquisition of the industrial camera, the dimming and focusing of the microscope magnification lens, and the movement of the electric bracket, thereby realizing automated control of the entire image acquisition mechanism.

[0075] The fill light mechanism 09 is located above the horizontal rotation mechanism 07. The fill light mechanism 09 includes a shell 31, a control circuit board 32, a light source board 33, a diffuser 34 and a driving power supply 35. The shell 31 is fixedly installed on the top of the profile frame 01. The control circuit board 32, the light source board 33 and the diffuser 34 are installed in the shell 31 in sequence. The driving power supply 35 is installed on the control circuit board 32. The driving power supply 35 is electrically connected to the light source board 33 to provide a power source for the light source board 33. The light control circuit on the control circuit board 32 controls the voltage and current of the light source board 33 and adjusts the light intensity of the fill light mechanism 09. The diffuser 34 is installed on the outside of the light source board 33 to diffuse and disperse the light emitted by the light source board 33.

[0076] Specifically, the housing is used to secure and encapsulate all components of the fill light mechanism. The light control circuit on the control circuit board 32 can control the voltage and current of the light source board 33, thereby adjusting the light intensity of the fill light mechanism 09. The light source board 33 is used to emit light and is the core component of the entire fill light mechanism 09. The diffuser 34 can diffuse or disperse the light emitted by the light source board 33 to ensure uniform illumination, avoid uneven illumination distribution in the image acquisition environment, and avoid strong light-dark contrast, strong shadows and highlights in the captured seed image, which affects the acquisition effect and, in turn, affects the detail recovery of the seed point cloud reconstruction. The driving power supply 35 is the direct power source for the light source board 33. In addition to supplying power, it is also responsible for converting the external power input into the appropriate voltage and current required by the lamp. The driving power supply can provide a stable power supply for the light source module to ensure its normal operation, and the driving power supply is an essential component for the control circuit to realize the dimming function. The control circuit of the control circuit board can adjust the brightness of the light source by controlling the current or modulating the pulse width to achieve the desired lighting effect. The external port is connected to the main control system, and the computer realizes automatic dimming.

[0077] The main control system includes a computer and a power supply 06. The computer is connected to the image acquisition mechanism control interface 27, the fill light mechanism control interface and the horizontal rotation mechanism control interface 17. The horizontal rotation mechanism 07, the image acquisition mechanism 08 and the fill light mechanism 09 are automatically controlled in an integrated manner through the computer operating console 05, and the seed image magnified by the microscope lens in the industrial camera 23 is observed through the computer display screen 04 to accurately extract the characteristics of the seed sample; the power supply 06 is connected to the horizontal rotation mechanism power supply interface 16, the image acquisition mechanism power supply interface 28 and the fill light mechanism power supply interface to provide stable power input for the computer, the horizontal rotation mechanism 07, the image acquisition mechanism 08 and the fill light mechanism 09.

[0078] Specifically, the computer possesses sufficient computing and training capabilities to perform point cloud reconstruction, automatic segmentation, and dimensional feature extraction. It also displays the camera system's captured images in real time on a display screen. This feedback allows the user to precisely adjust various parameters within the microscopic 3D reconstruction device to improve the acquisition effect. Furthermore, the computer is connected via a control interface to the dimmer of the fill light mechanism, the speed regulator of the horizontal rotation mechanism, and the control interface of the image acquisition mechanism. Controls are performed via buttons and knobs on the control panel, enabling integrated control of the entire microscopic 3D reconstruction device, including adjusting light intensity, camera position, speed, and acquisition. The power supply is the source of electricity for the entire microscopic 3D reconstruction device. The internal power input terminal can be connected to an external power source, and internal control circuitry distributes power to the computer and various systems of the microscopic 3D reconstruction device.

[0079] Based on the above-mentioned small-size seed microscopic three-dimensional reconstruction device, the present invention also provides a method for extracting size features of small-size seed microscopic three-dimensional reconstruction.

[0080] The extraction method comprises:

[0081] Step 1. Camera calibration: Use the image acquisition mechanism to take multiple pictures of the black and white checkerboard calibration plate, use the coordinate relationship to calculate the camera's distortion coefficient, and obtain the camera's intrinsic parameter matrix;

[0082] The specific process is:

[0083] Prepare a calibration plate with regularly arranged black and white checkerboards. Use the image acquisition mechanism to take multiple pictures of the calibration plate, and ensure that the corners of the checkerboard are clearly visible in each image. Scan the entire image, detect the corners of the checkerboard, and calculate the coordinates of all corners. Select a corner of the checkerboard as the origin, measure the actual distance between each corner point, and obtain the coordinates of the checkerboard corner point in the three-dimensional world coordinate system. Use the captured internal corner point coordinates and the corresponding world coordinates to calculate the camera's intrinsic parameters and distortion coefficients, and express the camera's intrinsic parameters as a matrix F:

[0084]

[0085] Among them, f x 、f y is the focal length of the camera, c x 、c y are the principal point coordinates of the camera.

[0086] Step 2. Obtain a magnified image of the seed: Place the seed to be tested on the platform of the horizontal rotating mechanism, start the main control system, fill light mechanism, and image acquisition mechanism, and adjust the fill light mechanism and image acquisition mechanism through the buttons or knobs on the computer console in the main control system to make the surface details of the seed clearly visible. Use the acquisition setting module in the computer software to set the parameters such as the speed, angle, camera exposure time, and acquisition quantity of the horizontal rotating mechanism to perform image acquisition.

[0087] The specific operation process is as follows:

[0088] Place the seeds to be tested on the platform of the horizontal rotating mechanism. After placement, close the door panel to ensure the closed environment for collection. Turn on the power, activate the main control system, enable the fill light mechanism with the help of computer software, and make preliminary adjustments to its light intensity. Start the image acquisition mechanism and open the image acquisition window through the computer software to monitor the image output of the industrial camera in real time. Use the buttons or knobs on the computer console to adjust the light intensity of the fill light mechanism and the aperture of the microscope lens until the picture displayed in the image acquisition window is bright enough. Adjust the camera position by controlling the electric joint to ensure that the industrial camera can fully capture the image of the seeds to be tested. Use the microscope lens to accurately focus until the surface details of the seeds are clearly visible. Use the acquisition setting module in the computer software to set the parameters such as the speed, angle, camera exposure time and number of acquisitions of the horizontal rotating mechanism to realize an automated image acquisition process.

[0089] Assume that the number of collected photos is n, the rotation speed is ω, the rotation angle is θ, and the camera exposure time is T. The relationship between the rotation angle θ and the number of collected photos n satisfies:

[0090]

[0091] The specific working conditions are as follows: the horizontal rotation mechanism starts and rotates horizontally by θ at a speed of ω, pauses for T time, and the image acquisition mechanism performs the first acquisition and saves the image as {img_1}, then rotates by θ again, pauses for T time again, acquires another image, and saves it as {img_2}, until the nth photo {img_n} is obtained. The automatic acquisition ends and the n acquired photos are saved in the main control system, ready for the next step.

[0092] Step 3. Deformation Correction: For each pixel in the image, calculate the ideal coordinates before distortion removal based on the camera intrinsic parameter matrix F and the distortion coefficient, and map the distorted coordinates to the undistorted image coordinates to obtain the corrected image:

[0093] The specific process is:

[0094] A distortion model is established based on the camera intrinsic parameter matrix and distortion coefficients. The corresponding position of each pixel in the initial image in the dedistorted image is calculated. Based on the calculated dedistortion mapping, each pixel position in the distorted image is remapped to the new image coordinates to obtain the corrected image.

[0095] Step 4. Point cloud reconstruction: The SIFT algorithm can be used to detect feature points using two rectified images, construct the camera's essential matrix, and extract the relative translation vector and rotation matrix between the two images by solving the camera's essential matrix and singular value decomposition. Repeat the above operations to calculate the absolute rotation matrix and displacement vector of all images in the three-dimensional world coordinate system, construct the relationship between the three-dimensional coordinates of the point in the world coordinate system and its projected two-dimensional coordinates in the image, and use the least squares method to solve the minimum value to obtain the optimal three-dimensional point coordinates, obtain a sparse point cloud three-dimensional model, and reconstruct a dense point cloud three-dimensional model containing more information based on the sparse point cloud three-dimensional model.

[0096] The specific process is:

[0097] The SIFT (Scale-Invariant Feature Transform) algorithm is used to detect feature points in the two images. Descriptors are generated based on the gradient information of the local area of the key points to ensure that they are scale and rotation invariant. The similarity between the descriptors is then used to match the key points to improve the matching accuracy. When enough matching points are obtained, the camera's essential matrix E is constructed:

[0098]

[0099] where p i (x i ,y i , 1) and p′ i (x i ′,y i ′, 1) represent the homogeneous coordinates of the i-th pair of matching points between the two images, and then the matching points are solved by constructing a system of equations to solve E; after solving the essential matrix E, the relative translation vector t and rotation matrix R between the two images are further extracted by singular value decomposition, which is in the form of:

[0100] E=U∑V T

[0101] Where U and V are orthogonal matrices, ∑ is a singular value matrix, and the singular vector corresponding to the minimum singular value is the translation vector t, and then:

[0102] E=[t]R

[0103] Recover the rotation matrix R, where [t] is the cross product matrix of t, defined as:

[0104]

[0105] where t x , t y , t zare the values of the translation vector t in three dimensions. Select an image to establish a three-dimensional world coordinate system, and calculate the absolute rotation matrix and displacement vector of all images in the three-dimensional world coordinate system by recovering the relative rotation matrix and displacement vector between each image.

[0106] Assume (X P , Y P , Z P ) is the three-dimensional coordinate of a point P in the three-dimensional world coordinate system, (x Pi ,y Pi ) is the two-dimensional coordinate of the projection point of point P in the image {img_i}, and the relationship between the two satisfies the projection equation:

[0107] [x Pi y Pi 1] T =F[R i |t i ][X P Y P Z P 1] T

[0108] where R i , t i are the absolute rotation matrix and displacement vector of the image {img_i} in the 3D world coordinate system. Then, the optimal 3D coordinates of point P are estimated by the least squares method to minimize the reprojection error of all projected points, thereby obtaining the optimal 3D coordinate estimate of point P:

[0109]

[0110] This equation represents the sum of the squares of the differences between the actual and predicted projected points in all images. Using this equation, a mathematical model is established to minimize the optimal 3D point coordinates P. The optimal coordinates of all 3D points are sequentially estimated to obtain a sparse point cloud 3D model. The depth value of each pixel is calculated using images from multiple viewpoints to generate a depth map. Each pixel in the depth map is then converted into a 3D point. A dense point cloud 3D model containing more information is reconstructed from the sparse point cloud 3D model.

[0111] Step 5. Automatic point cloud segmentation: Establish 7 types of information for the M points in the point cloud, including 3D coordinates (x, y, z), color values (R, G, B), and labels (Label), build a point cloud segmentation neural network model, use a multi-layer perceptron to extract the local features of each point, and then use the maximum pooling function to ensure that the model is insensitive to the order of input points and obtain global features at the same time. Merge the features of each point with the global features and segment them through an additional fully connected layer; segment the point cloud through the segmentation network, separate the seed 3D point cloud, and save it in the main control system in preparation for the next step.

[0112] Step 6. Data centering: Assume that there are N points p in the point cloud of a seed. i =[x i ,y i , z i ] T , 1≤i≤N; centralize the point cloud data, subtract the mean of all points from the coordinates of each point, and transform the entire point cloud into a new coordinate system so that the center of mass of the point cloud is at the origin of the new coordinate system:

[0113] p′ i =p i -μ

[0114] Among them, p′ i It is p i The new coordinates after data centralization, μ is the coordinate of the mean (center of mass) of the entire point cloud in the initial coordinate system;

[0115] Step 7. Calculate the covariance matrix: Construct the covariance matrix C of the point cloud data in the new coordinate system to further describe the data distribution characteristics in each direction of the point cloud:

[0116]

[0117] Among them, P′ is composed of N p′ i An N×3 matrix composed of vectors;

[0118] Step 8. Eigenvalue decomposition: Solve the eigenvalues and eigenvectors of the covariance matrix C, and find the main axis direction of the point cloud through eigenvalue decomposition:

[0119] Cv i =λ i v i

[0120] Among them, λ i is the i-th eigenvalue of the covariance matrix C, v i is its corresponding eigenvector, where the eigenvector specifies the direction of the principal axis of the point cloud, and the principal axis of the cloud is the principal component of the point cloud.

[0121] Step 9. Principal component sorting: The eigenvalues λ1, λ2, and λ3 reflect the variance of the seed point cloud model in the direction of its corresponding eigenvector. Sort the eigenvectors according to the size of the eigenvalues to determine the directions of the three principal axes that can represent the length, width, and height of the seed point cloud model:

[0122] λ1≥λ2≥λ3

[0123] Then, the eigenvectors (v1, v2, v3) corresponding to the eigenvalues are selected in turn as the length, width and height directions of the seed point cloud model.

[0124] Step 10. Model size calculation: Calculate the boundary of the point cloud along each principal component direction, project the point cloud onto the principal component direction, find the minimum and maximum values in these directions, and the difference between them is the 3D size information of the seed point cloud model:

[0125]

[0126] l m =l max -l min , w m =w max -w min , t m =t max -t min

[0127] Among them, l m , w m , t m Represent the length, width and height values of the seed point cloud model respectively.

[0128] Step 11. Correct the magnification factor and determine the seed geometry: Correct the length, width, and height of the seed point cloud model by the magnification factor to restore the true size of the seed:

[0129]

[0130] Among them, K is the magnification of the microscope lens, l, w, and t are the seed length, width, and thickness values extracted by the microscopic three-dimensional reconstruction size feature extraction method.

[0131] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present invention is limited to these examples. Within the scope of the present invention, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the present invention as described above, which are not provided in detail for the sake of simplicity.

[0132] Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A microscopic three-dimensional reconstruction device for small-size seeds, comprising a frame and an image acquisition mechanism (08), characterized in that: The invention also includes a horizontal rotation mechanism (07), a light-filling mechanism (09) and a main control system. The image acquisition mechanism (08), the horizontal rotation mechanism (07) and the light-filling mechanism (09) are all arranged in a frame. The horizontal rotation mechanism (07) is installed at the lower part of the frame. The seeds to be detected are placed on the horizontal rotation mechanism (07). The light-filling mechanism (09) and the image acquisition mechanism (08) are installed at the upper part of the frame to perform light-filling and image acquisition on the seeds to be detected. The main control system is arranged beside the frame. The image acquisition mechanism (08), the horizontal rotation mechanism (07) and the light-filling mechanism (09) are all electrically connected to the main control system.

2. The device for microscopic 3D reconstruction of small-size seeds according to claim 1, characterized in that: The frame comprises a profile frame (01), a panel (02) and a door panel (03); the panel (02) is mounted on the upper and lower surfaces, two side surfaces and the back of the profile frame (01); the door panel (03) is mounted on the front of the profile frame (01); and the image acquisition mechanism (08), the fill light mechanism (09) and the horizontal rotation mechanism (07) are located inside the profile frame (01).

3. The device for microscopic 3D reconstruction of small-size seeds according to claim 2, characterized in that: The horizontal rotation mechanism (07) includes a base (11), a bearing (12), a platform (13), an electric motor (14) and a speed regulator (15). The base (11) is fixedly connected to the panel (02) at the bottom of the profile frame (01). The base (11) is provided with a space for accommodating the electric motor (14). The electric motor (14) is fixedly connected to the base (11). The platform (13) is located above the base (11). The output shaft of the electric motor (14) is connected to the platform (13). The electric motor (14) drives the platform (13) to rotate. The bearing (12) is arranged between the base (11) and the platform (13). The outer ring of the bearing (12) is connected to the base (11). The inner ring of the bearing (12) is connected to the platform (13) and rotates with the platform (13). The speed regulator (15) is installed on the base and electrically connected to the electric motor (14) and is responsible for adjusting the speed of the electric motor (14).

4. The device for microscopic 3D reconstruction of small-size seeds according to claim 2 or 3, characterized in that: The image acquisition mechanism (08) includes an electric bracket, an industrial camera (23) and a microscope magnification lens. One end of the electric bracket is fixedly connected to the top of the profile frame (01) through a mounting plate. The industrial camera (23) is connected to the other end of the electric bracket. The microscope magnification lens is installed in front of the camera head of the industrial camera (23).

5. The device for microscopic three-dimensional reconstruction of small-size seeds according to claim 4, characterized in that: The electric bracket includes a deflection joint (21) and an elevation joint (22), an industrial camera (23) is mounted on the elevation joint (22), the elevation joint (22) is connected to the deflection joint (21), and the industrial camera (23) performs deflection and elevation motions; the micro-magnifying lens includes a light-adjusting ring (24), a focus ring (25) and a micro-magnifying lens (26), one end of the light-adjusting ring (24) is connected to the camera head of the industrial camera (23), the other end of the light-adjusting ring (24) is connected to the focus ring (25), and the focus ring (25) is connected to the micro-magnifying lens (26).

6. The device for microscopic three-dimensional reconstruction of small-size seeds according to claim 5, characterized in that: The fill light mechanism (09) is located above the horizontal rotation mechanism (07). The fill light mechanism (09) includes a shell (31), a control circuit board (32), a light source board (33), a light diffusion board (34) and a driving power supply (35). The shell (31) is fixedly mounted on the top of the profile frame (01). The control circuit board (32), the light source board (33) and the light diffusion board (34) are sequentially mounted in the shell (31). The driving power supply (35) is mounted on the control circuit board (32). The driving power supply (35) is electrically connected to the light source board (33) to provide a power source for the light source board (33). The light control circuit on the control circuit board (32) controls the voltage and current of the light source board (33) and adjusts the light intensity of the fill light mechanism (09). The light diffusion board (34) is mounted on the outside of the light source board (33) to diffuse and disperse the light emitted by the light source board (33).

7. The device for microscopic three-dimensional reconstruction of small-size seeds according to claim 6, characterized in that: The main control system includes a computer and a power supply (06). The computer is connected to an image acquisition mechanism control interface (27), a light-filling mechanism control interface and a horizontal rotation mechanism control interface (17). The horizontal rotation mechanism (07), the image acquisition mechanism (08) and the light-filling mechanism (09) are automatically and integratedly controlled through a computer operating console (05). The seed image magnified by a microscope lens in an industrial camera (23) is observed through a computer display screen (04) to accurately extract the characteristics of the seed sample. The power supply (06) is connected to a horizontal rotation mechanism power supply interface (16), an image acquisition mechanism power supply interface (28) and a light-filling mechanism power supply interface to provide a stable power input for the computer, the horizontal rotation mechanism (07), the image acquisition mechanism (08) and the light-filling mechanism (09).

8. A method for extracting size features from microscopic three-dimensional reconstruction of small-size seeds, characterized by: Based on the small-size seed microscopic three-dimensional reconstruction device according to any one of claims 1 to 7, the extraction method comprises: Step 1. Camera calibration: Use the image acquisition mechanism to take multiple pictures of the black and white checkerboard calibration plate, use the coordinate relationship to calculate the camera's distortion coefficient, and obtain the camera's intrinsic parameter matrix; Step 2. Obtaining a magnified image of the seed: Place a single or multiple seeds to be tested on the platform of the horizontal rotating mechanism, start the main control system, the fill light mechanism, and the image acquisition mechanism, and adjust the fill light mechanism and the image acquisition mechanism using the buttons or knobs on the computer console of the main control system so that the surface details of the seed are clearly visible. Use the acquisition setting module in the computer software to set the parameters such as the rotation speed, rotation angle, camera exposure time, and acquisition quantity of the horizontal rotating mechanism to perform image acquisition; Step 3. Distortion Correction: For each pixel in the image, calculate the ideal coordinates before distortion removal based on the camera intrinsic parameter matrix and the distortion coefficient, and map the distorted coordinates to the undistorted image coordinates to obtain the corrected image. Step 4. Point Cloud Reconstruction: Use the SIFT algorithm to detect feature points using the two rectified images, construct the camera's intrinsic matrix, and extract the relative translation vector and rotation matrix between the two images by solving the camera's intrinsic matrix and singular value decomposition. Repeat the above steps to calculate the absolute rotation matrix and displacement vector of all images in the 3D world coordinate system. Construct the relationship between the 3D coordinates of the point in the world coordinate system and its projected 2D coordinates in the image. Use the least squares method to solve the minimum value to obtain the optimal 3D point coordinates, obtain a sparse point cloud 3D model, and reconstruct a dense point cloud 3D model containing more information based on the sparse point cloud 3D model. Step 5. Automatic point cloud segmentation: For each of the M points in the point cloud, seven types of information are established: 3D coordinates (x, y, z), color values (R, G, B), and labels. A point cloud segmentation neural network model is constructed, using a multi-layer perceptron to extract local features for each point. A maximum pooling function is then used to ensure that the model is insensitive to the order of the input points while also obtaining global features. The features of each point are merged with the global features, and segmentation is performed through an additional fully connected layer. The point cloud is segmented using this segmentation network, separating the seed 3D point cloud and saving it in the main control system in preparation for the next step. Step 6. Data centering: Assume that there are N points p in the point cloud of a seed. i =[x i ,y i , z i ] T , 1≤i≤N; centralize the point cloud data, subtract the mean of all points from the coordinates of each point, and transform the entire point cloud into a new coordinate system so that the center of mass of the point cloud is at the origin of the new coordinate system: p′ i =p i -μ Among them, p′ i It is p i The new coordinates after data centralization, μ is the coordinate of the mean (center of mass) of the entire point cloud in the initial coordinate system; Step 7. Calculate the covariance matrix: Construct the covariance matrix C of the point cloud data in the new coordinate system to further describe the data distribution characteristics in each direction of the point cloud: Among them, P′ is composed of N p′ i An N×3 matrix composed of vectors; Step 8. Eigenvalue decomposition: solve the eigenvalues and eigenvectors of the covariance matrix C, and use eigenvalue decomposition to find the main axis direction of the point cloud: Cv i =λ i v i , where λ i is the i-th eigenvalue of the covariance matrix C, v i is its corresponding eigenvector, where the eigenvector specifies the direction of the point cloud principal axis, and the cloud principal axis is the principal component of the point cloud; Step 9. Principal component sorting: The eigenvalues λ1, λ2, and λ3 reflect the variance of the seed point cloud model in the direction of its corresponding eigenvector. Sort the eigenvectors according to the size of the eigenvalues to determine the directions of the three main axes that can represent the length, width, and height of the seed point cloud model: λ1 ≥ λ2 ≥ λ3. Then, select the eigenvectors (v1, v2, v3) corresponding to the eigenvalues in turn as the directions of the length, width, and height of the seed point cloud model respectively. Step 10. Model size calculation: Calculate the boundary of the point cloud along each principal component direction, project the point cloud onto the principal component direction, find the minimum and maximum values in these directions, and the difference between them is the 3D size information of the seed point cloud model: l m =l max -l min ,w m =w max -w min ,t m =t max -t min Among them, l m , w m , t m Represent the length, width and height values of the seed point cloud model respectively; Step 11. Correct the magnification factor and determine the seed geometry: Correct the length, width, and height of the seed point cloud model by the magnification factor to restore the true size of the seed: Among them, K is the magnification of the microscope lens, l, w, and t are the seed length, width, and thickness values extracted by the microscopic three-dimensional reconstruction size feature extraction method.

9. The method for extracting size features of small-size seeds by microscopic three-dimensional reconstruction according to claim 8, characterized in that: The specific process of step 1 is as follows: prepare a calibration plate with regularly arranged black and white checkerboard grids, use an image acquisition mechanism to take multiple pictures of the calibration plate, and ensure that the corner points of the checkerboard grid are clearly visible in each image; scan the entire image, detect the corner points of the checkerboard grid and calculate the coordinates of all corner points; select a corner of the checkerboard grid as the origin, measure the actual distance between each corner point, and obtain the coordinates of the checkerboard grid corner point in the three-dimensional world coordinate system; use the captured internal corner point coordinates and the corresponding world coordinates to calculate the camera's intrinsic parameters and distortion coefficients, and express the camera's intrinsic parameters as a matrix F: Among them, f x 、f y is the focal length of the camera, c x 、c y is the principal point coordinate of the camera; For each pixel in the image, the ideal coordinates before distortion removal are calculated based on the camera intrinsic parameter matrix F, and the distorted coordinates are mapped to the undistorted image coordinates to obtain the corrected image.

10. The method for extracting size features of small-size seeds by microscopic three-dimensional reconstruction according to claim 8, characterized in that: The specific process of step 4 is: perform feature point detection on the two images using the SIFT algorithm, generate descriptors based on the gradient information of the local area of the key points, and construct the camera's essential matrix E: where p i (x i ,y i , 1) and p′ i (x i ′,y i ′, 1) represent the homogeneous coordinates of the i-th pair of matching points between the two images, and then the matching points are solved by constructing a system of equations to solve E; after solving the essential matrix E, the relative translation vector t and rotation matrix R between the two images are further extracted by singular value decomposition, which is in the form of: E=U∑V T Where U and V are orthogonal matrices, ∑ is a singular value matrix, and the singular vector corresponding to the minimum singular value is the translation vector t, and then: E=[t]R Recover the rotation matrix R, where [t] is the cross product matrix of t, defined as: where t x , t y , t z are the values of the translation vector t in three dimensions. Select an image to establish a three-dimensional world coordinate system, and calculate the absolute rotation matrix and displacement vector of all images in the three-dimensional world coordinate system by recovering the relative rotation matrix and displacement vector between each image. Construct the three-dimensional coordinates (X P , Y P , Z P ) and the two-dimensional coordinates (x Pi ,y Pi ) and satisfies the projection equation: [x Pi y Pi 1] T =F[R i |t i ][X P Y P Z P 1] T The optimal three-dimensional coordinates of point P are estimated by the least squares method to minimize the reprojection error of all projection points, thereby obtaining the optimal three-dimensional coordinate estimate of point P: This method is used to estimate the optimal coordinates of all three-dimensional points in sequence to obtain a sparse point cloud three-dimensional model; the depth value of each pixel is calculated using images from multiple perspectives to generate a depth map, and each pixel in the depth map is converted into a three-dimensional point. Based on the sparse point cloud three-dimensional model, a dense point cloud three-dimensional model containing more information is reconstructed.

Images