Three-dimensional reconstruction method for crops and crop micro organs in close planting state
By setting rigid signs around rice plants and building a micro-scene environment, combining feature extraction and multi-view reconstruction technology, the problems of rice three-dimensional reconstruction and small and micro-organ reconstruction in densely planted state are solved, and high-quality three-dimensional model generation and texture are achieved Retention of information.
Patent Information
- Application Number
- CN202410993670.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-24
- Publication Date
- 2024-12-13
AI Technical Summary
The existing three-dimensional reconstruction method of field rice in densely planted state cannot effectively solve the problem of rice shading, and the reconstruction quality of small and micro organs such as grains is low, and cannot be completed in one go, and texture information is lacking.
Set up rigid signs around the target rice plant to add feature points, and take two-dimensional images through handheld sensors. Use feature extraction and matching techniques to generate pairing groups, combine motion recovery structure and multi-view reconstruction technology to generate three-dimensional models to build a micro-scene reconstruction environment. to support high-resolution reconstruction of small and micro organs.
Three-dimensional reconstruction of the whole rice plant in densely planted state and one-time high-quality reconstruction of small and micro organs was realized, rich texture information was retained, and the difficulty of reconstruction in densely planted state and the complexity of small and micro organs were solved.
Smart Images
Figure CN119131231A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a three-dimensional reconstruction method for crops and small micro-organs of crops in a densely planted state, and belongs to the technical field of three-dimensional reconstruction, computer vision and plant phenotype processing. Background Art
[0002] Multi-view stereo reconstruction technology: Use multi-view images to estimate the camera pose, project the image pixel information into 3D space according to the pose, and reconstruct the 3D scene or model. Typical algorithms include SfM, MVS, etc.
[0003] Plant phenotyping technology: a technology that uses a variety of high-throughput and high-precision tools and methods to quantitatively measure and analyze plant traits such as morphology, structure, function, and growth dynamics.
[0004] Drone multi-perspective reconstruction technology of rice fields: Use drones to take high-resolution images of rice fields, and reconstruct and analyze the growth conditions and terrain characteristics of rice fields through image processing and 3D modeling technology.
[0005] Drone radar reconstruction technology for rice fields: A technology that uses drones equipped with radar sensors to acquire and process radar data to perform three-dimensional reconstruction and analysis of the growth conditions and terrain features of rice fields.
[0006] The existing 3D reconstruction method of rice in the field under dense planting cannot solve the problem of rice occlusion. At the same time, the feature points of rice are highly repetitive, which makes it impossible to align them during reconstruction. This application proposal designs a reconstruction solution by adding rigid signs around the target rice plants to increase feature points, which can effectively solve the difficulty of rice reconstruction under dense planting.
[0007] In addition, the existing reconstruction schemes for small micro-organs, such as rice grains, cannot reconstruct the results in one go, requiring multiple alignments and merging, and the grain surface lacks texture, resulting in low reconstruction quality. There is no report on the reconstruction technology for rice flowers. Summary of the invention
[0008] One of the purposes of the present invention is to support the capture of plant characteristics in a densely planted state to achieve whole plant reconstruction.
[0009] The second purpose of the present invention is to support the one-time reconstruction of plant micro-organs and retain rich texture information.
[0010] To achieve the above object, the technical solution adopted by the present invention is as follows:
[0011] In one aspect, the present invention provides a method for three-dimensional reconstruction of crops in a densely planted state, comprising the following steps:
[0012] Step 1: Set up rigid signboards around the target plants;
[0013] Step 2: Use a handheld sensor to capture a two-dimensional image containing the sign and crop plant features;
[0014] Step 3: Extract the mixed features of each two-dimensional image using feature extraction technology;
[0015] Step 4: Use matching technology to match feature points of each two-dimensional image and generate a pairing group;
[0016] Step 5: Estimate the pose of each 2D image using the Structure from Motion (SfM) technique.
[0017] Step 6: Use multi-view reconstruction technology (MVS) to project the two-dimensional pixels into three-dimensional space to generate a three-dimensional model.
[0018] In step S1, the rigid signboard can be made of wood, PP board or other rigid materials. The signboard is inserted around the target plant, and the characteristics of the scene can be recorded as:
[0019] F=F 1 +F 2 ,
[0020] Among them, F 1 It is a characteristic provided by the crop plant, F 2 is a feature provided by rigid signs.
[0021] In step S2, the handheld sensor includes a monocular camera, a mobile phone or other RGB image capture sensor. In order to obtain a denser RGB image, data is generally collected in the form of a video, and then the video is framed to obtain an RGB data set. The process can be recorded as follows:
[0022] I=GetFrames(V,k)
[0023] Where I is the RGB image sequence, V is the video data captured using an RGB sensor, and k is an RGB image extracted every k frames.
[0024] In step S3, the scene features in S1 are extracted from the RGB image sequence. The process can be recorded as:
[0025] F'=FeaturesDetection(I),F'∈F
[0026] F' is the feature extracted from the RGB image sequence. The feature extraction method can be: SIFT, SURF, ORB or other feature point extraction algorithms.
[0027] The step S4 is recorded as:
[0028] M ij=PairMatching(I,F i ',F j ')
[0029] Among them, M ij It is the F in RGB dataset I i ',F j 'Pairing of two sets of features. The feature point matching algorithm can be KNN, ANN, etc.
[0030] The step S5 is recorded as:
[0031] P i =SfM(M ij ).
[0032] Among them, P i Contains the sensor's rotation and translation matrices, detailing the sensor's orientation and position relative to the scene at each frame captured.
[0033] The step S6 is recorded as:
[0034] C=MVS(P,I)
[0035] Where C is the reconstructed dense point cloud 3D model of the crop plant.
[0036] On the other hand, the present invention also provides a method for three-dimensional reconstruction of crop micro-organs, which comprises the following steps:
[0037] Step 1: Build a micro-scene reconstruction environment, including target support, micro-scene supplementary feature points, turntable and other hardware settings;
[0038] Step 2, using a macro camera to capture a two-dimensional image containing the supplementary features of the micro scene and the features of the small micro organs;
[0039] Step 3: Extract the mixed features of each two-dimensional image using feature extraction technology;
[0040] Step 4: Use matching technology to match feature points of each two-dimensional image and generate a pairing group;
[0041] Step 5: Estimate the pose of each 2D image using the Structure from Motion (SfM) technique.
[0042] Step 6: Use multi-view reconstruction technology (MVS) to project the two-dimensional pixels into three-dimensional space to generate a three-dimensional model.
[0043] In step 1, the microenvironment feature points can be recorded as:
[0044] F=F 1 +F 2 ,
[0045] Among them, F 1 It is a feature provided by small micro-organs (such as rice grains and flowers). 2 It is the characteristic point of the microenvironment.
[0046] In step 2, in order to obtain a denser RGB image, data is generally collected in the form of video first, and then the video is extracted to obtain an RGB data set. The process can be recorded as:
[0047] I=GetFrames(V,k)
[0048] Where I is the RGB image sequence, V is the video data captured by the macro camera, and k is an RGB image extracted every k frames.
[0049] In step 3, the scene features in S1 are extracted from the RGB image sequence, and the process can be recorded as:
[0050] F'=FeaturesDetection(I),F'∈F
[0051] F' is the feature extracted from the RGB image sequence. The feature extraction method can be: SIFT, SURF, ORB or other feature point extraction algorithms.
[0052] The step 4 is recorded as:
[0053] M ij =PairMatching(I,F i ',F j ')
[0054] Among them, M ij It is the F in RGB dataset I i ',F j 'Pairing of two sets of features. The feature point matching algorithm can be KNN, ANN, etc.
[0055] The step S5 is recorded as:
[0056] P i =SfM(M ij )
[0057] Among them, P i Contains the sensor's rotation and translation matrices, detailing the sensor's orientation and position relative to the scene at each frame captured.
[0058] The step 6 is recorded as:
[0059] C=MVS(P,I)
[0060] Where C is the reconstructed dense point cloud 3D model of the crop plant.
[0061] The beneficial effects of the present invention are as follows:
[0062] (1) The present invention designs a reconstruction scheme by adding rigid markers around the target rice plant to increase feature points, which can effectively solve the difficulty of rice reconstruction under dense planting conditions. The present invention supports capturing the characteristics of plants under dense planting conditions, and can achieve the purpose of whole plant reconstruction.
[0063] (2) The present invention designs a reconstruction scheme by building a micro-reconstruction scene around the micro-organ target, which can effectively solve the difficulty of one-time reconstruction of micro-organs. The present invention supports one-time reconstruction of plant micro-organs and retains rich texture information. BRIEF DESCRIPTION OF THE DRAWINGS
[0064] Figure 1 It is a flow chart of the present invention.
[0065] Figure 2 Middle: (a) Farmers’ rice under dense planting; (b) Cultivated rice under dense planting; (c) Rigid signboards set up in densely planted fields; (d) Reconstructed farmers’ rice; (e) Reconstructed cultivated rice.
[0066] Figure 3 Middle: (a) Textured 3D rice reconstruction model; (b) 3D rice flower reconstruction model. DETAILED DESCRIPTION
[0067] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.
[0068] Embodiment 1
[0069] This embodiment provides a method for three-dimensional reconstruction of rice in a densely planted field, which includes the following steps:
[0070] Step 1: Set up rigid signboards around the target plants;
[0071] Step 2: Use a handheld sensor to capture a two-dimensional image containing the sign and rice plant features;
[0072] Step 3: Extract the mixed features of each two-dimensional image using feature extraction technology;
[0073] Step 4: Use matching technology to match feature points of each two-dimensional image and generate a pairing group;
[0074] Step 5: Estimate the pose of each 2D image using the Structure from Motion (SfM) technique.
[0075] Step 6: Use multi-view reconstruction technology (MVS) to project the two-dimensional pixels into three-dimensional space to generate a three-dimensional model.
[0076] In step S1, the rigid signboard can be made of wood, PP board or other rigid materials. The signboard is inserted around the target plant, and the characteristics of the scene can be recorded as:
[0077] F=F 1 +F 2 ,
[0078] Among them, F 1 is a characteristic provided by the rice plant, F 2 is a feature provided by rigid signs.
[0079] In step S2, the handheld sensor includes a monocular camera, a mobile phone or other RGB image capture sensor. In order to obtain a denser RGB image, data is generally collected in the form of a video, and then the video is framed to obtain an RGB data set. The process can be recorded as follows:
[0080] I=GetFrames(V,k)
[0081] Where I is the RGB image sequence, V is the video data captured using an RGB sensor, and k is an RGB image extracted every k frames.
[0082] In step S3, the scene features in S1 are extracted from the RGB image sequence. The process can be recorded as:
[0083] F'=FeaturesDetection(I),F'∈F
[0084] F' is the feature extracted from the RGB image sequence. The feature extraction method can be: SIFT, SURF, ORB or other feature point extraction algorithms.
[0085] The step S4 is recorded as:
[0086] M ij =PairMatching(I,F i ',F j ')
[0087] Among them, M ij It is the F in RGB dataset I i ',F j 'Pairing of two sets of features. The feature point matching algorithm can be KNN, ANN, etc.
[0088] The step S5 is recorded as:
[0089] P i =SfM(M ij ).
[0090] Among them, P iContains the sensor's rotation and translation matrices, detailing the sensor's orientation and position relative to the scene at each frame captured.
[0091] The step S6 is recorded as:
[0092] C=MVS(P,I)
[0093] Where C is the reconstructed dense point cloud 3D model of the rice plant.
[0094] Embodiment 2
[0095] This embodiment provides a method for three-dimensional reconstruction of crop micro-organs of rice in the field, which comprises the following steps:
[0096] Step 1: Build a micro-scene reconstruction environment, including target support, micro-scene supplementary feature points, turntable and other hardware settings;
[0097] Step 2, using a macro camera to capture a two-dimensional image containing the supplementary features of the micro scene and the features of the small micro organs;
[0098] Step 3: Extract the mixed features of each two-dimensional image using feature extraction technology;
[0099] Step 4: Use matching technology to match feature points of each two-dimensional image and generate a pairing group;
[0100] Step 5: Estimate the pose of each 2D image using the Structure from Motion (SfM) technique.
[0101] Step 6: Use multi-view reconstruction technology (MVS) to project the two-dimensional pixels into three-dimensional space to generate a three-dimensional model.
[0102] In step 1, taking rice flowers and grains as an example, a vertical support rod is set under the receptacle or the grain, and the rod is fixed on the turntable with a clip. The feature point picture is arranged on the clip. The microenvironment feature points can be recorded as:
[0103] F=F 1 +F 2 ,
[0104] Among them, F 1 It is a feature provided by micro-organs (such as rice grains and flowers). 2 It is the characteristic point of the microenvironment.
[0105] In step 2, a macro camera is used to capture a two-dimensional image. The macro camera can focus at a very close distance (usually a few centimeters or even a few millimeters) to capture the details of the object. In order to obtain a denser RGB image, data is generally collected in the form of video, and then the video is framed to obtain an RGB data set. The process can be recorded as:
[0106] I=GetFrames(V,k)
[0107] Where I is the RGB image sequence, V is the video data captured by the macro camera, and k is an RGB image extracted every k frames.
[0108] In step 3, the scene features in S1 are extracted from the RGB image sequence, and the process can be recorded as:
[0109] F'=FeaturesDetection(I),F'∈F
[0110] F' is the feature extracted from the RGB image sequence. The feature extraction method can be: SIFT, SURF, ORB or other feature point extraction algorithms.
[0111] The step 4 is recorded as:
[0112] M ij =PairMatching(I,F i ',F j ')
[0113] Among them, M ij It is the F in RGB dataset I i ',F j 'Pairing of two sets of features. The feature point matching algorithm can be KNN, ANN, etc.
[0114] The step S5 is recorded as:
[0115] P i =SfM(M ij )
[0116] Among them, P i Contains the sensor's rotation and translation matrices, detailing the sensor's orientation and position relative to the scene at each frame captured.
[0117] The step 6 is recorded as:
[0118] C=MVS(P,I)
[0119] Where C is the reconstructed dense point cloud 3D model of the rice plant.
[0120] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the above embodiments do not limit the protection scope of the present invention in any form, and all technical solutions obtained by equivalent replacement and the like fall within the protection scope of the present invention.
[0121] The parts not involved in the present invention are the same as the prior art or can be implemented by using the prior art.
Claims
1. A method for three-dimensional reconstruction of crops in a densely planted state, characterized by: The following steps are involved: Step 1: Set up rigid signboards around the target plants; Step 2: Use a handheld sensor to capture a two-dimensional image containing the sign and crop plant features; Step 3: Extract the mixed features of each two-dimensional image using feature extraction technology; Step 4: Use matching technology to match feature points of each two-dimensional image and generate a pairing group; Step 5: Estimate the pose of each 2D image using the structure from motion technique. Step 6: Use multi-view reconstruction technology to project the two-dimensional pixels into three-dimensional space to generate a three-dimensional model.
2. The method for three-dimensional reconstruction of crops in a densely planted state according to claim 1, characterized in that: In step 1, the rigid sign is made of rigid material and is inserted around the target plant. The characteristics of the scene are recorded as follows: F=F1+F2, Among them, F1 is the characteristic provided by the crop plant, and F2 is the characteristic provided by the rigid sign.
3. The method for three-dimensional reconstruction of crops in a densely planted state according to claim 1, characterized in that: In step 2, the handheld sensor is an RGB image capture sensor including a monocular camera or a mobile phone. In order to obtain a denser RGB image, data is first collected in the form of a video, and then the video is framed to obtain an RGB data set. The process is recorded as: I = GetFrames(V, k) Where I is the RGB image sequence, V is the video data captured by the RGB sensor, and k is an RGB image extracted every k frames.
4. A method for three-dimensional reconstruction of crop micro-organs, characterized in that: The following steps are involved: Step 1: Build a micro-scene reconstruction environment; Step 2, using a macro camera to capture a two-dimensional image containing the supplementary features of the micro scene and the features of the small micro organs; Step 3: Extract the mixed features of each two-dimensional image using feature extraction technology; Step 4: Use matching technology to match feature points of each two-dimensional image and generate a pairing group; Step 5: Estimate the pose of each 2D image using the structure from motion technique. Step 6: Use multi-view reconstruction technology to project the two-dimensional pixels into three-dimensional space to generate a three-dimensional model.
5. The method for three-dimensional reconstruction of crop micro-organs according to claim 4, characterized in that: In step 1, the micro-environment feature points are recorded as: F=F1+F2, Among them, F1 is the feature provided by the micro-organ, and F2 is the microenvironment feature point.
6. The method for three-dimensional reconstruction of crop micro-organs according to claim 4, characterized in that: In step 2, a macro camera is used to capture details of the object in a two-dimensional image. To obtain a denser RGB image, data is first collected in the form of a video, and then the video is framed to obtain an RGB data set. The process is recorded as follows: I=GetFrames(V,k) Where I is the RGB image sequence, V is the video data captured by the macro camera, and k is an RGB image extracted every k frames.
7. The crop 3D reconstruction method according to claim 3 or 6, characterized in that: In step 3, the scene features in step 1 are extracted from the RGB image sequence, and the process is recorded as follows: F'=FeaturesDetection(I),F'∈F F' is the feature extracted from the RGB image sequence.
8. The crop 3D reconstruction method according to claim 1 or 4, characterized in that: The steps of step 4 are recorded as follows: Mij=PairMatching(I,Fi',Fj') Among them, Mij is the pairing of the two sets of features Fi' and Fj' in the RGB dataset I.
9. The crop 3D reconstruction method according to claim 1 or 4, characterized in that: The steps of step 5 are recorded as follows: Pi=SfM(Mij) where Pi contains the rotation and translation matrices of the sensor.
10. The crop 3D reconstruction method according to claim 1 or 4, characterized in that: The steps of step 6 are recorded as: C=MVS(P,I) Among them, C is the reconstructed dense point cloud 3D model of the crop plant.