Fruit three-dimensional model modeling method based on point cloud imaging
Patent Information
- Application Number
- CN202510018189.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-06
- Publication Date
- 2025-05-13
AI Technical Summary
In fruit harvesting, it is difficult for the prior art to effectively build a three-dimensional model of fruit, which affects the harvesting efficiency and quality.
Using a point cloud imaging method, a high-precision three-dimensional fruit model is generated by imaging, marking, scanning, splicing, denoising, diluting and grid processing of the fruit surface.
The accurate construction of the three-dimensional fruit model is achieved, providing a solid foundation for the design and optimization of agricultural machinery and equipment, and improving the efficiency and quality of fruit harvesting operations.
Smart Images
Figure CN119991995A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of three-dimensional modeling, and in particular to a method for modeling a three-dimensional fruit model based on point cloud imaging. Background Art
[0002] At present, the labor cost of orchards is gradually increasing, and mechanized operation in orchards has become a development trend. However, in mechanized harvesting operations, due to low harvesting efficiency and high fruit damage rate, subsequent storage and sales are affected, causing economic losses. How to improve the efficiency and quality of fruit harvesting operations is a key measure to ensure high and stable yields, improve quality and increase efficiency in the fruit industry.
[0003] Finite element simulation technology is crucial in the research of reducing losses and improving efficiency in fruit harvesting. By conducting simulation experiments on fruit picking, transportation, packaging and other operations, it can directly and effectively help complete the design and optimization of operating equipment. At the same time, the simulation experiment is not affected by the production time of fruit trees and the growth environment of orchards. Among them, the most critical is the construction of a three-dimensional model of the fruit.
[0004] Therefore, how to realize the construction of a three-dimensional model of fruits during fruit harvesting is a technical problem that needs to be urgently solved in order to reduce losses during fruit harvesting. Summary of the invention
[0005] In order to solve the above technical problems, this application proposes the following technical solutions:
[0006] In a first aspect, an embodiment of the present application provides a method for modeling a three-dimensional fruit model based on point cloud imaging, comprising:
[0007] Select a fruit with uniform shape and size and no obvious scars, image the surface of the fruit, and mark the eight positions of the fruit cross section divided into eight equal parts;
[0008] According to the distribution of the marking points, a scanner is used to perform a 360° scan of the fruit surface in the latitude direction to plan the scanning path, and data on the surface of different parts of the fruit are collected according to the scanning path planning to generate a point cloud array;
[0009] Import the collected data into CloudCompare software, and splice the point cloud arrays of different parts by matching the marking points;
[0010] The spliced complete fruit surface point cloud array is subjected to denoising to extract key shape features;
[0011] The point cloud array after denoising is diluted according to the proportion to obtain the fruit surface point cloud arrays with different dilution ratios;
[0012] The point cloud arrays after different dilution ratios are meshed to generate mesh surfaces of different sizes, which are then saved in stl. format;
[0013] Import the mesh surface in stl. format into SolidWorks software, fill the mesh surface, and generate a complete surface;
[0014] Select the surface with the best size, and thicken it vertically and outwards along the contour of the surface to determine the numerical value of the peel and pulp models;
[0015] The peel and pulp models are combined according to the model contour to obtain a complete three-dimensional model of the fruit.
[0016] In a possible implementation, the step of selecting a fruit with uniform shape and size and without obvious scars, imaging the surface of the fruit, and marking eight positions of the fruit cross section divided into eight equal parts includes:
[0017] Spraying white developer on the surface of the fruit to highlight the position of subsequent marking points so that the scanner can more easily identify the scanned object;
[0018] Divide the fruit into eight equal parts at intervals of 45° in the latitude direction;
[0019] In each interval, the fruit is randomly marked in the meridian direction.
[0020] In a possible implementation, the method of planning a scanning path by scanning the fruit surface 360° in the latitude direction using a scanner according to the distribution position of the marking points, collecting data on the surface of different parts of the fruit according to the scanning path planning and generating a point cloud array includes:
[0021] When planning the scanning path, the angle of the meridian direction is first determined, and the meridian directions are in a top-down order;
[0022] After determining the angle of the longitude direction, the fruit surface is scanned 360° in the latitude direction. During the scanning process, the single rotation angle of the fruit in the latitude direction is 10-20°;
[0023] After scanning for one circle, the single rotation angle in the meridian direction is 20 to 30 degrees, and the operation is repeated until the full range of the fruit surface is scanned;
[0024] After rotating to a certain angle, click the scan button. The binocular scanner will scan and store the geometric data of the fruit surface at this angle. Perform multiple rotation operations in the latitude direction, and the scanner will automatically match the marked points to generate a point cloud array.
[0025] In a possible implementation, the collected data is imported into the CloudCompare software, and the point cloud arrays of different parts are spliced according to the method of matching marking points, including:
[0026] According to the marked marking points, the marking points are numbered along the meridian direction. Two sets of point cloud arrays are imported each time, which are defined as the reference point cloud array and the point cloud array to be aligned. The marking points with obvious shape features in the two sets of point cloud arrays are roughly spliced;
[0027] Then, the iterative closest point ICP algorithm is used to further fine-tune the point clouds according to the geometric features between them. The objective function of fine-tune the point clouds is:
[0028]
[0029] Where: R is the rotation matrix in the optimal change matrix; t is the translation vector in the optimal change matrix; q i is the target point set; p i is the source point set; i is the point number, i∈(1,N).
[0030] In one possible implementation, the rough stitching operation process is as follows: select the alignment option on the CloudCompare software tool page, and the alignment mode is point pair selection. In the main interface, select the point pairs that need to be aligned in the reference point cloud array and the point cloud to be aligned, and add them one by one until the selected number of groups is completed. After the group selection is completed, select alignment to complete the rough stitching, wherein: select multiple groups or more obvious feature points in the reference point cloud array and the point cloud to be aligned.
[0031] In one possible implementation, the fine stitching workflow is as follows: select the reference point cloud array that has been roughly stitched and the point cloud array to be aligned at the same time, select the ICP option on the CloudCompare software tool page, set the iteration mode to iterate the nearest point, set the point cloud overlap ratio to be greater than or equal to 60%, the scaling ratio to 1:3, the iteration stop threshold to 1mm, the maximum number of iterations to 50, and do not filter noise points during calculation. During the fine stitching process, the position and angle of the point cloud will be automatically adjusted until the overlapping parts of the two sets of point clouds are optimally aligned.
[0032] In a possible implementation, the denoising process is performed on the spliced complete fruit surface point cloud array to extract key shape features, including:
[0033] Select the stitched point cloud array, select the Gaussian filter option, the Laplace operator, set the sigma, kernel size and number of iterations of the Laplace operator, where: adjust the number of iterations according to the noise level of the point cloud. The Gaussian filter denoising process formula is:
[0034]
[0035] Where: F(x, y, z) is the coordinate value of the point after filtering, i is the point number, i∈(1,N), p i is the coordinate value of the i-th point in the neighborhood, p o is the coordinate value of the current point, G(||p i -p o|| ) is the Gaussian weight of the i-th point, which is the distance p between the i-th point and the current point i -p o related.
[0036] In a possible implementation, the point cloud array after denoising is diluted according to a ratio to obtain fruit surface point cloud arrays with different dilution ratios, including:
[0037] Select the denoised point cloud array and use a spatial sampling method that matches the evenly distributed point scenario to determine the dilution option for subsequent point cloud meshing;
[0038] The dilution ratio is selected and set according to the geometric features of the point cloud, wherein point clouds with different geometric features correspond to different dilution ratios.
[0039] In a possible implementation, the point cloud arrays after different dilution ratios are meshed to generate mesh surfaces of different sizes, which are then saved in stl. format, including:
[0040] Select the point cloud array that has been diluted, and choose the Meshing option on the SolidWorks software editing page;
[0041] Set the best fitting surface and determine the maximum triangulation edge length to generate a mesh surface in stl. format.
[0042] In a possible implementation, the mesh surface in stl. format is imported into SolidWorks software, and the mesh surface is filled to generate a complete surface, including:
[0043] Select the surface model that needs to be converted into a solid and check whether the surface is completely closed;
[0044] If the surface is not completely closed, fill and trim the surface model in the surface options of SolidWorks software to complete the surface closure;
[0045] Then, the surface is converted into a mesh entity, and the surface is converted into an entity.
[0046] In the embodiment of the present application, the reverse engineering modeling method is used to complete the construction of a three-dimensional model of fruits with complex geometric shapes based on point cloud imaging technology, providing a solid foundation for the design and optimization of agricultural machinery and equipment. Applied to orchard fruit picking, a true description of the fruit shape can be prepared before picking, thereby improving the efficiency and quality of fruit harvesting operations during picking. BRIEF DESCRIPTION OF THE DRAWINGS
[0047] Figure 1 A schematic diagram of a process of a method for modeling a three-dimensional fruit model based on point cloud imaging provided in an embodiment of the present application;
[0048] Figure 2 A flow chart of the interface of the software for building a three-dimensional model of a Fuji apple based on point cloud imaging provided in an embodiment of the present application;
[0049] Figure 3 The three-dimensional models of Korla pear and Narcissus mango provided in the embodiments of the present application are constructed based on the method of the present application. DETAILED DESCRIPTION
[0050] The present solution is described below in conjunction with the accompanying drawings and specific implementation methods.
[0051] See also Figure 1 The method for modeling a three-dimensional fruit model based on point cloud imaging provided in this embodiment includes:
[0052] S101, selecting a fruit with uniform shape and size and without obvious scars, performing imaging processing on the surface of the fruit, and marking eight positions of the fruit cross section divided into eight equal parts.
[0053] In this embodiment, when selecting fruits, a variety of fruit types can be selected for modeling, including apples, pears, mangoes, kiwis, etc. In this embodiment, the fruit type is apple, specifically Red Fuji apple.
[0054] Spray white developer (DPT-5) on the surface of the fruit to highlight the position of the subsequent marking points, making it easier for the scanner to identify the scanned object. At the same time, the developer can make the geometric features of the fruit surface more obvious and the scanned data more detailed by enhancing the black and white color contrast effect. The fruit is divided into eight equal parts with 45° intervals in the latitude direction. In each interval, the marking points are randomly marked in the longitude direction of the fruit. This is to facilitate the subsequent splicing of the point cloud arrays of different parts of the fruit.
[0055] S102, based on the distribution position of the marking points, a scanner is used to perform a 360° scan on the latitude direction of the fruit surface to plan a scanning path, and data on the surface of different parts of the fruit are collected according to the scanning path planning to generate a point cloud array.
[0056] When planning the scanning path, first determine the angle of the meridian direction, and the meridian direction is in a top-down order. After determining the meridian angle, perform a 360° scan of the fruit surface in the latitude direction. During the scanning process, the single rotation angle of the fruit in the latitude direction is 10 to 20°. After scanning one circle, the single rotation angle in the meridian direction is 20 to 30°. Repeat the operation until the full-scale scan of the fruit surface is completed.
[0057] After rotating to a certain angle, click the scan button, and the binocular scanner will scan and store the geometric data of the fruit surface at this angle. After multiple rotations in the latitude direction, the scanner will automatically match the marked points to generate a point cloud array.
[0058] S103, importing the collected data into CloudCompare software, and splicing the point cloud arrays of different parts according to the method of matching marking points.
[0059] The stitching of point cloud arrays is divided into rough stitching and fine stitching. According to the marking points marked in Step 2, the marking points are numbered along the meridian direction (marking point 1, marking point 2...). Two sets of point cloud arrays are imported each time, defined as the reference point cloud array and the point cloud array to be aligned. To stitch the marking points with obvious shape features in the two sets of point cloud arrays, it is necessary to align the point clouds near the two sets of marking points (the number is not less than 5).
[0060] The rough stitching operation process is as follows: Select the Alignment option on the tool page, and the alignment method is point pair selection. In the main interface, select the point pairs to be aligned on the reference point cloud array and the point cloud to be aligned (usually select 5 or more obvious feature points, such as corner points or obvious shape feature points), and add them one by one until the selected number of groups is completed. After the group selection is completed, select Alignment to complete the rough stitching.
[0061] The fine stitching process is as follows: select the reference point cloud array and the point cloud array to be aligned after rough stitching at the same time, select the ICP option on the tool page, and the iteration method is to iterate the closest point. Set the overlap ratio of the point cloud to no less than 60%; the scaling ratio is 1:3; the iteration stop threshold is 1mm; the maximum number of iterations is 50 times; and noise points are not filtered during calculation. The position and angle of the point cloud will be automatically adjusted during the fine stitching process until the overlapping parts of the two sets of point clouds are optimally aligned.
[0062] Fine stitching uses the iterative closest point (ICP) algorithm to further align the point clouds according to their geometric features. Objective function:
[0063]
[0064] Where: R is the rotation matrix in the optimal change matrix; t is the translation vector in the optimal change matrix; q i is the target point set; p iis the source point set; i is the point number, i∈(1,N).
[0065] The following is the ICP algorithm code for implementing fine splicing in this embodiment:
[0066]
[0067]
[0068]
[0069]
[0070]
[0071]
[0072]
[0073]
[0074] S104, denoising the spliced complete fruit surface point cloud array to extract key shape features.
[0075] Select the stitched point cloud array, select the Gaussian filter option, the Laplace operator, set the sigma, kernel size and number of iterations of the Laplace operator, where: adjust the number of iterations according to the noise level of the point cloud. The Gaussian filter denoising process formula is:
[0076]
[0077] Where: F(x, y, z) is the coordinate value of the point after filtering, i is the point number, i∈(1,N), p i is the coordinate value of the i-th point in the neighborhood, p o is the coordinate value of the current point, G(||p i -p o|| ) is the Gaussian weight of the i-th point, which is the distance p between the i-th point and the current point i -p o related.
[0078] In this example, sigma is set to 0.003; kernel size is set to 2.5; number of iterations is set to 6 (adjusted according to the noise level of the point cloud, when the noise is large, the number of iterations can be appropriately increased); the model selects the precision mode. If the denoising effect is not ideal, you can reduce the kernel size or the number of iterations and denoise again.
[0079] Gaussian filtering algorithm for point cloud array denoising:
[0080]
[0081]
[0082] S105, diluting the point cloud array after denoising according to a ratio to obtain fruit surface point cloud arrays with different dilution ratios.
[0083] Select the denoised point cloud array and select the dilution option in the edit page. The method is Space-BasedSampling, which is suitable for scenes with evenly distributed points and used for subsequent point cloud meshing. Set the dilution ratios to 0.1197, 0.2394, 0.3592, 0.4789, 0.5986, and 0.7183. Select the appropriate dilution ratio based on the preservation of the point cloud geometric features. The dilution ratio selected in this example is 0.1197.
[0084] S106, meshing the point cloud arrays after different dilution ratios to generate mesh surfaces of different sizes, which are then saved in stl. format.
[0085] Select the diluted point cloud array, select the meshing option on the editing page, select Delaunay 3D (suitable for three-dimensional distributed point cloud arrays), set it to the best fit surface, and the maximum triangulation edge length is 2mm. The quality of meshing depends on the density and uniformity of the point cloud, which is related to the accuracy and smoothness of the denoising and dilution processing.
[0086] S107, importing the mesh surface in stl. format into SolidWorks software, filling the mesh surface, and generating a complete surface.
[0087] Select the surface model to be converted to a solid and check whether the surface is completely closed. In the surface option of the tool, fill and trim the surface model to complete the surface closure. Then perform mesh entity conversion to convert the surface into a solid. In the evaluation tool, you can see that the model converted to a solid already has surface area and volume attributes, indicating that the model entity conversion is successful.
[0088] S108, selecting a curved surface of optimal size, and thickening the surface vertically and outwardly along the contour of the curved surface to determine the numerical value to generate a peel and pulp model.
[0089] The process of converting a closed surface entity into a peel entity is as follows: In the surface option of the tool, complete the solid model thickening process. Set the peel thickening direction to Vertically outward and the thickness to 0.7mm (according to the actual peel thickness); set the pulp thickening direction to Vertically inward and the thickness to 40mm (according to the actual fruit axis diameter). Complete the operation of converting the surface entity into a peel and pulp solid model.
[0090] S109, combining the peel and pulp models according to the model outline to obtain a complete three-dimensional fruit model.
[0091] According to the model outline, the peel and pulp models are assembled to obtain a complete three-dimensional model of the Red Fuji apple. Figure 2 The flow chart of the software interface for building the three-dimensional model of Red Fuji apple based on point cloud imaging is given.
[0092] According to the method of the above embodiment, the present application also successively completed the construction of the three-dimensional model of Korla pear and Narcissus mango, such as Figure 3 shown.
[0093] In the embodiments of the present application, "at least one" refers to one or more, and "plurality" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. Among them, A and B can be singular or plural. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.
[0094] The above is only a specific implementation of the present application. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. The protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A method for modeling a three-dimensional fruit model based on point cloud imaging, characterized in that: include: Select a fruit with uniform shape and size and no obvious scars, image the surface of the fruit, and mark the eight positions of the fruit cross section divided into eight equal parts; According to the distribution of the marking points, a scanner is used to perform a 360° scan of the fruit surface in the latitude direction to plan the scanning path, and data on the surface of different parts of the fruit are collected according to the scanning path planning to generate a point cloud array; Import the collected data into CloudCompare software, and splice the point cloud arrays of different parts by matching the marking points; The spliced complete fruit surface point cloud array is subjected to denoising to extract key shape features; The point cloud array after denoising is diluted according to the proportion to obtain the fruit surface point cloud arrays with different dilution ratios; The point cloud arrays after different dilution ratios are meshed to generate mesh surfaces of different sizes, which are then saved in stl. format; Import the mesh surface in stl. format into SolidWorks software, fill the mesh surface, and generate a complete surface; Select the surface with the best size, and thicken it vertically and outwards along the contour of the surface to determine the numerical value of the peel and pulp models; The peel and pulp models are combined according to the model contour to obtain a complete three-dimensional model of the fruit.
2. The fruit three-dimensional modeling method based on point cloud imaging according to claim 1, characterized in that: The method comprises: selecting a fruit with uniform shape and size and without obvious scars, performing imaging on the surface of the fruit, and marking eight positions of the fruit cross section divided into eight equal parts, including: Spraying white developer on the surface of the fruit to highlight the position of subsequent marking points so that the scanner can more easily identify the scanned object; The fruit is divided into eight equal parts at intervals of 45° along the latitude; In each interval, the fruit is randomly marked in the meridian direction.
3. The fruit three-dimensional modeling method based on point cloud imaging according to claim 1, characterized in that, According to the distribution position of the marking points, a scanning path is planned by using a scanner to perform a 360° scanning on the latitude direction of the fruit surface, and data on the surface of different parts of the fruit are collected according to the scanning path planning to generate a point cloud array, including: When planning the scanning path, the angle of the meridian direction is first determined, and the meridian directions are in a top-down order; After determining the angle of the longitude direction, the fruit surface is scanned 360° in the latitude direction. During the scanning process, the single rotation angle of the fruit in the latitude direction is 10-20°; After scanning for one circle, the single rotation angle in the meridian direction is 20 to 30 degrees, and the operation is repeated until the full range of the fruit surface is scanned; After rotating to a certain angle, click the scan button. The binocular scanner will scan and store the geometric data of the fruit surface at this angle. Perform multiple rotation operations in the latitude direction, and the scanner will automatically match the marked points to generate a point cloud array.
4. The fruit three-dimensional modeling method based on point cloud imaging according to claim 3, characterized in that: The collected data is imported into the CloudCompare software, and the point cloud arrays of different parts are spliced according to the method of matching the marking points, including: According to the marked marking points, the marking points are numbered along the meridian direction. Two sets of point cloud arrays are imported each time, which are defined as the reference point cloud array and the point cloud array to be aligned. The marking points with obvious shape features in the two sets of point cloud arrays are roughly spliced; Then, the iterative closest point ICP algorithm is used to further fine-tune the point clouds according to the geometric features between them. The objective function of fine-tune the point clouds is: Where: R is the rotation matrix in the optimal change matrix; t is the translation vector in the optimal change matrix; q i is the target point set; p i is the source point set; i is the point number, i∈(1,N).
5. The fruit three-dimensional modeling method based on point cloud imaging according to claim 4, characterized in that: The rough stitching operation process is as follows: select the alignment option on the CloudCompare software tool page, and the alignment method is point pair selection. In the main interface, select the point pairs that need to be aligned in the reference point cloud array and the point cloud to be aligned, and add them one by one until the selected number of groups is completed. After the group selection is completed, select alignment to complete the rough stitching, among which: select multiple groups or more obvious feature points in the reference point cloud array and the point cloud to be aligned.
6. The fruit three-dimensional modeling method based on point cloud imaging according to claim 4, characterized in that: The workflow for fine stitching is as follows: select the reference point cloud array and the point cloud array to be aligned after rough stitching at the same time, select the ICP option on the CloudCompare software tool page, set the iteration mode to iterate the nearest point, set the overlap ratio of the point cloud to be greater than or equal to 60%, the scaling ratio to 1:3, the iteration stop threshold to 1mm, the maximum number of iterations to 50, and do not filter noise points during calculation. The position and angle of the point cloud will be automatically adjusted during the fine stitching process until the overlapping parts of the two sets of point clouds are optimally aligned.
7. The fruit three-dimensional modeling method based on point cloud imaging according to claim 1, characterized in that: The process of performing denoising on the spliced complete fruit surface point cloud array and extracting key shape features includes: Select the stitched point cloud array, select the Gaussian filter option, the Laplace operator, set the sigma, kernel size and number of iterations of the Laplace operator, where: adjust the number of iterations according to the noise level of the point cloud. The Gaussian filter denoising process formula is: Where: F(x, y, z) is the coordinate value of the point after filtering, i is the point number, i∈(1,N), p i is the coordinate value of the i-th point in the neighborhood, p o is the coordinate value of the current point, G(||p i -p o|| ) is the Gaussian weight of the i-th point, which is the distance p between the i-th point and the current point i -p o related.
8. The method for modeling a fruit three-dimensional model based on point cloud imaging according to claim 1, characterized in that: The point cloud array after the denoising process is diluted according to a ratio to obtain the fruit surface point cloud arrays with different dilution ratios, including: Select the denoised point cloud array and use a spatial sampling method that matches the evenly distributed point scenario to determine the dilution option for subsequent point cloud meshing; The dilution ratio is selected and set according to the geometric features of the point cloud, wherein point clouds with different geometric features correspond to different dilution ratios.
9. The fruit three-dimensional modeling method based on point cloud imaging according to claim 1, characterized in that: The point cloud arrays after different dilution ratios are meshed to generate mesh surfaces of different sizes, which are then saved in stl. format, including: Select the point cloud array that has been diluted, and choose the Meshing option on the SolidWorks software editing page; Set the best fitting surface and determine the maximum triangulation edge length to generate a mesh surface in stl. format.
10. The method for modeling a fruit three-dimensional model based on point cloud imaging according to claim 1, characterized in that: The mesh surface in stl. format is imported into SolidWorks software, and the mesh surface is filled to generate a complete surface, including: Select the surface model that needs to be converted into a solid and check whether the surface is completely closed; If the surface is not completely closed, fill and trim the surface model in the surface options of SolidWorks software to complete the surface closure; Then, the surface is converted into a mesh entity, and the surface is converted into an entity.