Method and device for picking and positioning berry fruits

By segmenting and reconstructing the point cloud data of strawberries and fruit stems, determining the picking points of strawberries, solving the problem of picking points error in the existing technology, and achieving accurate picking of strawberries.

CN118941630APending Publication Date: 2024-11-12NORTH CHINA UNIVERSITY OF TECHNOLOGY

Patent Information

Application Number
CN202410990215.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-23
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

When identifying strawberries, there is an error between the estimated picking points and the real picking points, resulting in the picking failure.

Method used

By dividing the berries into original point clouds, obtaining the sepal equivalent round surface and the statuary pose line, determining the fruit stalk picking point based on the line-to-surface relationship, and through steps such as point cloud edge extraction, principal component analysis and surface reconstruction, Accurately locate the picking points.

Benefits of technology

It realizes the accurate picking of strawberries, and can observe the connection points between strawberries and fruit stems from different angles to ensure the accuracy and completeness of picking.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118941630A_ABST
    Figure CN118941630A_ABST
Patent Text Reader

Abstract

The invention relates to a berry type fruit picking and positioning method and device. The method comprises the following steps that berry type fruits are divided into berry original point clouds and fruit stem original point clouds; obtaining an equivalent round surface of sepals based on the original point cloud of the berries; obtaining a fruit stem pose straight line based on the fruit stem original point cloud; determining a fruit stem picking point based on a line-plane relationship between the sepal equivalent circular surface and the fruit stem pose straight line; and picking and positioning the berry fruits based on the fruit stem picking points and the fruit stem pose straight lines. Through the method, the picking point can be quickly positioned for picking; complete strawberry outlines can be conveniently obtained from the angle of head-up observation, the number of strawberries can be conveniently obtained from the angle of long-distance observation, it can be conveniently guaranteed that connection points of strawberries to be picked and fruit stems are visible and complete fruit stems can be conveniently obtained from the angle of overlook observation, and high definition of the connection points and the fruit stems can be conveniently guaranteed from the angle of short-distance observation.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of agricultural picking, and in particular relates to a method for picking and positioning berry fruits. Background Art

[0002] Most of the existing picking positioning methods estimate the strawberry axis based on the identification of the strawberry and estimate the picking point on the fruit axis. However, the actual picking point is located on the fruit stalk. There is an error between the estimated picking point and the actual picking point position, which will lead to picking failure due to the existence of the error. Summary of the invention

[0003] The purpose of the present invention is to overcome the above problems and provide a method and device for picking and positioning berry fruits.

[0004] In order to achieve the above purpose, the technical solution adopted by the present invention is:

[0005] A method for picking and locating berry fruits, comprising the following steps:

[0006] Segment berry fruits into berry original point cloud and fruit stem original point cloud;

[0007] Obtaining a sepal equivalent circular surface based on the berry original point cloud;

[0008] Obtaining a fruit stalk posture straight line based on the fruit stalk original point cloud;

[0009] Determine the fruit stalk picking point based on the line-surface relationship between the sepal equivalent circular surface and the fruit stalk posture straight line;

[0010] The berry fruit is picked and positioned based on the fruit stalk picking point and the fruit stalk posture straight line.

[0011] As a further improvement, the following steps are also included:

[0012] Extract the edge of the point cloud of the original point cloud of berry fruits to obtain the edge of the point cloud of the fruit;

[0013] Performing principal component analysis on the edge of the fruit point cloud to obtain the primary axis and secondary axis of the fruit edge;

[0014] Taking a cross product of the major axis and the minor axis to obtain a plane normal vector;

[0015] The centroid of the point cloud is obtained by averaging the primary axis and the secondary axis;

[0016] The plane normal vector is rotated and the center of mass is translated to obtain the position and observation angle of the collected fruit data.

[0017] As a further improvement, obtaining the sepal equivalent circular surface based on the berry original point cloud comprises the following steps:

[0018] Performing surface reconstruction on the original point cloud of the fruit to obtain an equivalent point cloud of the fruit;

[0019] Extracting the edge of the fruit point cloud equivalent to obtain the edge of the fruit point cloud;

[0020] Screening out the top edge of the fruit point cloud on the edge of the fruit point cloud;

[0021] The top edge of the fruit point cloud is fitted into the top edge plane of the fruit point cloud;

[0022] Extract the projection line of the top edge of the fruit point cloud on the plane of the top edge of the fruit point cloud;

[0023] The projection line circle is fitted into the sepal equivalent circle surface.

[0024] As a further improvement, the step of obtaining the fruit stalk pose straight line based on the fruit stalk original point cloud comprises the following steps:

[0025] The original point cloud of the fruit stalk is reconstructed to obtain the equivalent point cloud of the fruit stalk;

[0026] Extracting the point cloud of the bottom of the fruit stalk from the equivalent point cloud of the fruit stalk to obtain the point cloud of the fruit stalk posture;

[0027] A straight line is obtained by performing straight line fitting on the fruit stalk posture point cloud.

[0028] As a further improvement, the method of determining the fruit stalk picking point based on the line-surface relationship between the sepal equivalent circular surface and the fruit stalk posture straight line comprises the following steps:

[0029] Determine the line-surface relationship between the fruit stalk posture straight line and the sepal equivalent circular surface to obtain the relationship between the fruit stalk and the fruit;

[0030] Based on the relationship between the fruit and the stalk, the picking point of the strawberry is calculated.

[0031] As a further improvement, the relationship between the fruit and the fruit stalk includes:

[0032] If the line-surface relationship is one intersection point, the fruit stalk belongs to the fruit; if the line-surface relationship is multiple intersection points, the corresponding straight line closest to the intersection point and the center of the circle is taken as the fruit stalk belonging to the fruit.

[0033] As a further improvement, the step of screening out the top edge of the fruit point cloud on the edge of the fruit point cloud includes:

[0034] Based on the edge of the strawberry and the half plane of the fruit, take the edge point above the half plane of the fruit;

[0035] Clustering the edge points and finding the centroid of each clustering result;

[0036] The category with the largest vertical axis component of the centroid is taken as the point cloud of the top of the strawberry.

[0037] As a further improvement, the observation angle of the fruit data is determined based on a plane passing through the main axis and a normal vector of the plane; the normal vector and the observation angle are fixed angles.

[0038] On the other hand, the present invention also provides a system for berry fruit picking and positioning, the system comprising at least one processor and a memory storing instructions, and when the instructions are executed by at least one processor, the method in the above technical solution is implemented.

[0039] A computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method described in the steps of the above technical solution.

[0040] A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method described in the steps of the above technical solution is implemented.

[0041] The beneficial effects of the present invention are:

[0042] By using the method and device, the picking point can be quickly located and the strawberries can be picked; a relatively complete outline of the strawberries can be obtained from a horizontal perspective, the number of strawberries can be obtained from a long distance, the connection point between the strawberries to be picked and the fruit stem can be seen and the complete fruit stem can be obtained from a top-down perspective, and the connection point and the fruit stem can be highly defined from a close distance.

[0043] By observing from a long distance and looking straight ahead, the position information of all strawberries in a certain area can be obtained, and the strawberry targets to be picked can be determined; by observing from a close distance and looking down, the fruit stalks around the strawberries to be picked and the connection points between the strawberries to be picked and the fruit stalks connected to them can be obtained, the connection relationship between the strawberries to be picked and the surrounding fruit stalks can be analyzed, and the fruit stalks connected to the strawberries to be picked can be found; by observing from different positions and angles, the strawberries, fruit stalks and the connection points between the two can be comprehensively observed, and a complete description of a single strawberry target can be completed, thereby achieving the task of accurately picking strawberries. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The present invention has the following accompanying drawings:

[0045] Figure 1is a flow chart of a method for picking and locating berry fruits in some specific embodiments of the present invention;

[0046] Figure 2 It is a schematic diagram of the structure of a device for picking and positioning berry fruits in some specific embodiments of the present invention;

[0047] Figure 3 A schematic diagram of a system for picking and positioning berries of the present invention;

[0048] Figure 4 Schematic diagram of the boundary of a strawberry point cloud according to some specific embodiments of the present invention;

[0049] Figure 5 Schematic diagram of the main direction of the PCA calculated point cloud in some specific embodiments of the present invention;

[0050] Figure 6 is a schematic diagram of a point cloud edge in some specific embodiments of the present invention;

[0051] Figure 7 are schematic diagrams of some specific embodiments of the present invention viewed from the side;

[0052] Figure 8 The figure is a schematic diagram of the posture of a strawberry observed from the side in some specific embodiments of the present invention. Figure 1 ;

[0053] Fig. 9 The figure is a schematic diagram of the posture of a strawberry observed from the side in some specific embodiments of the present invention. Figure 2 ;

[0054] Fig.10 is a schematic diagram of a point cloud located above a plane according to some specific embodiments of the present invention;

[0055] Fig.11 It is a schematic diagram of the results after clustering segmentation in some specific embodiments of the present invention. DETAILED DESCRIPTION

[0056] The following examples further illustrate the content of the present invention, but should not be construed as limiting the present invention. Without departing from the spirit and substance of the present invention, modifications or substitutions made to the methods, steps or conditions of the present invention all fall within the scope of the present invention.

[0057] Embodiment 1:

[0058] like Figure 1 As shown, the present invention provides a method for picking and locating berry fruits, comprising the following steps:

[0059] Step S101: Segmenting berry fruits into berry original point clouds and fruit stem original point clouds;

[0060] For example, after determining the collection position in advance, the berry data is collected, and then the segmentation module is used to split the berries into berry original point clouds and fruit stalk original point clouds. In this way, different technologies can be used to process these two parts separately to obtain the corresponding sepal equivalent circular surface and fruit stalk posture data.

[0061] Step S102: obtaining a sepal equivalent circular surface based on the berry original point cloud;

[0062] For example, after obtaining the original point cloud of the berry, the equivalent point cloud of the berry fruit is obtained by using the surface reconstruction technology of the point cloud data. At this time, the point cloud includes the edge point cloud on the central half-section of the fruit and the point cloud data on other half-planes; in order to obtain the edge point cloud of the fruit, the overall point cloud must be extracted and the contours must be screened, so that the point cloud data of the top edge of the point cloud can be obtained; then the edge point cloud fitting plane technology is used to obtain the plane where the edge is located, and then the edge point cloud is projected onto the plane of the edge line, and the circle fitting technology is used to obtain the equivalent circular surface of the sepals.

[0063] Step S103: obtaining a fruit stalk posture straight line based on the fruit stalk original point cloud;

[0064] For example, the original point cloud of the fruit stalk is reconstructed using surface reconstruction technology to obtain the equivalent point cloud of the fruit stalk, the point cloud at the bottom of the fruit stalk is extracted to obtain the fruit stalk pose point cloud, and then the fruit stalk pose straight line is obtained through straight line fitting processing.

[0065] Step S104: determining a fruit stalk picking point based on the line-surface relationship between the sepal equivalent circular surface and the fruit stalk posture straight line;

[0066] Step S105: Position the berry fruit for picking based on the stalk picking point and the stalk posture straight line. In the present invention, the stalk picking point and stalk posture straight line are used for positioning, because a fruit has only one stalk on the sepals, the corresponding berry is found through the stalk, and then the picking point is found on the berry for positioning and picking. After the position is determined, the robot arm is moved to the designated position for picking: after the picking point is generated by the algorithm, the picking mechanism at the end of the robot arm moves to the picking point position, and the strawberry stalk is sheared by the picking mechanism to complete the strawberry picking task.

[0067] In some embodiments of the present invention, the following steps are also included:

[0068] Step S11: extracting the edge of the point cloud of the berry fruit original point cloud to obtain the edge of the fruit point cloud;

[0069] Step S12: performing principal component analysis on the edge of the fruit point cloud to obtain the primary axis and secondary axis of the fruit edge;

[0070] Step S13: Calculate the cross product of the major axis and the minor axis to obtain a plane normal vector;

[0071] Step S14: averaging the primary axis and the secondary axis to obtain the point cloud centroid;

[0072] Step S15: rotating the plane normal vector and translating the center of mass to obtain the position and observation angle of the collected fruit data.

[0073] The above steps S11 to S15 are to find the shooting angle and shooting position to facilitate the collection of berry data. Because when the berries are observed from different angles, the patterns that can be seen are different. For some, the connection point between the pedicel and the sepal can be seen, but for some angles, it cannot be seen.

[0074] For example, find the shooting angle and shooting position, move the robot arm to the specified position to observe the connection point between the strawberry and the stalk and the stalk. After the algorithm determines the target strawberry to be picked, move the camera at the end of the robot arm to a position close to the strawberry to be picked, and observe the connection point between the strawberry and the stalk and the stalk from a bird's-eye view.

[0075] For example, the observation angle is determined according to the posture of a strawberry (a berry with fruit, sepals, and pedicels). The strawberry point cloud data is half of the real space surface of the strawberry, and the main axis of the strawberry is located in the plane where half of the strawberry is located; the main direction of the strawberry point cloud is calculated by principal component analysis technology to obtain the main axis component and the secondary axis main component. The plane normal vector is obtained by cross product of the main axis component and the secondary axis component. The main axis direction of the strawberry point cloud is calculated using the point cloud boundary points and all the points of the point cloud, respectively, to obtain the main axis and the secondary axis, respectively. Two plane normal vectors are obtained by cross product of the main axis and the secondary axis. Among them, when judging whether the main axis and the secondary axis are collinear, the cross product of the two is approximately equal to the zero vector, so the two can be considered equal, that is, collinear or coplanar. Find the center of mass of the point cloud boundary, translate the main axis and the secondary axis of the strawberry point cloud to the center of mass position, and draw it in the original strawberry point cloud image.

[0076] Through analysis, we can get three different postures of the strawberry in the side view direction, which are divided into three types: a, b, and c. The observation direction of the camera is at a fixed angle with the normal vector of the half plane of the strawberry (an empirical value should be obtained from experiments). Therefore, the plane normal vector is rotated downward by a fixed angle as the observation angle.

[0077] Step S102 includes the following steps:

[0078] Step S1021: performing surface reconstruction on the original point cloud of the fruit to obtain an equivalent point cloud of the fruit;

[0079] Step S1022: extracting the edge of the point cloud for the fruit equivalent point cloud to obtain the edge of the fruit point cloud;

[0080] Step S1023: Filtering out the top edge of the fruit point cloud on the edge of the fruit point cloud;

[0081] For example, point cloud extraction of the top of a strawberry: Based on the edge of the strawberry and the half-plane of the strawberry, find the plane where the edge of the strawberry is located, take the edge points above the half-plane, cluster them, find the centroid of each clustering result, and take the one with the largest vertical axis component of the centroid as the point cloud of the top of the strawberry.

[0082] Step S1024: the top edge of the fruit point cloud is fitted into a top edge plane of the fruit point cloud;

[0083] Step S1025: extracting the projection line of the top edge of the fruit point cloud on the plane of the top edge of the fruit point cloud;

[0084] Step S1026: fitting the projection line circle into a sepal equivalent circular surface.

[0085] Step S103 includes the following steps:

[0086] Step S1031: performing surface reconstruction on the original point cloud of the fruit stalk to obtain an equivalent point cloud of the fruit stalk;

[0087] Step S1032: extracting the point cloud of the bottom of the fruit stalk from the fruit stalk equivalent point cloud to obtain the fruit stalk posture point cloud;

[0088] Step S1033: performing straight line fitting on the fruit stalk posture point cloud to obtain a fruit stalk posture straight line.

[0089] Step S104 includes the following steps:

[0090] Step S1041: determining the line-surface relationship between the fruit stalk posture straight line and the sepal equivalent circular surface, and obtaining the relationship between the fruit stalk and the fruit;

[0091] Step S1042: Calculate the picking point of the strawberry based on the relationship between the fruit and the stalk.

[0092] In some embodiments of the present invention, the relationship between the fruit and the fruit stalk includes:

[0093] If the line-surface relationship is one intersection point, the fruit stalk belongs to the fruit; if the line-surface relationship is multiple intersection points, the corresponding straight line closest to the intersection point and the center of the circle is taken as the fruit stalk belonging to the fruit.

[0094] In some embodiments of the present invention, the step of screening out the top edge of the fruit point cloud on the edge of the fruit point cloud comprises:

[0095] Based on the edge of the strawberry and the half plane of the fruit, take the edge point above the half plane of the fruit;

[0096] Clustering the edge points and finding the centroid of each clustering result;

[0097] The category with the largest vertical axis component of the centroid is taken as the point cloud of the top of the strawberry.

[0098] In some embodiments of the present invention, the observation angle of the fruit data is determined based on a plane passing through the main axis and a normal vector of the plane; the normal vector and the observation angle are fixed angles.

[0099] Embodiment 2:

[0100] like Figure 2 As shown, the present invention provides a device for picking and positioning berry fruits, comprising:

[0101] A segmentation module is used to segment berry fruits into berry original point clouds and fruit stem original point clouds;

[0102] The first obtaining module is used to obtain the sepal equivalent circular surface based on the original point cloud of the berry;

[0103] The second obtaining module is used to obtain the fruit stalk posture straight line based on the fruit stalk original point cloud;

[0104] A determination module, for determining a fruit stalk picking point based on a line-surface relationship between the sepal equivalent circular surface and the fruit stalk posture straight line;

[0105] A positioning module is used to locate the berry fruit picking based on the fruit stalk picking point and the fruit stalk posture straight line.

[0106] In some embodiments of the present invention, it further comprises:

[0107] Extract the edge of the point cloud of the original point cloud of berry fruits to obtain the edge of the point cloud of the fruit;

[0108] Performing principal component analysis on the edge of the fruit point cloud to obtain the primary axis and secondary axis of the fruit edge;

[0109] Taking a cross product of the major axis and the minor axis to obtain a plane normal vector;

[0110] The centroid of the point cloud is obtained by averaging the primary axis and the secondary axis;

[0111] The plane normal vector is rotated and the center of mass is translated to obtain the position and observation angle of the collected fruit data.

[0112] In some embodiments of the present invention, the first obtaining module includes:

[0113] Performing surface reconstruction on the original point cloud of the fruit to obtain an equivalent point cloud of the fruit;

[0114] Extracting the edge of the fruit point cloud equivalent to obtain the edge of the fruit point cloud;

[0115] Screening out the top edge of the fruit point cloud on the edge of the fruit point cloud;

[0116] The top edge of the fruit point cloud is fitted into the top edge plane of the fruit point cloud;

[0117] Extract the projection line of the top edge of the fruit point cloud on the plane of the top edge of the fruit point cloud;

[0118] The projection line circle is fitted into the sepal equivalent circle surface.

[0119] In some embodiments of the present invention, the second obtaining module includes:

[0120] The original point cloud of the fruit stalk is reconstructed to obtain the equivalent point cloud of the fruit stalk;

[0121] Extracting the point cloud of the bottom of the fruit stalk from the equivalent point cloud of the fruit stalk to obtain the point cloud of the fruit stalk posture;

[0122] A straight line is obtained by performing straight line fitting on the fruit stalk posture point cloud.

[0123] In some embodiments of the present invention, the determining module comprises the following steps:

[0124] Determine the line-surface relationship between the fruit stalk posture straight line and the sepal equivalent circular surface to obtain the relationship between the fruit stalk and the fruit;

[0125] Based on the relationship between the fruit and the stalk, the picking point of the strawberry is calculated.

[0126] In some embodiments of the present invention, the relationship between the fruit and the fruit stalk includes:

[0127] If the line-surface relationship is one intersection point, the fruit stalk belongs to the fruit; if the line-surface relationship is multiple intersection points, the corresponding straight line closest to the intersection point and the center of the circle is taken as the fruit stalk belonging to the fruit.

[0128] In some embodiments of the present invention, the step of screening out the top edge of the fruit point cloud on the edge of the fruit point cloud comprises:

[0129] Based on the edge of the strawberry and the half plane of the fruit, take the edge point above the half plane of the fruit;

[0130] Clustering the edge points and finding the centroid of each clustering result;

[0131] The category with the largest vertical axis component of the centroid is taken as the point cloud of the top of the strawberry.

[0132] In some embodiments of the present invention, the observation angle of the fruit data is determined based on a plane passing through the main axis and a normal vector of the plane; the normal vector and the observation angle are fixed angles.

[0133] For example, the classification of strawberry sepals:

[0134] 1. Fit;

[0135] 2. The tips of sepals are curled back;

[0136] 3. Sepals are partially recurved;

[0137] 4. Completely reverse.

[0138] In one embodiment, the main research areas are: 1. fitting, 2. curling.

[0139] In the horizontal perspective, the strawberry sepals are in two different states. When the sepals are attached, the connection point between the pedicel and the sepals cannot be observed. However, when the sepals are curled, the connection point between the pedicel and the sepals cannot be observed due to the obstruction of the sepals.

[0140] If you switch the viewing angle and observe the strawberry from a bird's-eye view, you can see the connection point between the pedicel and the sepals regardless of whether the sepals are in a closed or curled state.

[0141] The connection point between the sepals and the fruit stalk cannot be seen from a horizontal angle, but it can be seen from a top-down angle.

[0142] like Figure 5 As shown, the observation angle is determined according to the posture of the strawberry; according to the above analysis, the observation angle of the strawberry should be a top-down view; the strawberry point cloud data is half of the real space surface of the strawberry, and the principal axis of the strawberry is located in the plane where half of the strawberry is located; PCA calculates the main direction of the strawberry point cloud to obtain the first principal component and the second principal component; the plane normal vector is obtained by doing a cross product of the first principal component and the second principal component; the main directions of the strawberry point cloud are calculated using the boundary points of the point cloud and all the points of the point cloud, respectively, to obtain the first principal component and the second principal component, respectively.

[0143] like Figure 6 As shown, two plane normal vectors (regioncross and edge cross) are obtained by cross product of the first principal component and the second principal component.

[0144] Parallelism test

[0145] To determine whether the two are collinear, the cross product (res cross) of the two is approximately equal to the zero vector, so the two can be considered equal.

[0146] Find the centroid of the point cloud boundary, translate the first and second principal components of the main direction of the strawberry point cloud to the centroid position, and draw them in the original strawberry point cloud image.

[0147] like Figure 7 As shown in the figure, the posture of the strawberry is observed from the side view. Through analysis, we can get three different postures of the strawberry from the side view, which are divided into three types: a, b, and c. Figure 8 , 9 In the figure, the bottom arc represents the strawberry, the middle of the strawberry represents the cross section of the strawberry, the line perpendicular to the cross section is the normal line, and the ray coming down from the upper right is the viewing direction of the camera. In the three postures of observing the strawberry from the side, the viewing direction and the normal line form a fixed angle α, and each time by Figure 8 Transform to Fig. 9 The transformation of the observation angle of the three postures is the rotation of the fixed angle θ. α is a fixed angle (an empirical value should be obtained from experiments). Therefore, the plane normal vector is rotated downward by a fixed angle θ as the observation angle.

[0148] like Figure 8-9 As shown, there are three postures of strawberries when viewed from the side.

[0149] like Figure 4 , 10 -11 shows the extraction of the point cloud of the top of the strawberry; the edge of the strawberry and the half-plane of the strawberry; finding the plane where the boundary of the strawberry is located, taking the points above the plane, clustering them, and finding the centroid of each clustering result; taking the class with the largest centroid y component as the point cloud of the top of the strawberry.

[0150] By using the method and device, the picking point can be quickly located and the strawberries can be picked; a relatively complete outline of the strawberries can be obtained from a horizontal perspective, the number of strawberries can be obtained from a long distance, the connection point between the strawberries to be picked and the fruit stem can be seen and the complete fruit stem can be obtained from a top-down perspective, and the connection point and the fruit stem can be highly defined from a close distance.

[0151] By observing from a long distance and looking straight ahead, the position information of all strawberries in a certain area can be obtained, and the strawberry targets to be picked can be determined; by observing from a close distance and looking down, the fruit stalks around the strawberries to be picked and the connection points between the strawberries to be picked and the fruit stalks connected to them can be obtained, the connection relationship between the strawberries to be picked and the surrounding fruit stalks can be analyzed, and the fruit stalks connected to the strawberries to be picked can be found; by observing from different positions and angles, the strawberries, fruit stalks and the connection points between the two can be comprehensively observed, and a complete description of a single strawberry target can be completed, thereby achieving the task of accurately picking strawberries.

[0152] In some specific embodiments, Figure 3As shown, a system for picking and locating berry fruits includes at least one processor and a memory storing instructions to implement any one of the methods for picking and locating berry fruits in the above technical solutions.

[0153] In some specific embodiments, a computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps in the above technical solution.

[0154] In some specific embodiments, a computer program product is provided, the computer program product includes a computer program, and when the computer program is executed by a processor, the method of implementing the steps in the above technical solution is implemented.

[0155] The embodiments and functional operations of the subject matter described in this specification may be implemented in: digital electronic circuitry, tangibly implemented computer software or firmware, computer hardware, including the structures disclosed in this specification and their structural equivalents, or a combination of more than one of the above. The embodiments of the subject matter described in this specification may be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on one or more tangible non-transitory program carriers, for execution by a data processing device or to control the operation of a data processing device.

[0156] Alternatively or additionally, the program instructions may be encoded on an artificially generated propagation signal, such as a machine-generated electrical, optical, or electromagnetic signal, which is generated to encode information for transmission to an appropriate receiver device for execution by a data processing device. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of the above devices.

[0157] A computer program (which may also be referred to or described as a program, software, software application, module, software module, script or code) may be written in any form of programming language, including compiled or interpreted languages ​​or declarative or procedural languages, and the computer program may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment. A computer program may, but need not, correspond to a file in a file system. A program may be stored in a portion of a file that stores other programs or data, for example, one or more scripts stored in the following: in a markup language document; in a single file dedicated to a related program; or in multiple collaborative files, for example, files storing one or more modules, subroutines or code portions. A computer program may be deployed to execute on one or more computers, the computers being located in one place or distributed to multiple locations and interconnected by a communication network.

[0158] The contents not described in detail in this specification belong to the prior art known to professional and technical personnel in this field.

[0159] Although the present invention has been described in detail above by general description, specific implementation methods and experiments, it is obvious to those skilled in the art that some modifications or improvements can be made to the present invention. Therefore, these modifications or improvements made without departing from the spirit of the present invention all belong to the scope of protection claimed by the present invention.

Claims

1. A method for picking and locating berry fruits, characterized in that: The following steps are involved: Segment berry fruits into berry original point cloud and fruit stem original point cloud; Obtaining a sepal equivalent circular surface based on the berry original point cloud; Obtaining a fruit stalk posture straight line based on the fruit stalk original point cloud; Determine the fruit stalk picking point based on the line-surface relationship between the sepal equivalent circular surface and the fruit stalk posture straight line; The berry fruit is picked and positioned based on the fruit stalk picking point and the fruit stalk posture straight line.

2. A method for picking and locating berry fruits according to claim 1, characterized in that: The following steps are also included: Extract the edge of the point cloud of the original point cloud of berry fruits to obtain the edge of the point cloud of the fruit; Performing principal component analysis on the edge of the fruit point cloud to obtain the primary axis and secondary axis of the fruit edge; Taking a cross product of the major axis and the minor axis to obtain a plane normal vector; The centroid of the point cloud is obtained by averaging the primary axis and the secondary axis; The plane normal vector is rotated and the center of mass is translated to obtain the position and observation angle of the collected fruit data.

3. The method for picking and locating berry fruits according to claim 1, characterized in that: The method of obtaining a sepal equivalent circular surface based on the berry original point cloud comprises the following steps: Performing surface reconstruction on the original point cloud of the fruit to obtain an equivalent point cloud of the fruit; Extracting the edge of the fruit point cloud equivalent to obtain the edge of the fruit point cloud; Screening out the top edge of the fruit point cloud on the edge of the fruit point cloud; The top edge of the fruit point cloud is fitted into the top edge plane of the fruit point cloud; Extract the projection line of the top edge of the fruit point cloud on the plane of the top edge of the fruit point cloud; The projection line circle is fitted into the sepal equivalent circle surface.

4. The method for picking and locating berry fruits according to claim 1, characterized in that: The step of obtaining a fruit stalk posture straight line based on the fruit stalk original point cloud comprises the following steps: Perform surface reconstruction on the original point cloud of the fruit stalk to obtain the equivalent point cloud of the fruit stalk; Extracting the point cloud of the bottom of the fruit stalk from the equivalent point cloud of the fruit stalk to obtain the point cloud of the fruit stalk posture; A straight line is obtained by performing straight line fitting on the fruit stalk posture point cloud.

5. The method for picking and locating berry fruits according to claim 1, characterized in that: The method of determining the fruit stalk picking point based on the line-surface relationship between the sepal equivalent circular surface and the fruit stalk posture straight line comprises the following steps: Determine the line-surface relationship between the fruit stalk posture straight line and the sepal equivalent circular surface to obtain the relationship between the fruit stalk and the fruit; Based on the relationship between the fruit and the stalk, the picking point of the strawberry is calculated.

6. The method for picking and locating berry fruits according to claim 5, characterized in that: The relationship between the fruit and the fruit stalk includes: If the line-surface relationship is one intersection point, the fruit stalk belongs to the fruit; if the line-surface relationship is multiple intersection points, the corresponding straight line closest to the intersection point and the center of the circle is taken as the fruit stalk belonging to the fruit.

7. The method for picking and locating berry fruits according to claim 3, characterized in that: The step of screening out the top edge of the fruit point cloud on the edge of the fruit point cloud comprises: Based on the edge of the strawberry and the half plane of the fruit, take the edge point above the half plane of the fruit; Clustering the edge points and finding the centroid of each clustering result; The category with the largest vertical axis component of the centroid is taken as the point cloud of the top of the strawberry.

8. The method for picking and locating berry fruits according to claim 2, characterized in that: The observation angle of the fruit data is determined based on a plane passing through the main axis and a normal vector of the plane; the normal vector and the observation angle are fixed angles.

9. A system for berry fruit picking and positioning, the system comprising at least one processor and a memory storing instructions, which, when executed by at least one processor, implements a method for berry fruit picking and positioning as described in any one of claims 1-8.

Citation Information

Patent Citations

  • Method for positioning fruit stem shear point of bunched tomato picking robot

    CN117893599A

Cited By

  • Fruit pose real-time estimation method based on multi-frame information fusion

    CN121962239A