Single fruit spatial pose detection method, system and equipment based on local point cloud

Abstract

The present invention discloses a method, system, and device for detecting the spatial pose of a single fruit based on a local point cloud. The method comprises: collecting and receiving local smooth point cloud data of a single fruit and constructing an enclosing rectangular frame; determining the shape of the enclosing rectangular frame; if it is a square, extracting the point cloud coordinates and normal vector direction corresponding to the coordinate index of the center point of the enclosing rectangular frame as the fruit's spatial pose; if it is a rectangle, obtaining the center of a fitting sphere as the center of mass and constructing a Cartesian coordinate system; performing Delaunary triangulation on the point cloud; obtaining the spatial coordinates of all circumcenters and generating a Voronoi diagram; constructing fitting arcs; calculating the curvature radius of the fitting arcs; selecting the fitting arc with the largest curvature radius and calculating the spatial coordinates of its arc center; and using the symmetry axis of the target point cloud as the fruit's spatial pose. The present invention detects the spatial pose of a single fruit target by adopting a classification pose substitution strategy, overcoming the drawback of single fruit recognition that cannot accurately locate the real-time spatial pose, thereby improving the efficiency of automated fruit picking.

Description

Technical Field

[0001] The present invention belongs to the technical field of artificial intelligence, and specifically relates to a method, system and device for detecting the spatial pose of a single fruit based on a local point cloud. Background Art

[0002] Automated fruit harvesting has long been a key topic in agricultural production. The quality of fruit harvested by harvesting robots directly impacts the fruit's storage and transportation lifespan, and thus its economic value. However, in natural environments, the growth posture of fruit varies significantly due to factors such as soil, season, and weather, severely impacting the precise positioning of the picking point and the execution of the harvest. Literature indicates that determining the fruit's posture is a crucial step in the natural fruit harvesting process by harvesting robots. For spherical fruits such as pitaya, citrus, and apples, accurate posture positioning directly impacts the final yield of the fruit.

[0003] Currently, the more common methods for detecting fruit posture include posture detection methods based on convolutional neural networks, posture recognition methods based on binocular vision, and natural image symmetry axis detection methods based on edge feature learning. However, they still have many problems, such as: inverting high-latitude posture through two-dimensional grasping posture without directly using three-dimensional point clouds, resulting in large iteration errors; the method of directly using three-dimensional point clouds has high requirements on point cloud accuracy, and the learning time is too long, which is not convenient for the picking process of picking robots.

[0004] Therefore, how to provide a single fruit spatial pose detection technology based on local point clouds that can directly utilize three-dimensional point clouds and reduce the learning time to realize the picking process of picking robots is a technical problem that needs to be urgently solved by technical personnel in this field. Summary of the Invention

[0005] The main purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art and provide a method, system and equipment for detecting the spatial posture of a single fruit based on a local point cloud. The method realizes the detection of the spatial posture of a single fruit target by adopting a classification posture replacement strategy. When the method is applied to the field of fruit picking, the spatial posture of the fruit can be accurately found based on the image of the single fruit taken in the natural scene, overcoming the defect that the previous recognition of the single fruit cannot accurately locate the real-time spatial posture, facilitating the grasping of the robot arm, improving the efficiency of the automatic picking of the fruit, and reducing the damage to the fruit.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] In a first aspect, the present invention provides a method for detecting the spatial pose of a single fruit based on a local point cloud, comprising the following steps:

[0008] Collect and receive local smooth point cloud data of a single fruit, and construct an outer rectangular frame for the local smooth point cloud data of the single fruit; the outer rectangular frame is used for condition judgment before the spatial posture detection of the fruit;

[0009] Determine the shape of the outer rectangular frame. When the ratio of the adjacent sides of the outer rectangular frame is equal to or approximately equal to 1, the outer rectangular frame is identified as a square rectangular frame, and the point cloud coordinates and normal vector direction corresponding to the coordinate index of the center point of the square rectangular frame are extracted as the fruit spatial pose; otherwise, the shape of the outer rectangular frame is identified as a rectangle, and the single fruit symmetry axis is further constructed as the fruit spatial pose;

[0010] As a preferred technical solution, the further construction of a single fruit symmetry axis specifically includes the following steps:

[0011] The Ransac algorithm is used to process the local smooth point cloud data of a single fruit into a sphere, and the center of the sphere is used as the center of mass, and a Cartesian coordinate system is constructed with the center of mass as the coordinate origin.

[0012] Traverse the single-fruit local smooth point cloud data, perform Delaunay triangulation on it, and construct a Delaunay triangulation network from all the iteratively formed triangles;

[0013] Traverse the Delaunay triangulation network, calculate the center of each Delaunay triangle's circumcircle and record its spatial coordinates, connect the centers of the circumcircles of adjacent triangles in sequence, construct a Voronoi edge set and generate a Voronoi diagram;

[0014] Traverse the Voronoi diagram, use the Delaunay triangle circumcenter to construct a preliminary fitting arc, and calculate the curvature radius of the preliminary fitting arc;

[0015] Select the prepared fitting arc with the largest curvature radius as the selected fitting arc, and record the coordinates of the center of the selected fitting arc;

[0016] The symmetry axis of the local smooth point cloud data of a single fruit is obtained by connecting the center of the selected fitting arc and the center of mass of the fitting sphere, and the symmetry axis of the local smooth point cloud data of a single fruit is used as the spatial pose of the fruit.

[0017] As a preferred technical solution, the method of traversing the single-fruit local smooth point cloud data, performing Delaunay triangulation on it, and constructing a Delaunay triangulation network from all triangles formed by iteration further includes the following steps:

[0018] Calculate the distance d between each point and the centroid, and find the point with the largest distance d max The farthest point a; take point a as the starting point, take n preliminary points near the set positive integer preset value k, and construct the point set V;

[0019] Calculate the distance between each preliminary point in the point set V and point a, find the point b with the minimum distance, and then connect it with a to form a baseline edge of the Delaunay triangle;

[0020] Construct a Delaunay triangle with any preparatory point. If the triangle does not contain other point clouds, record the preparatory point as c. Otherwise, search again.

[0021] The line connecting b and c is used as the new baseline edge, and the Delaunay triangle is iteratively constructed and the preliminary points in the triangle that do not contain other point clouds are searched. The iteration terminates when all points in the point set V are included in the Delaunay triangulation network.

[0022] The iteration terminates when all points in the point set V, which contains the preliminary points of other point clouds, are included in the Delaunay triangulation network.

[0023] As a preferred technical solution, the method of constructing a preliminary fitting arc using the circumcenter of a Delaunay triangle and calculating the curvature radius of the preliminary fitting arc further includes the following steps:

[0024] Calculate the distance between each point and the centroid, find the farthest point a with the largest distance, and record the coordinates of point a as (x1, y1, z1); traverse the circumcenter of the Delaunay triangle starting from point a, select m circumcenters as preliminary points near the preset positive integer value k, and construct the point set M;

[0025] Randomly select two preliminary points from the point set M and record their coordinates as (x2, y2, z2) and (x3, y3, z3) respectively;

[0026] Use these two points and a to construct preliminary fitting arcs Calculate the curvature radius R of the prepared fitting arc;

[0027] Furthermore, the curvature radius R of the preliminary fitting arc is calculated as follows:

[0028] A1=y1·z2-y1·z3-z1·y2+z1·y3+y2·z3-y3·z2;

[0029] B1=-x1·z2+x1·z3+z1·x2-z1·x3-x2·z3+x3·z2;

[0030] C1=x1·y2-x1·y3-y1·x2+y1·x3+x2·y3-x3·y2;

[0031] D1=-x1·y2·z3+x1·y3·z2+x2·y1·z3-x3·y1·z2-x2·y3·z1+x3·y2·z1;

[0032] A2=2×(x2-x1);

[0033] B2=2×(y2-y1);

[0034] C2=2×(z2-z1);

[0035] D2=x1 2 +y1 2 +z1 2 -x2 2 -y2 2 -z2 2 ;

[0036] A3=2×(x3-x1);

[0037] B3=2×(y3-y1);

[0038] C3=2×(z3-z1);

[0039] D3=x1 2 +y1 2 +z1 2 -x3 2 -y3 2 -z3 2 ;

[0040]

[0041]

[0042] where x a ,y a ,z a To prepare the fitting arc center α′(x a ,y a ,z a ), A1, A2, A3, B1, B2, B3, C1, C2, and C3 represent the matrix parameters for calculating the coordinates of the fitting circle center.

[0043] In a second aspect, the present invention further provides a single fruit spatial pose detection system based on local point cloud, which is applied to a single fruit spatial pose detection method based on local point cloud, comprising an acquisition module, an identification and judgment module, a first processing module, a second processing module, a third processing module, a fourth processing module, a fifth processing module, and an execution module;

[0044] The acquisition module is used to collect and receive local smooth point cloud data of a single fruit, and construct an outer rectangular frame based on the local smooth point cloud data of the single fruit;

[0045] The identification and judgment module compares the outer rectangular frame with the data in the local smooth point cloud data of the single fruit, and judges the ratio of the side length of the outer rectangular frame: when the value of the adjacent side ratio of the outer rectangular frame is equal to or approximately equal to 1, the outer rectangular frame is identified as a square rectangular frame, and the point cloud coordinates and normal vector direction corresponding to the coordinate index of the center point of the outer rectangular frame are extracted as the fruit space pose; otherwise, the outer rectangular frame is identified as a rectangular rectangular frame, and the single fruit symmetry axis is further constructed as the fruit space pose; when the outer rectangular frame is identified as a square rectangular frame, the extracted parameters are directly transmitted to the execution module;

[0046] The first processing module uses the Ransac algorithm to process the single fruit local smooth point cloud data to fit it into a sphere, takes the center of the sphere as the center of mass, and constructs a Cartesian coordinate system with the center of mass as the coordinate origin;

[0047] The second processing module traverses the single-fruit local smooth point cloud data, performs Delaunay triangulation on it, and constructs a Delaunay triangulation network from all triangles formed by iteration;

[0048] The third processing module traverses the Delaunay triangulation network, calculates the center of the circumscribed circle of each Delaunay triangle and records its spatial coordinates, sequentially connects the centers of the circumscribed circles of adjacent triangles, constructs a Voronoi edge set and generates a Voronoi diagram;

[0049] The fourth processing module traverses the Voronoi diagram, constructs a preliminary fitting arc using the circumcenter of the Delaunay triangle, and calculates the curvature radius of the preliminary fitting arc;

[0050] The fifth processing module selects the prepared fitting arc with the largest curvature radius as the selected fitting arc, and records the coordinates of the center of the selected fitting arc;

[0051] The execution module is used to receive data from the recognition and judgment module, obtain the fruit's spatial posture when the outer rectangular frame is identified as a square rectangular frame, and connect the selected fitting arc center and the center of mass of the fitting sphere to obtain the symmetry axis of the single fruit local smooth point cloud data, and use the symmetry axis of the single fruit local smooth point cloud data as the fruit's spatial posture.

[0052] In a third aspect, the present invention provides an electronic device, comprising:

[0053] at least one processor; and,

[0054] a memory communicatively connected to the at least one processor; wherein,

[0055] The memory stores computer program instructions that can be executed by the at least one processor, and the computer program instructions are executed by the at least one processor so that the at least one processor can execute the single-fruit spatial pose detection method based on local point cloud.

[0056] In a fourth aspect, the present invention provides a computer-readable storage medium storing a program, which, when executed by a processor, implements the single-fruit spatial pose detection method based on local point cloud.

[0057] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0058] (1) The present invention discloses a method for detecting the spatial pose of a single fruit based on a local point cloud. The method realizes the detection of the spatial pose of a single fruit target by adopting a classification pose replacement strategy. When the method is applied to the field of fruit picking, the spatial pose of the fruit can be accurately found based on the image of the single fruit taken in the natural scene. This overcomes the defect that the previous recognition of the single fruit cannot accurately locate the real-time spatial pose, facilitates the grasping of the robot arm, improves the efficiency of the automatic picking of the fruit, and reduces the damage to the fruit.

[0059] (2) The present invention constructs an outer rectangular frame based on the local smooth point cloud data of a single fruit. By capturing the image of a single fruit taken in a natural scene, the spatial posture of the fruit is detected by comparing the frame ratio, which reduces the dependence on the posture detection method of the convolutional neural network, avoids the large iterative error of the two-dimensional image deconvolution and the high point cloud accuracy requirements and long learning time required by directly using three-dimensional images. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0061] Figure 1 This is a flow chart of a method for detecting the spatial pose of a single fruit based on a local point cloud according to an embodiment of the present invention;

[0062] Figure 2 A schematic diagram of extracting a fitting sphere from point cloud data according to an embodiment of the present invention;

[0063] Figure 3 A schematic diagram of extracting the centroid of a point cloud by fitting a sphere according to an embodiment of the present invention;

[0064] Figure 4 A schematic diagram of extracting a symmetry axis from point cloud data according to an embodiment of the present invention;

[0065] Figure 5 Schematic diagram of the structure of a single fruit spatial pose detection system based on local point cloud according to an embodiment of the present invention;

[0066] Figure 6 2 is a structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0067] In order to enable those skilled in the art to better understand the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments in the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0068] References to "embodiments" in this application mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0069] See also Figure 1 The present embodiment provides a method for detecting the spatial pose of a single fruit based on a local point cloud, comprising the following steps:

[0070] S1. Collect and receive single-fruit local smooth point cloud data, and construct an outsourcing rectangular frame based on the single-fruit local smooth point cloud data; the outsourcing rectangular frame is used to determine the condition before the fruit spatial posture detection;

[0071] Optionally, the single fruit local smooth point cloud data includes the single fruit local smooth point cloud data in the database, the single fruit local smooth point cloud data collected in real time, and the data transmitted by other devices.

[0072] S2. Determine the shape of the outer rectangular frame. When the ratio of the adjacent sides of the outer rectangular frame is equal to or approximately equal to 1, the outer rectangular frame is identified as a square rectangular frame, and the point cloud coordinates and normal vector direction corresponding to the coordinate index of the center point of the outer rectangular frame are extracted as the fruit spatial pose; otherwise, the shape of the outer rectangular frame is identified as a rectangle, and the single fruit symmetry axis is further constructed as the fruit spatial pose;

[0073] Furthermore, constructing a single fruit symmetry axis specifically includes the following steps:

[0074] S21. Use the Ransac algorithm to process the single fruit local smooth point cloud data into a sphere, take the center of the sphere as the center of mass, and construct a Cartesian coordinate system with the center of mass as the coordinate origin, as shown in the following example: Figure 2 、 Figure 3 As shown;

[0075] Optionally, the sphere is fitted by 4 randomly selected point clouds from the surface of the single fruit local smooth point cloud data;

[0076] Optionally, the sphere is formed by fitting a plurality of randomly selected point clouds from the surface of a single local smooth point cloud data;

[0077] S22. traverse the single-fruit local smooth point cloud data, perform Delaunay triangulation on it, and construct a Delaunay triangulation network from all the triangles formed by iteration;

[0078] Preferably, the single local smooth point cloud data is traversed and Delaunay triangulated, and a triangle network is formed by all the iteratively constructed Delaunay triangles, such as Figure 4 As shown, specifically:

[0079] S221. Calculate the distance d between each point and the centroid, and find the point with the largest distance d max The farthest point a; take point a as the starting point, take n preliminary points near the set positive integer preset value k, and construct the point set V;

[0080] Optional, the value of the preparation point is {n|>10, n∈N +};

[0081] S222. Calculate the distance between each preliminary point and point a, find point b with the minimum distance, and then connect it with point a to form a baseline edge of the Delaunay triangle;

[0082] S223. Construct a Delaunay triangle with any preparatory point. If the triangle does not contain other point clouds, mark the preparatory point as c. Otherwise, search again.

[0083] S224. Use the line connecting b and c as the new baseline edge, iteratively construct a Delaunay triangle, and search for a preliminary point in the triangle that does not contain other point clouds. The iteration terminates when all points in the point set V are included in the Delaunay triangulation network.

[0084] S23. Traverse the Delaunay triangulation network, calculate the center of each Delaunay triangle's circumscribed circle and record its spatial coordinates, sequentially connect the centers of the circumscribed circles of adjacent triangles, construct a Voronoi edge set, and generate a Voronoi diagram;

[0085] S24. traverse the Voronoi diagram, construct a preliminary fitting arc using the circumcenter of the Delaunay triangle, and calculate the curvature radius of the preliminary fitting arc;

[0086] Preferably, the Delaunay triangle circumcenter is used to construct a preliminary fitting arc, and the curvature radius of the preliminary fitting arc is calculated, specifically,

[0087] S241. Calculate the distance between each point and the centroid, find the farthest point a with the largest distance, and record the coordinates of point a as (x1, y1, z1);

[0088] S242. Starting from point a, traverse the circumcenter of the Delaunay triangle, select m circumcenters near a preset positive integer value k as preliminary points, and construct a point set M;

[0089] Furthermore, the curvature radius R of the prepared fitting arc is characterized by being calculated as follows:

[0090] A1=y1·z2-y1·z3-z1·y2+z1·y3+y2·z3-y3·z2;

[0091] B1=-x1·z2+x1·z3+z1·x2-z1·x3-x2·z3+x3·z2;

[0092] C1=x1·y2-x1·y3-y1·x2+y1·x3+x2·y3-x3·y2;

[0093] D1=-x1·y2·z3+x1·y3·z2+x2·y1·z3-x3·y1·z2-x2·y3·z1+x3·y2·z1;

[0094] A2=2×(x2-x1);

[0095] B2=2×(y2-y1);

[0096] C2=2×(z2-z1);

[0097] D2=x1 2 +y1 2 +z1 2 -x2 2 -y2 2 -z2 2 ;

[0098] A3=2×(x3-x1);

[0099] B3=2×(y3-y1);

[0100] C3=2×(z3-z1);

[0101] D3=x1 2 +y1 2 +z1 2 -x3 2 -y3 2 -z3 2 ;

[0102]

[0103]

[0104] where x a ,y a ,z a To prepare the fitting arc center α′(x a ,y a ,z a ), A1, A2, A3, B1, B2, B3, C1, C2, and C3 represent the matrix parameters for calculating the coordinates of the fitting circle center.

[0105] S25. Select the prepared fitting arc with the largest curvature radius as the selected fitting arc, and record the coordinates of the center of the selected fitting arc;

[0106] Optionally, iterate n2-n times, select the preliminary fitting arc with the largest curvature radius as the selected fitting arc, and record the center of the fitting arc;

[0107] S26. Connect the center of the selected fitting arc and the center of mass of the fitting sphere to obtain the symmetry axis of the local smooth point cloud data of the single fruit, and use the symmetry axis of the local smooth point cloud data of the single fruit as the spatial pose of the fruit.

[0108] It should be noted that, for the sake of convenience, the aforementioned method embodiments are all expressed as a series of action combinations, but those skilled in the art should know that the present invention is not limited to the described order of actions, because according to the present invention, certain steps can be performed in other orders or simultaneously.

[0109] Based on the same idea as the single fruit spatial pose detection method based on local point cloud in the above-mentioned embodiment, the present invention also provides a single fruit spatial pose detection system based on local point cloud, which can be used to execute the above-mentioned single fruit spatial pose detection method based on local point cloud. For ease of explanation, the structural diagram of the embodiment of the single fruit spatial pose detection system based on local point cloud only shows the parts related to the embodiment of the present invention. Those skilled in the art will understand that the illustrated structure does not constitute a limitation of the device, and it may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0110] See also Figure 5In another embodiment of the present application, a method and system 10 for detecting a single fruit spatial pose based on a local point cloud is provided. The system includes an acquisition module 11, an identification and judgment module 12, a first processing module 13, a second processing module 14, a third processing module 15, a fourth processing module 16, a fifth processing module 17, and an execution module 18.

[0111] The acquisition module 11 is used to collect and receive local smooth point cloud data of a single fruit, and construct an outer rectangular frame based on the local smooth point cloud data of the single fruit;

[0112] The identification and judgment module 12 compares the outer rectangular frame with the data in the local smooth point cloud data of the single fruit, and judges the ratio of the side length of the outer rectangular frame: when the value of the adjacent side ratio of the outer rectangular frame is equal to or approximately equal to 1, the outer rectangular frame is identified as a square rectangular frame, and the point cloud coordinates and normal vector direction corresponding to the coordinate index of the center point of the outer rectangular frame are extracted as the fruit spatial pose; otherwise, the outer rectangular frame is identified as a rectangular rectangular frame, and the single fruit symmetry axis is further constructed as the fruit spatial pose; when the outer rectangular frame is identified as a square rectangular frame, the extracted parameters are directly transmitted to the execution module;

[0113] The first processing module 13 processes the single fruit local smooth point cloud data using the Ransac algorithm to fit it into a sphere, takes the center of the sphere as the center of mass, and constructs a Cartesian coordinate system with the center of mass as the coordinate origin;

[0114] The second processing module 14 traverses the single-fruit local smooth point cloud data, performs Delaunay triangulation on it, and constructs a Delaunay triangulation network from all the iteratively formed triangles;

[0115] The third processing module 15 traverses the Delaunay triangulation network, calculates the center of the circumscribed circle of each Delaunay triangle and records its spatial coordinates, sequentially connects the centers of the circumscribed circles of adjacent triangles, constructs a Voronoi edge set and generates a Voronoi diagram;

[0116] The fourth processing module 16 traverses the Voronoi diagram, constructs a preliminary fitting arc using the circumcenter of the Delaunay triangle, and calculates the curvature radius of the preliminary fitting arc;

[0117] The fifth processing module 17 selects the prepared fitting arc with the largest curvature radius as the selected fitting arc, and records the coordinates of the center of the selected fitting arc;

[0118] The execution module 18 is used to receive data from the recognition and judgment module, obtain the fruit's spatial posture when the outer rectangular frame is identified as a square rectangular frame, and connect the selected fitting arc center and the center of mass of the fitting sphere to obtain the symmetry axis of the single fruit local smooth point cloud data, and use the symmetry axis of the single fruit local smooth point cloud data as the fruit's spatial posture.

[0119] It should be noted that the single fruit spatial pose detection system based on local point cloud of the present invention corresponds one to one with the single fruit spatial pose detection method based on local point cloud of the present invention. The technical features and beneficial effects described in the above-mentioned embodiment of the single fruit spatial pose detection method based on local point cloud are applicable to the embodiment of the single fruit spatial pose detection method based on local point cloud. For specific contents, please refer to the description in the embodiment of the method of the present invention. No further details will be given here. This is hereby declared.

[0120] In addition, in the implementation of the single-fruit spatial pose detection system based on local point cloud in the above embodiment, the logical division of each program module is only an example. In actual application, the above functions can be assigned to different program modules as needed, for example, for the convenience of corresponding hardware configuration requirements or software implementation. That is, the internal structure of the single-fruit spatial pose detection system based on local point cloud is divided into different program modules to complete all or part of the functions described above.

[0121] See also Figure 6 In one embodiment, an electronic device for implementing a single fruit spatial pose detection method based on a local point cloud is provided. The electronic device 20 may include a first processor 21, a first memory 22 and a bus, and may also include a computer program stored in the first memory 22 and executable on the first processor 21, such as a single fruit spatial pose detection program 23 based on a local point cloud.

[0122] The first memory 22 includes at least one type of readable storage medium, including flash memory, a mobile hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the first memory 22 may be an internal storage unit of the electronic device 20, such as a mobile hard disk of the electronic device 20. In other embodiments, the first memory 22 may also be an external storage device of the electronic device 20, such as a plug-in mobile hard disk, a smart memory card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 20. Furthermore, the first memory 22 may include both an internal storage unit of the electronic device 20 and an external storage device. The first memory 22 can be used not only to store application software and various types of data installed in the electronic device 20, such as the code of the single-fruit spatial pose detection program 23 based on the local point cloud, but also to temporarily store data that has been output or is about to be output.

[0123] In some embodiments, the first processor 21 may be composed of an integrated circuit, such as a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and a combination of various control chips. The first processor 21 is the control core (Control Unit) of the electronic device, connecting the various components of the entire electronic device using various interfaces and lines, and executing or executing programs or modules stored in the first memory 22, as well as calling data stored in the first memory 22, to perform various functions of the electronic device 20 and process data.

[0124] Figure 6 Only the electronic device with components is shown, and it can be understood by those skilled in the art that Figure 5 The structure shown does not constitute a limitation on the electronic device 20 , and the electronic device 20 may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0125] The single-point-cloud-based spatial pose detection program 23 stored in the first memory 22 of the electronic device 20 is a combination of multiple instructions. When executed in the first processor 21, the following can be achieved:

[0126] Used to collect and receive local smooth point cloud data of a single fruit, and construct an outer rectangular frame based on the local smooth point cloud data of the single fruit;

[0127] The outer rectangular frame is compared with the data in the local smooth point cloud data of a single fruit, and the side length ratio of the outer rectangular frame is judged: when the adjacent side ratio value of the outer rectangular frame is equal to or approximately equal to 1, the outer rectangular frame is identified as a square rectangular frame, and the point cloud coordinates and normal vector direction corresponding to the coordinate index of the center point of the outer rectangular frame are extracted as the fruit spatial pose; otherwise, the outer rectangular frame is identified as a rectangular frame, and the single fruit symmetry axis is further constructed as the fruit spatial pose; when the outer rectangular frame is identified as a square rectangular frame, the extracted parameters are directly transmitted to the execution module;

[0128] The Ransac algorithm is used to process the local smooth point cloud data of a single fruit into a sphere, and the center of the sphere is used as the center of mass, and a Cartesian coordinate system is constructed with the center of mass as the coordinate origin.

[0129] Traverse the single-fruit local smooth point cloud data, perform Delaunay triangulation on it, and construct a Delaunay triangulation network from all the iteratively formed triangles;

[0130] Traverse the Delaunay triangulation network, calculate the center of each Delaunay triangle's circumcircle and record its spatial coordinates, connect the centers of the circumcircles of adjacent triangles in sequence, construct a Voronoi edge set and generate a Voronoi diagram;

[0131] Traverse the Voronoi diagram, use the Delaunay triangle circumcenter to construct a preliminary fitting arc, and calculate the curvature radius of the preliminary fitting arc;

[0132] Select the prepared fitting arc with the largest curvature radius as the selected fitting arc, and record the coordinates of the center of the selected fitting arc;

[0133] Receive data from the recognition and judgment module, obtain the fruit's spatial pose when the outer rectangular frame is identified as a square rectangular frame, and connect the selected fitting arc center and the center of mass of the fitting sphere to obtain the symmetry axis of the single fruit local smooth point cloud data, and use the symmetry axis of the single fruit local smooth point cloud data as the fruit's spatial pose.

[0134] Furthermore, if the modules / units integrated in the electronic device 20 are implemented as software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium. The computer-readable medium may include any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0135] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0136] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0137] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be considered as equivalent replacement methods and are included in the scope of protection of the present invention.

Claims

1. A single fruit spatial pose detection method based on local point cloud, characterized in that: The following steps are included: Collecting and receiving local smooth point cloud data of a single fruit, and constructing an outer rectangular frame for the local smooth point cloud data of the single fruit; the outer rectangular frame is used for condition judgment before the spatial posture detection of the fruit; Determine the shape of the outer rectangle. When the adjacent side ratio of the outer rectangle is equal to 1, the outer rectangle is identified as a square rectangle. Extract the point cloud coordinates and normal vector direction corresponding to the coordinate index of the center point of the square rectangle as the spatial pose of the fruit. Otherwise, the outer rectangular frame shape is identified as a rectangle, and a single fruit symmetry axis is further constructed as the fruit spatial posture; the further construction of the single fruit symmetry axis specifically includes the following steps: The Ransac algorithm is used to process the local smooth point cloud data of a single fruit into a sphere, and the center of the sphere is used as the center of mass, and a Cartesian coordinate system is constructed with the center of mass as the coordinate origin. Traverse the single-fruit local smooth point cloud data, perform Delaunay triangulation on it, and construct a Delaunay triangulation network from all the iteratively formed triangles; Traverse the Delaunay triangulation network, calculate the center of each Delaunay triangle's circumcircle and record its spatial coordinates, connect the centers of the circumcircles of adjacent triangles in sequence, construct a Voronoi edge set and generate a Voronoi diagram; Traverse the Voronoi diagram, use the Delaunay triangle circumcenter to construct a preliminary fitting arc, and calculate the curvature radius of the preliminary fitting arc; Select the prepared fitting arc with the largest curvature radius as the selected fitting arc, and record the coordinates of the center of the selected fitting arc; The symmetry axis of the local smooth point cloud data of a single fruit is obtained by connecting the center of the selected fitting arc and the center of mass of the fitting sphere, and the symmetry axis of the local smooth point cloud data of a single fruit is used as the spatial pose of the fruit.

2. The method for detecting the spatial pose of a single fruit based on a local point cloud according to claim 1, characterized in that: The method traverses the single-fruit local smooth point cloud data, performs Delaunay triangulation on it, and constructs a Delaunay triangulation network from all the triangles formed by iteration, specifically: Calculate the distance d between each point and the centroid, and find the point with the largest distance d max The farthest point a; take point a as the starting point, take n preliminary points near the set positive integer preset value k, and construct the point set V; Calculate the distance between each preliminary point in the point set V and point a, find the point b with the minimum distance, and then connect it with a to form a baseline edge of the Delaunay triangle; Construct a Delaunay triangle with any preparatory point. If the triangle does not contain other point clouds, record the preparatory point as c. Otherwise, search again. The line connecting b and c is used as the new baseline edge, and the Delaunay triangle is iteratively constructed and the preliminary points in the triangle that do not contain other point clouds are searched. The iteration terminates when all points in the point set V are included in the Delaunay triangulation network.

3. The method for detecting the spatial pose of a single fruit based on a local point cloud according to claim 1, wherein: The method of constructing a preliminary fitting arc using the circumcenter of the Delaunay triangle and calculating the curvature radius of the preliminary fitting arc is as follows: Calculate the distance between each point and the centroid, find the farthest point a with the largest distance, and record the coordinates of point a as (x1, y1, z1); Starting from point a, traverse the circumcenter of the Delaunay triangle, select m circumcenters near the preset positive integer value k as preliminary points, and construct the point set M; Randomly select two preliminary points from the point set M and record their coordinates as (x2, y2, z2) and (x3, y3, z3) respectively; Use these two points and a to construct preliminary fitting arcs Calculate the curvature radius R of the prepared fitting arc.

4. The method for detecting the spatial pose of a single fruit based on a local point cloud according to claim 3, characterized in that: The curvature radius R of the preparatory fitting arc is calculated as follows: A1=y1·z2-y1·z3-z1·y2+z1·y3+y2·z3-y3·z2; B1=-x1·z2+x1·z3+z1·x2-z1·x3-x2·z3+x3·z2; C1=x1·y2-x1·y3-y1·x2+y1·x3+x2·y3-x3·y2; D1=-x1·y2·z3+x1·y3·z2+x2·y1·z3-x3·y1·z2-x2·y3·z1+x3·y2·z1; A2=2×(x2-x1); B2=2×(y2-y1); C2=2×(z2-z1); <h2 style=";text-align:left;direction:ltr">D2 = x1<h2 style=";text-align:left;direction:ltr"> 2 <h2 style=";text-align:left;direction:ltr"> +y1<h2 style=";text-align:left;direction:ltr"> 2 <h2 style=";text-align:left;direction:ltr"> +z1<h2 style=";text-align:left;direction:ltr"> 2 <h2 style=";text-align:left;direction:ltr"> -x2<h2 style=";text-align:left;direction:ltr"> 2 <h2 style=";text-align:left;direction:ltr"> -y2<h2 style=";text-align:left;direction:ltr"> 2 <h2 style=";text-align:left;direction:ltr"> -z2<h2 style=";text-align:left;direction:ltr"> 2 <h2 style=";text-align:left;direction:ltr"> ; A3=2×(x3-x1); B3=2×(y3-y1); C3=2×(z3-z1); <h2 style=";text-align:left;direction:ltr">D3 = x1<h2 style=";text-align:left;direction:ltr"> 2 <h2 style=";text-align:left;direction:ltr"> +y1<h2 style=";text-align:left;direction:ltr"> 2 <h2 style=";text-align:left;direction:ltr"> +z1<h2 style=";text-align:left;direction:ltr"> 2 <h2 style=";text-align:left;direction:ltr"> -x3<h2 style=";text-align:left;direction:ltr"> 2 <h2 style=";text-align:left;direction:ltr"> -y3<h2 style=";text-align:left;direction:ltr"> 2 <h2 style=";text-align:left;direction:ltr"> -z3<h2 style=";text-align:left;direction:ltr"> 2 <h2 style=";text-align:left;direction:ltr"> ; where x a ,y a ,z a To prepare the fitting arc center α′(x a ,y a ,z a ), A1, A2, A3, B1, B2, B3, C1, C2, and C3 represent the matrix parameters for calculating the coordinates of the fitting circle center.

5. Single fruit spatial pose detection system based on local point cloud, characterized by: A single fruit spatial pose detection method based on a local point cloud applied to any one of claims 1-4, comprising an acquisition module, a recognition and judgment module, a first processing module, a second processing module, a third processing module, a fourth processing module, a fifth processing module, and an execution module; The acquisition module is used to collect and receive local smooth point cloud data of a single fruit, and construct an outer rectangular frame based on the local smooth point cloud data of the single fruit; The identification and judgment module compares the outer rectangular frame with the data in the local smooth point cloud data of the single fruit, and judges the ratio of the side length of the outer rectangular frame: when the value of the adjacent side ratio of the outer rectangular frame is equal to 1, the outer rectangular frame is identified as a square rectangular frame, and the point cloud coordinates and normal vector direction corresponding to the coordinate index of the center point of the outer rectangular frame are extracted as the fruit space pose; otherwise, the outer rectangular frame is identified as a rectangular rectangular frame, and the single fruit symmetry axis is further constructed as the fruit space pose; when the outer rectangular frame is identified as a square rectangular frame, the extracted parameters are directly transmitted to the execution module; The first processing module uses the Ransac algorithm to process the single fruit local smooth point cloud data to fit it into a sphere, takes the center of the sphere as the center of mass, and constructs a Cartesian coordinate system with the center of mass as the coordinate origin; The second processing module traverses the single-fruit local smooth point cloud data, performs Delaunay triangulation on it, and constructs a Delaunay triangulation network from all triangles formed by iteration; The third processing module traverses the Delaunay triangulation network, calculates the center of the circumscribed circle of each Delaunay triangle and records its spatial coordinates, sequentially connects the centers of the circumscribed circles of adjacent triangles, constructs a Voronoi edge set and generates a Voronoi diagram; The fourth processing module traverses the Voronoi diagram, constructs a preliminary fitting arc using the circumcenter of the Delaunay triangle, and calculates the curvature radius of the preliminary fitting arc; The fifth processing module selects the prepared fitting arc with the largest curvature radius as the selected fitting arc, and records the coordinates of the center of the selected fitting arc; The execution module is used to receive data from the recognition and judgment module, obtain the fruit's spatial posture when the outer rectangular frame is identified as a square rectangular frame, and connect the selected fitting arc center and the center of mass of the fitting sphere to obtain the symmetry axis of the single fruit local smooth point cloud data, and use the symmetry axis of the single fruit local smooth point cloud data as the fruit's spatial posture.

6. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores computer program instructions that can be executed by the at least one processor, and the computer program instructions are executed by the at least one processor so that the at least one processor can execute the single-fruit spatial pose detection method based on local point cloud as described in any one of claims 1-4.

7. A computer-readable storage medium storing a program, characterized in that: When the program is executed by the processor, the single fruit spatial pose detection method based on local point cloud described in any one of claims 1 to 4 is implemented.

Images