A 3D point cloud-based pig rib row cutting path planning method

Abstract

The application discloses a kind of pig rib row cutting path planning method based on 3D point cloud, belong to intelligent cutting and robot automation technical field.The high-precision original point cloud of pig carcass rib row area is acquired in real time by 3D vision sensor, then by introducing the adaptive growth factor based on local point cloud density, dynamic adjustment segmentation threshold, so that robust, accurate the point cloud after pre-processing is segmented into rib and rib meat area, and spine midline and intercostal space are positioned simultaneously.Based on this segmentation result, further through voxelization clustering and graph theory connectivity analysis, multiple independent rib meat connected components are automatically identified and separated.For each component, determine its main extension direction by constructing axis-aligned bounding box, and generate a series of discrete path points closely fitting the actual surface of point cloud along the direction.Use cubic polynomial curve to fit the space, generate smooth, continuous and highly fitted dissection path of anatomical structure, and directly converted into robot executable trajectory.

Description

Technical Field

[0001] This invention relates to the field of intelligent cutting and robotic automation technology, specifically to a method for planning the cutting path of pork ribs based on 3D point clouds. Background Technology

[0002] In the field of modern livestock and poultry slaughtering and processing, with population growth and consumption upgrading, the demand for refined meat processing is becoming increasingly urgent, driving the industry to transform from a traditional labor-intensive model to intelligent and automated processes. Currently, pork cutting still relies heavily on manual operation, which is not only labor-intensive and involves harsh working conditions, but also has low cutting efficiency, makes it difficult to guarantee product quality consistency, and results in poor controllability of meat hygiene and safety, seriously restricting industrial upgrading and large-scale development. Therefore, realizing intelligent cutting and processing to standardize production management, maintain product quality, reduce labor intensity, and improve production efficiency has become a core issue that the industry urgently needs to address.

[0003] However, existing intelligent segmentation technologies still have significant drawbacks. First, their perception and modeling are rigid: most technologies rely on pre-built standardized 3D models or require offline modeling using massive amounts of sample data. This is not only labor-intensive and costly, but also difficult to adapt to the natural differences in body shape, fatness, and skeletal morphology among individual pigs, resulting in poor generalization ability. In actual production, manual intervention is often required for correction, making true real-time adaptive processing impossible. Second, their segmentation strategies are rudimentary: existing methods are mostly based on simple image processing or fixed geometric rules for path planning, lacking a deep semantic understanding of complex biological anatomy. For example, they struggle to effectively distinguish the transition area between ribs and rib meat, failing to achieve the "soft segmentation" along natural physiological sutures required by dietary culture. This can easily lead to bone loss or excessive meat retention, affecting yield and appearance. Finally, there is a disconnect between planning and execution: the generated cutting paths are often too idealized or simplistic, poorly conforming to the actual 3D curved surface of the ribs. They fail to fully consider the smoothness of robot execution and obstacle avoidance requirements, resulting in cutting jitter, trajectory deviation, or low efficiency. These shortcomings collectively result in existing automated systems lacking precision, adaptability, and reliability, making it difficult to meet the high standards required for industrial applications. Therefore, there is an urgent need for a new intelligent cutting method capable of real-time sensing, precise understanding, and autonomous planning to fit individual shapes. Summary of the Invention

[0004] The purpose of this invention is to provide a method for planning the cutting path of pork ribs based on 3D point clouds, so as to solve the problems mentioned in the background art.

[0005] To address the aforementioned technical problems, this invention provides a method for planning the cutting path of pork ribs based on 3D point clouds, comprising:

[0006] S100. Obtain 3D point cloud data of the rib area of ​​the pig carcass: Use a 3D vision sensor to scan the rib area of ​​the pig carcass fixed on the workbench to obtain the original three-dimensional point cloud data including the spine, ribs, intercostal spaces and surface tissues.

[0007] S200. Preprocess the original 3D point cloud data: sequentially perform region of interest segmentation to remove the background, statistical filtering to remove outlier noise points, and voxel downsampling to uniformly simplify the point cloud to obtain the preprocessed point cloud.

[0008] The raw point cloud data, which is unprocessed, contains a lot of background and noise information, which can seriously interfere with the cutting path planning of pork ribs. Therefore, we first use region of interest segmentation to remove the background, then use statistical filtering to remove outliers, and finally use a voxel downsampling algorithm to reduce the amount of point cloud data.

[0009] Since the point cloud data of pork ribs acquired by the line laser scanner contains a large amount of irrelevant point cloud data, in order to remove the interference of irrelevant point cloud data on the semantic segmentation of ribs, a region of interest is set to remove the background point cloud. The region of interest is set according to the relative position of the object and the line laser scanner.

[0010] Set in 3D point cloud data , , The region of interest for the axis is defined, and a point cloud segmentation threshold is set. 3D point cloud data within the threshold range is retained, while background point cloud data is removed. The specific steps are as follows:

[0011] Setting the region of interest: using a six-element vector ,set up , , The region of interest for the axis.

[0012] Traverse the original 3D point cloud data and extract the point cloud indexes within the region of interest.

[0013] Remove 3D point cloud data outside the region of interest. After the above steps, relatively clean target point cloud data is obtained.

[0014] Outlier removal is achieved using statistical filtering. Based on the distribution of 3D point cloud data within the statistical filtering neighborhood and a statistical distance threshold, 3D point cloud data exceeding the statistical distance threshold are identified as outliers and removed. The specific formula is shown below:

[0015] ;

[0016] ;

[0017] ;

[0018] In the formula , , Representing points respectively 3D coordinates; , , Representing points respectively The three-dimensional coordinates of the neighboring points. Point The Euclidean distance between it and its neighboring points. Point The mean distance to neighboring points. Point The standard deviation of the distances to neighboring points. This represents the total number of points in the point cloud.

[0019] In 3D point cloud data processing, the original point cloud often suffers from problems such as excessive data volume and uneven local point cloud density due to factors such as the accuracy of the acquisition equipment and environmental interference, resulting in low efficiency of subsequent feature extraction and model inference.

[0020] Using voxel downsampling as an efficient downsampling technique, point cloud data is simplified and optimized by simulating the discretization process in three-dimensional space. The specific process is as follows:

[0021] S201, Define voxel resolution parameters Using the minimum coordinates of the point cloud bounding box as the origin, a three-dimensional raster coordinate system is constructed, and the entire point cloud space is uniformly divided into several grids with side lengths of... Cube unit.

[0022] S202, For any point in the point cloud Calculate the index of its corresponding voxel. To achieve the mapping between point clouds and voxels:

[0023] ;

[0024] In the formula, Point clouds are respectively Minimum coordinate value in the axial direction.

[0025] S203. Perform statistical processing on all points within each voxel and calculate the mean coordinates of all points within the voxel. As a representative point for smoothing local noise, the formula is as follows:

[0026] ;

[0027] In the formula, voxels The number of original points contained within. These are the coordinates of a single point within a voxel.

[0028] S204. Collect representative points of all non-empty voxels to form a downsampled point cloud set.

[0029] Not only can it reduce the amount of point cloud data to the original Furthermore, voxel division can ensure the uniformity of point cloud spatial distribution, laying the foundation for subsequent processing.

[0030] S300. Semantic segmentation and key anatomical structure extraction of preprocessed point cloud: An improved region growing adaptive semantic segmentation method is used to segment the preprocessed point cloud into rib region and rib muscle region.

[0031] Region growing algorithms were initially used primarily for segmentation of two-dimensional images. With the development of laser scanning and three-dimensional reconstruction technologies, they have begun to be used for segmentation and feature extraction of three-dimensional point clouds.

[0032] The core of traditional region growing algorithms is to start from a seed point, judge the similarity of neighboring points based on geometric features, gradually merge similar points into the same region, and achieve point cloud segmentation through iterative expansion.

[0033] However, the shape distribution characteristics of pork ribs, including the morphological differences between ribs and rib meat, the complex transition of the contact surface, non-uniform point cloud density, and local curvature changes, pose a great challenge to the segmentation effect of region growing algorithms.

[0034] These shape factors make it difficult for traditional region growth to stably and accurately divide ribs and rib meat, leading to problems such as blurred division boundaries, misjudgment, and breakage.

[0035] Therefore, when handling point cloud segmentation tasks with such complex shape distributions, it is necessary to combine richer features and more intelligent algorithm design to improve segmentation performance. Specifically, this includes:

[0036] S301. Calculate the eigenvalues ​​of each point in the preprocessed point cloud based on the eigenvalue analysis of the covariance matrix. The curvature is such that the point with the smallest curvature value is set as the initial seed point.

[0037] A kd-tree is a binary search tree structure used to organize k-dimensional spatial data. Its main function is to efficiently and quickly query data points in multi-dimensional space.

[0038] To establish topological relationships for point cloud data and improve the search efficiency and segmentation speed of point elements within the three-dimensional space of the research object, a k-dTree was used to establish topological relationships for the point cloud data of pork ribs.

[0039] In traditional region growing algorithms, geometric features such as the normal vector and curvature of each point in the target point cloud are first calculated, while outliers and noise are filtered out to ensure the stability of subsequent processing.

[0040] Secondly, an initial seed point is randomly selected and added to the current growth region. At the same time, its neighboring points are added to the candidate list. For each point in the candidate list, the k-nearest neighbor search is used to find its neighboring points.

[0041] Then, the candidate points are compared with the current region points in terms of features such as the angle between their normals and curvature differences. If a preset threshold is met, the point is considered to belong to the current region. Finally, points that meet the growth conditions are added to the current region, and their neighboring points are added to the candidate list.

[0042] Repeat the above operation until no new points meet the growth conditions. The current region growth is then complete, and the region result is saved. Simultaneously, randomly select new seed points from the unassigned points and continue repeating the above process for clustering one region until all points are assigned or the stopping condition is met.

[0043] This concludes the region growing process and outputs the clustering results for all regions. Compared to traditional region growing algorithms, the improved adaptive semantic segmentation method mainly refines the selection of initial seed points and the neighbor point similarity judgment criteria to adapt to semantic segmentation tasks involving point clouds with complex shapes, such as pork ribs.

[0044] Setting points and its neighborhood set The specific steps for calculating curvature are as follows:

[0045] Centroid of neighboring points :

[0046] ;

[0047] covariance matrix :

[0048] ;

[0049] Extracting eigenvalues ​​of the covariance matrix ,set up Calculation points curvature at :

[0050] ;

[0051] In the formula, Representative point The The coordinates of the neighboring points Represents the number of neighboring points. ...

[0052] S302, using the initial seed point Start growth and calculate the initial seed point. and neighboring points normal vector and Then, the angle between their normal vectors is calculated. :

[0053] ;

[0054] S303, Calculate neighborhood points Adaptive growth factor at the location and according to Dynamically adjust the threshold of the angle between the normal vectors and curvature threshold .

[0055] In the semantic segmentation of pork ribs, the normal vector angle and curvature of most rib regions are significantly different from those of the rib meat region. Therefore, the rib region and rib meat region of pork ribs can be divided by these two features.

[0056] However, some rib areas have smaller normal vector angles and curvatures, making it difficult to distinguish them from the rib meat based solely on these two characteristics.

[0057] Experimental studies revealed that the point cloud density in the rib region was significantly higher than that in the rib area. Based on this characteristic, an adaptive growth factor was introduced. This allows for dynamic adjustment of the normal vector angle threshold and curvature threshold for this region.

[0058] As the local density of the point cloud increases, growth factors Increasing the threshold for the angle between the normal vectors and the curvature threshold makes the similarity judgment condition between the seed point and the neighboring points more stringent, resulting in fewer neighboring points being assigned to the current growth region.

[0059] As the local density of the point cloud decreases, growth factors This reduces the thresholds for the angle between the normal vectors and the curvature, making the similarity judgment conditions between the seed point and its neighbors more lenient, resulting in more neighboring points being included in the current growth region.

[0060] The specific calculation steps are as follows:

[0061] Get Number of points in the neighborhood and space volume Calculate point cloud density :

[0062] ;

[0063] Preset minimum growth factor respectively and maximum growth factor Calculate growth factors :

[0064] ;

[0065] Dynamic normal vector angle threshold :

[0066] ;

[0067] Dynamic curvature threshold :

[0068] ;

[0069] In the formula, and These are the global minimum and maximum local point cloud densities of the entire point cloud, respectively. and These are the initial normal vector angle reference threshold and the initial curvature reference threshold, respectively.

[0070] S304. Determine the angle between normal vectors. Is it less than And point Is the curvature difference from the current seed point less than If so, then the point will be... Merge it into the current growth region and add it to the new seed point sequence.

[0071] S305. Repeat steps S302 to S304 until the current region is grown. Repeat step S301 from the unassigned points to select a new initial seed point and start the next region growth until all points are assigned, thus completing the semantic segmentation of the rib and rib meat regions.

[0072] Based on the segmentation results, the midline of the spine is located, and the intercostal spaces are identified along both sides of the spine. The center point sequence of the intercostal spaces on the left and right sides is extracted.

[0073] S400. Generating cutting paths based on semantic segmentation results: Extracting all point sets labeled as "rib meat" from the semantic segmentation results, and performing connected component segmentation on this point set to obtain multiple independent connected components for "rib meat". Specifically, this includes:

[0074] S401. Construct the target point set:

[0075] ;

[0076] In the formula, This represents the set of all three-dimensional points marked as ribs. Representing the Coordinates of a point in three-dimensional space. Three real coordinates The set of all points that constitute it. The tag represents the semantic meaning of rib meat.

[0077] Because the rib muscles within each intercostal space are spatially separated, they form independent strip-like structures. If the entire... Treating a path as a single area will cause the tool path to cross multiple gaps, resulting in incorrect cutting or invalid movement.

[0078] Therefore, it is necessary to Connectivity component segmentation is performed to identify each individual intercostal space region. Specifically, segmentation is achieved using voxel-based clustering combined with graph-based connectivity analysis.

[0079] S402, Collect the rib meat. Mapped to a regular voxel grid with preset side lengths. Each non-empty voxel is treated as a graph node.

[0080] S403. If two voxels share a surface in three-dimensional space, then an undirected edge is established between these two nodes to construct an undirected graph.

[0081] S404. Traverse the undirected graph using a depth-first search algorithm or a disjoint-set data structure algorithm, grouping the voxels corresponding to interconnected nodes into the same connected component. Finally, obtain... A spatially independent rib-connecting component:

[0082] ;

[0083] In the formula, This represents the number of individual intercostal spaces detected. Representing the The intercostal spaces correspond to the rib muscle connecting components, and satisfy the following conditions: .

[0084] Build an axis-aligned bounding box for each rib connected component and determine its main extension direction.

[0085] After obtaining the rib point cloud connectivity component Next, a local geometric reference frame needs to be established to determine the direction of the cutting path. Considering that the rib meat has a slender, arc-shaped structure, its main extension direction can be approximated by the longest axis of its spatial distribution.

[0086] Each rib connects to the component Construct axis-aligned bounding box Further calculate the lengths of the three sides of the bounding box, and take the direction corresponding to the longest side as the main extension direction of the rib region. The specific calculation formula is as follows:

[0087] ;

[0088] ;

[0089] In the formula, , Representing the rib meat connecting components All points in The set of coordinates on the axis; These represent the bounding box in The length of the direction.

[0090] Discrete path points are generated along the main extension direction to fit the point cloud surface of the rib meat connectivity component. Based on all discrete path points, the final pork rib cutting path is generated through curve fitting.

[0091] After determining the main extension direction of each rib region, a concise and effective cutting trajectory needs to be generated on its anatomical boundary. Considering the efficiency and stability requirements of pork rib processing, while also taking into account the sparsity and noise characteristics of point cloud data, a lightweight discrete path generation strategy is adopted:

[0092] For each individual connected rib component generated, perform the following operations to generate its cutting path:

[0093] First, determine the main extension direction of the connected component.

[0094] Then, along this main extension direction, within the axis-aligned bounding box of the connected component, U equidistant sampling positions are uniformly generated.

[0095] Next, at each sampling location, a virtual sampling plane perpendicular to the main extension direction is constructed.

[0096] Subsequently, the projection boundary or geometric center of the point cloud data of the rib connected component on the sampling plane is calculated, and the calculated points are used as the key discrete path points of the path segment.

[0097] Finally, a cubic polynomial curve is used to spatially fit all discrete path points, generating a smooth and continuous three-dimensional spatial curve as the final cutting path for the rib meat region.

[0098] The formula for curve fitting of a cubic polynomial is:

[0099] ;

[0100] in, ~ , ~ , ~ These are the polynomial coefficients obtained by fitting discrete path points. It is a path parameter, and satisfies =1, 2, ..., , The number of selected point cloud coordinates.

[0101] S500 Output and execute cutting path: Convert the generated cutting path into a trajectory format executable by the robot control system, and drive the cutting actuator to complete the separation of pork ribs.

[0102] The optimized path is sent to the robot control system in a standard format, driving the robotic arm to carry the cutting tool along the planned path to complete the separation of pork ribs.

[0103] Compared with the prior art, the beneficial effects achieved by the present invention are:

[0104] This invention achieves accurate adaptive semantic segmentation of complex biological tissues: Addressing the challenges of complex geometric features and blurred transition regions in the rib and rib meat regions of pork ribs, this invention abandons traditional fixed-threshold region growing algorithms and pioneers the introduction of an adaptive growth factor based on local point cloud density. This mechanism dynamically adjusts the angle between normal vectors and the curvature threshold, tightening merging conditions in densely populated rib regions to suppress overgrowth, and relaxing conditions in sparsely populated rib regions to ensure region integrity. This fundamentally solves the problems of inaccurate segmentation and blurred boundaries caused by uneven point cloud distribution and gradual feature changes, achieving robust and high-precision semantic segmentation without any prior models, laying a reliable foundation for core path planning.

[0105] This invention achieves fully adaptive individualized processing without calibration: it completely eliminates the reliance on standard templates or scaling factors. Through voxel clustering and graph connectivity analysis, the system can automatically identify and separate each independent rib-vein connected component (corresponding to each rib gap) directly from the point cloud of a single scan. The cutting path of each component is calculated entirely based on its own point cloud data, including the main extension direction, bounding box size, and surface geometry. This "scan-and-plan" mode gives the system a powerful ability to handle individual morphological differences, eliminating the need for manual parameter presets or adjustments, and truly achieving all-weather, fully adaptive intelligent operation.

[0106] A smooth cutting trajectory highly conforming to anatomical morphology was generated: the cutting path planned in this invention is not a simple straight line or a fixed curve, but is generated based on the actual three-dimensional shape of each rib muscle connecting component. By sampling along the main extension direction, key discrete points on the conforming point cloud surface are obtained, and then spatial fitting is performed using a cubic polynomial curve, ultimately generating a smooth, continuous cutting path in three-dimensional space that closely follows the natural curvature and undulations of the rib muscle. This path not only meets the process requirements of "soft segmentation" to the greatest extent, reducing damage to the bone and waste of meat, but its high-order continuity also ensures smooth and stable motion during robot execution, improving cutting quality and equipment lifespan.

[0107] An end-to-end fully automated intelligent cutting closed-loop system has been constructed: This invention integrates all key technologies from 3D visual perception, intelligent point cloud processing, semantic understanding, path planning to robot trajectory generation, forming a complete solution. The system input is only the raw 3D point cloud, and the output is the control command that can directly drive the actuator, without any human intervention throughout the entire process. This greatly improves production efficiency and operational consistency, reduces reliance on manpower and labor intensity, and provides solid technical support for realizing truly unmanned and intelligent operation in slaughtering and processing workshops. Attached Figure Description

[0108] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0109] Figure 1 This is a flowchart illustrating a method for planning the cutting path of pork ribs based on 3D point clouds according to the present invention.

[0110] Figure 2 This is a schematic diagram comparing the two region growth methods before and after the improvement in this invention. Detailed Implementation

[0111] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0112] Example 1: Please refer to Figure 1 and Figure 2 This invention provides a method for planning the cutting path of pork ribs based on 3D point clouds, including:

[0113] S100. Obtain 3D point cloud data of the rib area of ​​the pig carcass: Use a 3D vision sensor to scan the rib area of ​​the pig carcass fixed on the workbench to obtain the original three-dimensional point cloud data including the spine, ribs, intercostal spaces and surface tissues.

[0114] In the specific implementation process, a point cloud data acquisition platform for pork ribs was built. The acquisition platform mainly consists of a conveying device, a line laser scanner, an electrical control box, photoelectric switches, and a host computer.

[0115] Data acquisition was performed using Bsmart AR, the accompanying software for the line laser scanner. After multiple trials and adjustments, the position of the top line laser scanner was adjusted to be located on the central axis of the conveyor platform and 60cm above the conveyor belt, thus ensuring the acquisition of complete point cloud data of the pork ribs.

[0116] The line laser scanner and the host computer transmit point cloud data via Ethernet, saving the data in TXT format. Subsequent point cloud data processing uses software such as MATLAB 2023a (MathWorks) and the open-source point cloud processing software Cloud Compare (PCL point cloud library).

[0117] During the point cloud data acquisition process, the pork rib sample is first placed at the starting position of the conveyor platform and the conveyor device is started. The sample moves along the conveyor belt at a speed of 0.1 m / s. When the sample blocks the photoelectric switch, the line laser scanner performs a scanning operation to obtain the point cloud data of the sample's upper surface.

[0118] The scanning operation ends 2 seconds after the sample moves away from the photoelectric switch, and then the transport is stopped. The acquired point cloud data of pork ribs is viewed and saved using Bsmart AR software.

[0119] S200. Preprocess the original 3D point cloud data: sequentially perform region of interest segmentation to remove the background, statistical filtering to remove outlier noise points, and voxel downsampling to uniformly simplify the point cloud to obtain the preprocessed point cloud.

[0120] In the specific implementation process, the original point cloud is sequentially subjected to denoising, downsampling, normal vector estimation, and local plane fitting:

[0121] The Statistical Outlier Removal (SOR) algorithm is used to remove isolated noise points.

[0122] Use voxel grid filtering to uniformly downsample the point cloud to an appropriate density.

[0123] Calculate the normal vector of each point based on K-nearest neighbors (K=30).

[0124] A small plane is fitted within a local neighborhood for subsequent curvature and edge feature extraction.

[0125] The raw point cloud data, which is unprocessed, contains a lot of background and noise information, which can seriously interfere with the cutting path planning of pork ribs. Therefore, we first use region of interest segmentation to remove the background, then use statistical filtering to remove outliers, and finally use a voxel downsampling algorithm to reduce the amount of point cloud data.

[0126] Since the point cloud data of pork ribs acquired by the line laser scanner contains a large amount of irrelevant point cloud data, in order to remove the interference of irrelevant point cloud data on the semantic segmentation of ribs, a region of interest is set to remove the background point cloud. The region of interest is set according to the relative position of the object and the line laser scanner.

[0127] In the specific implementation process, settings are made in the 3D point cloud data. , , The region of interest for the axis is defined, and a point cloud segmentation threshold is set. 3D point cloud data within the threshold range is retained, while background point cloud data is removed. The specific steps are as follows:

[0128] Setting the region of interest: using a six-element vector ,set up , , The region of interest for the axis.

[0129] Traverse the original 3D point cloud data and extract the point cloud indexes within the region of interest.

[0130] Remove 3D point cloud data outside the region of interest. After the above steps, relatively clean target point cloud data is obtained.

[0131] Outlier removal is achieved using statistical filtering. Based on the distribution of 3D point cloud data within the statistical filtering neighborhood and a statistical distance threshold, 3D point cloud data exceeding the statistical distance threshold are identified as outliers and removed. The specific formula is shown below:

[0132] ;

[0133] ;

[0134] ;

[0135] In the formula , , Representing points respectively 3D coordinates; , , Representing points respectively The three-dimensional coordinates of the neighboring points. Point The Euclidean distance between it and its neighboring points. Point The mean distance to neighboring points. Point The standard deviation of the distances to neighboring points. This represents the total number of points in the point cloud.

[0136] In 3D point cloud data processing, the original point cloud often suffers from problems such as excessive data volume and uneven local point cloud density due to factors such as the accuracy of the acquisition equipment and environmental interference, resulting in low efficiency of subsequent feature extraction and model inference.

[0137] In the specific implementation process, voxel downsampling is used as an efficient downsampling technique. By simulating the discretization process in three-dimensional space, the point cloud data is simplified and optimized. The specific process is as follows:

[0138] S201, Define voxel resolution parameters Using the minimum coordinates of the point cloud bounding box as the origin, a three-dimensional raster coordinate system is constructed, and the entire point cloud space is uniformly divided into several grids with side lengths of... Cube unit.

[0139] S202, For any point in the point cloud Calculate the index of its corresponding voxel. To achieve the mapping between point clouds and voxels:

[0140] ;

[0141] In the formula, Point clouds are respectively Minimum coordinate value in the axial direction.

[0142] S203. Perform statistical processing on all points within each voxel and calculate the mean coordinates of all points within the voxel. As a representative point for smoothing local noise, the formula is as follows:

[0143] ;

[0144] In the formula, voxels The number of original points contained within. These are the coordinates of a single point within a voxel.

[0145] S204. Collect representative points of all non-empty voxels to form a downsampled point cloud set.

[0146] Not only can it reduce the amount of point cloud data to the original (In an ideal uniform point cloud scenario), it can also ensure the uniformity of the spatial distribution of the point cloud through voxel division, laying the foundation for subsequent processing.

[0147] S300. Semantic segmentation and key anatomical structure extraction of preprocessed point cloud: An improved region growing adaptive semantic segmentation method is used to segment the preprocessed point cloud into rib region and rib muscle region.

[0148] Region growing algorithms were initially used primarily for segmentation of two-dimensional images. With the development of laser scanning and three-dimensional reconstruction technologies, they have begun to be used for segmentation and feature extraction of three-dimensional point clouds.

[0149] The core of traditional region growing algorithms is to start from a seed point, judge the similarity of neighboring points based on geometric features (such as normals and curvature), gradually merge similar points into the same region, and achieve point cloud segmentation through iterative expansion.

[0150] However, the shape distribution characteristics of pork ribs, including the morphological differences between ribs and rib meat, the complex transition of the contact surface, non-uniform point cloud density, and local curvature changes, pose a great challenge to the segmentation effect of region growing algorithms.

[0151] These shape factors make it difficult for traditional region growth to stably and accurately divide ribs and rib meat, leading to problems such as blurred division boundaries, misjudgment, and breakage.

[0152] Therefore, when handling point cloud segmentation tasks with such complex shape distributions, it is necessary to combine richer features and more intelligent algorithm design to improve segmentation performance. Specifically, this includes:

[0153] S301. Calculate the eigenvalues ​​of each point in the preprocessed point cloud based on the eigenvalue analysis of the covariance matrix. The curvature is such that the point with the smallest curvature value is set as the initial seed point.

[0154] A kd-tree (k-dimensional tree) is a binary search tree structure used to organize k-dimensional spatial data. Its main function is to efficiently and quickly query data points in multidimensional space.

[0155] In the specific implementation process, in order to establish topological relationships on point cloud data and improve the search efficiency and segmentation speed of point elements in the three-dimensional space of the research object, k-dTree was used to establish topological relationships on the point cloud data of pork ribs.

[0156] In traditional region growing algorithms, geometric features such as the normal vector and curvature of each point in the target point cloud are first calculated, while outliers and noise are filtered out to ensure the stability of subsequent processing.

[0157] Secondly, an initial seed point is randomly selected and added to the current growth region. At the same time, its neighboring points are added to the candidate list. For each point in the candidate list, the k-nearest neighbor search is used to find its neighboring points.

[0158] Then, the candidate points are compared with the current region points in terms of features such as the angle between their normals and curvature differences. If a preset threshold is met, the point is considered to belong to the current region. Finally, points that meet the growth conditions are added to the current region, and their neighboring points are added to the candidate list.

[0159] Repeat the above operation until no new points meet the growth conditions. The current region growth is then complete, and the region result is saved. Simultaneously, randomly select new seed points from the unassigned points and continue repeating the above process for clustering one region until all points are assigned or the stopping condition is met.

[0160] This concludes the region growing process and outputs the clustering results for all regions. Compared to traditional region growing algorithms, the improved adaptive semantic segmentation method mainly refines the selection of initial seed points and the neighbor point similarity judgment criteria to adapt to semantic segmentation tasks involving point clouds with complex shapes, such as pork ribs.

[0161] Setting points and its neighborhood set The specific steps for calculating curvature are as follows:

[0162] Centroid of neighboring points :

[0163] ;

[0164] covariance matrix :

[0165] ;

[0166] Extracting eigenvalues ​​of the covariance matrix ,set up Calculation points curvature at :

[0167] ;

[0168] In the formula, Representative point The The coordinates of the neighboring points Represents the number of neighboring points. ...

[0169] S302, using the initial seed point Start growth and calculate the initial seed point. and neighboring points normal vector and Then, the angle between their normal vectors is calculated. :

[0170] ;

[0171] S303, Calculate neighborhood points Adaptive growth factor at the location and according to Dynamically adjust the threshold of the angle between the normal vectors and curvature threshold .

[0172] In the specific implementation process, during the semantic segmentation of pork ribs, the normal vector angle and curvature of most rib regions are significantly different from those of the rib meat region. Therefore, the rib region and rib meat region of pork ribs can be divided by these two features.

[0173] However, some rib areas have smaller normal vector angles and curvatures, making it difficult to distinguish them from the rib meat based solely on these two characteristics.

[0174] Experimental studies revealed that the point cloud density in the rib region was significantly higher than that in the rib area. Based on this characteristic, an adaptive growth factor was introduced. This allows for dynamic adjustment of the normal vector angle threshold and curvature threshold for this region.

[0175] As the local density of the point cloud increases, growth factors Increasing the threshold for the angle between the normal vectors and the curvature threshold makes the similarity judgment condition between the seed point and the neighboring points more stringent, resulting in fewer neighboring points being assigned to the current growth region.

[0176] As the local density of the point cloud decreases, growth factors This reduces the thresholds for the angle between the normal vectors and the curvature, making the similarity judgment conditions between the seed point and its neighbors more lenient, resulting in more neighboring points being included in the current growth region.

[0177] The specific calculation steps are as follows:

[0178] Get Number of points in the neighborhood and space volume Calculate point cloud density :

[0179] ;

[0180] Preset minimum growth factor respectively and maximum growth factor Calculate growth factors :

[0181] ;

[0182] Dynamic normal vector angle threshold :

[0183] ;

[0184] Dynamic curvature threshold :

[0185] ;

[0186] In the formula, and These are the global minimum and maximum local point cloud densities of the entire point cloud, respectively. and These are the initial normal vector angle reference threshold and the initial curvature reference threshold, respectively.

[0187] S304. Determine the angle between normal vectors. Is it less than And point Is the curvature difference from the current seed point less than If so, then the point will be... Merge it into the current growth region and add it to the new seed point sequence.

[0188] S305. Repeat steps S302 to S304 until the current region is grown. Repeat step S301 from the unassigned points to select a new initial seed point and start the next region growth until all points are assigned, thus completing the semantic segmentation of the rib and rib meat regions.

[0189] In the specific implementation process, based on the segmentation results, the midline of the spine is located, and the intercostal spaces are identified along both sides of the spine, and the center point sequence of the intercostal spaces on the left and right sides is extracted.

[0190] S400. Generating cutting paths based on semantic segmentation results: Extracting all point sets labeled as "rib meat" from the semantic segmentation results, and performing connected component segmentation on this point set to obtain multiple independent connected components for "rib meat". Specifically, this includes:

[0191] S401. Construct the target point set:

[0192] ;

[0193] In the formula, This represents the set of all three-dimensional points marked as ribs. Representing the Coordinates of a point in three-dimensional space. Three real coordinates The set of all points that constitute it. The tag represents the semantic meaning of rib meat.

[0194] In practice, because the rib muscles within each intercostal space are spatially separated, they form independent strip-like structures. If the entire... Treating a path as a single area will cause the tool path to cross multiple gaps, resulting in incorrect cutting or invalid movement.

[0195] Therefore, it is necessary to Connectivity component segmentation is performed to identify each individual intercostal space region. Specifically, segmentation is achieved using voxel-based clustering combined with graph-based connectivity analysis.

[0196] S402, Collect the rib meat. Mapped to a regular voxel grid with preset side lengths. Each non-empty voxel is treated as a graph node.

[0197] S403. If two voxels share a surface in three-dimensional space, then an undirected edge is established between these two nodes to construct an undirected graph.

[0198] S404. Traverse the undirected graph using a depth-first search algorithm or a disjoint-set data structure algorithm, grouping the voxels corresponding to interconnected nodes into the same connected component. Finally, obtain... A spatially independent rib-connecting component:

[0199] ;

[0200] In the formula, This represents the number of individual intercostal spaces detected. Representing the The intercostal spaces correspond to the rib muscle connecting components, and satisfy the following conditions: .

[0201] Build an axis-aligned bounding box for each rib connected component and determine its main extension direction.

[0202] After obtaining the rib point cloud connectivity component Next, a local geometric reference frame needs to be established to determine the direction of the cutting path. Considering that the rib meat has a slender, arc-shaped structure, its main extension direction can be approximated by the longest axis of its spatial distribution.

[0203] Each rib connects to the component Construct axis-aligned bounding box Further calculate the lengths of the three sides of the bounding box, and take the direction corresponding to the longest side as the main extension direction of the rib region. The specific calculation formula is as follows:

[0204] ;

[0205] ;

[0206] In the formula, , Representing the rib meat connecting components All points in The set of coordinates on the axis; These represent the bounding box in The length of the direction.

[0207] In the specific implementation process, discrete path points are generated along the main extension direction to fit the point cloud surface of the rib meat connectivity component. Based on all discrete path points, the final pork rib cutting path is generated through curve fitting.

[0208] After determining the main extension direction of each rib region, a concise and effective cutting trajectory needs to be generated on its anatomical boundary. Considering the efficiency and stability requirements of pork rib processing, while also taking into account the sparsity and noise characteristics of point cloud data, a lightweight discrete path generation strategy is adopted:

[0209] For each individual connected rib component generated, perform the following operations to generate its cutting path:

[0210] First, determine the main extension direction of the connected component.

[0211] Then, along this main extension direction, within the axis-aligned bounding box of the connected component, U equidistant sampling positions are uniformly generated.

[0212] Next, at each sampling location, a virtual sampling plane perpendicular to the main extension direction is constructed.

[0213] Subsequently, the projection boundary or geometric center of the point cloud data of the rib connected component on the sampling plane is calculated, and the calculated points are used as the key discrete path points of the path segment.

[0214] Finally, a cubic polynomial curve is used to spatially fit all discrete path points, generating a smooth and continuous three-dimensional spatial curve as the final cutting path for the rib meat region.

[0215] The formula for curve fitting of a cubic polynomial is:

[0216] ;

[0217] in, ~ , ~ , ~ These are the polynomial coefficients obtained by fitting discrete path points. It is a path parameter, and satisfies =1, 2, ..., , The number of selected point cloud coordinates.

[0218] S500 Output and execute cutting path: Convert the generated cutting path into a trajectory format executable by the robot control system, and drive the cutting actuator to complete the separation of pork ribs.

[0219] The optimized path is sent to the robot control system in a standard format (such as ROS's trajectory_msgs / JointTrajectory or URScript), driving the robotic arm to carry the cutting tool along the planned path to complete the separation of pork ribs.

[0220] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0221] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for planning the cutting path of pork ribs based on 3D point clouds, characterized in that: The method includes: S100. Obtain 3D point cloud data of the rib area of ​​the pig carcass: Use a 3D vision sensor to scan the rib area of ​​the pig carcass fixed on the workbench to obtain the original three-dimensional point cloud data including the spine, ribs, intercostal spaces and surface tissues. S200. Preprocess the original three-dimensional point cloud data: sequentially perform region of interest segmentation to remove the background, statistical filtering to remove outlier noise points, and voxel downsampling to uniformly simplify the point cloud to obtain a preprocessed point cloud. S300. Semantic segmentation and key anatomical structure extraction of the preprocessed point cloud: An improved region growing adaptive semantic segmentation method is used to segment the preprocessed point cloud into rib region and rib muscle region; based on the segmentation results, the midline of the spine is located, and the intercostal spaces are identified along both sides of the spine, and the center point sequence of the intercostal spaces on the left and right sides is extracted. The improved region-growing adaptive semantic segmentation method includes: S301. Calculate the eigenvalues ​​of each point in the preprocessed point cloud based on the eigenvalue analysis of the covariance matrix. The curvature is such that the point with the smallest curvature value is set as the initial seed point; let point... and its neighborhood set The steps for calculating curvature are as follows: Centroid of neighboring points : ； covariance matrix : ； Extracting eigenvalues ​​of the covariance matrix ,set up Calculation points curvature at : ； In the formula, Representative point The The coordinates of the neighboring points Represents the number of neighboring points. These represent the minimum, middle, and maximum eigenvalues ​​of the covariance matrix, respectively. S302, using the initial seed point Start growth and calculate the initial seed point. and neighboring points normal vectors F and F i Then, the angle between their normal vectors is calculated. : ； S303, Calculate neighborhood points Adaptive growth factor at the location and according to Dynamically adjust the threshold of the angle between the normal vectors and curvature threshold ; The calculation steps for the dynamic normal vector angle threshold and the dynamic curvature threshold are as follows: Get Number of points in the neighborhood and space volume Calculate point cloud density : ； Preset minimum growth factor respectively and maximum growth factor Calculate growth factors : ； Dynamic normal vector angle threshold : ； Dynamic curvature threshold : ； In the formula, and These are the global minimum and maximum local point cloud densities of the entire point cloud, respectively. and These are the initial normal vector angle reference threshold and the initial curvature reference threshold, respectively; S304. Determine the angle between the normal vectors. Is it less than And point Is the curvature difference from the current seed point less than If so, then the point will be... Merge it into the current growth region and add it to the new seed point sequence; S305. Repeat steps S302 to S304 until the current region is grown. Repeat step S301 from the unassigned points to select a new initial seed point and start the next region growth until all points are assigned, thus completing the semantic segmentation of the rib and rib meat regions. S400. Generate cutting path based on semantic segmentation results: Extract all point sets marked as rib meat from the semantic segmentation results, perform connected component segmentation on the point set to obtain multiple independent rib meat connected components; construct an axis-aligned bounding box for each rib meat connected component and determine its main extension direction; generate discrete path points that fit the point cloud surface of the rib meat connected component along the main extension direction; generate the final pork rib cutting path based on all discrete path points through curve fitting. S500, Output and execute the cutting path: Convert the generated cutting path into a trajectory format executable by the robot control system, and drive the cutting actuator to complete the separation of pork ribs.

2. The method for planning the cutting path of pork ribs based on 3D point clouds according to claim 1, characterized in that: In S200, the voxel downsampling specifically includes: S201, Define voxel resolution parameters Using the minimum coordinates of the point cloud bounding box as the origin, a three-dimensional raster coordinate system is constructed, and the entire point cloud space is uniformly divided into several grids with side lengths of... Cube unit; S202, For any point in the point cloud Calculate the index of its corresponding voxel. To achieve the mapping between point clouds and voxels: ； In the formula, Point clouds are respectively Minimum coordinate value in the axial direction; S203. Perform statistical processing on all points within each voxel and calculate the mean coordinates of all points within the voxel. As a representative point for smoothing local noise, the formula is as follows: ； In the formula, voxels The number of original points contained within. The coordinates of a single point within a voxel; S204. Collect representative points of all non-empty voxels to form a downsampled point cloud set.

3. The method for planning the cutting path of pork ribs based on 3D point clouds according to claim 1, characterized in that: The S400 includes: S401. Extract all points marked as "rib meat" from the semantic segmentation results to form the target point set: ； In the formula, This represents the set of all three-dimensional points marked as ribs. Representing the Coordinates of a point in three-dimensional space. Three real coordinates The set of all points that constitute it. The tag represents the semantic meaning of rib meat; S402, Collect the rib meat. Map to a regular voxel grid with a preset side length; treat each non-empty voxel as a graph node; S403. If two voxels share a surface in three-dimensional space, then an undirected edge is established between these two nodes to construct an undirected graph. S404. Traverse the undirected graph using a depth-first search algorithm or a disjoint-set data structure algorithm, grouping the voxels corresponding to interconnected nodes into the same connected component; finally, obtain... A spatially independent rib-connecting component: ； In the formula, This represents the number of individual intercostal spaces detected. Representing the The intercostal spaces correspond to the rib muscle connecting components, and satisfy the following conditions: .

4. The method for planning the cutting path of pork ribs based on 3D point clouds according to claim 3, characterized in that: Connecting components for each rib Construct axis-aligned bounding box Further calculate the lengths of the three sides of the bounding box, and take the direction corresponding to the longest side as the main extension direction of the rib region; the specific calculation formula is as follows: ； ； In the formula, , Representing the rib meat connecting components All points in The set of coordinates on the axis; These represent the bounding box in The length of the direction.

5. The method for planning the cutting path of pork ribs based on 3D point clouds according to claim 4, characterized in that: For each individual connected rib component generated, perform the following operations to generate its cutting path: First, determine the main extension direction of the connected component; Then, along this main extension direction, within the axis-aligned bounding box of the connected component, U equidistant sampling positions are uniformly generated; Next, at each sampling location, a virtual sampling plane perpendicular to the main extension direction is constructed; Subsequently, the projection boundary or geometric center of the point cloud data of the rib connected component on the sampling plane is calculated, and the calculated points are used as the key discrete path points of the path segment. Finally, a cubic polynomial curve is used to spatially fit all discrete path points to generate a smooth and continuous three-dimensional spatial curve, which serves as the final cutting path for the rib meat region. The formula for fitting the cubic polynomial curve is: ； in, ~ , ~ , ~ These are the polynomial coefficients obtained by fitting discrete path points. It is a path parameter, and satisfies =1, 2, ..., , The number of selected point cloud coordinates.

Images