Livestock point cloud segmentation denoising method based on adaptive parameter Hough transformation

Abstract

The invention discloses a livestock point cloud segmentation denoising method based on adaptive parameter Hough transformation. The method comprises the following steps: acquiring livestock point cloud data; preprocessing the point cloud data to obtain two pieces of point cloud data containing livestock and handrails on one side, and performing feature extraction on the point cloud data to obtain a point cloud feature vector; inputting the point cloud feature vector into a Hough transformation parameter adaptive model, and performing adaptive Hough transformation railing detection to obtain a potential railing straight line; de-noising and segmenting the livestock point cloud by taking the handrail straight line as a reference to obtain a unilateral livestock point cloud; and finally, carrying out point cloud registration, and outputting complete point cloud of the livestock. According to the method, a parameter adaptive model based on a small amount of annotated data is constructed, the problem of annotated data scarcity is solved by adopting transfer learning and small sample learning technologies, nonlinear mapping from a point cloud feature space to a Hough parameter space is established, and adaptive parameter adjustment of different scenes is adapted.

Description

Technical Field

[0001] This invention relates to image processing technology, specifically to a method for livestock point cloud segmentation and denoising based on adaptive parameter Hough transform. Background Technology

[0002] With the development of 3D technology, point cloud data is being used in intelligent agricultural applications, such as non-contact measurement of livestock body size, live animal monitoring, health assessment, and behavioral analysis. Livestock farm environments are complex, and the acquired livestock point clouds contain complex backgrounds, passageway fences, and other noise. While conditional filtering and clustering algorithms can remove complex backgrounds and noise points far from the livestock being measured, they cannot remove interference from fences directly adjacent to the livestock. Given the complexity of livestock farm environments and the ubiquitous presence of fences, removing noise from fences and other sources directly impacts target segmentation accuracy in livestock point cloud data processing. Existing point cloud processing methods mainly include those based on traditional geometric features and those based on deep learning.

[0003] Traditional geometric feature-based methods include Euclidean clustering, RANSAC (Random Sample Consensus), and region growing. While these methods can filter out most noise, they cannot effectively filter out fences that are close to livestock without affecting the target point cloud, and they require manual parameter tuning and cannot adapt to different environments.

[0004] Deep learning-based methods include: PointNet / PointNet++, which directly processes point clouds, but has weak local feature extraction capabilities and struggles to segment thin, elongated railings; 3DCNN (voxelation method), which converts point clouds into meshes, but suffers from resolution loss leading to broken railings; and Graph Convolutional Networks (GCN), which have high computational complexity and are difficult to process large-scale point clouds in real time. All deep learning-based methods require large amounts of labeled data, and currently, livestock point cloud data collection is difficult, with the available data volume insufficient to meet these requirements.

[0005] To address the problem that existing point cloud segmentation algorithms struggle to effectively filter out interfering objects such as linear fences adjacent to livestock without affecting the target point cloud, a joint KD-Tree and Hough transform-driven algorithm can efficiently segment livestock point clouds and filter out noise.

[0006] To address the issue that manual parameter tuning is insufficient for detecting different railings, a parameter adaptive model based on a limited amount of labeled data is constructed. Transfer learning and few-shot learning techniques are used to solve the problem of scarce labeled data, and a nonlinear mapping from the point cloud feature space to the Hough parameter space is established to adapt to adaptive parameter adjustments for different scenarios. Summary of the Invention

[0007] Purpose of the Invention: To address the problem that existing point cloud segmentation algorithms struggle to effectively filter out interfering objects such as linear fences adjacent to livestock without affecting the target point cloud, this invention provides a livestock point cloud segmentation and denoising method based on adaptive parameter Hough transform. Utilizing a joint KD-Tree and Hough transform-driven algorithm, it can efficiently segment livestock point clouds and filter out noise. To address the difficulty of manually adjusting parameters to meet the detection requirements of different fences, a parameter adaptive model based on a limited amount of labeled data is constructed. Transfer learning and few-shot learning techniques are employed to solve the problem of scarce labeled data, establishing a nonlinear mapping from the point cloud feature space to the Hough parameter space, adapting to adaptive parameter adjustments for different scenarios.

[0008] Technical solution: The livestock point cloud segmentation and denoising method based on adaptive parameter Hough transform of the present invention includes the following steps:

[0009] Step 1: Obtain livestock point cloud data;

[0010] First, herd the livestock into the guard pen and ensure that they are positioned on the aisle scale. Then, use a FemtoBolt depth camera to collect point cloud data of the livestock from both sides of the guard pen. (The following steps can be completed by merging the point clouds from both sides through point cloud registration. Considering that there are fences on both sides, it is more difficult to detect after registration. Therefore, the data from both sides can be detected and segmented separately before point cloud registration.)

[0011] Step 2: Preprocess the point cloud data obtained in Step 1. First, use outlier removal to obtain the point cloud of livestock closely attached to the fence. Then, use voxel mesh downsampling to uniformly downsample the data. Finally, output two point cloud datasets containing livestock and fences on one side. ;

[0012] Step 3: Extract features from the preprocessed point cloud data, i.e., input the point cloud of livestock with fences. For the same livestock point cloud P with fences, its corresponding spatial distribution features, geometric shape features, density statistics features, and color distribution features are obtained sequentially. Then, the spatial distribution features, geometric shape features, density statistics features, and color distribution features are concatenated and normalized to finally output a point cloud feature vector that comprehensively describes its characteristics. ; This refers to the total number of points in the same point cloud;

[0013] Step 4: Convert the point cloud feature vector The pre-constructed adaptive model of Hough transform parameters is input, and a mapping function is learned. The above Indicates the Hough transform parameters; This refers to distance resolution, θ refers to angular resolution, and threshold refers to the threshold value. The specific method is as follows:

[0014] First, construct a small sample support set. Then use a feature encoder Point cloud feature vectors Mapped to a discriminative low-dimensional embedding space; then the query point cloud is computed. Embedded To obtain the prediction parameters ; predict parameters The Hough transform parameters are input to the detection railing; In this context, the subscript K represents the number of samples, i.e. how many point clouds are labeled, P represents one of the point clouds, and N represents the number of points in a point cloud.

[0015] Step 5: Perform adaptive Hough transform for railing detection, which involves projecting the 3D point cloud of the railing onto a 2D plane to obtain a 2D image. Then, use the Canny operator to perform edge detection on the 2D image and combine it with the predicted parameters obtained in Step 4 to detect potential railing lines.

[0016] Step 6: Using the fence line obtained from the Hough transform fence detection in Step 5 as a reference, perform KD-Tree denoising and segmentation on the livestock point cloud to obtain a single-sided livestock point cloud.

[0017] Step 7: Perform point cloud registration and output the complete point cloud of livestock.

[0018] Furthermore, the detailed method for preprocessing the obtained point cloud data in step 2 is as follows:

[0019] Step 2.1: Remove outliers. First, using the distance between the depth camera and the guardrail, set point cloud filtering conditions to filter out the scene point clouds that are far from the livestock outside the guardrail, retaining the point clouds of the guardrail, the aisle scale, and the livestock. Then, using the point cloud density difference and distance difference, use the DBSCAN clustering algorithm to extract the point cloud with the largest cluster, remove small clusters and some noise, and segment and extract the point clouds of the livestock with higher density and the adjacent guardrail and aisle scale. Finally, use the RANSAC algorithm for planar detection to filter out the aisle scale point cloud and segment and extract the point clouds of the livestock and the point clouds adjacent to the guardrail.

[0020] Step 2.2, Voxel Mesh Downsampling

[0021] VoxelGrid is used to uniformly downsample the point cloud of livestock with fences, preserving geometric features while reducing data volume.

[0022] Furthermore, in step 3, the feature vector The methods for obtaining spatial distribution characteristics, geometric shape characteristics, density statistical characteristics, and color distribution characteristics are as follows:

[0023] For spatial distribution characteristics, a point cloud of livestock with fences is defined. The size ratio of the point cloud 3D spatial bounding box The point cloud centroid coordinates (cx, cy, cz) and the eigenvalues ​​(λ1, λ2, λ3) of the point cloud covariance matrix are used to describe the extensibility and principal direction of the point cloud.

[0024] Bounding box size ratio The calculation formula is as follows:

[0025] ; ; ;

[0026] , , The length, width, and height of the 3D bounding box;

[0027] The centroid coordinates c(cx, cy, cz) of the point cloud are obtained by taking the average of the three-dimensional coordinates of all points.

[0028] ;

[0029] The eigenvalues ​​(λ1, λ2, λ3) of the covariance matrix are calculated by describing the principal orientation and extensibility of the point cloud in descending order:

[0030] ;

[0031] p i Let be the three-dimensional coordinates of the i-th point; eigenvalue decomposition yields λ1≥λ2≥λ3;

[0032] Geometric features include linearity Flatness and scattering ;

[0033] linearity Flatness Scattering ;

[0034] The density statistical features are obtained by calculating the overall point density D and the average distance from the point cloud to the centroid. and standard deviation ;

[0035] ;

[0036] ;

[0037] ;

[0038] The method for obtaining color distribution features is as follows: use the K-means clustering method to group similar colors into one category, find the main colors and their proportions, and use them to determine the color distribution features of the scene.

[0039] Extract dominant color using K-means clustering: ;

[0040] Where K represents the number of clusters, i.e. the number of color categories. Let i be the RGB color value of the i-th point in the point cloud. This represents the k-th cluster center. Let k represent the set of points belonging to the k-th class. Indicates color similarity.

[0041] Furthermore, the detailed process of step 4 is as follows:

[0042] Step 4.1: Construct a support set. Using self-collected livestock point cloud data from the farm, manually label 50 point cloud data blocks (fences, livestock) and their optimal Hough parameters, and expand the support set by data augmentation techniques such as random rotation, scaling, and adding noise.

[0043] Constructing a small support set of 50 samples ;

[0044] Step 4.2: Feature embedding network. Train a feature encoder E using a multilayer perceptron (MLP). ϕ The point cloud features F are mapped to a low-dimensional, discriminative embedding space. The training objective is to make point cloud features of the same scene type close in distance in the embedding space, while features of different scenes are far apart.

[0045] Step 4.3, Parameter Prediction; For a query point cloud P q Calculate its embedding ;

[0046] In the embedding space, find the support set and z q The most similar M samples (measured by cosine similarity) are used to calculate the weighted average of their corresponding optimal parameters, which yields the prediction parameters. :

[0047] , ;

[0048] Where sim() is the cosine similarity and τ is the temperature coefficient, which is a hyperparameter used to control the "sharpness" or "smoothness" of the weight distribution when performing weighted averaging. It refers to the feature representation of the query sample (new point cloud) in the embedding space; This refers to the feature representation of the k-th labeled sample in the embedding space; This refers to the feature representation of the j-th labeled sample in the embedding space.

[0049] Prediction parameters The detection bar serves as the parameter input for the Hough transform. In this invention, the core parameters of the Hough transform are distance resolution (ρ), angle resolution (θ), and threshold, which respectively represent the distance step size, angle step size, and the vote threshold for determining whether a line is a valid line in the accumulator.

[0050] Further, step 5 uses the Canny operator to perform edge detection on the image; performs Hough transform to detect lines lines=cv2.HoughLines(edges,ρ,θ,threshold), detecting potential railing lines; and adds spatial constraints to avoid detecting non-railing lines.

[0051] Step 5.1: Considering that the railings are mostly horizontally distributed, use an angle threshold to filter out horizontally distributed straight lines, setting the angle threshold to 15 degrees. o ;

[0052] Horizontal vector along the x-axis Let the unit direction vector of the detected line be... Calculate the angle θ between the x-axis and the horizontal vector, where θ ≤ 15°. o ,Right now ;

[0053] in, Straight lines that meet the conditions are retained;

[0054] Step 5.2: Check that the straight line meets the minimum length of the railing.

[0055] The horizontal length of the guardrail in Baoding is 2 meters. Considering factors such as point cloud loss, the guardrail length is set to be greater than 0.5 meters.

[0056] The coordinates of the starting and ending points of the detected line are (x1, y1, z1) and (x2, y2, z2), respectively.

[0057] straight-line distance Lines that satisfy L≥0.5 are retained.

[0058] Furthermore, in step 6, when performing KD-Tree denoising and segmentation on the livestock point cloud, the fence line obtained by Hough transform is used as a reference. A distance threshold of 1.5 times the fence diameter is set. All points are traversed, and the distance from each point to the nearest fence line is quickly queried using KD-Tree. Points with a distance less than the threshold are classified as "fence point cloud", and those with a distance greater than the threshold are classified as "livestock point cloud". The segmentation is completed, and the final pure one-sided livestock point cloud is retained.

[0059] Furthermore, the specific method for point cloud registration in step 7 is as follows:

[0060] Because the coordinate systems of the two cameras are different, a registration method is needed to obtain the transformation matrix between the different coordinate systems, transform the collected data from different coordinate systems to the same coordinate system, and obtain a complete point cloud.

[0061] The PnP-RANSAC algorithm can calculate the camera's rotation matrix and translation vector using known camera intrinsic parameters and several feature points extracted from the image, thereby determining the camera's position and orientation in the world coordinate system, and is used to solve the camera pose estimation problem. Since the poses of two cameras differ significantly, such as... Figure 12 As shown, automatic feature point matching suffers from significant errors. This paper proposes a calibration method combining manually labeled feature points with the PnP-RANSAC algorithm. First, four pairs of two-dimensional feature points are manually labeled on the two-dimensional color images from the two viewpoints to be matched. The coordinates of these four pairs of points define a three-dimensional space. Then, using camera parameters and the corresponding depth map, the three-dimensional coordinates corresponding to the feature points from the first viewpoint are calculated. The three-dimensional feature point coordinates from the first viewpoint and the two-dimensional feature point coordinates from the second viewpoint are used as input values ​​for the PnP-RANSAC algorithm to calculate the transformation matrix between the two viewpoint coordinate systems. (See figure...) Figure 13 Each pair of feature points shown can be correctly matched. The transformation matrix calculated using this method can be used to register and fuse the point cloud images from the left and right views of the livestock to obtain the complete point cloud of the livestock.

[0062] Beneficial Effects: This invention combines the robustness of traditional algorithms with the adaptability of data-driven approaches to solve the complex, variable, and data-scarce practical application scenario of livestock point cloud segmentation. Compared with existing methods, it can quickly filter out noise with regular shapes such as fences without affecting the integrity of the livestock point cloud, segment and extract the livestock point cloud, consumes less energy, is more efficient, and eliminates the need for tedious work such as extensive annotation, making the operation simpler. Compared with the prior art, this invention has the following advantages:

[0063] (1) More specific adaptability: This invention combines machine learning to establish the relationship between point cloud features and Hough transform parameters, thus making up for the limitation of Hough transform relying on manual parameter tuning.

[0064] (2) Stronger noise and occlusion resistance: Considering that livestock (such as cattle and pigs) will severely occlude fences in the point cloud, and that the livestock's own outlines, ears, legs, and other parts will generate a large number of messy noise points, the Hough transform mechanism used in this invention makes it insensitive to missing data and noise. Even if part of the fence is completely occluded by the livestock's body, as long as there are enough points in the remaining visible line segment to accumulate votes, it is still possible to detect it as a complete "candidate" line in the parameter space. Compared with segmentation methods that are directly based on point cloud geometric features (such as normals and curvature), this method is more robust to partial occlusion and can detect interrupted linear features.

[0065] (3) Low requirement for labeled data volume: Existing deep learning models (such as PointNet++ and segmentation networks) typically require a large amount of diverse and accurately labeled point cloud data for training. Livestock point cloud data collection and labeling is a heavy and costly task. This invention can establish adaptive rules using a small amount of data. It greatly reduces the cost and threshold of data collection and labeling, and is suitable for fields such as agriculture and animal husbandry where specialization is high and labeling resources are limited.

[0066] (4) Low energy consumption and high speed. Compared with large-scale deep learning models, this invention does not require high-performance hardware support and may achieve real-time or near-real-time processing on edge devices with limited computing resources (such as embedded systems in aquaculture sites). Attached Figure Description

[0067] Figure 1 This is a schematic diagram of the overall processing flow of the present invention;

[0068] Figure 2 This is a schematic diagram of the framework of the present invention;

[0069] Figure 3 This is a schematic diagram of the livestock point cloud acquisition device in the embodiment;

[0070] Figure 4 The image shows the original color image of the pigs collected in the example.

[0071] Figure 5 This is a depth map in the embodiment;

[0072] Figure 6 This is a point cloud image containing noisy points in the example;

[0073] Figure 7 This is a schematic diagram of the passageway detection and segmentation in the embodiment;

[0074] Figure 8 The point cloud preprocessing result is shown in the example (point cloud of livestock containing fences on the right).

[0075] Figure 9This is a schematic diagram of some of the labeled data in the embodiment;

[0076] Figure 10 The images show the detection effect of the railing in the embodiment (left and right sides);

[0077] Figure 11 This is a schematic diagram of the automatic matching results of the PnPRansac algorithm in the embodiment;

[0078] Figure 12 This is a schematic diagram illustrating the integration of manually calibrated feature point matching results in the embodiment;

[0079] Figure 13 This is the complete livestock point cloud after registration in the example. Detailed Implementation

[0080] The technical solution of the present invention will be described in detail below, but the scope of protection of the present invention is not limited to the embodiments described.

[0081] like Figure 1 As shown, the livestock point cloud segmentation and denoising method based on adaptive parameter Hough transform of the present invention includes the following steps:

[0082] Step 1: Obtain livestock point cloud data;

[0083] First, drive the livestock into the Baoding pen 4 and ensure that they are located on the aisle scale 3. Then, use the FemtoBolt depth camera to collect point cloud data of the livestock from both sides of the Baoding pen 4.

[0084] Step 2: Preprocess the point cloud data obtained in Step 1. First, use outlier removal to obtain the point cloud of livestock closely attached to the fence. Then, use voxel mesh downsampling to uniformly downsample the data, finally outputting point cloud data containing livestock and fence on one side. ;

[0085] Step 3: Extract features from the preprocessed point cloud data, i.e., input the point cloud of livestock with fences. Dot the clouds for the same domesticated livestock with fences The corresponding spatial distribution features, geometric shape features, density statistics features, and color distribution features are obtained sequentially. Then, these features are concatenated and normalized to finally output the point cloud feature vector. ; This refers to the total number of points in the same point cloud;

[0086] Step 4: Convert the point cloud feature vector The pre-constructed adaptive model of Hough transform parameters is input, and a mapping function is learned. The above Indicates the Hough transform parameters; This refers to distance resolution, θ refers to angular resolution, and threshold refers to the threshold value. The specific method is as follows:

[0087] First, construct a small sample support set. Then use a feature encoder Point cloud feature vectors Mapped to a discriminative low-dimensional embedding space; then the query point cloud is computed. Embedded To obtain the prediction parameters ;

[0088] Step 5: Perform adaptive Hough transform for railing detection. This involves projecting the 3D point cloud of the legal entity with railings, which has been preprocessed in Step 2, onto a 2D plane to obtain a 2D image. Then, the Canny operator is used to perform edge detection on the 2D image to detect potential railing lines.

[0089] Step 6: Using the fence line obtained from the Hough transform fence detection in Step 5 as a reference, perform KD-Tree denoising and segmentation on the livestock point cloud to obtain a single-sided livestock point cloud.

[0090] Step 7: Perform point cloud registration and output the complete point cloud of livestock.

[0091] This invention utilizes a depth camera to collect raw livestock point clouds, including livestock, the guard pens 4, the ground, and other background environments within the farm. Then, conditional filtering, DBSCAN (Density-Based Spatial Clustering of Applications with Noise) clustering algorithm, and RANSAC algorithm are used in sequence to filter out some redundant background, noise, and ground point clouds. However, it is difficult to filter out the point clouds of the guard pens that are close to the livestock without affecting the integrity of the livestock point cloud.

[0092] Considering that the railings of Baoding 4 are straight lines, the Hough transform can detect straight lines, circles, or other geometric shapes in the image. By mapping points in the image space to the parameter space, a voting mechanism is used to find the most likely geometric shape parameters, exhibiting strong noise resistance and robustness to local occlusion. The 3D point cloud containing the railings is projected onto a 2D plane to obtain a 2D image. The Hough transform is used to detect straight lines, and horizontal straight lines are filtered out using an angle threshold; these lines can be considered railings. Black lines are drawn along the detected railing lines on the image to cover the railing portion. In 3D space, based on the projection, the black lines represent the 3D point cloud of the railings. Then, the KD-Tree algorithm is used to filter out points identified as railings and their neighbors from the original point cloud, segmenting the 3D point cloud of the livestock.

[0093] Since the size and distribution of fences vary in different livestock collection scenarios, the Hough transform parameters cannot be standardized. By utilizing the spatial features of point clouds and labeling them to construct a small sample dataset, the Hough transform parameters are automatically adjusted to improve the efficiency and accuracy of fence detection.

[0094] The detailed method for preprocessing the obtained point cloud data in step 2 of the embodiment is as follows:

[0095] Step 2.1: Remove outliers. First, using the distance between the depth camera and the guardrail 4, set point cloud filtering conditions to filter out scene point clouds that are far from the livestock outside the guardrail 4, retaining the point clouds of the guardrail 4, the aisle scale, and the livestock. Then, using the point cloud density difference and distance difference, use the DBSCAN clustering algorithm to extract the point cloud with the largest cluster, remove small clusters and some noise, and segment and extract the point clouds of the livestock with higher density and the adjacent guardrail 4 and aisle scale. Then, use the RANSAC algorithm for planar detection to filter out the aisle scale point cloud and segment and extract the point clouds of the livestock and the point clouds adjacent to the guardrail. In this embodiment, a control box 5 is also provided on one side of the guardrail 4 to house an industrial control computer for connecting the camera to collect and save data using a data transmission cable.

[0096] Step 2.2, Voxel Mesh Downsampling

[0097] Use voxelized meshes to uniformly downsample the point cloud of livestock with fences.

[0098] Feature vector in step 3 of the embodiment The methods for obtaining spatial distribution characteristics, geometric shape characteristics, density statistical characteristics, and color distribution characteristics are as follows:

[0099] For spatial distribution characteristics, a point cloud of livestock with fences is defined. The size ratio of the point cloud 3D spatial bounding box The point cloud centroid coordinates (cx, cy, cz) and the eigenvalues ​​(λ1, λ2, λ3) of the point cloud covariance matrix are used to describe the extensibility and principal direction of the point cloud.

[0100] Bounding box size ratio The calculation formula is as follows:

[0101] ; ; ;

[0102] , , The length, width, and height of the 3D bounding box;

[0103] The centroid coordinates c(cx, cy, cz) of the point cloud are obtained by taking the average of the three-dimensional coordinates of all points.

[0104] ;

[0105] The eigenvalues ​​of the covariance matrix (λ1, λ2, λ3) are calculated by arranging λ1≥λ2≥λ3 in descending order:

[0106] ;

[0107] p i Let be the three-dimensional coordinates of the i-th point; eigenvalue decomposition yields λ1≥λ2≥λ3;

[0108] Geometric features include linearity Flatness and scattering ;

[0109] linearity Flatness Scattering ;

[0110] The density statistical features are obtained by calculating the overall point density D and the average distance from the point cloud to the centroid. and standard deviation ;

[0111] ;

[0112] ;

[0113] ;

[0114] The K-means clustering method is used to group similar colors into one category, identify the main colors and their proportions, and obtain the color distribution characteristics.

[0115] Extract dominant color using K-means clustering: ;

[0116] Where K represents the number of clusters, i.e. the number of color categories. Let be the RGB color value of the i-th point in the point cloud. This represents the k-th cluster center. Let k represent the set of points belonging to the k-th class. Indicates color similarity.

[0117] The detailed process of step 4 in the embodiment is as follows:

[0118] Step 4.1: Construct the support set. Using self-collected livestock point cloud data from farms, manually label 50 point cloud data blocks and their optimal Hough parameters. Then, randomly rotate, scale, and add noise to expand the support set; finally, 50 small sample support sets are formed. ;

[0119] Step 4.2: Feature embedding network. Train a feature encoder E using a multilayer perceptron (MLP). ϕ Mapping the point cloud features F to the embedding space;

[0120] Step 4.3, Parameter Prediction; For a query point cloud P q Calculate its embedding ;

[0121] In the embedding space, find the support set and z q The most similar M samples are used to calculate the weighted average of their corresponding optimal parameters to obtain the predicted parameters. :

[0122] , ;

[0123] Where sim() is the cosine similarity and τ is the temperature coefficient;

[0124] Final prediction parameters The parameters are used for railing detection as adaptive Hough transform detection parameters.

[0125] In step 5 of the example, the Canny operator is used to perform edge detection on the image; the Hough transform is performed to detect straight lines lines=cv2.HoughLines(edges,ρ,θ,threshold), and potential railing lines are detected.

[0126] Step 5.1: Use an angle threshold to filter out horizontally distributed straight lines, setting the angle threshold to 15. o ;

[0127] Horizontal vector along the x-axis Let the unit direction vector of the detected line be... Calculate the angle θ between the x-axis and the horizontal vector, where θ ≤ 15°. o ,Right now ;

[0128] in, Straight lines that meet the conditions are retained;

[0129] Step 5.2: Check that the straight line meets the minimum length of the railing;

[0130] Set the starting and ending coordinates of the detection line to (x1, y1, z1) and (x2, y2, z2) respectively, and then calculate the line distance L. ;

[0131] The straight distance is greater than the preset railing length. reserve.

[0132] In step 6 of the embodiment, when performing KD-Tree denoising and segmentation on the livestock point cloud, the fence line obtained by Hough transform is used as a reference. A distance threshold of 1.5 times the fence diameter is set. All points are traversed, and the distance from each point to the nearest fence line is quickly queried using KD-Tree. Points with a distance less than the threshold are classified as fence point clouds, and those with a distance greater than the threshold are classified as livestock point clouds. The segmentation is completed, and the final pure one-sided livestock point cloud is retained.

[0133] The specific method for point cloud registration in step 7 of the embodiment is as follows:

[0134] First, with the camera position fixed, take a set of color calibration board photos. Manually calibrate 4 pairs of 2D feature points in the 2D color images of the two perspectives that need to be matched. The coordinates of the 4 pairs of points can determine a 3D space.

[0135] Then, using the camera parameters and the corresponding depth map, the 3D coordinates of the feature points in the first view are calculated. The 3D coordinates of the feature points in the first view and the 2D coordinates of the feature points in the second view are used as input values ​​for the PnP-RANSAC algorithm to calculate the transformation matrix of the two view coordinate systems.

[0136] By using a transformation matrix, the point cloud images from the left and right sides of the livestock are registered and fused to obtain the complete point cloud of the livestock.

[0137] To further verify the technical feasibility and effectiveness of this invention, a pig point cloud was used as an example for implementation. Video data of 50 pigs were collected at the farm. To successfully collect the complete point cloud of the pigs, the pigs were driven into pen 4 in sequence during the collection process.

[0138] like Figures 1 to 3 As shown, a depth camera (first depth camera 1 and second depth camera 2) is installed on each side of the pig pen 4. After the pigs are driven to the weighing scale 3 in the aisle, data is collected from both sides of the pigs. After collection, Open3D is used to cut the video into JPG color images. Figure 4 ) and PNG depth map ( Figure 5 ), combined with camera intrinsic parameters, to generate a single-sided PCD point cloud map ( Figure 6 The point cloud map includes point clouds of scenes such as pigs, Baoding pen 4, ground, and walls.

[0139] To avoid the left and right side railings affecting the detection results after registration, the railings are detected on one side of the point cloud before the point clouds on both sides are registered.

[0140] Due to the imaging principle of cameras, the collected livestock point clouds are inverted. Rotate the point cloud 180 degrees counterclockwise along the x-axis. oObtain point clouds of livestock standing upright; use conditional filtering, setting the conditional filtering parameters according to the actual physical distance between the camera and the pigs: x_range = (-0.9, 0.5); y_range = (-0.8, 0.1); z_range = (-1.9, -1.2);

[0141] Then, point cloud images of the left and right sides of the pig and the four points in the Baoding pen were obtained respectively, and point clouds of distant scenes were filtered out.

[0142] Then, the DBSCAN clustering algorithm is used to extract the point cloud with the largest cluster size, removing small clusters and some noise points to obtain the point cloud of pigs with higher density and adjacent horizontal fences and aisle scales, while filtering out noise points such as those on the top and vertical fences. The RANSAC algorithm is then used for planar detection to filter out the ground scale point cloud and segment and extract the point cloud of pigs and adjacent horizontal fences, with the following results: Figure 7 As shown, Figure 7 The blue area represents ground-based point clouds. Figure 8 The extracted point cloud is the point cloud containing livestock fences.

[0143] Next, the point cloud of livestock containing fences is labeled. Due to the small sample size, 3D straight lines are directly labeled to improve accuracy. In this embodiment, CloudCompare software is used to manually label and segment the fences. Figure 9 As shown in the diagram, blue dots labeled 0 represent livestock point clouds, green dots labeled 1 represent the upper railing point cloud, and red dots labeled 2 represent the lower railing point cloud. A Python program will then be written to generate the 3D line parameters.

[0144] The global features of the point cloud across the entire scene are analyzed. An MLP model is trained to create a mapping between parameters and point cloud features. Using the trained model's regressor, the point cloud to be detected is input, and the global features are analyzed to predict the parameters of the Hough transform. The predicted parameters are then used to perform a Hough transform on the point cloud containing the fence to detect the fence. The detection results are as follows: Figure 10 As shown.

[0145] Using the fence line obtained by Hough transform as a reference, a distance threshold of 1.5 times the fence diameter is set. All points are traversed, and the distance from each point to the nearest fence line is quickly queried using a KD-Tree algorithm. Points with a distance less than the threshold are classified as "fence point clouds," and those with a distance greater than the threshold are classified as "pig point clouds," thus completing the segmentation and retaining the final pure unilateral pig point cloud. The PnP-RANSAC algorithm, combined with manually calibrated feature points, is used to register the pig point clouds on both sides, obtaining the complete pig point cloud, as shown below. Figure 13 As shown.

Claims

1. A livestock point cloud segmentation denoising method based on adaptive parameter Hough transform, characterized in that, Comprise the following steps: Step 1, obtain livestock point cloud data; First, drive the livestock into the holding pen to ensure that it is located on the passageway scale, and use the FemtoBolt depth camera from both sides of the holding pen to collect the corresponding point cloud data of the livestock; Step 2, the point cloud data obtained in step 1 is preprocessed, that is, first, the point cloud of livestock close to the fence is obtained by using an outlier removal method, then uniform down-sampling is performed on the point cloud by using a voxel grid down-sampling method, and finally, the point cloud data containing livestock and the fence on one side is output ; Step 3, feature extraction is performed on the pretreated point cloud data, that is, input the livestock point cloud with the fence , the same point cloud with the fence , the corresponding spatial distribution features, geometric shape features, density statistical features and color distribution features are obtained in sequence, and then the spatial distribution features, geometric shape features, density statistical features and color distribution features are spliced and normalized, and finally the point cloud feature vector is output ; , refers to the number of all points in the same point cloud; Step 4, the point cloud feature vector Input the constructed Hough transform parameter adaptive model, and then learn a mapping function ; the above Indicates the Hough transform parameter; Refers to the distance resolution, theta refers to the angle resolution, and threshold refers to the threshold. First construct a small sample support set Then use a feature encoder Map the point cloud feature vector To a discriminative low-dimensional embedding space; then compute the query point cloud Embedding To get the predicted parameters ; Step 5, adaptive Hough transform railing detection is performed, that is, the three-dimensional point cloud of one side with a railing after pre-processing in step 2 is projected to a two-dimensional plane to obtain a two-dimensional image, and then a Canny operator is used to perform edge detection on the two-dimensional image to obtain parameter eages, and the predicted parameter obtained in step 4 is combined A potential railing straight line is detected. Step 6, using the rail straight line obtained by Hough transform in step 5 as a reference, performing spatial constraint verification, executing KD-Tree denoising and segmentation on the livestock point cloud, and obtaining single-side livestock point cloud; Step 7, performing point cloud registration, and outputting complete livestock point cloud.

2. The livestock point cloud segmentation and denoising method based on adaptive parameter Hough transform according to claim 1, characterized in that, The detailed method of preprocessing the obtained point cloud data in step 2 is: Step 2.1, remove outliers, first set the point cloud filtering condition using the distance between the depth camera and the holding pen to filter out the scene point cloud far away from the livestock outside the holding pen, and retain the holding pen, passageway scale and livestock point cloud; then use the point cloud density difference and distance difference to extract the cluster maximum point cloud using the DBSCAN clustering algorithm, remove small clusters and part of the noise, and segment and extract the livestock and the closely attached holding pen and passageway scale point cloud with higher density; then perform plane detection through the RANSAC algorithm to filter out the passageway scale point cloud and segment and extract the livestock and closely attached rail point cloud; Step 2.2, voxel grid downsampling Uniformly downsample the livestock point cloud with rail using the voxel grid.

3. The livestock point cloud segmentation and denoising method based on adaptive parameter Hough transform according to claim 1, characterized in that, The feature vector in step 3 The obtaining method of the spatial distribution feature, the geometric shape feature, the density statistical feature and the color distribution feature is as follows: For the spatial distribution feature, set the livestock point cloud with a railing The size of the point cloud three-dimensional space bounding box , the point cloud barycenter coordinates (cx, cy, cz), the eigenvalues of the point cloud covariance matrix (λ1, λ2, λ3); Border box size ratio The calculation formula is as follows: ； ； ； , , are the length, width, and height of the three-dimensional bounding box; The center of gravity coordinates c(cx, cy, cz) of the point cloud are obtained by taking the three-dimensional coordinate mean value of all points: ； Arrange λ1≥λ2≥λ3 in descending order to calculate the eigenvalues (λ1, λ2, λ3) of the covariance matrix: ； p i three-dimensional coordinates of the i-th point; Eigenvalue decomposition, λ1≥λ2≥λ3 is obtained; Geometric features include linearity , flatness , and scatter ; linearity ; flatness ; scattering degree ; The method for obtaining the density statistical features is: calculating the overall point density D, the average distance of the point cloud to the center of gravity and the standard deviation respectively. ； ； ； Use the K-means clustering method to classify similar colors into one category, find out the main colors and their proportion, and obtain the color distribution characteristics; Extracting dominant colors using K-means clustering: ; wherein K represents the number of clusters, i.e. the number of color classes, RGB color value of the i-th point cloud in the point cloud, denotes the k-th cluster center, denotes the set of points belonging to the k-th class, denotes the color similarity.

4. The livestock point cloud segmentation and denoising method based on adaptive parameter Hough transform according to claim 1, characterized in that, The detailed process of step 4 is as follows: Step 4.1, construct support set, use self-cultured farm livestock point cloud data, manually label 50 point cloud data blocks and their optimal Hough parameters, and randomly rotate, scale, and add noise to expand the support set; ultimately form 50 small sample support sets ; Step 4.2, Feature embedding network, train a feature encoder E using a multi-layer perceptron, MLP ϕ map the point cloud features F to the embedding space; Step 4.3, parameter prediction; for a query point cloud P q , compute its embedding ; In the embedding space, find the M most similar samples in the support set to z q The M most similar samples, and their corresponding optimal parameters are weighted and averaged to obtain the predicted parameters : ， ； wherein sim() is cosine similarity, and τ is a temperature coefficient; denotes the feature representation of the query sample in the embedding space; denotes the feature representation of the k-th labeled sample in the support set in the embedding space; denotes the feature representation of the j-th labeled sample in the support set in the embedding space; Final prediction parameters Railing detection is performed as a parameter of adaptive Hough transform detection.

5. The livestock point cloud segmentation and denoising method based on adaptive parameter Hough transform according to claim 1, characterized in that, Step 5 uses the Canny operator to perform edge detection on the image; and performs Hough transform to detect straight lines lines=cv2.HoughLines(edges, ρ, θ, threshold), to detect potential rail straight lines; Step 5.1, screening out the horizontal distribution of straight lines using angle threshold, set the angle threshold of 15 o ; Horizontal transverse vector along the x-axis , let the unit direction vector of the straight line be , calculate the included angle θ≤15 o , that is ; wherein , the straight line satisfying the condition is retained; Step 5.2, the detected straight line satisfies the minimum length of the rail; Set the start point coordinates and end point coordinates of the detection straight line as (x1, y1, z1) and (x2, y2, z2) respectively, and then calculate the straight line distance L, the straight line distance ; a straight line distance greater than a preset railing length reserved.

6. The livestock point cloud segmentation and denoising method based on adaptive parameter Hough transform according to claim 1, characterized in that, When performing KD-Tree denoising and segmentation on the livestock point cloud in step 6, the rail straight line obtained by Hough transform is used as a reference, a distance threshold of 1.5 times the rail diameter is set, all points are traversed, and the distance from each point to the nearest rail straight line is quickly queried through KD-Tree; points with a distance less than the threshold are classified as rail point cloud, and points with a distance greater than the threshold are classified as livestock point cloud, the segmentation is completed, and the final pure single-side livestock point cloud is retained.

7. The livestock point cloud segmentation and denoising method based on adaptive parameter Hough transform according to claim 1, characterized in that, The specific method of point cloud registration in step 7 is: First, under the condition that the camera position is fixed, a set of color calibration plates are shot, and 4 pairs of two-dimensional feature points are manually calibrated in the two-dimensional color graphs under the two viewing angles that need to be matched, and the 4 pairs of point coordinates can determine a three-dimensional space; Then, the three-dimensional point coordinates corresponding to the feature point pairs in the first viewing angle are calculated using the camera parameters and the corresponding depth map, and the three-dimensional feature point coordinates in the first viewing angle and the two-dimensional feature point coordinates in the second viewing angle are used as input values of the PnP-RANSAC algorithm to calculate the transformation matrix of the coordinate systems of the two viewing angles; The left and right two-view point cloud graphs of the livestock are registered and fused through the transformation matrix to obtain the complete point cloud of the livestock.

Images