Sheep body three-dimensional reconstruction method

Abstract

The invention provides a sheep body three-dimensional reconstruction method, and relates to the technical field of three-dimensional point cloud segmentation and registration. The method comprises the step of obtaining a three-dimensional reconstruction image of a sheep body based on a background segmentation model and a registration model obtained through training. The background segmentation model is obtained based on a training sample, the training sample is obtained by obtaining sample point cloud data, down-sampling the sample point cloud data by adopting a non-playback random sampling method to obtain sampling points, and the sample point cloud data is obtained by filtering initial sample point cloud data by simultaneously and respectively using four set filtering modes; for each sampling point, determining a local feature vector of the sampling point according to the three-dimensional coordinate of the sampling point, the distance between the sampling point and each neighborhood data point and the point pair feature; and obtaining a training sample according to the initial sample point cloud data and the local feature vector of each sampling point. According to the method, the manual annotation amount is reduced by using the traditional algorithm data annotation, and the model training is performed by applying the deep learning algorithm, so that the generalization ability of the model is improved.

Description

Technical Field

[0001] The present application relates to the technical field of three-dimensional point cloud segmentation and registration, and specifically, to a method for three-dimensional reconstruction of a sheep body. Background Art

[0002] In computer vision, 3D reconstruction refers to the process of reconstructing 3D information based on single-view or multi-view images. Compared with 2D images, 3D point clouds are less sensitive to color, texture, and light, so the 3D point cloud method is more universal than 2D. However, the environment is usually complex during data collection, and the point cloud contains a large number of data points in non-target areas. Therefore, in order to obtain a cleaner target detection object, it is necessary to apply different methods to the original point cloud according to the acquisition characteristics, such as through filtering, k-nearest neighbor algorithm, and other conditional filtering methods, and perform multiple repeated filtering. In this process, because multiple depth cameras with different perspectives are usually used when collecting target detection objects, the data collection of the target detection object is complete. Therefore, it is necessary to align the point cloud data of different perspectives after filtering. For this purpose, algorithms such as ICP, 4PCS (4-Points Congruent Sets, four-point set) + ICP are applied to 3D registration.

[0003] Then, the above existing filtering and registration methods are all based on traditional algorithms. In terms of filtering, the robustness of traditional algorithms is insufficient. When the type or equipment of the collected data changes, it is necessary to find a new filtering algorithm based on the characteristics of the collected data. In terms of registration, that is, 3D reconstruction, traditional algorithms are sensitive to the position of the initial point cloud or the order of the point set, and are prone to fall into a local optimal solution, thus failing to achieve a good registration effect. Summary of the invention

[0004] The purpose of this application is to solve at least one of the above-mentioned technical deficiencies.

[0005] In one aspect, a method for three-dimensional reconstruction of a sheep body is provided, the method comprising:

[0006] Acquire original point cloud data and a background segmentation model of a target sheep body, and segment the background data in the original point cloud data based on the background segmentation model to obtain sheep body point cloud data, wherein the sheep body point cloud data includes left view point cloud data, right view point cloud data, and top view point cloud data of the target sheep body;

[0007] Input the sheep body point cloud data into the registration model to obtain the upper left view point cloud data and the upper right view point cloud data of the target sheep body, and merge the upper left view point cloud data and the upper right view point cloud data to obtain a three-dimensional reconstruction image of the target sheep body;

[0008] The background segmentation model is trained based on at least one training sample, and each training sample is obtained in the following way:

[0009] Acquire sample point cloud data, and downsample the sample point cloud data at least once using a random sampling method without playback to obtain at least one sampling point, wherein the sample point cloud data is obtained by filtering the initial sample point cloud data using four set filtering methods at the same time, and the initial sample point cloud data includes three views of the sample sheep body, and the three views include a left view, a right view, and a top view;

[0010] For each sampling point, the local feature vector of the sampling point is determined according to the three-dimensional coordinates of the sampling point, the distance between the sampling point and each neighboring data point, and the point pair features;

[0011] The training samples are obtained according to the initial sample point cloud data and the local feature vector of each sampling point.

[0012] Optionally, obtain sample point cloud data, including:

[0013] Acquire initial sample point cloud data, and filter the initial sample point cloud data of three views in the initial sample point cloud data using four set filtering methods at the same time to obtain target initial sample point cloud data;

[0014] According to the initial sample point cloud data and the target initial sample point cloud data, the sample background point cloud data is obtained;

[0015] Different identifiers are used to identify the last columns of the target initial sample point cloud data and the sample background point cloud data, respectively, and the identified target initial sample point cloud data and the sample background point cloud data are merged to obtain the sample point cloud data.

[0016] Optionally, four filter settings include through filtering, region growing, radius filtering, and DBSCAN.

[0017] Optionally, a local feature vector of the sampling point is determined according to the three-dimensional coordinates of the sampling point, the distance between the sampling point and each neighboring data point, and the PPF feature, including:

[0018] Determine at least one sampling point group corresponding to the sampling point according to the set radius, the sampling point group including neighborhood data points within a spherical area determined by the sampling point with the set radius;

[0019] For each sampling point group, the geometric features corresponding to the sampling point group are obtained through multi-layer perception and maximum pooling encoding operations according to the three-dimensional coordinates of the sampling point, the distance between the sampling point and each neighboring data point in the sampling point group, and the PPF feature;

[0020] The geometric features of at least one sampling point group are fused according to a set radius to obtain a local feature vector of the sampling point.

[0021] Optionally, the PPF features include the angle between the sampling point and the line connecting the point pair of each neighboring point, the angle between the normal vector of each neighboring point and the line connecting the sampling point pair, the angle between the sampling point and the normal vectors of two points of each neighboring point, and the Euclidean distance between the sampling point and each neighboring point.

[0022] Optionally, training samples are obtained based on the initial sample point cloud data and the local feature vector of each sampling point, including:

[0023] The local feature vector of each sampling point is fused into the initial sample point cloud data by upsampling to obtain the corresponding global feature vector;

[0024] The global feature vector and the local feature vector of each sampling point are concatenated to obtain the training sample.

[0025] Optionally, the background segmentation model is trained by:

[0026] Obtain at least one training sample, a labeled classification result corresponding to each data point in each training sample, and an MLP model, wherein the labeled classification result is a sheep body data point identifier or a background data point identifier;

[0027] The MLP model is trained based on each training sample until the loss function corresponding to the MLP model converges, and the MLP model at the end of the training is used as the background segmentation model;

[0028] The input of the MLP model is the training sample, the output is the classification result of the training sample, the loss function is the cross entropy loss function, and the value of the loss function is used to characterize the difference between the classification result of each training sample output by the model and the labeled classification result corresponding to each training sample.

[0029] Optionally, the registration model is trained by at least one training dataset, and the training dataset is obtained by:

[0030] Acquire registration parameters and a source point cloud data set, where the source point cloud data set includes initial training point cloud data of three views of a sample sheep body, where the three views include a left view, a right view, and a top view;

[0031] The initial training point cloud data of the three views are roughly registered according to the registration parameters to obtain the training point cloud data after the three views are roughly registered;

[0032] The training point cloud data after the rough registration of the three views are precisely registered according to the ICP algorithm to obtain a reference point cloud data set, which includes the training data of the upper left view, the training data of the upper right view, and the training data of the upper left and right views;

[0033] The source point cloud dataset and the reference point cloud dataset are used as training datasets.

[0034] Optionally, the registration model is trained by:

[0035] Acquire a training data set and an initial rigid change matrix, and transform a source point cloud data set in the training data set according to the initial rigid change matrix to obtain a transformed source point cloud data set;

[0036] Determine a corresponding permutation matrix according to the converted source point cloud data set and the reference point cloud data set in the training data set;

[0037] For each data point in the source point cloud data set, determine the weight of the data point according to the three-dimensional coordinates of the data point corresponding to the data point in the reference point cloud data set;

[0038] The covariance matrix is ​​obtained according to the weight of each data point in the reference point cloud data set, and the covariance matrix is ​​subjected to singular value decomposition of the permutation matrix to obtain the rigid change matrix;

[0039] The rigid change matrix is ​​used as the initial rigid change matrix for iterative training to a set number of times, and the trained rigid change matrix is ​​used as the registration model.

[0040] Optionally, determining a corresponding permutation matrix according to the converted source point cloud dataset and the reference point cloud dataset in the training dataset includes:

[0041] Extract features from the conversion source point cloud dataset and the reference point cloud dataset in the training dataset, aggregate and normalize the extracted features to obtain feature vectors;

[0042] According to the conversion source point cloud data set and the reference point cloud data set, corresponding annealing parameters and outlier parameters are obtained;

[0043] A matching matrix is ​​obtained according to the converted source point cloud data set, the reference point cloud data set, the annealing parameters and the outlier parameters, and the matching matrix is ​​normalized to obtain a permutation matrix.

[0044] On the other hand, a device for three-dimensional reconstruction of a sheep body is provided, the device comprising:

[0045] A segmentation module is used to obtain the original point cloud data of the target sheep body and the background segmentation model, and based on the background segmentation model, the background data in the original point cloud data is segmented to obtain the sheep body point cloud data, which includes the left view point cloud data, the right view point cloud data and the top view point cloud data of the target sheep body;

[0046] A registration module is used to input the sheep body point cloud data into the registration model to obtain the upper left view point cloud data and the upper right view point cloud data of the target sheep body, and merge the upper left view point cloud data and the upper right view point cloud data to obtain a three-dimensional reconstruction image of the target sheep body;

[0047] The training module is used to train a background segmentation model based on at least one training sample, where each training sample is obtained by:

[0048] Acquire sample point cloud data, and downsample the sample point cloud data at least once using a random sampling method without playback to obtain at least one sampling point, wherein the sample point cloud data is obtained by filtering the initial sample point cloud data using four set filtering methods at the same time, and the initial sample point cloud data includes three views of the sample sheep body, and the three views include a left view, a right view, and a top view;

[0049] For each sampling point, the local feature vector of the sampling point is determined according to the three-dimensional coordinates of the sampling point, the distance between the sampling point and each neighboring data point, and the point pair features;

[0050] The training samples are obtained according to the initial sample point cloud data and the local feature vector of each sampling point.

[0051] The beneficial effects of the technical solution provided by the embodiment of the present application are:

[0052] In the embodiment of the present application, the sample point cloud data for the background segmentation model is obtained after four kinds of filtering. At this time, the sample point cloud data obtained after filtering will include ten data representing the sample features, and the data quality will be greatly improved and the robustness will be improved, so that the subsequent model training will be more accurate. Further, the sample point cloud data will be downsampled at least once by a random sampling method without playback, that is, each data point can only be selected once, so that the sampling efficiency is improved while the implementation method is simple, and the diversity and uniformity of the sampling can be ensured. At the same time, the PPF feature is added to the training sample to enhance the expression ability of the local features of the sampling point, and because the PPF feature combines the ideas of global modeling and local matching, it can still show high recognition performance when the sheep body data is in the case of noise, clutter and partial occlusion. Moreover, traditional algorithms are used for data preprocessing in the segmentation and registration stages for data annotation, which reduces the workload of manual annotation. At the same time, deep learning algorithms are applied to train the segmentation and registration models, which improves the generalization ability of the model and can quickly adapt to new scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in describing the embodiments of the present application are briefly introduced below.

[0054] Figure 1 A schematic diagram of a process for three-dimensional reconstruction of a sheep body provided in an embodiment of the present application;

[0055] Figure 2 A schematic diagram of original point cloud data of a target sheep body provided in an embodiment of the present application;

[0056] Figure 3 A schematic diagram of a three-dimensional reconstruction of a sheep body provided in an embodiment of the present application;

[0057] Figure 4 A schematic diagram of the PPF feature provided in an embodiment of the present application;

[0058] Figure 5 A schematic diagram of the complete steps of the sheep body three-dimensional reconstruction method provided in an embodiment of the present application;

[0059] Figure 6 A schematic diagram of the classified sheep body and background point cloud data provided in an embodiment of the present application;

[0060] Figure 7 A schematic diagram of a point cloud registration provided in an embodiment of the present application;

[0061] Figure 8 A schematic diagram of the structure of a device for three-dimensional reconstruction of a sheep body provided in an embodiment of the present application;

[0062] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0063] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be interpreted as limiting the present invention.

[0064] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0065] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0066] The present application embodiment provides a method for three-dimensional reconstruction of a sheep body, such as Figure 1 As shown, the method includes:

[0067] Step S101, obtaining the original point cloud data and background segmentation model of the target sheep body, and based on the background segmentation model, segmenting the background data in the original point cloud data to obtain the sheep body point cloud data, which includes the left view point cloud data, the right view point cloud data and the top view point cloud data of the target sheep body.

[0068] Optionally, the target sheep body refers to a sample that needs to be 3D reconstructed. The method of obtaining the original point cloud data of the target sheep is not limited in this application, for example, it can be obtained by taking pictures with a depth camera. Figure 2As shown in , at this time, the original point cloud data acquired based on the depth camera (i.e., the original point cloud image in the figure) will also include other background point cloud data except the sheep body point cloud data. At this time, the acquired original point cloud data can be input into the trained background segmentation model, and the background segmentation model can determine that each data point in the original point cloud data is specifically a background data point suitable for the target sheep body. The background point cloud data can be further removed to obtain the sheep body point cloud data, which includes the left view point cloud data, the right view point cloud data and the top view point cloud data of the target sheep body.

[0069] Step S102, input the sheep body point cloud data into the registration model to obtain the upper left view point cloud data and the upper right view point cloud data of the target sheep body, and merge the upper left view point cloud data and the upper right view point cloud data to obtain a three-dimensional reconstructed image of the target sheep body.

[0070] Optional, such as Figure 3 As shown, after obtaining the sheep body point cloud data, the sheep body point cloud data can be input into the registration model, which will perform registration based on the left view point cloud data and the top view point cloud data in the sheep body point cloud data to obtain the left top view point cloud data (i.e. Figure 3 (b) left-top part point cloud in the sheep body point cloud data), and the right view point cloud data and the top view point cloud data in the sheep body point cloud data are aligned to obtain the right top view point cloud data (i.e. Figure 3 (c) right-top part point cloud in the figure), and then merge the upper left view point cloud data and the upper right view point cloud data to obtain the 3D reconstruction of the target sheep body (i.e. Figure 3 (a) Overall point cloud).

[0071] The background segmentation model mentioned above is obtained based on at least one training sample, and each training sample is obtained in the following way:

[0072] Acquire sample point cloud data, and downsample the sample point cloud data at least once using a random sampling method without playback to obtain at least one sampling point, wherein the sample point cloud data is obtained by filtering the initial sample point cloud data using four set filtering methods at the same time, and the initial sample point cloud data includes three views of the sample sheep body, and the three views include a left view, a right view, and a top view;

[0073] For each sampling point, the local feature vector of the sampling point is determined according to the three-dimensional coordinates of the sampling point, the distance between the sampling point and each neighboring data point, and the point pair feature (PPF feature for short).

[0074] The training samples are obtained according to the initial sample point cloud data and the local feature vector of each sampling point.

[0075] Optionally, initial sample point cloud data including the initial sample point cloud data of the left view, the initial sample point cloud data of the right view, and the initial sample point cloud data of the top view of the sample sheep body can be obtained, and then the initial sample point cloud data of the left view, the initial sample point cloud data of the right view, and the initial sample point cloud data of the top view are filtered using four set filtering methods at the same time to obtain sample point cloud data. In an embodiment of the present application, the four set filtering methods may include pass-through filtering, regional growing method, radius filtering, and DBSCAN. At this time, the sample point cloud data obtained after filtering will include ten data representing sample characteristics, and the data quality will be greatly improved and the robustness will be enhanced, so that subsequent model training will be more accurate.

[0076] In an optional embodiment of the present application, obtaining sample point cloud data may include:

[0077] Acquire initial sample point cloud data, and filter the initial sample point cloud data of three views in the initial sample point cloud data using four set filtering methods at the same time to obtain target initial sample point cloud data;

[0078] According to the initial sample point cloud data and the target initial sample point cloud data, the sample background point cloud data is obtained;

[0079] Different identifiers are used to identify the last columns of the target initial sample point cloud data and the sample background point cloud data, respectively, and the identified target initial sample point cloud data and the sample background point cloud data are merged to obtain the sample point cloud data.

[0080] Optionally, after the initial sample point cloud data acquired by the depth camera is filtered using pass-through filtering, region growing method, radius filtering and DBSCAN respectively, the target initial sample point cloud data can be obtained. At this time, the target initial sample point cloud data can be sent to the initial sample point cloud data, and the corresponding sample background point cloud data can be obtained; further, since the obtained target initial sample point cloud data and sample background point cloud data are files in a table format, the last column of the target initial sample point cloud data and the sample background point cloud data can be marked at this time, for example, the last column of the target initial sample point cloud data is marked 1, and the last column of the sample background point cloud data is marked 0, and then the marked target initial sample point cloud data and the last column of the sample background point cloud data are merged into a new point cloud file. At this time, the new point cloud file obtained is a sample point cloud data.

[0081] Furthermore, each sample point cloud data can be down-sampled. In the present application, the farthest sampling method is modified to a random sampling method without replacement for down-sampling, that is, each data point can only be selected once. This not only improves the sampling efficiency, but also has a simple implementation method and ensures the diversity and uniformity of the sampling.

[0082] In an optional embodiment of the present application, the local feature vector of the sampling point is determined according to the three-dimensional coordinates of the sampling point, the distance between the sampling point and each neighborhood data point, and the PPF feature, including:

[0083] Determine at least one sampling point group corresponding to the sampling point according to the set radius, the sampling point group including neighborhood data points within a spherical area determined by the sampling point with the set radius;

[0084] For each sampling point group, the geometric features corresponding to the sampling point group are obtained through multi-layer perception and maximum pooling encoding operations according to the three-dimensional coordinates of the sampling point, the distance between the sampling point and each neighboring data point in the sampling point group, and the PPF feature;

[0085] The geometric features of at least one sampling point group are fused according to a set radius to obtain a local feature vector of the sampling point.

[0086] Optionally, after sampling to obtain a sampling point, at least one sampling point group of the sampling point can be determined according to a preset radius. Specifically, after the sampling point is selected, a fixed number of points (i.e., neighborhood data points) within a spherical region with a surrounding radius r can be selected as a group. For example, during the first sampling, the radius that can be selected is [0.1, 0.2, 0.4], and the number of samples (i.e., the number of neighborhood data points) corresponding to the sampling point group is [32, 64, 128].

[0087] Furthermore, for each sampling point group, the three-dimensional coordinates of the sampling point and each neighborhood data point in the sampling point group can be determined, and then the distance and PPF features between the sampling point and each neighborhood data point in the sampling point group can be determined based on the three-dimensional coordinates of the sampling point and each neighborhood data point in the sampling point group.

[0088] In an optional embodiment of the present application, the PPF features include the angle between the sampling point and the point-pair connecting the sampling point of each neighborhood point, the angle between the normal vector of each neighborhood point and the point-pair connecting the sampling point, the angle between the normal vectors of the sampling point and two points of each neighborhood point, and the Euclidean distance between the sampling point and each neighborhood point.

[0089] For example, assuming that the three-dimensional coordinates of the sampling point are xyz, the distance between the sampling point and each neighboring data point in the group and the PPF features between the sampling point and each neighboring point in the group can be as follows: Figure 4As shown in the figure. Each large circle in the figure is a sampling point group, the center point in the circle (point c in the figure) is the sampling point, and the points in the circle and the line connecting the sampling points (points P1 to P8 in the figure) are the neighborhood points for which distance and PPF features need to be calculated. c A neighborhood point x i Take the distance between the sampling point and each neighboring data point in the group as an example to illustrate the PPF characteristics between the sampling point and each neighboring point in the group. c With neighboring point x i The Euclidean distance d is |x c -x i |, sampling point x c With neighboring point x i The angle F1 between the point pair connection line and the sampling point x c With neighboring point x i The angle F3 between the point pairs and the line connecting them, the sampling point x c With neighboring point x i The normal vector n of two points i and n c The angle F2 between them, at this time the PPF feature is PPF(x c ,x i )=(||d||,∠(n c -d),∠(n i -d),∠(n c -n i )).

[0090] Furthermore, the geometric features of the region can be encoded through multi-layer perception and maximum pooling, and then the features can be spliced ​​by setting different radii, and these features can be aggregated into local feature vectors of the sampling points.

[0091] In this application, PPF features are added to the training samples, which can enhance the expression ability of the local features of the sampling points. And because the PPF features combine the ideas of global modeling and local matching, they can still show high recognition performance when the sheep body data is noisy, cluttered and partially occluded.

[0092] In the optional example of the application, a training sample is obtained based on the initial sample point cloud data and the local feature vector of each sampling point, including:

[0093] The local feature vector of each sampling point is fused into the initial sample point cloud data by upsampling to obtain the corresponding global feature vector;

[0094] The global feature vector and the local feature vector of each sampling point are concatenated to obtain the training sample.

[0095] Optionally, for the obtained local features, the local feature vector of each sampling point can be fused to the initial sample point cloud data by reverse interpolation, i.e. upsampling, to obtain the global feature vector corresponding to each sampling point, and then the global feature and the previous local feature are concatenated to obtain the training sample.

[0096] In an optional embodiment of the present application, the background segmentation model is trained in the following manner:

[0097] Obtain at least one training sample, a labeled classification result corresponding to each data point in each training sample, and an MLP model, wherein the labeled classification result is a sheep body data point identifier or a background data point identifier;.

[0098] The MLP model is trained based on each training sample until the loss function corresponding to the MLP model converges, and the MLP model at the end of the training is used as the background segmentation model;

[0099] The input of the MLP model is the training sample, the output is the classification result of the training sample, the loss function is the cross entropy loss function, and the value of the loss function is used to characterize the difference between the classification result of each training sample output by the model and the labeled classification result corresponding to each training sample.

[0100] Optionally, after obtaining the training samples, each training sample can be input into the MLP model to train the MLP model until the loss function corresponding to the MLP model converges. At this time, the MLP model will output whether each data point in the training sample is a sheep body data point or a background data point. For example, for a data point, if 0 represents a background data point and 1 represents a sheep body data point, the probability of 0 or 1 can be output at this time, and then the data point can be determined whether it is a background data point or a sheep body data point based on the probability of the output being 0 or 1. In addition, the value of the loss function in the above represents the difference between the classification results of each training sample output by the model and the labeled classification results corresponding to each training sample. If the loss function converges, it means that the training can be ended. At this time, the MLP model at the end of the training is used as the background segmentation model.

[0101] Optionally, since the loss function in the present application is modified from the negative log-likelihood loss function NLLLoss to a cross-entropy loss with a relatively stable derivative, this helps to propagate effective gradients in the deep layers of the network, making the gradients in back propagation relatively stable, which helps to optimize the convergence of the algorithm and helps the network to better learn features.

[0102] In an optional embodiment of the present application, the registration model is obtained by training at least one training data set, and the training data set is obtained in the following manner:

[0103] Acquire registration parameters and a source point cloud data set, where the source point cloud data set includes initial training point cloud data of three views of a sample sheep body, where the three views include a left view, a right view, and a top view;

[0104] The initial training point cloud data of the three views are roughly registered according to the registration parameters to obtain the training point cloud data after the three views are roughly registered;

[0105] The training point cloud data after the rough registration of the three views are precisely registered according to the ICP algorithm to obtain a reference point cloud data set, which includes the training data of the upper left view, the training data of the upper right view, and the training data of the upper left and right views;

[0106] The source point cloud dataset and the reference point cloud dataset are used as training datasets.

[0107] Optionally, the registration model required in the data registration process can also be trained based on at least one training data set. In this case, the initial training point cloud data including the left view, right view and top view of the sample sheep body can be obtained as the source point cloud data set, and then the calibration box is used as a reference to obtain its transformation matrix, i.e., the registration parameter, and the initial training point cloud data of the left view, right view and top view of the sample sheep body after background segmentation are roughly registered with the initial training point cloud data of the top view as a reference. Further, the ICP algorithm is used to perform precise registration on the initial training point cloud data of the left view, the initial training point cloud data of the right view and the initial training point cloud data of the top view, and the registration results are merged to obtain three point cloud files, i.e., the training data of the upper left view, the training data of the upper right view and the training data of the upper left and right views as reference point cloud data sets. Further, the source point cloud data set and the reference point cloud data set are used as training data sets and used to train the registration model.

[0108] In an embodiment of the present application, the point cloud data is registered using traditional methods as training data, and after coarse registration using the calibration method, it is finely registered using the ICP method, which can prevent falling into a local optimal solution and improve data quality, while reducing the process of manual labeling.

[0109] In an optional example of this application, the registration model is trained in the following manner:

[0110] Acquire a training data set and an initial rigid change matrix, and transform a source point cloud data set in the training data set according to the initial rigid change matrix to obtain a transformed source point cloud data set;

[0111] Determine a corresponding permutation matrix according to the converted source point cloud data set and the reference point cloud data set in the training data set;

[0112] For each data point in the source point cloud data set, determine the weight of the data point according to the three-dimensional coordinates of the data point corresponding to the data point in the reference point cloud data set;

[0113] The covariance matrix is ​​obtained according to the weight of each data point in the reference point cloud data set, and the covariance matrix is ​​subjected to singular value decomposition of the permutation matrix to obtain the rigid change matrix;

[0114] The rigid change matrix is ​​used as the initial rigid change matrix for iterative training to a set number of times, and the trained rigid change matrix is ​​used as the registration model.

[0115] Optionally, the acquired training data set can be transformed based on the initial rigid change matrix to obtain a transformed source point cloud data set. At this time, the corresponding permutation matrix can be determined based on the transformed source point cloud data set and the reference point cloud data set. Furthermore, for each data point in the source point cloud data set, the three-dimensional coordinates of the data point corresponding to the data point in the reference point cloud data set are calculated. Since not every data point in the source point cloud data set can find a corresponding data point in the reference point cloud data set, the weight can be obtained by summing, and then the covariance matrix can be calculated based on the weight of each data point obtained, and the permutation matrix is ​​singularly decomposed based on the obtained covariance matrix to obtain the rigid change matrix.

[0116] Correspondingly, the obtained rigid change matrix is ​​used as the initial rigid change matrix for iterative training until the set number of times is reached, and the trained rigid change matrix is ​​used as the registration model. At this time, based on the obtained registration model, the registration images of the left view and the top view of the sheep body and the registration images of the right view and the top view of the sheep body can be obtained respectively, so as to obtain the final three-dimensional reconstruction image of the sheep body.

[0117] Optionally, the number of iterations can be set according to actual needs, and this application does not limit it. For example, the number of iterations is set to 2 during registration model training and 5 during testing.

[0118] In an optional embodiment of the present application, determining a corresponding permutation matrix according to converting a source point cloud dataset and a reference point cloud dataset in a training dataset includes:

[0119] Extract features from the conversion source point cloud dataset and the reference point cloud dataset in the training dataset, aggregate and normalize the extracted features to obtain feature vectors;

[0120] According to the conversion source point cloud data set and the reference point cloud data set, corresponding annealing parameters and outlier parameters are obtained;

[0121] A matching matrix is ​​obtained according to the converted source point cloud data set, the reference point cloud data set, the annealing parameters and the outlier parameters, and the matching matrix is ​​normalized to obtain a permutation matrix.

[0122] Optionally, feature extraction is performed on the conversion source point cloud dataset and the reference point cloud dataset respectively. The specific feature extraction method can be: extract the features of 64 neighborhood points in the neighborhood sphere within the extraction radius of 0.3 for each data point. The features obtained at this time include the three-dimensional coordinates xyz of each data point, the distance between the data point and each neighborhood point in the neighborhood, and the PPF features of the data point and the neighborhood point. Then, the geometric features of the local area are encoded by sharing a multi-layer perceptron and a maximum pooling operation, and these features are aggregated and normalized to obtain a feature vector. Among them, for how to obtain the PPF feature, please refer to the description above, and the embodiments of the present application will not be repeated here.

[0123] Furthermore, a column of feature 0 can be added to the conversion source point cloud dataset, a column of feature 1 can be added to the reference point cloud dataset, and the features of the added column can be spliced. The spliced ​​data is processed by the MLP (multi-layer perceptron) with shared parameters, and then the maximum pooling operation is performed, and then the MLP and Softplus function activation processing are performed to obtain the annealing parameter β and the outlier parameter α respectively. Accordingly, the matching matrix is ​​calculated according to the features of the conversion source point cloud dataset and the reference point cloud dataset, the annealing parameter β and the outlier parameter α, and the matching matrix is ​​Sinkhorn normalized to become a permutation matrix.

[0124] In the embodiment of the present application, after the calibration method is used for coarse alignment, ICP is used for fine alignment. While improving the quality of training data, the traditional data preprocessing method is combined with deep learning. Deep learning is used to automatically learn the complex features in the data. When the initial value is good, the model convergence is accelerated and the accuracy of point cloud alignment is improved.

[0125] In order to better understand the sheep body 3D reconstruction method provided in the embodiment of the present application, the following is a detailed description in conjunction with a specific embodiment. Specifically, in this example, it can include three parts: data preprocessing, background segmentation, and point cloud registration. Figure 5 Provide detailed explanation.

[0126] (1) Original point cloud data of the sheep body: The original point cloud data (i.e. Figure 5), and then use four filtering methods, namely, straight-through filtering, region growing method, radius filtering, and DBSCAN, to obtain the point cloud data of the left, upper, and right parts of the target detection object sheep body (i.e., the sheep body in the figure: the target detection object sheep body is obtained by using four filtering methods, namely, straight-through filtering, region growing method, radius filtering, and DBSCAN).

[0127] (2) Background acquisition: The filtered sheep body point cloud data including the three views are respectively sent to the corresponding original point cloud data, and the corresponding background data is obtained (that is, the filtered sheep body is obtained in the original point cloud data).

[0128] (3) Sheep body identification: The last column of the sheep body point cloud data preprocessed in step (1) is filled with 1 and identified (i.e. Figure 5 In the sheep body point cloud data, the last column is filled and marked with 1).

[0129] (4) Background identification: The last column of the background point cloud data obtained in step (2) is filled with 0 and identified (i.e. Figure 5 In the figure, the last column of the background point cloud data is filled and marked with 0).

[0130] (5) Data merging: Merge the marked sheep body and background data in steps (3) and (4) along the last column to form a new point cloud file (i.e., sample point cloud data) for data training (i.e., Figure 5 The marked sheep body and background data are merged into a new point cloud file for data training).

[0131] In this process, the sheep body point cloud data is obtained using traditional geometric methods, and then the background data is obtained from the original point cloud file based on the point cloud data. Then, traditional preprocessing methods are used to remove noise and improve data quality, while reducing the manual labeling process.

[0132] (6) Train the segmentation model based on the training samples (i.e. background segmentation in the figure):

[0133] (6-1) Sampling: The farthest sampling method is modified to a random sampling method without replacement for downsampling. Each data point can only be selected once, which improves the sampling efficiency and ensures the diversity and uniformity of the sampling. The advantage of this step is that compared with the farthest sampling method, the random sampling method without replacement has high sampling efficiency and is simple to implement.

[0134] (6-2) Grouping: After the sampling points are selected, a fixed number of points (i.e., neighborhood points) in the spherical area with a radius R are selected as a group. Optionally, when sampling for the first time, the selected radius is [0.1, 0.2, 0.4], and the number of samples (i.e., neighborhood points) in the corresponding cluster is [32, 64, 128].

[0135] (6-3) Feature extraction: For each group, a small PointNet network is used to extract features, which are: sampling point xyz coordinates, neighborhood point dyz and PPF. The geometric features of the local area are encoded by establishing a multi-layer perceptron and a maximum pooling operation, and these features are aggregated into a feature vector of the sampling point. Among them, the PPF feature is the angle between the point-to-point connection line of the sampling point and the neighborhood point; the angle between the normal vector of the neighborhood point and the line connecting the sampling point pair; the angle between the normal vectors of the two points of the sampling point and the neighborhood point; the Euclidean distance between the two points of the sampling point and the neighborhood point. For details, please refer to Figure 4 shown.

[0136] It can be understood that the advantage of this step is that the PPF feature is added, which enhances the expressiveness of the local features of the sampling points, and the PPF feature combines the ideas of global modeling and local matching. This method shows high recognition performance when dealing with noise, clutter and partial occlusion.

[0137] Optionally, in this example, a downsampled point set can be obtained by downsampling and grouping twice in steps (6-1), (6-2), and (6-3), and the global features of the point cloud can be gradually extracted. These feature vectors are used for subsequent segmentation tasks.

[0138] (6-4) Decoding: The downsampled points output by the encoder are returned to the original point cloud (i.e., original point cloud data) through reverse interpolation, i.e., upsampling and feature fusion.

[0139] (6-6) Segmentation: The global features and the point feature vectors output by the decoder are sent to one or more fully connected layers (i.e., MLP) to predict the segmentation task. For each point, the output is a probability of 0 or 1, where 0 represents the background and 1 represents the sheep body. The result can be shown as follows: Figure 6 As shown in the figure, the data points in the circle are the point cloud data of the sheep body, and the others are the point cloud data of the background.

[0140] Preferably, in step (6) of training the segmentation model, the original negative log-likelihood loss function NLLLoss is modified to a cross entropy loss to calculate the classification error of each point in the output stage of the network. The advantage is that the derivative of the cross entropy loss is relatively stable, which helps to propagate effective gradients in the deep layers of the network, making the gradients in the back propagation relatively stable, which helps to optimize the convergence of the algorithm and helps the network to better learn features.

[0141] It can be understood that the traditional preprocessing method in this example can retain the key features in the point cloud data and improve the data quality. Combined with deep learning, it can automatically learn the complex features in the data, thereby improving the accuracy of point cloud segmentation and improving the robustness of the segmentation model for data under different conditions, especially when processing complex scenes and noisy data.

[0142] (7) Source point cloud X: The left view point cloud data and the right view point cloud data of the sheep body after segmentation in step (6) are defined as the source point cloud X (i.e., the source point cloud data set, the point cloud of the left or right view after segmentation in the figure).

[0143] (8) The files of the upper left view point cloud data, upper right view point cloud data, and upper left and right view point cloud data that are coarsely registered using the calibration method and finely registered using ICP are defined as reference point Y (i.e., reference point cloud data set), as follows: Figure 7 As shown, the specific steps may be as follows:

[0144] (8-1) Using the calibration box as a reference, obtain its transformation matrix, i.e., the registration parameters, and perform rough registration on the point clouds of the left, top (also called top), and right views of the sheep body after background segmentation, and perform rough registration using the top view as a reference.

[0145] (8-2) Use the ICP algorithm to precisely align the left and top, right and top, and merge the results after alignment to obtain three point cloud files: left top view point cloud data, right top view point cloud data, and left and right view reference point cloud data sets as reference point cloud Y, which is used as training data for the alignment model. Among them, right top view point cloud + top view point cloud + left view point cloud = completed point cloud image.

[0146] It can be understood that in this example, the traditional method is used to align the point cloud data as training data. After rough alignment through the calibration method, the ICP method is used for fine alignment to prevent falling into the local optimal solution and improve data quality while reducing the manual labeling process.

[0147] (9) The RPMnet algorithm is used to align the left view, top view, and right view. The training of the alignment model specifically includes the following sub-steps.

[0148] (9-1) Initial rigidity change {R i-1 , t i-1} (i.e. the transformation matrix of the previous iteration); the changed source point cloud Transform the source point cloud X by the transformation matrix of the previous iteration.

[0149] (9-2) Feature extraction and Y feature extraction: sampling point xyz coordinates, neighborhood point dyz and PPF, by establishing a multi-layer perceptron and maximum pooling operation to encode the geometric features of the local area, and aggregate these features into a feature vector for the sampling point. Among them, the PPF feature is the angle between the point-to-point connection line of the sampling point and the neighborhood point; the angle between the normal vector of the neighborhood point and the line connecting the sampling point pair; the angle between the normal vectors of the two points of the sampling point and the neighborhood point; the Euclidean distance between the two points of the sampling point and the neighborhood point. For details, please refer to Figure 4 As shown, no further details are given here.

[0150] (9-2) Parameter prediction: Add a column of feature 0 to the point cloud, add a column of feature 1 to Y, and splice along the features of the added column. The spliced ​​data is processed by MLP with shared parameters, and then the maximum pooling operation is performed. After that, it is activated by MLP and Softplus function (not shown in the figure) to obtain the annealing parameter β and the outlier parameter α.

[0151] (9-4) Calculate the matching matrix: According to and Y features, as well as β, α values ​​to calculate the matching matrix M i , and for M i Perform Sinkhorn normalization to transform it into a permutation matrix.

[0152] (9-5) Estimation of rigidity change {R i , t i}: For each point in the X set, calculate its corresponding reference point in Y, because not every x j You can find the corresponding point y j , so by summing m jk Get the weight w j , where m jk It means (x j ,y j ), k refers to the number of all possible matching points, and the singular value decomposition (SVD) is used to calculate the singular values ​​and singular vectors of the covariance matrix to obtain the transformation matrix {R i , t i}.

[0153] (9-6) The transformation matrix obtained in step (9-5) is returned to step (9-1) for a new round of iteration, where the number of iterations is 2 during model training and 5 during testing.

[0154] (9-7) Apply the final transformation matrix to obtain the registration map of the left view and the top view, and the registration map of the right view and the top view, respectively. Finally, merge the two registered point cloud files (i.e., the registered sheep body point cloud in the figure, the point cloud data of the top left view and the point cloud data of the top right view mentioned above) to obtain the final sheep body 3D reconstructed image.

[0155] In this example, after the calibration method is used for rough registration, ICP is used for fine registration. While improving the quality of training data, the traditional data preprocessing method is combined with deep learning, and deep learning is used to automatically learn the complex features in the data. With good initial values, the model convergence is accelerated to improve the accuracy of point cloud registration. In addition, after testing, the accuracy of the segmentation algorithm based on this application can reach 0.98, IoU can reach 0.955, the average conversion error of the registration algorithm is 0.01004, and the chamfer error is 0.0004339. Its conversion accuracy is much higher than that of the traditional algorithm.

[0156] The present application embodiment provides a device for three-dimensional reconstruction of a sheep body, such as Figure 8 As shown, the device 80 may include: a segmentation module 801, a registration module 802 and a training module 803, wherein:

[0157] A segmentation module is used to obtain the original point cloud data of the target sheep body and the background segmentation model, and based on the background segmentation model, the background data in the original point cloud data is segmented to obtain the sheep body point cloud data, which includes the left view point cloud data, the right view point cloud data and the top view point cloud data of the target sheep body;

[0158] A registration module is used to input the sheep body point cloud data into the registration model to obtain the upper left view point cloud data and the upper right view point cloud data of the target sheep body, and merge the upper left view point cloud data and the upper right view point cloud data to obtain a three-dimensional reconstruction image of the target sheep body;

[0159] The training module is used to train a background segmentation model based on at least one training sample, where each training sample is obtained by:

[0160] Acquire sample point cloud data, and downsample the sample point cloud data at least once using a random sampling method without playback to obtain at least one sampling point, wherein the sample point cloud data is obtained by filtering the initial sample point cloud data using four set filtering methods at the same time, and the initial sample point cloud data includes three views of the sample sheep body, and the three views include a left view, a right view, and a top view;

[0161] For each sampling point, the local feature vector of the sampling point is determined according to the three-dimensional coordinates of the sampling point, the distance between the sampling point and each neighboring data point, and the point pair features;

[0162] The training samples are obtained according to the initial sample point cloud data and the local feature vector of each sampling point.

[0163] Optionally, when acquiring sample point cloud data, the training module is specifically used to:

[0164] Acquire initial sample point cloud data, and filter the initial sample point cloud data of three views in the initial sample point cloud data using four set filtering methods at the same time to obtain target initial sample point cloud data;

[0165] According to the initial sample point cloud data and the target initial sample point cloud data, the sample background point cloud data is obtained;

[0166] Different identifiers are used to identify the last columns of the target initial sample point cloud data and the sample background point cloud data, respectively, and the identified target initial sample point cloud data and the sample background point cloud data are merged to obtain the sample point cloud data.

[0167] Optionally, four filter settings include through filtering, region growing, radius filtering, and DBSCAN.

[0168] Optionally, when the training module determines the local feature vector of the sampling point according to the three-dimensional coordinates of the sampling point, the distance between the sampling point and each neighborhood data point, and the PPF feature, it is specifically used to:

[0169] Determine at least one sampling point group corresponding to the sampling point according to the set radius, the sampling point group including neighborhood data points within a spherical area determined by the sampling point with the set radius;

[0170] For each sampling point group, the geometric features corresponding to the sampling point group are obtained through multi-layer perception and maximum pooling encoding operations according to the three-dimensional coordinates of the sampling point, the distance between the sampling point and each neighboring data point in the sampling point group, and the PPF feature;

[0171] The geometric features of at least one sampling point group are fused according to a set radius to obtain a local feature vector of the sampling point.

[0172] Optionally, the PPF features include the angle between the sampling point and the line connecting the point pair of each neighboring point, the angle between the normal vector of each neighboring point and the line connecting the sampling point pair, the angle between the sampling point and the normal vectors of two points of each neighboring point, and the Euclidean distance between the sampling point and each neighboring point.

[0173] Optionally, when the training module obtains the training sample according to the initial sample point cloud data and the local feature vector of each sampling point, it is specifically used to:

[0174] The local feature vector of each sampling point is fused into the initial sample point cloud data by upsampling to obtain the corresponding global feature vector;

[0175] The global feature vector and the local feature vector of each sampling point are concatenated to obtain the training sample.

[0176] Optionally, the background segmentation model is trained by:

[0177] Obtain at least one training sample, a labeled classification result corresponding to each data point in each training sample, and an MLP model, wherein the labeled classification result is a sheep body data point identifier or a background data point identifier;

[0178] The MLP model is trained based on each training sample until the loss function corresponding to the MLP model converges, and the MLP model at the end of the training is used as the background segmentation model;

[0179] The input of the MLP model is the training sample, the output is the classification result of the training sample, the loss function is the cross entropy loss function, and the value of the loss function is used to characterize the difference between the classification result of each training sample output by the model and the labeled classification result corresponding to each training sample.

[0180] Optionally, the registration model is trained by at least one training dataset, and the training dataset is obtained by:

[0181] Acquire registration parameters and a source point cloud data set, where the source point cloud data set includes initial training point cloud data of three views of a sample sheep body, where the three views include a left view, a right view, and a top view;

[0182] The initial training point cloud data of the three views are roughly registered according to the registration parameters to obtain the training point cloud data after the three views are roughly registered;

[0183] The training point cloud data after the rough registration of the three views are precisely registered according to the ICP algorithm to obtain a reference point cloud data set, which includes the training data of the upper left view, the training data of the upper right view, and the training data of the upper left and right views;

[0184] The source point cloud dataset and the reference point cloud dataset are used as training datasets.

[0185] Optionally, the registration model is trained by:

[0186] Acquire a training data set and an initial rigid change matrix, and transform a source point cloud data set in the training data set according to the initial rigid change matrix to obtain a transformed source point cloud data set;

[0187] Determine a corresponding permutation matrix according to the converted source point cloud data set and the reference point cloud data set in the training data set;

[0188] For each data point in the source point cloud data set, determine the weight of the data point according to the three-dimensional coordinates of the data point corresponding to the data point in the reference point cloud data set;

[0189] The covariance matrix is ​​obtained according to the weight of each data point in the reference point cloud data set, and the covariance matrix is ​​subjected to singular value decomposition of the permutation matrix to obtain the rigid change matrix;

[0190] The rigid change matrix is ​​used as the initial rigid change matrix for iterative training to a set number of times, and the trained rigid change matrix is ​​used as the registration model.

[0191] Optionally, when the training module determines the corresponding permutation matrix according to the conversion source point cloud dataset and the reference point cloud dataset in the training dataset, it is specifically used to:

[0192] Extract features from the conversion source point cloud dataset and the reference point cloud dataset in the training dataset, aggregate and normalize the extracted features to obtain feature vectors;

[0193] According to the conversion source point cloud data set and the reference point cloud data set, corresponding annealing parameters and outlier parameters are obtained;

[0194] A matching matrix is ​​obtained according to the converted source point cloud data set, the reference point cloud data set, the annealing parameters and the outlier parameters, and the matching matrix is ​​normalized to obtain a permutation matrix.

[0195] The device for three-dimensional reconstruction of a sheep body in this embodiment can execute the three-dimensional reconstruction method of a sheep body shown in the embodiment of the present application. The implementation principle is similar and will not be repeated here.

[0196] The present application embodiment provides an electronic device, such as Fig. 9 As shown, Fig. 9 The electronic device 2000 shown includes: a processor 2001 and a memory 2003. The processor 2001 and the memory 2003 are connected, such as through a bus 2002. Optionally, the electronic device 2000 may also include a transceiver 2004. It should be noted that in actual applications, the transceiver 2004 is not limited to one, and the structure of the electronic device 2000 does not constitute a limitation on the embodiments of the present application.

[0197] The processor 2001 is used in the embodiment of the present application to implement Figure 6 The functionality of the AA module is shown.

[0198] Processor 2001 may be a CPU, a general-purpose processor, a DSP, an ASIC, an FPGA or other programmable logic device, a transistor logic device, a hardware component or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. Processor 2001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0199] The bus 2002 may include a path to transmit information between the above components. The bus 2002 may be a PCI bus or an EISA bus, etc. The bus 2002 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig. 9 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0200] The memory 2003 may be a ROM or other type of static storage device that can store static information and instructions, a RAM or other type of dynamic storage device that can store information and instructions, or an EEPROM, a CD-ROM or other optical disk storage, an optical disk storage (including a compressed optical disk, a laser disk, an optical disk, a digital versatile disk, a Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to these.

[0201] The memory 2003 is used to store the application code for executing the solution of the present application, and the execution is controlled by the processor 2001. The processor 2001 is used to execute the application code stored in the memory 2003 to implement Figure 8 The illustrated embodiments provide for the operation of the apparatus.

[0202] An embodiment of the present application provides a computer-readable storage medium, which is used to store computer instructions. When the computer instructions are executed on a computer, the computer can execute a method for three-dimensional reconstruction of a sheep body.

[0203] The nouns and implementation principles involved in a computer-readable storage medium in the present application can be specifically referred to a calling method for three-dimensional reconstruction of a sheep body in an embodiment of the present application, and will not be repeated here.

[0204] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0205] The above descriptions are only some embodiments of the present invention. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A sheep body three-dimensional reconstruction method, characterized in that: include: Acquire original point cloud data and a background segmentation model of a target sheep body, and segment the background data in the original point cloud data based on the background segmentation model to obtain sheep body point cloud data, wherein the sheep body point cloud data includes left view point cloud data, right view point cloud data, and top view point cloud data of the target sheep body; Inputting the sheep body point cloud data into the registration model to obtain the upper left view point cloud data and the upper right view point cloud data of the target sheep body, and merging the upper left view point cloud data and the upper right view point cloud data to obtain a three-dimensional reconstructed image of the target sheep body; The background segmentation model is obtained by training based on at least one training sample, and each training sample is obtained by the following method: Acquire sample point cloud data, and downsample the sample point cloud data at least once using a random sampling method without playback to obtain at least one sampling point, wherein the sample point cloud data is obtained by filtering the initial sample point cloud data using four set filtering methods at the same time, and the initial sample point cloud data includes initial sample point cloud data of three views of the sample sheep body, and the three views include a left view, a right view, and a top view; For each of the sampling points, determining a local feature vector of the sampling point according to the three-dimensional coordinates of the sampling point, the distance between the sampling point and each neighborhood data point, and a point pair feature (PPF); The training sample is obtained according to the initial sample point cloud data and the local feature vector of each sampling point.

2. The method according to claim 1, characterized in that The obtaining of sample point cloud data comprises: Acquire the initial sample point cloud data, and filter the initial sample point cloud data of three views in the initial sample point cloud data using four set filtering methods respectively at the same time to obtain target initial sample point cloud data; Obtaining sample background point cloud data according to the initial sample point cloud data and the target initial sample point cloud data; The last columns of the target initial sample point cloud data and the sample background point cloud data are respectively identified by using different identifiers, and the identified target initial sample point cloud data and sample background point cloud data are merged to obtain the sample point cloud data.

3. The method according to claim 1 or 2, characterized in that: The four setting filtering methods include through filtering, region growing method, radius filtering, and density-based clustering algorithm (Density-Based Spatial Clustering of Applications with Noise, DBSCAN).

4. The method according to claim 1, characterized in that Determining the local feature vector of the sampling point according to the three-dimensional coordinates of the sampling point, the distance between the sampling point and each neighborhood data point, and the PPF feature includes: Determine at least one sampling point group corresponding to the sampling point according to a set radius, the sampling point group including neighborhood data points within a spherical area determined by the sampling point with the set radius; For each of the sampling point groups, according to the three-dimensional coordinates of the sampling point, the distance between the sampling point and each neighborhood data point in the sampling point group, and the PPF feature, a multi-layer perception and maximum pooling encoding operation is performed to obtain the geometric features corresponding to the sampling point group; The geometric features of the at least one sampling point group are fused according to the set radius to obtain a local feature vector of the sampling point.

5. The method according to claim 4, characterized in that The PPF features include the angle between the sampling point and the line connecting the point pair of each neighboring point, the angle between the normal vector of each neighboring point and the line connecting the sampling point pair, the angle between the normal vectors of the sampling point and two points of each neighboring point, and the Euclidean distance between the sampling point and each neighboring point.

6. The method according to claim 1, characterized in that The step of obtaining the training sample according to the initial sample point cloud data and the local feature vector of each sampling point comprises: The local feature vector of each sampling point is fused into the initial sample point cloud data by upsampling to obtain a corresponding global feature vector; The global feature vector and the local feature vector of each sampling point are connected in series to obtain the training sample.

7. The method according to claim 1, characterized in that The background segmentation model is trained in the following way: Obtaining at least one of the training samples, a labeled classification result corresponding to each data point in each of the training samples, and a multi-layer perceptron (MLP) model, wherein the labeled classification result is a sheep body data point identifier or a background data point identifier; The MLP model is trained based on each of the training samples until the loss function corresponding to the MLP model converges, and the MLP model at the end of the training is used as the background segmentation model; Among them, the input of the MLP model is the training sample, the output is the classification result of the training sample, the loss function is the cross entropy loss function, and the value of the loss function is used to characterize the difference between the classification result of each training sample output by the model and the labeled classification result corresponding to each training sample.

8. The method according to claim 1, characterized in that The registration model is obtained by training at least one training data set, and the training data set is obtained in the following manner: Acquire registration parameters and a source point cloud data set, wherein the source point cloud data set includes initial training point cloud data of three views of a sample sheep body, wherein the three views include a left view, a right view, and a top view; Performing coarse registration on the initial training point cloud data of the three views according to the registration parameters to obtain the coarsely registered training point cloud data of the three views; Performing fine registration on the training point cloud data after rough registration of the three views according to the ICP algorithm to obtain a reference point cloud data set, wherein the reference point cloud data set includes training data of the upper left view, training data of the upper right view, and training data of the upper left and right views; The source point cloud dataset and the reference point cloud dataset are used as the training datasets.

9. The method according to claim 8, characterized in that The registration model is trained in the following way: Acquire the training data set and an initial rigid change matrix, and transform the source point cloud data set in the training data set according to the initial rigid change matrix to obtain a transformed source point cloud data set; Determining a corresponding permutation matrix according to the conversion source point cloud dataset and the reference point cloud dataset in the training dataset; For each data point in the source point cloud data set, determining a weight of the data point according to the three-dimensional coordinates of a data point corresponding to the data point in the reference point cloud data set; Obtaining a covariance matrix according to the weight of each data point in the reference point cloud data set, and performing singular value decomposition on the covariance matrix and the permutation matrix to obtain a rigid change matrix; The rigid change matrix is ​​used as the initial rigid change matrix for iterative training to a set number of times, and the trained rigid change matrix is ​​used as the registration model.

10. The method according to claim 9, characterized in that The step of determining a corresponding permutation matrix according to the conversion source point cloud dataset and the reference point cloud dataset in the training dataset includes: Extracting features from the conversion source point cloud data set and the reference point cloud data set in the training data set, and aggregating and normalizing the extracted features to obtain a feature vector; According to the conversion source point cloud data set and the reference point cloud data set, corresponding annealing parameters and outlier parameters are obtained; A matching matrix is ​​obtained according to the conversion source point cloud data set, the reference point cloud data set, the annealing parameters and the outlier parameters, and the matching matrix is ​​normalized to obtain the permutation matrix.

Images