Pig body three-dimensional point cloud reconstruction method and system based on non-full coverage view angle and storage medium

Abstract

The invention provides a pig body three-dimensional point cloud reconstruction method and system based on a non-full-coverage view angle and a storage medium, and the method comprises the steps: reconstructing an originally obtained non-full-coverage small-amount view angle point cloud containing a pig body through a deep learning network, and achieving the feature extraction of the point cloud and a depth map through a PigSVFNet network; then, three-dimensional point cloud reconstruction of a non-full-coverage pig body view angle is completed by using a PigSDG network; according to the invention, only a small number of non-full-coverage cameras are used for data acquisition and pig body point cloud reconstruction, so that the equipment configuration requirement is simplified, and a better and more accurate measurement result can be obtained in a smaller space; the pig body three-dimensional fine point cloud obtained through reconstruction can be widely applied to the aspects of pig body point cloud segmentation, body size measurement, volume estimation, weight estimation, grouping, seed selection and the like, and has wide application scenes and practical value.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional point cloud processing, and more specifically, to a method, system and storage medium for reconstructing a three-dimensional point cloud of a pig body based on a non-full coverage viewing angle. Background Art

[0002] In modern animal husbandry, 3D point cloud data is of great significance. Through 3D point cloud technology, the body shape and posture of livestock and poultry can be accurately captured, thus providing reliable data support for body measurement, health monitoring, weight estimation and behavior analysis. At the same time, 3D point cloud data can achieve all-round and multi-angle observation of individual animals, avoiding human errors in traditional methods and improving the accuracy and repeatability of data.

[0003] With the development of 3D vision technology, using pig point cloud to study pigs has become the mainstream method. However, the collection of complete 3D point cloud usually requires four to five perspectives, which not only requires the camera to capture the point cloud data instantly and synchronously, but also requires a very spacious collection space, which makes it difficult to promote and apply in most pig farms. Summary of the invention

[0004] In order to overcome the defects of the above-mentioned prior art that capturing a complete point cloud of a pig body requires simultaneous capture from multiple perspectives, the collection space is too large, too many cameras need to be installed, and it is time-consuming and labor-intensive, the present invention provides a method, system and storage medium for reconstructing a three-dimensional point cloud of a pig body based on a non-full coverage perspective. Only a small number of cameras with incomplete coverage are used to collect data and reconstruct the pig body point cloud, which not only simplifies the equipment configuration requirements, but also can obtain better and more accurate measurement results in a smaller space.

[0005] In order to solve the above technical problems, the technical solution of the present invention is as follows:

[0006] A method for reconstructing a pig body three-dimensional point cloud based on a non-full coverage perspective comprises the following steps:

[0007] S1: photograph the pig body from different non-full coverage perspectives to obtain initial point cloud data from at least two perspectives;

[0008] S2: performing background removal, clustering and denoising on the initial point cloud data of at least two viewing angles respectively, to obtain three-dimensional point cloud data of the pig body of at least two viewing angles;

[0009] S3: registering the three-dimensional point cloud data of the pig body in the at least two viewing angles to obtain the registered point cloud data of the pig body;

[0010] S4: inputting the registered pig point cloud data into a pre-trained PigSVFNet network, extracting the 3D global features and 2D local features of the pig, and generating pig point cloud skeleton points by combining the 3D global features and 2D local features of the pig;

[0011] S5: Input the pig body point cloud skeleton points into the pre-trained PigSDG network for fine reconstruction to obtain a reconstructed three-dimensional complete point cloud of the pig body.

[0012] Preferably, in step S1, at least two light-sensitive depth cameras with different shooting angles are used to synchronously capture one side of the pig body to obtain initial point cloud data of at least two angles with non-full coverage;

[0013] The initial point cloud data of each viewing angle includes a three-dimensional non-background point cloud M1 and a two-dimensional background point cloud M2.

[0014] Preferably, in step S2, for the initial point cloud data of each viewing angle, the following steps are performed respectively:

[0015] Calculate the shortest distance from each point in M1 to the plane where M2 is located, remove the points whose shortest distance from M1 to the plane where M2 is located is less than a preset threshold, obtain the point cloud M3 after removing the background, and complete the background removal;

[0016] Clustering the point cloud M3 after removing the background by using a density-based clustering algorithm, and selecting the cluster point of the largest category in M3 as the point cloud data M4 corresponding to the pig body;

[0017] Statistical outlier removal and radius outlier removal are performed on the point cloud data M4 corresponding to the pig body to remove noise points around M4. After denoising, the final three-dimensional point cloud data of the pig body is obtained.

[0018] Preferably, in step S3, the three-dimensional point cloud data of the pig body of the at least two viewing angles are registered using a cubic calibration object to obtain the registered point cloud data of the pig body.

[0019] Preferably, the step S4 comprises:

[0020] S4.1: projecting the registered pig body point cloud data onto the X-axis, Y-axis and Z-axis planes respectively to generate a three-dimensional vertical depth map;

[0021] S4.2: Input the registered pig point cloud data into the three-dimensional feature extractor of the PigSVFNet network to obtain the three-dimensional global feature F p ;

[0022] The three-dimensional vertical depth map is input into the two-dimensional feature extractor of the PigSVFNet network to obtain the two-dimensional local feature F v ;

[0023] S4.3: Using the attention mechanism to transform the three-dimensional global feature F p and the two-dimensional local feature F vPerform feature fusion to obtain fusion feature F V ′;

[0024] S4.4: The fusion feature F V ′ performs the maximum pooling operation and combines the pooled fusion features with the three-dimensional global features F p Perform splicing and obtain splicing feature F g ;

[0025] S4.5: The splicing feature F g Input into the encoder of the PigSVFNet network to obtain the pig body point cloud skeleton point P l .

[0026] Preferably, the step S4.3 comprises:

[0027] Feature transformation: transform the three-dimensional global feature F p Make a copy and combine it with the two-dimensional local feature F v Perform splicing to obtain the converted feature F pu , as shown below:

[0028] F pu =Concat(F p ,F v )

[0029] Among them, Concat represents the feature concatenation operation;

[0030] The transformed feature F pu Transformed into query Q, key K and value V through linear projection:

[0031] Q=W q F pu

[0032] K=W k F pu

[0033] V=W v F pu

[0034] Among them, W q , W k and W v are the first linear projection matrices of query Q, key K and value V respectively;

[0035] Introduction of position information: The perspective information VP is converted into position information through linear projection, and element-wise summed with the query Q and key K respectively to obtain the modified query Q′ and key K′, which can be expressed as:

[0036] Q′=Q+linear(VP)

[0037] K′=K+linear(VP)

[0038] Among them, linear represents the linear projection operation;

[0039] Use the modified query Q′ and key K′ to calculate the attention weight and perform feature fusion to obtain the fused feature F V ′:

[0040]

[0041] Among them, d k is the dimension of key K′.

[0042] Preferably, the step S5 comprises:

[0043] S5.1: Use the fully connected layer of the PigSDG network to transform the pig body point cloud skeleton point P l With the splicing feature F g Connect and then embed through the linear layer to get a set of point-by-point features F l ;

[0044] S5.2: Point-wise feature F l The incompleteness-aware self-attention layer fed into the PigSDG network introduces the distance vector d i And calculate the pig body point cloud skeleton point P l Incomplete information F Q ;

[0045] S5.3: The registered pig point cloud data P in Input three sequentially connected edge convolutional layers in the PigSDG network to extract a set of downsampled point-by-point features F in ;

[0046] S5.4: The F Q and F in The common input cross attention layer is used for feature exchange, and the cross attention layer outputs point-by-point features F H ; In the cross attention layer, F in As key and value vectors, the query matrix is ​​composed of F Q generate;

[0047] S5.5: F Q and F H Combine to obtain the final fine feature F out , F out Project it into high-dimensional space and upsample it to obtain the reconstructed three-dimensional complete point cloud of the pig body.

[0048] Preferably, in step S5.2, the point-by-point feature F l Recorded as N l F l The number of feature points in the Q Recorded as

[0049] The distance vector d introduced i Specifically:

[0050]

[0051] Among them, d i is the i-th feature point f i And the registered pig point cloud data P in The shortest distance between all points; y is the pig point cloud data P after registration in points within;

[0052] The calculation of the pig body point cloud skeleton point P l Incomplete information F Q Specifically:

[0053]

[0054] Among them, f i is the i-th feature point; q i is the incomplete information of the i-th point; W′ Q , W′ K and W V ′ are the second linear projection matrices of query Q, key K and value V respectively.

[0055] The present invention also provides a pig body three-dimensional point cloud reconstruction system based on a non-full coverage perspective, which uses the above-mentioned pig body three-dimensional point cloud reconstruction method based on a non-full coverage perspective, including:

[0056] Point cloud acquisition unit: used to photograph the pig body from different non-full coverage viewing angles to obtain initial point cloud data of at least two viewing angles;

[0057] Point cloud segmentation and denoising unit: used to perform background removal, clustering and denoising on the initial point cloud data of at least two viewing angles, respectively, to obtain three-dimensional point cloud data of the pig body of at least two viewing angles;

[0058] Point cloud registration unit: used for registering the three-dimensional point cloud data of the pig body from the at least two viewing angles to obtain the registered point cloud data of the pig body;

[0059] Skeleton point generation unit: used for inputting the registered pig point cloud data into the pre-trained PigSVFNet network, extracting the 3D global features and 2D local features of the pig, and generating pig point cloud skeleton points by combining the 3D global features and 2D local features of the pig;

[0060] Complete point cloud reconstruction unit: used for inputting the pig body point cloud skeleton points into the pre-trained PigSDG network for fine reconstruction to obtain a reconstructed three-dimensional complete point cloud of the pig body.

[0061] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the above method are implemented.

[0062] The present invention provides a method, system and storage medium for reconstructing a pig body three-dimensional point cloud based on a non-full coverage perspective. The present invention uses a deep learning network to reconstruct a small amount of originally acquired non-full coverage perspective point clouds containing a pig body, uses a PigSVFNet network to extract point cloud and depth map features, and then uses a PigSDG network to complete the reconstruction of the three-dimensional point cloud of the non-full coverage pig body perspective.

[0063] The present invention only uses a small number of cameras with non-full coverage to collect data and reconstruct pig body point clouds, which not only simplifies the equipment configuration requirements, but also can obtain better and more accurate measurement results in a smaller space; the three-dimensional fine point cloud of the pig body reconstructed by the present invention can be widely used in pig body point cloud segmentation, body size measurement, volume estimation, weight estimation, grouping, breeding, etc., and has a wide range of application scenarios and practical value. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 This is a flow chart of a method for reconstructing a pig body three-dimensional point cloud based on a non-full coverage perspective provided in Example 1.

[0065] Figure 2 This is a schematic diagram of photographing a pig body from two different non-fully covered viewing angles provided in Example 2.

[0066] Figure 3 This is a top view of the pig body point cloud data after non-full coverage perspective registration provided in Example 2.

[0067] Figure 4 This is a side view of the pig body point cloud data after non-full coverage perspective registration provided in Example 2.

[0068] Figure 5 This is the PigSVFNet network structure diagram provided in Example 2.

[0069] Figure 6 This is the PigSDG network structure diagram provided in Example 2.

[0070] Figure 7 This is a side view of the reconstructed three-dimensional complete point cloud of the pig body provided in Example 2.

[0071] Figure 8This is a top view of the reconstructed three-dimensional complete point cloud of the pig body provided in Example 2.

[0072] Fig. 9 This is a structural diagram of a pig body three-dimensional point cloud reconstruction system based on non-full coverage perspective provided in Example 3. DETAILED DESCRIPTION

[0073] The drawings are for illustrative purposes only and should not be construed as limiting the present patent;

[0074] In order to better illustrate the present embodiment, some parts in the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product;

[0075] It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0076] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0077] Example 1

[0078] like Figure 1 As shown, this embodiment provides a method for reconstructing a pig body three-dimensional point cloud based on a non-full coverage perspective, comprising the following steps:

[0079] S1: photograph the pig body from different non-full coverage perspectives to obtain initial point cloud data from at least two perspectives;

[0080] S2: performing background removal, clustering and denoising on the initial point cloud data of at least two viewing angles respectively, to obtain three-dimensional point cloud data of the pig body of at least two viewing angles;

[0081] S3: registering the three-dimensional point cloud data of the pig body in the at least two viewing angles to obtain the registered point cloud data of the pig body;

[0082] S4: inputting the registered pig point cloud data into a pre-trained PigSVFNet network, extracting the 3D global features and 2D local features of the pig, and generating pig point cloud skeleton points by combining the 3D global features and 2D local features of the pig;

[0083] S5: Input the pig body point cloud skeleton points into the pre-trained PigSDG network for fine reconstruction to obtain a reconstructed three-dimensional complete point cloud of the pig body.

[0084] In the specific implementation process, this embodiment takes two different perspectives on the same side of a pig body as an example, firstly photographs the pig body from two different perspectives that are not fully covered, and obtains initial point cloud data of the two perspectives;

[0085] Then, the initial point cloud data of the two perspectives are subjected to background removal, clustering and denoising respectively to obtain the three-dimensional point cloud data of the pig body of the two perspectives;

[0086] After completing the point cloud segmentation and denoising, the two-view point clouds are registered and combined. Since the network model uses two-view point clouds (such as the right view and the top view) to restore the three-view pig point cloud (such as the left view, the top view and the right view), the pig point cloud needs to be registered in two views; after registering the pig three-dimensional point cloud data of the two views, the registered pig point cloud data is obtained;

[0087] The registered pig point cloud data is input into the pre-trained PigSVFNet network to extract the 3D global features and 2D local features of the pig body, and then the 3D global features and 2D local features of the pig body are combined to generate the pig point cloud skeleton points;

[0088] Finally, the pig body point cloud skeleton points are input into the pre-trained PigSDG network for fine reconstruction to obtain the reconstructed three-dimensional complete point cloud of the pig body;

[0089] This method uses a deep learning network to reconstruct a small number of originally acquired point clouds containing incomplete coverage of the pig body, uses the PigSVFNet network to extract point cloud and depth map features, and then uses the PigSDG network to complete the reconstruction of the three-dimensional point cloud of the incomplete coverage pig body perspective;

[0090] This method only uses a small number of cameras with non-full coverage to collect data and reconstruct pig body point clouds, which not only simplifies the equipment configuration requirements, but also can obtain better and more accurate measurement results in a smaller space; the three-dimensional fine point cloud of the pig body reconstructed by this method can be widely used in pig body point cloud segmentation, body size measurement, volume estimation, weight estimation, grouping, breeding, etc., and has a wide range of application scenarios and practical value.

[0091] Example 2

[0092] This embodiment provides a method for reconstructing a pig body three-dimensional point cloud based on a non-full coverage perspective, comprising the following steps:

[0093] S1: photographing the pig body from different non-full coverage perspectives. In this embodiment, the pig body is photographed from two different perspectives on the same side of the pig body to obtain initial point cloud data from the two perspectives.

[0094] S2: performing background removal, clustering and denoising on the initial point cloud data of the two perspectives respectively, to obtain three-dimensional point cloud data of the pig body of the two perspectives;

[0095] S3: registering the pig body three-dimensional point cloud data of the two perspectives to obtain registered pig body point cloud data;

[0096] S4: inputting the registered pig point cloud data into a pre-trained PigSVFNet network, extracting the 3D global features and 2D local features of the pig, and generating pig point cloud skeleton points by combining the 3D global features and 2D local features of the pig;

[0097] S5: Input the pig body point cloud skeleton points into the pre-trained PigSDG network for fine reconstruction to obtain a reconstructed three-dimensional complete point cloud of the pig body.

[0098] In the step S1, one side of the pig body is synchronously captured using two light-sensitive depth cameras with different shooting angles to obtain initial point cloud data of the two angles;

[0099] The initial point cloud data of each viewing angle includes a three-dimensional non-background point cloud M1 and a two-dimensional background point cloud M2;

[0100] In step S2, for the initial point cloud data of each viewing angle, the following steps are performed respectively:

[0101] Calculate the shortest distance from each point in M1 to the plane where M2 is located, remove the points whose shortest distance from M1 to the plane where M2 is located is less than a preset threshold, obtain the point cloud M3 after removing the background, and complete the background removal;

[0102] Clustering the point cloud M3 after removing the background by using a density-based clustering algorithm, and selecting the cluster point of the largest category in M3 as the point cloud data M4 corresponding to the pig body;

[0103] Perform statistical outlier removal and radius outlier removal on the point cloud data M4 corresponding to the pig body, remove noise points around M4, and obtain the final three-dimensional point cloud data of the pig body after denoising;

[0104] In the step S3, the three-dimensional point cloud data of the pig body at the two perspectives are registered using a cubic calibration object to obtain the registered point cloud data of the pig body;

[0105] The step S4 comprises:

[0106] S4.1: projecting the registered pig body point cloud data onto the X-axis, Y-axis and Z-axis planes respectively to generate a three-dimensional vertical depth map;

[0107] S4.2: Input the registered pig point cloud data into the three-dimensional feature extractor of the PigSVFNet network to obtain the three-dimensional global feature F p ;

[0108] The three-dimensional vertical depth map is input into the two-dimensional feature extractor of the PigSVFNet network to obtain the two-dimensional local feature F v ;

[0109] S4.3: Using the attention mechanism to transform the three-dimensional global feature F p and the two-dimensional local feature F v Perform feature fusion to obtain fusion feature F V ′;

[0110] S4.4: The fusion feature F V ′ performs the maximum pooling operation and combines the pooled fusion features with the three-dimensional global features F p Perform splicing and obtain splicing feature F g ;

[0111] S4.5: The splicing feature F g Input into the encoder of the PigSVFNet network to obtain the pig body point cloud skeleton point P l ;

[0112] The step S4.3 comprises:

[0113] Feature transformation: transform the three-dimensional global feature F p Make a copy and combine it with the two-dimensional local feature F v Perform splicing to obtain the converted feature F pu , as shown below:

[0114] F pu =Concat(F p ,F v )

[0115] Among them, Concat represents the feature concatenation operation;

[0116] The transformed feature F pi Transformed into query Q, key K and value V through linear projection:

[0117] Q=W q F pu

[0118] K=W k F pu

[0119] V=W v F pu

[0120] Among them, W q , W k and W v are the first linear projection matrices of query Q, key K and value V respectively;

[0121] Introduction of position information: The perspective information VP is converted into position information through linear projection, and element-wise summed with the query Q and key K respectively to obtain the modified query Q′ and key K′, which can be expressed as:

[0122] Q′=Q+linear(VP)

[0123] K′=K+linear(VP)

[0124] Among them, linear represents the linear projection operation;

[0125] Use the modified query Q′ and key K′ to calculate the attention weight and perform feature fusion to obtain the fused feature F V ′:

[0126]

[0127] Among them, d k is the dimension of the key K′;

[0128] The step S5 comprises:

[0129] S5.1: Use the fully connected layer of the PigSDG network to transform the pig body point cloud skeleton point P l With the splicing feature F g Connect and then embed through the linear layer to get a set of point-by-point features F l ;

[0130] S5.2: Point-wise feature F l The incompleteness-aware self-attention layer fed into the PigSDG network introduces the distance vector d i And calculate the pig body point cloud skeleton point P l Incomplete information F Q ;

[0131] S5.3: The registered pig point cloud data P in Input three sequentially connected edge convolutional layers in the PigSDG network to extract a set of downsampled point-by-point features F in ;

[0132] S5.4: The F Q and F in The common input cross attention layer is used for feature exchange, and the cross attention layer outputs point-by-point features F H ; In the cross attention layer, F in As key and value vectors, the query matrix is ​​composed of F Q generate;

[0133] S5.5: F Q and F H Combine to obtain the final fine feature F out , F out Project to high-dimensional space and upsample to obtain the reconstructed three-dimensional complete point cloud of the pig body;

[0134] In step S5.2, the point-by-point feature F l Recorded as N l F l The number of feature points in the Q Recorded as

[0135] The distance vector d introduced i Specifically:

[0136]

[0137] Among them, d i is the i-th feature point f i And the registered pig point cloud data P in The shortest distance between all points; y is the pig point cloud data P after registration in points within;

[0138] The calculation of the pig body point cloud skeleton point P l Incomplete information F Q Specifically:

[0139]

[0140] Among them, f i is the i-th feature point; q i is the incomplete information of the i-th point; W′ Q , W′ K and W V ′ are the second linear projection matrices of query Q, key K and value V respectively.

[0141] In the specific implementation process, two light-sensitive depth cameras with different shooting angles are first used to synchronously capture the pig body, such as Figure 2 As shown, the viewing angles in this embodiment are the upper viewing angle and the right viewing angle, and the initial point cloud data of the two viewing angles are obtained;

[0142] The initial point cloud data of each viewing angle includes a three-dimensional non-background point cloud M1 and a two-dimensional background point cloud M2;

[0143] Then, the initial point cloud data of the two perspectives are subjected to background removal, clustering and denoising respectively to obtain the three-dimensional point cloud data of the pig body of the two perspectives, specifically:

[0144] Due to the disorder of the point cloud, the closest distance from each point in M1 to the plane where M2 is located is calculated. The formula is as follows:

[0145]

[0146] Eliminate the points in M1 whose closest distance to the plane where M2 is located is less than a preset threshold, obtain the point cloud M3 after removing the background, and complete the background removal; in this embodiment, if the closest distance between M1 and M2 is less than 0.04, it is considered that the point in M1 is the background point in M2, so the point with a distance greater than 0.04 is found and saved as the point cloud M3 after removing the background;

[0147] The point cloud M3 after background removal is clustered using a density-based clustering algorithm, and the cluster point with the largest category in M3 is selected as the point cloud data M4 corresponding to the pig body;

[0148] When clustering, first randomly select a point p from M3 and calculate the neighborhood of p in radius ∈, N ∈ (p)={q∈M3|dist(p,q)≤∈};

[0149]

[0150] And determine whether point p is a core point. The judgment condition is:

[0151] |N ∈ (p)|≥minPts

[0152] If it is a core point, a new cluster is formed and all density-reachable points are added to the cluster; continue to select unprocessed points and repeat the above process until all points are processed;

[0153] After clustering the point cloud, the largest cluster is selected as the pig body data M4, and then the point cloud data M4 corresponding to the pig body is statistically removed and the radius outlier is removed to remove the noise points around M4. After the denoising is completed, the final pig body three-dimensional point cloud data is obtained;

[0154] After completing the point cloud segmentation and denoising, the two-view point clouds are registered and combined. Since the network model uses two-view point clouds (such as the right view and the top view) to restore the three-view pig point cloud (such as the left view, the top view and the right view), the pig point cloud needs to be registered in two views; after registering the pig 3D point cloud data of the two views, the registered pig point cloud data is obtained, such as Figure 3 and 4 As shown, they are the top view and side view of the registered pig point cloud data;

[0155] The registered pig point cloud data is input into the pre-trained PigSVFNet network to extract the 3D global features and 2D local features of the pig body, and then the 3D global features and 2D local features of the pig body are combined to generate the pig point cloud skeleton points;

[0156] The structure of the PigSVFNet network is as follows Figure 5As shown in the figure, in this step, the pig body point cloud is divided into two parts for feature extraction. One part is the three-dimensional point cloud P in , and the other part is the three vertical depth maps V generated from the point cloud; specifically, these two parts are put into the network PigSVFNet at the same time, and feature extraction is performed through the three-dimensional feature extractor and the two-dimensional feature extractor to obtain F p and F v ;

[0157] After extracting the global feature F p and local features F v After that, we need to transform the global feature F p and local features F v The information is effectively fused to achieve a strong perception of the pig point cloud; in the fusion process, position information is introduced to understand the relationship between features at different spatial positions, so as to achieve better feature fusion;

[0158] Feature transformation, the global shape feature F p Make a copy and combine it with the view feature F v To splice:

[0159] F pu =Concat(F p ,F v )

[0160] The concatenated features are converted into query Q, key K and value V through linear projection:

[0161] Q=W q F pu

[0162] K=W k F pu

[0163] V=W v F pu

[0164] Position information is introduced, the viewpoint VP is converted into position information through linear projection, and the elements are added to the query Q and key K respectively, and then the features are further fused;

[0165] Q′=Q+linear(VP)

[0166] K′=K+linear(VP)

[0167] Use the modified query Q′ and key K′ to calculate the attention weights and perform feature fusion:

[0168]

[0169] where dk is the dimension of the key;

[0170] After feature fusion, F v Each feature in F p Under the guidance of V ′; Finally, use the maximum pooling to fused feature F V ′ is pooled, and the pooled feature F V ′ and the global shape feature F p Splice to get F g , the encoder then converts F g Converted into predicted skeleton point P l ;

[0171] P l =Encoder(Concat(MaxPool(F V ′),F b ))

[0172] Finally, the pig body point cloud skeleton points are input into the pre-trained PigSDG network for fine reconstruction to obtain the reconstructed three-dimensional complete point cloud of the pig body;

[0173] After predicting the pig body point cloud skeleton P1, it is necessary to l Perform layer-by-layer upsampling to restore the fine structure of the pig body. Considering that the pig body point cloud has its own geometric symmetry, the geometric features of the point cloud on the left can be used to restore the geometric details of the point cloud on the right.

[0174] Specifically, the structure of the PigSDG network is as follows: Figure 6 As shown in the figure, in the process of generating point cloud by PigSDG network, an incompleteness-aware self-attention layer is embedded to pay more attention to the missing point cloud on the right; first, point P1 is combined with shape feature F g Connect and then embed through a linear layer to get a set of point-by-point features These points are then fed into the incompleteness-aware self-attention layer to obtain a set of features Encoding point-by-point incomplete information;

[0175] Since the missing perspectives are often at greater distances from the partial inputs, the distance vector d is added to the formula i , better calculate the incomplete information of skeleton point cloud.

[0176]

[0177] After analyzing the pig body structure, P in Similar local features in the previous step F are used to align Q Specifically, firstin Three edge convolutional layers are used to extract a set of downsampled point-wise features F in ; Since there are long-range similar structures on the left and right sides of the pig body, cross attention is used for feature exchange; where F in As key and value vectors, the query matrix is ​​composed of F Q Generation; Cross-attention layer outputs point-wise features F H , it learns P in Similar local structures in the , integrated into the coarse shape P l For each point in

[0178] After completing the geometric symmetry analysis and similar feature alignment, F Q and F H Combined into F out , where F l It contains two sources of shape information: one is the learned shape prior knowledge, and the other is P in Similar geometric patterns found in out It is projected into a high-dimensional space and upsampled into a more refined pig shape. The final reconstructed fine pig point cloud is as follows: Figure 7 and Figure 8 As shown, they are the side view and top view ( Figure 8 The point cloud on the left is the original intact pig point cloud, i.e. the true value, and the point cloud on the right is the pig point cloud reconstructed by this method, i.e. the predicted value);

[0179] This method uses a deep learning network to reconstruct the original three-view point cloud containing the non-full coverage of the pig body, uses the PigSVFNet network to extract point cloud and depth map features, and then uses the PigSDG network to complete the reconstruction of the three-dimensional point cloud of the non-full coverage pig body perspective;

[0180] This method only uses two cameras under non-full coverage to collect data and reconstruct the pig body point cloud, which not only simplifies the equipment configuration requirements, but also can obtain better and more accurate measurement results in a smaller space; the three-dimensional fine point cloud of the pig body reconstructed by this method can be widely used in pig body point cloud segmentation, body size measurement, volume estimation, weight estimation, grouping, breeding, etc., and has a wide range of application scenarios and practical value.

[0181] Example 3

[0182] like Fig. 9 As shown, this embodiment provides a pig body three-dimensional point cloud reconstruction system based on a non-full coverage perspective, and applies a pig body three-dimensional point cloud reconstruction method based on a non-full coverage perspective described in Embodiment 1 or 2, including:

[0183] Point cloud acquisition unit 301: used to photograph the pig body from different non-full coverage viewing angles to acquire initial point cloud data of at least two viewing angles;

[0184] Point cloud segmentation and denoising unit 302: used to perform background removal, clustering and denoising on the initial point cloud data of at least two viewing angles, respectively, to obtain three-dimensional point cloud data of the pig body of at least two viewing angles;

[0185] Point cloud registration unit 303: used to register the pig body three-dimensional point cloud data of the at least two viewing angles to obtain the registered pig body point cloud data;

[0186] Skeleton point generating unit 304: used for inputting the registered pig body point cloud data into the pre-trained PigSVFNet network, extracting the 3D global features and 2D local features of the pig body, and generating pig body point cloud skeleton points by combining the 3D global features and 2D local features of the pig body;

[0187] The complete point cloud reconstruction unit 305 is used to input the pig body point cloud skeleton points into the pre-trained PigSDG network for fine reconstruction to obtain a reconstructed three-dimensional complete point cloud of the pig body.

[0188] In the specific implementation process, first, the point cloud acquisition unit 301 photographs the pig body from at least two different non-full coverage viewing angles to obtain initial point cloud data of the two viewing angles;

[0189] Then the point cloud segmentation and denoising unit 302 performs background removal, clustering and denoising on the initial point cloud data of at least two viewing angles, respectively, to obtain three-dimensional point cloud data of the pig body of two viewing angles;

[0190] After completing the point cloud segmentation and denoising, the two-view point clouds are registered and combined. Since the network model uses the two-view point clouds to restore the three-view pig point cloud, the pig point cloud needs to be registered in two views. After the point cloud registration unit 303 registers the pig three-dimensional point cloud data of at least two views, the registered pig point cloud data is obtained.

[0191] The skeleton point generating unit 304 inputs the registered pig point cloud data into the pre-trained PigSVFNet network, extracts the 3D global features and 2D local features of the pig body, and generates the pig point cloud skeleton points by combining the 3D global features and 2D local features of the pig body;

[0192] Finally, the complete point cloud reconstruction unit 305 inputs the pig body point cloud skeleton points into the pre-trained PigSDG network for fine reconstruction to obtain a reconstructed three-dimensional complete point cloud of the pig body;

[0193] This system uses a deep learning network to reconstruct the original two-view point cloud containing the pig's body under non-full coverage, uses the PigSVFNet network to extract point cloud and depth map features, and then uses the PigSDG network to complete the three-dimensional point cloud reconstruction of the non-full coverage pig's body perspective;

[0194] This system only uses a small number of cameras under non-full coverage to collect data and reconstruct pig body point clouds, which not only simplifies the equipment configuration requirements, but also can obtain better and more accurate measurement results in a smaller space; the three-dimensional fine point cloud of the pig body reconstructed by this system can be widely used in pig body point cloud segmentation, body size measurement, volume estimation, weight estimation, grouping, breeding, etc., and has a wide range of application scenarios and practical value.

[0195] The same or similar reference numerals correspond to the same or similar components;

[0196] The terms used in the drawings to describe positional relationships are only used for illustrative purposes and should not be construed as limiting this patent;

[0197] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A method for reconstructing a pig body three-dimensional point cloud based on a non-full coverage perspective, characterized in that: The following steps are involved: S1: photograph the pig body from different non-full coverage perspectives to obtain initial point cloud data from at least two perspectives; S2: performing background removal, clustering and denoising on the initial point cloud data of at least two viewing angles respectively, to obtain three-dimensional point cloud data of the pig body of at least two viewing angles; S3: registering the three-dimensional point cloud data of the pig body in the at least two viewing angles to obtain the registered point cloud data of the pig body; S4: inputting the registered pig point cloud data into a pre-trained PigSVFNet network, extracting the 3D global features and 2D local features of the pig, and generating pig point cloud skeleton points by combining the 3D global features and 2D local features of the pig; S5: Input the pig body point cloud skeleton points into the pre-trained PigSDG network for fine reconstruction to obtain a reconstructed three-dimensional complete point cloud of the pig body.

2. According to claim 1, a method for reconstructing a pig body three-dimensional point cloud based on a non-full coverage perspective is characterized in that: In the step S1, one side of the pig body is synchronously captured using at least two light-sensitive depth cameras with different shooting angles to obtain initial point cloud data of at least two angles with non-full coverage; The initial point cloud data of each viewing angle includes a three-dimensional non-background point cloud M1 and a two-dimensional background point cloud M2.

3. The method for reconstructing a pig body three-dimensional point cloud based on a non-full coverage perspective according to claim 2, characterized in that: In step S2, for the initial point cloud data of each viewing angle, the following steps are performed respectively: Calculate the shortest distance from each point in M1 to the plane where M2 is located, remove the points whose shortest distance from M1 to the plane where M2 is located is less than a preset threshold, obtain the point cloud M3 after removing the background, and complete the background removal; Clustering the point cloud M3 after removing the background by using a density-based clustering algorithm, and selecting the cluster point of the largest category in M3 as the point cloud data M4 corresponding to the pig body; Statistical outlier removal and radius outlier removal are performed on the point cloud data M4 corresponding to the pig body to remove noise points around M4. After denoising, the final three-dimensional point cloud data of the pig body is obtained.

4. The method for reconstructing a pig body three-dimensional point cloud based on a non-full coverage perspective according to claim 1, characterized in that: In the step S3, the three-dimensional point cloud data of the pig body of the at least two viewing angles are registered using a cubic calibration object to obtain the registered point cloud data of the pig body.

5. The method for reconstructing a pig body three-dimensional point cloud based on a non-full coverage perspective according to claim 1, characterized in that: The step S4 comprises: S4.1: projecting the registered pig body point cloud data onto the X-axis, Y-axis and Z-axis planes respectively to generate a three-dimensional vertical depth map; S4.2: Input the registered pig point cloud data into the three-dimensional feature extractor of the PigSVFNet network to obtain the three-dimensional global feature F p ; The three-dimensional vertical depth map is input into the two-dimensional feature extractor of the PigSVFNet network to obtain the two-dimensional local feature F v ; S4.3: Using the attention mechanism to transform the three-dimensional global feature F p and the two-dimensional local feature F v Perform feature fusion to obtain fusion feature F v ′ ; S4.4: The fusion feature F v ′ Perform the maximum pooling operation and combine the pooled fusion features with the three-dimensional global features F p Perform splicing and obtain splicing feature F g ; S4.5: The splicing feature F g Input into the encoder of the PigSVFNet network to obtain the pig body point cloud skeleton point P l .

6. The method for reconstructing a pig body three-dimensional point cloud based on a non-full coverage perspective according to claim 5, characterized in that: The step S4.3 comprises: Feature transformation: transform the three-dimensional global feature F p Make a copy and combine it with the two-dimensional local feature F v Perform splicing to obtain the converted feature F pu , as shown below: F pu =Concat(F p ,F v ) Among them, Concat represents the feature concatenation operation; The transformed feature F pu Transformed into query Q, key K and value V through linear projection: Among them, W q , W k and W v are the first linear projection matrices of query Q, key K and value V respectively; Position information introduction: The perspective information VP is converted into position information through linear projection, and then element-wise summed with the query Q and key K to obtain the modified query Q ′ and key K ′ , expressed as: Q ′ =+linear(VP) K ′ =+linear(VP) Among them, linear represents the linear projection operation; Use the modified query Q ′ and key K ′ Calculate the attention weight and perform feature fusion to obtain the fused feature F V ′ : Among them, d k Yes Key K ′ Dimension.

7. The method for reconstructing a pig body three-dimensional point cloud based on a non-full coverage perspective according to claim 5, characterized in that: The step S5 comprises: S5.1: Use the fully connected layer of the PigSDG network to transform the pig body point cloud skeleton point P l With the splicing feature F g Connect and then embed through the linear layer to get a set of point-by-point features F l ; S5.2: Point-wise feature F l The incompleteness-aware self-attention layer fed into the PigSDG network introduces the distance vector d i And calculate the pig body point cloud skeleton point P l Incomplete information F Q ; S5.3: The registered pig point cloud data P in Input three sequentially connected edge convolutional layers in the PigSDG network to extract a set of downsampled point-by-point features F in ; S5.4: The F Q and F in The common input cross attention layer is used for feature exchange, and the cross attention layer outputs point-by-point features F H ; In the cross attention layer, F in As key and value vectors, the query matrix is ​​composed of F Q generate; S5.5: F Q and F H Combine to obtain the final fine feature F out , F out Project it into high-dimensional space and upsample it to obtain the reconstructed three-dimensional complete point cloud of the pig body.

8. The method for reconstructing a pig body three-dimensional point cloud based on a non-full coverage perspective according to claim 7, characterized in that: In step S5.2, the point-by-point feature F l Recorded as N l F l The number of feature points in the Q Recorded as The distance vector d introduced i Specifically: Among them, d i is the i-th feature point f i And the registered pig point cloud data P in The shortest distance between all points; y is the pig point cloud data P after registration in points within; The calculation of the pig body point cloud skeleton point P l Incomplete information F Q Specifically: Among them, f i is the i-th feature point; q i is the incomplete information of the i-th point; W ′ Q , W ′ K and W V ′ The second linear projection matrices for query Q, key K, and value V respectively.

9. A pig body three-dimensional point cloud reconstruction system based on non-full coverage perspective, using a pig body three-dimensional point cloud reconstruction method based on non-full coverage perspective as described in any one of claims 1 to 8, characterized in that: include: Point cloud acquisition unit: used to photograph the pig body from different non-full coverage viewing angles to obtain initial point cloud data of at least two viewing angles; Point cloud segmentation and denoising unit: used to perform background removal, clustering and denoising on the initial point cloud data of at least two viewing angles, respectively, to obtain three-dimensional point cloud data of the pig body of at least two viewing angles; Point cloud registration unit: used for registering the three-dimensional point cloud data of the pig body from the at least two viewing angles to obtain the registered point cloud data of the pig body; Skeleton point generation unit: used for inputting the registered pig point cloud data into the pre-trained PigSVFNet network, extracting the 3D global features and 2D local features of the pig, and generating pig point cloud skeleton points by combining the 3D global features and 2D local features of the pig; Complete point cloud reconstruction unit: used for inputting the pig body point cloud skeleton points into the pre-trained PigSDG network for fine reconstruction to obtain a reconstructed three-dimensional complete point cloud of the pig body.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method described in any one of claims 1 to 8 are implemented.

Images