Multi-view sheep point cloud data processing method and device

Abstract

The invention provides a multi-view sheep point cloud data processing method and device. The multi-view sheep point cloud data processing method comprises the following steps: acquiring multi-view point cloud data of goats with different postures in a goat captivity channel; registering the multi-view point cloud data to obtain registration data, and filtering the registration data to obtain filtering data; segmenting the filtering data according to different parts of the goat to obtain a plurality of part point cloud clusters; and extracting corresponding part features from each part point cloud cluster, and performing point cloud completion on the plurality of part point cloud clusters according to the part features, the target search radius, the target sampling radius and the target sampling step length to obtain target point cloud data. According to the method provided by the invention, the goat missing point cloud can be quickly reconstructed with high precision through the distributed point cloud completion method, and the quality of goat point cloud data and the accuracy of point cloud reconstruction are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to a multi-view sheep point cloud data processing method and device. Background Art

[0002] The body shape of goats is closely related to their developmental status, physiological functions, production performance, and genetic characteristics. Therefore, accurately obtaining the body shape data of meat goats at different development stages during their growth can effectively evaluate mutton production and breeding potential.

[0003] In order to solve various problems caused by manual measurement, computer vision technology is gradually being used in the field of animal husbandry, and non-contact, automated methods of acquiring livestock body shape data are gradually being proposed.

[0004] In related technologies, scholars have proposed foreground extraction and measurement point recognition algorithms based on yak images, and used data mining to achieve automatic measurement of yak body size; or used a three-dimensional scanner to collect point cloud data and measure the body shape of sheep specimens. This method requires the sheep to remain motionless during the collection process, but in actual breeding environments, due to interference factors such as equipment, railings, dust, or missing point clouds caused by goat movement, the precise shape characteristics of the animals cannot be presented, resulting in low measurement accuracy of body shape, weight, etc., and the inability to complete the extraction and reconstruction of the complete animal point cloud. Summary of the invention

[0005] The present invention provides a multi-view sheep point cloud data processing method and device, which are used to solve the defects of the prior art in which, during the process of reconstructing goat point cloud data, the collected goat point cloud data is affected by the interference of goat movement, environment and obstructions, resulting in low quality of collected goat point cloud data and low accuracy of animal point cloud reconstruction, thereby improving the quality of goat point cloud data and the accuracy of point cloud reconstruction.

[0006] The present invention provides a multi-view sheep point cloud data processing method, comprising: Obtain multi-view point cloud data of goats in different postures in the goat enclosure channel; The multi-view point cloud data is registered to obtain registered data, and the registered data is filtered to obtain filtered data; wherein the filtering includes filtering out at least one of the following: filtering out adhesion noise between the goat and the railing, filtering out noise in non-collection areas, filtering out ground noise, railing noise, random noise, and redundant point processing; The filtered data is segmented according to different parts of the goat to obtain a plurality of part point cloud clusters; corresponding part features are extracted from each part point cloud cluster, and point cloud completion is performed on the plurality of part point cloud clusters according to the part features according to a target search radius, a target sampling radius, and a target sampling step size to obtain target point cloud data; wherein the different parts include at least two of the head, neck, trunk, limbs, and tail of the goat; and the trunk includes at least one of the chest, abdomen, back, and pelvis.

[0007] According to a multi-view point cloud data processing method for goats provided by the present invention, high-frequency ear tag readers and grating trigger sensors are respectively arranged on both sides of the door frame of the goat enclosure channel, and at least three depth cameras are arranged in the goat enclosure channel for acquiring three-dimensional images of different targets in the goat enclosure channel; The method of obtaining multi-view point cloud data of goats in different postures in the goat enclosure channel includes: When there are goats in the goat enclosure channel, control the high-frequency ear tag reader to collect ear tag data of each goat, control the grating trigger sensor to collect position data of the goat, and control the at least three depth cameras to collect multi-view depth images of the goat; When the ear mark data and the position data corresponding to the target goat are received, the at least three depth cameras are controlled to convert the multi-view depth image data of the goat into the multi-view point cloud data.

[0008] According to a multi-view point cloud data processing method for goats provided by the present invention, three depth cameras are provided in the goat enclosure channel, which are respectively used to obtain the top-down point cloud data, the left-side point cloud data and the right-side point cloud data of the goats; The registering the multi-view point cloud data to obtain the registered data comprises: Taking the top-down point cloud data as a reference, the angle to be rotated and the translation distance corresponding to the left-side point cloud data are calculated to obtain a left-side rotation matrix, and the angle to be rotated and the translation distance corresponding to the right-side point cloud data are calculated to obtain a right-side rotation matrix; The left-side perspective point cloud data is transformed according to the left-side rotation matrix, and the right-side perspective point cloud data is transformed according to the right-side rotation matrix, and the transformed left-side perspective point cloud data, the transformed right-side perspective point cloud data and the overhead perspective point cloud data are merged to obtain the registration data.

[0009] According to a multi-view sheep point cloud data processing method provided by the present invention, the filtering includes filtering out at least one of the adhesion noise between the goat and the railing, filtering out the noise in the non-collection area, filtering out the ground noise, the railing noise, the random noise and the redundant point processing; The filtering of the registration data to obtain filtered data comprises: Removing the adhesion noise points between the goat and the railing from the registration data based on the Euclidean distance to obtain the first filtered data; Filtering out noise points in non-collection areas from the first filtered data based on a straight-through filtering method to obtain second filtered data; filtering out redundant points from the second filtered data based on a voxel downsampling method to obtain third filtered data; filtering out ground noise points from the third filtered data based on a random sampling one-shot algorithm to obtain fourth filtered data; The railing noise points and random noise points are filtered out from the fourth filtered data based on the point quantity and point difference distribution diagram of the point cloud clusters of each part to obtain the filtered data.

[0010] According to a multi-view sheep point cloud data processing method provided by the present invention, after obtaining the plurality of part point cloud clusters, the method further comprises: The multiple point cloud clusters are fitted based on the logistic regression model to obtain the fitting results: The plurality of part point cloud clusters are screened according to the fitting result and a preset threshold to obtain a valid part point cloud cluster.

[0011] According to a multi-view sheep point cloud data processing method provided by the present invention, after obtaining the target point cloud data, the method further includes: The measurement accuracy is calculated according to the target point cloud data and the measured body shape data of the goat, and the three-dimensional reconstructed model of the goat is evaluated according to the measurement accuracy.

[0012] The present invention also provides a multi-view sheep point cloud data processing device, comprising: A point cloud acquisition module is used to acquire multi-view point cloud data of goats in different postures in the goat enclosure channel; A point cloud registration and filtering module, used to register the multi-view point cloud data to obtain registration data, and filter the registration data to obtain filtered data; wherein the filtering includes filtering out at least one of the following: filtering out adhesion noise between goats and railings, filtering out noise in non-collection areas, filtering out ground noise, railing noise, random noise, and redundant point processing; The point cloud completion module is used to segment the filtered data according to different parts of the goat to obtain multiple part point cloud clusters; extract corresponding part features from each part point cloud cluster, and perform point cloud completion on the multiple part point cloud clusters according to the part features according to a target search radius, a target sampling radius, and a target sampling step size to obtain target point cloud data; wherein the different parts include at least two of the head, neck, trunk, limbs, and tail of the goat; and the trunk includes at least one of the chest, abdomen, back, and pelvis.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements any of the multi-view sheep point cloud data processing methods described above.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the multi-view sheep point cloud data processing methods described above.

[0015] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the multi-view sheep point cloud data processing methods described above.

[0016] The multi-view point cloud data processing method and device provided by the present invention obtain multi-view point cloud data of goats with different postures in a goat enclosure channel; align and filter the multi-view point cloud data, and then segment the filtered data according to different parts of the goat to obtain multiple part point cloud clusters, and extract corresponding part features from each part point cloud cluster, and according to the part features, point cloud completion is performed on the multiple part point cloud clusters according to the target search radius, the target sampling radius and the target sampling step size to obtain target point cloud data. The distributed point cloud completion method can quickly and accurately reconstruct the missing point cloud of the goat, thereby improving the quality of the goat point cloud data and the accuracy of point cloud reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 This is one of the flow charts of the multi-view sheep point cloud data processing method provided by the present invention.

[0019] Figure 2It is a schematic diagram of the point cloud registration interface provided by the present invention.

[0020] Figure 3 It is a schematic diagram of the segmentation position range provided by the present invention.

[0021] Figure 4 It is a flow chart of the distributed point cloud supplementation method provided by the present invention.

[0022] Figure 5 It is a flow chart of a method for obtaining multi-view sheep point cloud data provided by the present invention.

[0023] Figure 6 Schematic diagram of five different point cloud filtering processes provided by the present invention.

[0024] Figure 7 This is the second flow chart of the multi-view sheep point cloud data processing method provided by the present invention.

[0025] Figure 8 It is a structural schematic diagram of the multi-view sheep point cloud data processing device provided by the present invention.

[0026] Fig. 9 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] Combine the following Figure 1-Figure 8 The multi-view sheep point cloud data processing method and device of the present invention are described.

[0029] Figure 1 This is one of the flow charts of the multi-view sheep point cloud data processing method provided by the present invention, such as Figure 1 As shown, the multi-view sheep point cloud data processing method includes the following steps: Step 110: Acquire multi-view point cloud data of goats in different postures in the goat enclosure channel.

[0030] In this step, the goats in different postures may be moving goats or stationary goats, and the moving goats may be running goats or walking slowly goats.

[0031] In this embodiment, a plurality of point cloud data acquisition devices may be arranged at different positions of the goat enclosure passage to obtain multi-view point cloud data of the goats.

[0032] In this embodiment, the point cloud data acquisition device includes a depth camera, a laser radar (LiDAR), a structured light scanner, a time of flight (ToF) sensor, or an ultrasonic sensor‌.

[0033] For example, a depth camera is installed on both sides of the gantry of the goat enclosure passage, and a depth camera is installed in the middle of the top beam of the gantry. A total of three depth cameras are set up to collect point cloud data of goats from a bird's-eye view and left and right side view.

[0034] Step 120, align the multi-view point cloud data to obtain aligned data, and filter the aligned data to obtain filtered data; wherein the filtering includes filtering out at least one of the adhesion noise between the goat and the railing, filtering out the noise in the non-collected area, filtering out the ground noise, the railing noise, the random noise, and the redundant point processing.

[0035] In this embodiment, multiple point cloud data sets acquired from different perspectives or times are aligned into a unified coordinate system through point cloud registration, providing data support for subsequent creation of a complete three-dimensional model, precise measurement and analysis.

[0036] In this embodiment, the registration of multi-view point cloud data can be a method of combining coarse registration with fine registration, that is, a rough registration method (such as NDT algorithm) is used to quickly estimate a rough transformation matrix without initial transformation information, so that the two sets of point cloud data are roughly aligned, and the transformation matrix is ​​further optimized through a fine registration method (such as ICP algorithm) to obtain a more accurate registration result; in addition, fine registration usually requires a good initial estimate to avoid falling into a local minimum.

[0037] Figure 2 is a schematic diagram of the point cloud registration interface provided by the present invention. Figure 2 In the illustrated embodiment, three depth cameras arranged in the goat enclosure channel are used to obtain the top-view point cloud data, left-view point cloud data and right-view point cloud data of the goats respectively; the multi-view point cloud data are registered to obtain the registered data, including: taking the top-view point cloud data as a reference, calculating the angle to be rotated and the translation distance corresponding to the left-view point cloud data to obtain the left rotation matrix, and calculating the angle to be rotated and the translation distance corresponding to the right-view point cloud data to obtain the right rotation matrix; converting the left-view point cloud data according to the left rotation matrix, converting the right-view point cloud data according to the right rotation matrix, and merging the converted left-view point cloud data, the converted right-view point cloud data and the top-view point cloud data to obtain the registered data (i.e., the merged point cloud).

[0038] It should be noted that the multi-view point cloud data of goats not only includes the point cloud data of the goats themselves, but also includes the environmental point cloud. For example, the goat enclosure channel also includes the point cloud data corresponding to the railings, the ground, the manure or the sensor bracket, as well as the noise points of each environmental point cloud. These data will affect the extraction and reconstruction of the goat point cloud.

[0039] In this embodiment, the above-mentioned filtering is used for the method of identifying and filtering environmental noise in the goat enclosure channel, specifically, filtering out adhesion noise between sheep and railings, filtering out noise in useless spatial areas, processing redundant points, identifying and filtering ground noise, and identifying and filtering railing noise and other random noise.

[0040] Step 130, segmenting the filtered data according to different parts of the goat to obtain multiple part point cloud clusters; extracting corresponding part features from each part point cloud cluster, and completing the point clouds of the multiple part point cloud clusters according to the part features according to the target search radius, the target sampling radius and the target sampling step size to obtain target point cloud data; wherein the different parts include at least two of the goat's head, neck, trunk, limbs and tail; the trunk includes at least one of the chest, abdomen, back and pelvis.

[0041] It should be noted that in the process of point cloud data collection, there are various unavoidable environmental interferences in the farm environment, such as dust, railings, hair, etc., which makes the obtained sheep point cloud data partially missing.

[0042] In this embodiment, the above-mentioned filtered data is distributedly completed through the following steps: 1. Determine the point cloud segmentation position; In order to determine the location of the sheep head, neck and limbs, multiple groups of sheep point cloud data are calculated separately by the following formula Leg cutting position Distance to the ground Distance from the sheep’s overall point cloud Ratio : ; Figure 3 is a schematic diagram of the segmentation position range provided by the present invention, Figure 3 In the embodiment shown, laboratory data research shows that the ratio of the distance from the leg cutting position to the ground to the distance of the sheep's overall point cloud is always Therefore, the leg cutting position is initially determined in the X area, and b is the theoretical leg cutting position.

[0043] In this embodiment, since the top view is used as the reference in the registration process (the shooting direction of the top camera is perpendicular to the ground), it is only necessary to traverse all the extracted sheep point clouds. , find the maximum value of its z coordinate and , ; Then, along the z-axis at intervals of 20 mm, within the X region, point cloud clusters are gradually extracted using the following formula: { }, and obtain multiple point cloud clusters: ; This example extracts 243 point cloud clusters from 44 groups of sheep point clouds, of which 44 point cloud clusters are cutting locations (ACC) and the rest are non-cutting locations (non-ACC). Then, three typical features are selected to describe the definition interval: (1) The actual cutting position ratio of the point cloud cluster , ground to cutting point cloud position Total interval with sheep The z-coordinate ratio of

[0044] ; (2) Spread density , the point cloud cluster With the previous point cloud cluster Point difference and width of cut point cloud cluster along z axis Ratio

[0045] ; (3) Points Percentage , the number of cut point cloud clusters Total point cloud points Ratio

[0046] ; In this embodiment, the three typical features mentioned above can be used to accurately distinguish between actual cutting point cloud clusters (ACC) and non-cutting point cloud clusters (non-ACC).

[0047] 2. Sheep point cloud completion; Figure 4 is a flow chart of the distributed point cloud supplementation method provided by the present invention. Figure 4In the embodiment shown, the maximum value of the y coordinate in the point cloud cluster is first selected through the actual cutting point cloud cluster (ACC), and a cutting plane h perpendicular to the y axis is formed based on the value. The sheep's head and neck can be effectively separated from the body through the cutting plane h; at the same time, the maximum value of the z coordinate in the point cloud cluster is selected, and a cutting plane l perpendicular to the z axis is formed based on the value. The body and legs can be effectively separated through the cutting plane l. Finally, the leg area L, the body area T and the head and neck H are obtained respectively; because the data point distribution characteristics of the leg, body and head and neck point clouds are different, different search radius, sampling radius and sampling step size are set respectively, and they are complemented according to their respective characteristics.

[0048] Specifically, in the point cloud cluster , for each point find its search radius The set of all points in the neighborhood of

[0049] ; Then define a new plane : ; Using the least squares method, the objective function Minimum, to calculate the corresponding , the objective function It is expressed as: ; Finally, according to the sampling radius The range is calculated by: ; Sampling step The distance is calculated by the following formula: ; The newly generated point is determined by the following formula for: .

[0050] in, m and n It is an integer, which determines the relative position of the new point on the plane, and finally completes the result of the goat. Then, voxel downsampling is used to restore the completed goat point cloud data to the previous density to complete the reconstruction of the goat point cloud.

[0051] In this embodiment, the research object of the multi-view sheep point cloud data processing method can be goats, or can be animals suitable for captivity such as cows, pigs or donkeys.

[0052] The multi-view point cloud data processing method provided by the embodiment of the present invention obtains multi-view point cloud data of goats in different postures in a goat enclosure channel; aligns and filters the multi-view point cloud data, and then divides the filtered data according to different parts of the goat to obtain multiple part point cloud clusters, and extracts corresponding part features from each part point cloud cluster, and according to the part features, point clouds are completed for the multiple part point cloud clusters according to the target search radius, target sampling radius and target sampling step size to obtain target point cloud data. The above-mentioned distributed point cloud completion method can quickly and accurately reconstruct the missing point cloud of the goat, thereby improving the quality of the goat point cloud data and the accuracy of point cloud reconstruction.

[0053] In some embodiments, a high-frequency ear tag reader and a grating trigger sensor are respectively arranged on both sides of the door frame of the goat confinement channel, and at least three depth cameras are arranged in the goat confinement channel for obtaining three-dimensional images of different targets in the goat confinement channel; obtaining multi-perspective point cloud data of goats in different postures in the goat confinement channel includes: when there are goats in the goat confinement channel, controlling the high-frequency ear tag reader to collect ear tag data of each goat, controlling the grating trigger sensor to collect position data of the goat, and controlling at least three depth cameras to collect multi-perspective depth images of the goat; when the ear tag data and position data corresponding to the target goat are received, controlling at least three depth cameras to convert the multi-perspective depth image data of the goat into multi-perspective point cloud data.

[0054] In this embodiment, a feasible goat enclosure channel is 160 cm high and 170 cm wide. The channel portal sheep body three-dimensional point cloud acquisition equipment consists of an industrial computer, a high-frequency ear tag reader, a group of opposing grating sensors and 3 Kinect DK depth cameras; wherein, the high-frequency ear tag reader is arranged on the right side of the channel portal frame to identify which mutton sheep passes through the channel; the grating trigger sensor is arranged on the left side of the channel portal frame to determine the specific position of the mutton sheep; two Kinect DKs are deployed on both sides of the equipment rack, and the last Kinect DK is placed in the middle of the top beam; the three depth cameras respectively obtain data from the left, right and top directions.

[0055] In this embodiment, after preliminary testing in a sheep breeding farm, the cables connecting various devices are deployed in the bracket, and the USB 3.0 cable is used to transmit the data to an industrial computer to avoid affecting the animals; the key parameters of the equipment are shown in Table 1: Table 1: Key parameters of equipment

[0056] In this embodiment, multi-view sheep point cloud data collection is performed in a sheep pen channel of a goat farm; specifically, 14 Boer goats and Yunshang black goats with high-frequency ear tags and ages between 6 and 12 months are selected; the ground of the sheep pen is made of concrete, and there are interferences such as dust, feces, railings and equipment; the collection method of this embodiment can use C++ developed based on the K4A library and the PCL library as the processing language.

[0057] Figure 5 is a flow chart of a method for obtaining multi-view sheep point cloud data provided by the present invention. Figure 5 In the illustrated embodiment, according to the bidirectional distributed trigger logic, the reading of the high-frequency ear tag and the grating trigger are separated. In the process of the sheep passing through the device, the high-frequency ear tag reader continuously reads the ear tag on the sheep body until it is read, and the grating is used to ensure the specific position of the sheep; when the sheep walks in the channel and at the moment of triggering the grating, the three Kinect DKs start to collect the depth image data of the sheep, and then determine whether the ear tag information is obtained. When the ear tag information and the grating signal are successfully obtained at the same time, the depth image data is converted into corresponding point cloud data through the internal parameter calculation of the camera.

[0058] The multi-perspective sheep point cloud data processing method provided by the embodiment of the present invention controls a high-frequency ear tag reader to collect ear tag data of each goat, controls a grating trigger sensor to collect position data of the goat, and controls at least three depth cameras to collect multi-perspective depth images of the goat; when the ear tag data and position data corresponding to the target goat are received, at least three depth cameras are controlled to convert the multi-perspective depth image data of the goat into multi-perspective point cloud data, which can improve the quality of the collected sheep point cloud data and improve the efficiency of point cloud data collection when the sheep pass through the channel at a fast speed and with large posture changes.

[0059] In some embodiments, filtering includes filtering out at least one of adhesion noise between goats and railings, filtering out noise in non-collected areas, filtering out ground noise, railing noise, random noise, and redundant point processing; filtering the registration data to obtain filtered data includes: removing adhesion noise between goats and railings from the registration data based on Euclidean distance; obtaining first filtered data; filtering out non-collected area noise from the first filtered data based on a straight-through filtering method to obtain second filtered data; filtering out redundant points from the second filtered data based on a voxel downsampling method to obtain third filtered data; filtering out ground noise from the third filtered data based on a random sampling one-time algorithm to obtain fourth filtered data; filtering out railing noise and random noise from the fourth filtered data based on the number of points in the point cloud clusters of each part and the point difference distribution map to obtain filtered data.

[0060] Figure 6 Schematic diagram of five different point cloud filtering processes provided by the present invention. Figure 6 In the embodiment shown in A, in order to preliminarily remove the adhesion noise between the sheep and the railing and the influence of a large number of irrelevant noise points, filtering processing is performed according to the number of adjacent points within the radius range of the data point in three-dimensional space to obtain the first filtered data.

[0061] Specifically, in the collected point cloud data P{ }, calculated by the following formula p i In radius r The number of neighbor points in :

[0062] in, express arrive The Euclidean distance of .

[0063] Next, compare and the minimum number of neighbors k, if Keep Otherwise, it is considered as an outlier and removed.

[0064] exist Figure 6 In the embodiment shown in B, in order to further remove environmental noise points in non-collection areas such as equipment and debris, a unique point cloud target area extraction method is designed based on the physical position of the equipment in the sheep farm collection environment to obtain the second filtered data.

[0065] Specifically, by analyzing the spatial distribution characteristics of environmental noise points in the data, the target area is designed as a spatial matrix, and the target point cloud in the spatial matrix is ​​calculated using the straight-through filtering algorithm through the following formula: , : ; The space matrix is ​​bounded by the railings on both sides. (including railings), the length is extended in front and behind the equipment according to the direction of the sheep passage , height from the top camera down to the extended ground .

[0066] exist Figure 6 In the embodiment shown in C, the point cloud collected by multiple depth cameras usually has a large amount of data, and because the multi-view point cloud data is collected and synthesized by cameras with multiple different viewpoints, there is a large amount of redundancy in the data points, which has a certain impact on subsequent calculations. In order to ensure the integrity of the target point cloud features and reduce the amount of data at the same time, the voxel downsampling method can be used to divide the target point cloud into cubes of equal size.

[0067] Specifically, let the size of each voxel be , the voxel index is calculated by the following formula And clustering is done using this: , , ; Assume that a voxel has d points { }, the centroid of d points is calculated by the following formula: ; The final filtered data after removing redundant points is: .

[0068] exist Figure 6 In the embodiment shown in D, since the ground noise points in the sheep farm have relatively unique plane geometric shape characteristics, a random sample consensus (RANSAC) algorithm is used to automatically identify them and determine the third filtering data.

[0069] Specifically, (1) first select 3 points at random from the target sheep body point cloud and perform plane fitting according to the following formula: ; (2) Calculation parameters , and then calculate the distance D from all points to the fitting plane by the following formula: ; (3) Set a threshold , when satisfied When the plane is fitted, set the number of iterations. ， Repeat the above steps.

[0070] exist Figure 6 In the embodiment shown in E, in order to further remove the interference noise of the railing and other random noise to obtain the fourth filtered data, the specific method is as follows: (1) Determine the location of the railing and assume that the point cloud after the ground is removed is , along the x-axis at intervals of 6 mm, the longitudinal point cloud clusters are gradually extracted by the following formula { }: ; (2) Calculate the number of data points in each point cloud cluster separately { }, and get the number of data points in each point cloud cluster.

[0071] (3) In order to preliminarily identify the L and R rail regions (RL and RR), the point cloud difference before and after the L region is calculated by the following formula: { } and R area before and after point cloud difference { }: ; .

[0072] (4) Target point cloud For analysis, due to the human error of placing rails in the middle of the equipment in different sheep farms, it is impossible to ensure that each rail is straight and standard. In order to further increase the robustness of the algorithm, the sudden drop point cloud that must appear in the L and R areas is selected. b 1 and c 2 As the initial identification point for railing removal, the sheep point cloud in the middle is retained by the following formula { }: .

[0073] In this embodiment, since the railings and other noise points are not completely removed, in order to further remove other noise points, an empty point cloud cluster set is set , for each unprocessed data point ,if If it is not in any processed collection, create a new point cloud cluster: ; Then calculate the point by the following formula To any sheep point cloud The Euclidean distance of: ; Set a distance threshold ,when , then it is believed that and Belong to the same point cloud cluster, and expand the cluster by the following formula: ; After the expansion is completed, determine whether the following formula is satisfied: ; and are the minimum and maximum number of point clouds set for the sheep point cloud. If the conditions are met, u will be added to the point cloud cluster set. , otherwise discard; then keep executing the above steps until all points are processed.

[0074] In this embodiment, the remaining railings and possible interference points are successfully removed by using Euclidean clustering extraction, and the sheep point cloud is accurately extracted.

[0075] The multi-view sheep point cloud data processing method provided by the embodiment of the present invention reduces the interference of environmental factors in sheep farm collection by filtering out the adhesion noise between goats and railings, filtering out noise in non-collection areas, filtering out ground noise, railing noise, random noise and redundant points in the registration data, thereby further improving the quality of goat point cloud data.

[0076] In some embodiments, after obtaining multiple part point cloud clusters, the method further includes: fitting the multiple part point cloud clusters based on a logistic regression model to obtain fitting results: screening the multiple part point cloud clusters according to the fitting results and a preset threshold to obtain valid part point cloud clusters.

[0077] In this embodiment, since the typical features of the actual cutting point cloud cluster (ACC) and the non-cutting point cloud cluster (non-ACC) are significantly different, but there is still overlap, the direct use of the threshold cannot effectively distinguish them. The logistic regression model can be used to identify the actual cutting point cloud cluster (ACC), and the corresponding fitting result can be expressed by the following formula: ; Then the probability P is calculated by the following formula: ; In this embodiment, the preset threshold can be set according to user needs. For example, the preset threshold is 0.24. When the value of P≥0.24, the point cloud cluster is marked as ACC, otherwise it is marked as non-ACC.

[0078] The multi-view sheep point cloud data processing method provided in the embodiment of the present invention fits multiple part point cloud clusters through a logistic regression model to obtain fitting results, and screens multiple part point cloud clusters according to the fitting results and a preset threshold to obtain valid part point cloud clusters, thereby improving the reliability of the multiple part point cloud clusters and providing data support for subsequent distributed completion of the multiple part point cloud clusters.

[0079] In some embodiments, after obtaining the target point cloud data, the method further includes: calculating the measurement accuracy according to the target point cloud data and the measured body shape data of the goat, and evaluating the three-dimensional reconstructed model of the goat according to the measurement accuracy.

[0080] This study uses sheep point cloud data collected by a depth camera, and the actual distance corresponds to the sheep point cloud coordinates. In order to verify the specific accuracy of the extracted and reconstructed sheep point cloud, 10 experimental sheep were selected, and three different postures of point cloud data were randomly selected for each sheep. The body height, body oblique length, and chest girth of the sheep were calculated by manually calibrating the measurement points, and compared with the direct measurement using a tape measure and a measuring rod in an actual breeding environment, as shown in Table 1. In Table 2 below, id represents the number of each sheep, M represents the median, L represents the manual measurement data, all in millimeters, and E represents the average error, in units of %; Table 2 Sheep posture parameter table

[0081] From the results in the table, we can see that compared with manual measurement using tools, the average error of the measurement points does not exceed 3.1%, and the absolute error of the median is within 25 mm. The results show that the environmentally adaptive point cloud extraction method and the distributed point cloud completion algorithm have a high degree of restoration, which can provide better three-dimensional data for sheep phenotypic feature extraction, non-contact body measurement and other research, reduce the stress response of the measured sheep, and reduce manual labor.

[0082] In some embodiments, in order to prove that the distributed point cloud completion algorithm can accurately locate the sheep point cloud segmentation position and then better complete the sheep point cloud data, the processing process of the algorithm is analyzed in detail as follows: (1) Sheep point cloud segmentation position analysis

[0083] For the sheep point cloud leg section I and head and neck section e, the recognition accuracy of the segmentation section l is assumed to be , the recognition accuracy of the segmentation surface e is , such as

[0084] ; ; and Respectively represent the z and y coordinates of the segmentation plane predicted by the algorithm, and They represent the coordinates of the dividing surface z and y in the ideal state, and Indicates the range of the z and y coordinates of the entire sheep point cloud; based on the experience of segmentation results, when and When the recognition accuracy is less than 0.05, the segmentation planes l and e are the segmentation positions of the legs and head and neck, respectively, indicating that the segmentation is successful. The recognition accuracy of the segmentation planes is shown in the table. From Table 3, it can be seen that in the 112 goat point clouds, only one position is recognized incorrectly. Since the l and e segmentation planes are both determined by a point cloud cluster, when the position recognition of this point cloud cluster is wrong, the two section recognitions will usually be wrong. Therefore, the recognition accuracy of the two sections is 99.1% and the average error is less than 1%. The overall accuracy is high and the average error is small, which can meet the needs of goat point cloud data collection.

[0085] In this embodiment, in a farm environment, three cameras can be used to collect sheep point clouds. However, interference and occlusion such as railings and dust cannot be avoided, and point clouds such as the torso, legs, and head may randomly have certain omissions. This embodiment uses a completion method based on the least squares method to upsample the sheep point cloud, fits the local surface, estimates the missing parameters of the surface, and then generates new sampling points.

[0086] Specifically, we first determine the segmentation position of the sheep point cloud, and then set different search radii, sampling radii and sampling steps according to the surface curvatures of the leg, torso, and head and neck point clouds for completion. This completion method can automatically complete the missing part of the point cloud data while ensuring that the phenotypic characteristics of the sheep point cloud remain basically unchanged; for the empty point cloud of the body caused by other reasons such as the occlusion of the railing or the inability to capture the bottom data, the chest and abdomen point clouds commonly used in body measurement are manually cut out as shown in the figure. It can be clearly seen that the lack of chest and abdomen point clouds will definitely affect the subsequent calculation results of other commonly used body parameters such as chest circumference and abdominal circumference. Therefore, the distributed sheep point cloud completion algorithm can be used to better complete these missing parts while being closer to the real point and without destroying the original sheep point cloud phenotypic characteristics; the error between the size of the chest circumference after point cloud completion and the true value is used as one of the evaluation indicators to quantitatively analyze the results after point cloud completion. This method not only completes the missing part of the point cloud, but also performs a certain smoothing process on the curved surface of the sheep body, removing the factors that may affect the accuracy of the collection results due to wool erection. Compared with other completion algorithms, this algorithm is faster and can meet the time requirements of actual application.

[0087] Figure 7 This is the second flow chart of the multi-view sheep point cloud data processing method provided by the present invention. Figure 7In the illustrated embodiment, a sheep body three-dimensional point cloud acquisition device is set in the goat enclosure channel, and a high-frequency ear tag reader and a grating trigger sensor are arranged according to a bidirectional distributed trigger logic, and a plurality of depth cameras are arranged at different positions to realize multi-view sheep point cloud data acquisition. After the sheep point cloud data of each view is registered, the registered data is obtained, and the adhesion noise between the goat and the railing, the noise in the non-collection area, the ground noise, the railing noise, the random noise and the redundant points are filtered out from the registered data to realize the recognition and filtering of the noise in the sheep farm environment, and the filtered data is obtained. Then, the sheep point cloud data is completed by a distributed point cloud completion algorithm (including determining the point cloud segmentation position and the sheep point cloud completion) to obtain the target point cloud data, so as to realize the extraction and reconstruction of the sheep point cloud.

[0088] The multi-view sheep point cloud data processing device provided by the present invention is described below. The multi-view sheep point cloud data processing device described below and the multi-view sheep point cloud data processing method described above can be referenced to each other.

[0089] Figure 8 : is a schematic diagram of the structure of the multi-view sheep point cloud data processing device provided by the present invention, such as Figure 8 As shown, the multi-view sheep point cloud data processing device includes: a point cloud acquisition module 810, a point cloud registration and filtering module 820 and a point cloud completion module 830.

[0090] The point cloud acquisition module 810 is used to acquire multi-view point cloud data of goats in different postures in the goat enclosure channel; The point cloud registration and filtering module 820 is used to register the multi-view point cloud data to obtain the registered data, and filter the registered data to obtain the filtered data; wherein the filtering includes filtering out at least one of the adhesion noise between the goat and the railing, filtering out the noise in the non-collection area, filtering out the ground noise, the railing noise, the random noise and the redundant point processing; The point cloud completion module 830 is used to segment the filtered data according to different parts of the goat to obtain multiple part point cloud clusters; extract corresponding part features from each part point cloud cluster, and perform point cloud completion on the multiple part point cloud clusters according to the part features according to the target search radius, target sampling radius and target sampling step size to obtain target point cloud data; wherein the different parts include at least two of the goat's head, neck, trunk, limbs and tail; the trunk includes at least one of the chest, abdomen, back and pelvis.

[0091] The multi-view point cloud data processing device provided by the embodiment of the present invention obtains multi-view point cloud data of goats in different postures in a goat enclosure channel; aligns and filters the multi-view point cloud data, and then divides the filtered data according to different parts of the goat to obtain multiple part point cloud clusters, and extracts corresponding part features from each part point cloud cluster, and according to the part features, point clouds are completed for the multiple part point cloud clusters according to the target search radius, target sampling radius and target sampling step size to obtain target point cloud data. The above-mentioned distributed point cloud completion method can quickly and accurately reconstruct the missing point cloud of the goat, thereby improving the quality of the goat point cloud data and the accuracy of point cloud reconstruction.

[0092] Fig. 9 is a schematic diagram of the structure of the electronic device provided by the present invention, such as Fig. 9 As shown, the electronic device may include: a processor (processor) 910 , a communication interface (Communications Interface) 920 , a memory (memory) 930 and a communication bus 940 , wherein the processor 910 , the communication interface 920 , and the memory 930 communicate with each other through the communication bus 940 . The processor 910 can call the logic instructions in the memory 930 to execute the multi-view point cloud data processing method of goats, which includes: obtaining multi-view point cloud data of goats in different postures in the goat enclosure channel; aligning the multi-view point cloud data to obtain alignment data, and filtering the alignment data to obtain filtered data; wherein the filtering includes filtering out at least one of adhesion noise between the goat and the railing, filtering out noise in non-collection areas, filtering out ground noise, railing noise, random noise, and redundant point processing; segmenting the filtered data according to different parts of the goat to obtain multiple part point cloud clusters; extracting corresponding part features from each part point cloud cluster, and completing the point cloud of the multiple part point cloud clusters according to the part features according to the target search radius, target sampling radius, and target sampling step size to obtain target point cloud data; wherein the different parts include at least two of the goat's head, neck, trunk, limbs, and tail; the trunk includes at least one of the chest, abdomen, back, and pelvis.

[0093] In addition, the logic instructions in the above-mentioned memory 930 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0094] On the other hand, the present invention further provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the multi-view point cloud data processing method provided by the above methods, the method comprising: obtaining multi-view point cloud data of goats in different postures in a goat enclosure channel; registering the multi-view point cloud data to obtain registered data, and filtering the registered data to obtain filtered data; wherein the filtering includes filtering out the adhesion noise between the goat and the railing, At least one of filtering out noise points in non-collection areas, filtering out ground noise points, filtering out railing noise points, filtering out random noise points, and processing redundant points is performed; segmenting the filtered data according to different parts of the goat to obtain a plurality of part point cloud clusters; extracting corresponding part features from each part point cloud cluster, and completing point clouds for the plurality of part point cloud clusters according to the part features according to a target search radius, a target sampling radius, and a target sampling step size to obtain target point cloud data; wherein the different parts include at least two of the goat's head, neck, trunk, limbs, and tail; and the trunk includes at least one of the chest, abdomen, back, and pelvis.

[0095] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented by a processor to execute the multi-view point cloud data processing method provided by the above methods when executed by the processor, the method comprising: obtaining multi-view point cloud data of goats in different postures in a goat enclosure channel; registering the multi-view point cloud data to obtain registered data, and filtering the registered data to obtain filtered data; wherein the filtering comprises filtering out at least one of adhesion noise between the goat and the railing, filtering out noise in non-collected areas, filtering out ground noise, railing noise, random noise, and redundant point processing; segmenting the filtered data according to different parts of the goat to obtain a plurality of part point cloud clusters; extracting corresponding part features from each part point cloud cluster, and completing the point cloud of the plurality of part point cloud clusters according to the part features according to the target search radius, the target sampling radius, and the target sampling step size to obtain target point cloud data; wherein the different parts include at least two of the goat's head, neck, trunk, limbs, and tail; and the trunk includes at least one of the chest, abdomen, back, and pelvis.

[0096] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0097] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0098] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-view sheep point cloud data processing method, characterized in that: include: Obtain multi-view point cloud data of goats in different postures in the goat enclosure channel; The multi-view point cloud data is registered to obtain registered data, and the registered data is filtered to obtain filtered data; wherein the filtering includes filtering out at least one of the following: filtering out adhesion noise between the goat and the railing, filtering out noise in non-collection areas, filtering out ground noise, railing noise, random noise, and redundant point processing; The filtered data is segmented according to different parts of the goat to obtain a plurality of part point cloud clusters; corresponding part features are extracted from each part point cloud cluster, and point cloud completion is performed on the plurality of part point cloud clusters according to the part features according to a target search radius, a target sampling radius, and a target sampling step size to obtain target point cloud data; wherein the different parts include at least two of the head, neck, trunk, limbs, and tail of the goat; and the trunk includes at least one of the chest, abdomen, back, and pelvis.

2. The multi-view sheep point cloud data processing method according to claim 1, characterized in that: A high-frequency ear tag reader and a grating trigger sensor are respectively arranged on both sides of the door frame of the goat enclosure passage, and at least three depth cameras are arranged in the goat enclosure passage for obtaining three-dimensional images of different targets in the goat enclosure passage; The method of obtaining multi-view point cloud data of goats in different postures in the goat enclosure channel includes: When there are goats in the goat enclosure channel, control the high-frequency ear tag reader to collect ear tag data of each goat, control the grating trigger sensor to collect position data of the goat, and control the at least three depth cameras to collect multi-view depth images of the goat; When the ear mark data and the position data corresponding to the target goat are received, the at least three depth cameras are controlled to convert the multi-view depth image data of the goat into the multi-view point cloud data.

3. The multi-view sheep point cloud data processing method according to claim 1, characterized in that: Three depth cameras are provided in the goat enclosure channel, which are respectively used to obtain the top-down point cloud data, the left-side point cloud data and the right-side point cloud data of the goat; The registering the multi-view point cloud data to obtain the registered data comprises: Taking the top-down point cloud data as a reference, the angle to be rotated and the translation distance corresponding to the left-side point cloud data are calculated to obtain a left-side rotation matrix, and the angle to be rotated and the translation distance corresponding to the right-side point cloud data are calculated to obtain a right-side rotation matrix; The left-side perspective point cloud data is transformed according to the left-side rotation matrix, and the right-side perspective point cloud data is transformed according to the right-side rotation matrix, and the transformed left-side perspective point cloud data, the transformed right-side perspective point cloud data and the overhead perspective point cloud data are merged to obtain the registration data.

4. The multi-view sheep point cloud data processing method according to claim 1, characterized in that: The filtering includes filtering out at least one of the following: adhesion noise between the goat and the railing, filtering out noise in non-collection areas, filtering out ground noise, railing noise, random noise, and redundant point processing; The filtering of the registration data to obtain filtered data comprises: Removing the adhesion noise points between the goat and the railing from the registration data based on the Euclidean distance to obtain the first filtered data; Filtering out noise points in non-collection areas from the first filtered data based on a straight-through filtering method to obtain second filtered data; filtering out redundant points from the second filtered data based on a voxel downsampling method to obtain third filtered data; filtering out ground noise points from the third filtered data based on a random sampling one-shot algorithm to obtain fourth filtered data; The railing noise points and random noise points are filtered out from the fourth filtered data based on the point quantity and point difference distribution diagram of the point cloud clusters of each part to obtain the filtered data.

5. The multi-view sheep point cloud data processing method according to claim 1, characterized in that: After obtaining the plurality of part point cloud clusters, the method further includes: The multiple point cloud clusters are fitted based on the logistic regression model to obtain the fitting results: The plurality of part point cloud clusters are screened according to the fitting result and a preset threshold to obtain a valid part point cloud cluster.

6. The multi-view sheep point cloud data processing method according to claim 1, characterized in that: After obtaining the target point cloud data, the method further includes: The measurement accuracy is calculated according to the target point cloud data and the measured body shape data of the goat, and the three-dimensional reconstructed model of the goat is evaluated according to the measurement accuracy.

7. A multi-view sheep point cloud data processing device, characterized in that: include: A point cloud acquisition module is used to acquire multi-view point cloud data of goats in different postures in the goat enclosure channel; A point cloud registration and filtering module, used to register the multi-view point cloud data to obtain registration data, and filter the registration data to obtain filtered data; wherein the filtering includes filtering out at least one of the following: filtering out adhesion noise between goats and railings, filtering out noise in non-collection areas, filtering out ground noise, railing noise, random noise, and redundant point processing; The point cloud completion module is used to segment the filtered data according to different parts of the goat to obtain multiple part point cloud clusters; extract corresponding part features from each part point cloud cluster, and perform point cloud completion on the multiple part point cloud clusters according to the part features according to a target search radius, a target sampling radius, and a target sampling step size to obtain target point cloud data; wherein the different parts include at least two of the head, neck, trunk, limbs, and tail of the goat; and the trunk includes at least one of the chest, abdomen, back, and pelvis.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the multi-view sheep point cloud data processing method as described in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the multi-view sheep point cloud data processing method as described in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the multi-view sheep point cloud data processing method as described in any one of claims 1 to 6 is implemented.

Images