Animal point cloud multi-view real-time acquisition and 3D reconstruction method, device and system

Abstract

The present invention provides a method, device and system for real-time multi-view point cloud acquisition and 3D reconstruction of animals. The method includes: by setting up depth cameras on both sides and the top side of the channel, triggering infrared grating sensing when the animal passes, automatically driving the depth camera to synchronously acquire the original point cloud data of the animal, and identifying the unique RFID tag of the animal; aligning multiple initial point cloud data to generate the overall point cloud data of the animal to be tested. The method simultaneously acquires animal point cloud data through a multi-view depth camera. Since each point cloud data is acquired synchronously, errors caused by interference caused by animal activities are avoided. The point cloud data is calculated from multi-view depth images acquired on both sides and the top of the channel, which can effectively avoid the problem of inaccurate restoration of the animal's 3D model due to local occlusion and other interference. After aligning multiple initial point cloud data, the shape characteristics of the animal can be accurately restored, which is conducive to the accurate 3D reconstruction of the animal.

Description

Technical Field

[0001] The present invention relates to the field of image recognition, and in particular to a method, device and system for real-time multi-viewing angle acquisition and 3D reconstruction of animal point clouds. Background Art

[0002] As one of the main sources of protein for humans, the demand for meat such as beef has increased in the process of socio-economic development, thus stimulating the demand for precision breeding. In fact, precision breeding relies on the successful estimation of livestock body parameters, including the size, weight and body condition score of cattle. The most common method of collecting body information is manual measurement. However, if cattle are driven and confined in a closed space, it will cause great stress to the cattle and seriously hinder the weight gain process. Recent studies have shown that animal point cloud data and depth images can be used for the segmentation and extraction of body parts. There is currently an automatic algorithm to extract measurement points such as the back and buttocks of animals. The surface point cloud of pigs can be used to automatically calculate the heart circumference, and the body parameters such as body height and chest circumference of dairy cows are calculated through the collected 3D images, while 3D image acquisition and measurement point marking need to be done manually.

[0003] Local point clouds cannot restore the precise shape features of animals, so animal 3D reconstruction shows obvious advantages. For example, a current scanning system uses a laser beam to scan the body of an animal to achieve high-precision 3D reconstruction. Although the system has high accuracy, it requires the animal to remain completely still for 5 seconds during the point cloud data collection process. This process is obviously difficult for animals to achieve, resulting in low detection accuracy in actual applications of the system. Summary of the invention

[0004] In view of the problems existing in the prior art, the present invention provides a method, device and system for real-time multi-viewpoint acquisition and 3D reconstruction of animal point clouds.

[0005] The present invention provides a method for real-time multi-viewpoint acquisition and 3D reconstruction of animal point clouds, comprising: simultaneously acquiring initial point cloud data of multiple different angles when an animal to be tested passes through by using depth cameras arranged on both sides and the top side of a channel; aligning the multiple initial point cloud data to generate overall point cloud data of the animal to be tested.

[0006] According to an embodiment of the present invention, a method for real-time multi-viewpoint acquisition and 3D reconstruction of animal point clouds is provided, wherein the initial point cloud data at multiple different angles when the animal to be tested passes through is simultaneously acquired through depth cameras arranged on both sides and the top side of a channel, including: acquiring a passing signal of the animal to be tested through a reflected grating arranged on both sides of the channel; if the passing signal is detected, acquiring the initial point cloud data at multiple different angles when the animal to be tested passes through, and acquiring a unique identifier of the passing animal through a radio frequency identification RFID reader; wherein acquiring the initial point cloud data at multiple different angles when the animal to be tested passes through includes synchronously acquiring by driving the depth camera through a synchronous trigger line.

[0007] According to an embodiment of the present invention, the method for real-time multi-viewpoint acquisition and 3D reconstruction of animal point clouds, before aligning the multiple initial point cloud data to generate the overall point cloud data of the animal to be tested, also includes: for each initial point cloud data, based on the three-dimensional vector of the initial point cloud, calculating the neighborhood of each point according to the three-dimensional Gaussian distribution, and removing abnormal points according to the size of the neighborhood.

[0008] According to an embodiment of the present invention, the method for real-time multi-perspective acquisition and 3D reconstruction of animal point clouds, before registering the multiple initial point cloud data to generate the overall point cloud data of the animal to be tested, also includes: for each initial point cloud data, extracting plane point cloud data based on the random sample consensus RANSAC algorithm to filter out the point cloud data on the ground.

[0009] According to an embodiment of the present invention, a method for real-time multi-viewpoint acquisition and 3D reconstruction of animal point clouds is described, wherein the multiple initial point cloud data are registered to generate overall point cloud data of the animal to be tested, including: for multiple point cloud data on each side, any camera point cloud data is used as the source point cloud, and the other camera point cloud data are used as the target point cloud in turn, and the Super-4PCS algorithm is used to register the source point cloud and the target point cloud based on the affine invariance of point pairs to obtain preliminary registered point cloud data on the left, right and top sides respectively; the preliminary registered point cloud data are segmented by parallel planes at a preset height from the ground, and according to the point cloud data of the part above the segmented plane, the Super-4PCS algorithm is used to perform secondary registration on the preliminary registered point cloud data in three directions based on the affine invariance of point pairs to obtain final registered point cloud data.

[0010] According to an embodiment of the present invention, the method for real-time multi-perspective acquisition and 3D reconstruction of animal point clouds, after registering the multiple initial point cloud data to generate the overall point cloud data of the animal to be tested, further includes: down-sampling the overall point cloud data using octree point cloud processing, extracting the main direction of the animal point cloud through the principal component analysis (PCA) algorithm, and performing coordinate calibration.

[0011] The present invention also provides an animal point cloud multi-perspective real-time acquisition and 3D reconstruction device, including: an acquisition module, used to simultaneously obtain initial point cloud data of multiple different angles when the animal to be tested passes through through depth cameras arranged on both sides and the top side of the channel; a processing module, used to align the multiple initial point cloud data to generate overall point cloud data of the animal to be tested.

[0012] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of any of the above-mentioned methods for multi-perspective real-time acquisition and 3D reconstruction of animal point clouds are implemented.

[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above-described methods for multi-perspective real-time acquisition and 3D reconstruction of animal point clouds.

[0014] The present invention also provides an animal point cloud multi-perspective real-time acquisition and 3D reconstruction system, comprising: a gantry, multiple depth cameras, a grating trigger sensor, an RFID reader and the above-mentioned electronic equipment; the grating trigger sensor is arranged on both sides of the gantry to obtain animal passing signals; the RFID reader is used to read the animal's unique identification; at least two depth cameras are arranged on both sides of the gantry, and at least one depth camera is arranged on the top; the depth camera is used to obtain depth images of the animal to be tested at multiple different angles when the animal passes, and convert the depth images at different angles into initial point cloud data corresponding to each depth camera.

[0015] The animal point cloud multi-perspective real-time acquisition and 3D reconstruction method, device and system provided by the present invention simultaneously acquire animal point cloud data through a multi-perspective depth camera. Since each point cloud data is acquired synchronously and instantaneously, errors caused by interference from animal activities are avoided. The point cloud data is calculated from multi-perspective depth images on both sides and the top of the channel, which can effectively avoid the problem of inaccurate restoration of the animal 3D model due to local occlusion and other interferences. After aligning multiple initial point cloud data, the shape characteristics of the animal can be accurately restored, which is conducive to the accurate 3D reconstruction of the animal. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1It is a flow chart of the method for real-time multi-viewpoint acquisition and 3D reconstruction of animal point cloud provided by the present invention;

[0018] Figure 2 This is an application scenario diagram of the animal point cloud multi-view real-time acquisition and 3D reconstruction method provided by the present invention;

[0019] Figure 3 It is a comparison diagram of image point processing results before and after point cloud filtering provided by the present invention;

[0020] Figure 4 It is a schematic diagram of environmental point cloud filtering provided by the present invention;

[0021] Figure 5 It is a schematic diagram of ground point cloud fitting and extraction provided by the present invention;

[0022] Figure 6 is a registration point relationship diagram provided by the present invention;

[0023] Figure 7 is a schematic diagram of the horizontal point cloud registration result provided by the present invention;

[0024] Figure 8 is a schematic diagram of the point cloud registration result provided by the present invention;

[0025] Fig. 9 Schematic diagram of point cloud downsampling results provided by the present invention;

[0026] Fig.10 It is a schematic diagram of point cloud acquisition results with different light intensities provided by the present invention;

[0027] Fig.11 It is a flowchart of the system execution provided by the present invention;

[0028] Fig.12 It is a structural schematic diagram of the device for real-time multi-view point cloud acquisition and 3D reconstruction provided by the present invention;

[0029] Fig.13 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] Combine the following Figure 1-Figure 13The present invention describes the method, device and system for real-time multi-view point cloud acquisition and 3D reconstruction of animals. The present invention can be applied to point cloud data acquisition and 3D reconstruction of animals, such as pigs, cows and sheep. Figure 1 : is a flow chart of the method for real-time multi-view point cloud acquisition and 3D reconstruction of animals provided by the present invention, such as Figure 1 As shown, the present invention provides a method for real-time multi-viewpoint acquisition and 3D reconstruction of animal point clouds, including:

[0032] 101. By setting up depth cameras on both sides and the top side of the channel, the initial point cloud data of multiple different angles when the animal to be tested passes through is obtained simultaneously.

[0033] Figure 2 This is an application scenario diagram of the animal point cloud multi-view real-time acquisition and 3D reconstruction method provided by the present invention, combined with Figure 2 The present invention provides a gantry point animal cloud data acquisition system that can be combined with the farm's transfer corridor. The gantry in the system is 2.30m high and 2.32m wide, and consists of a computer, an RFID reader, a group of opposing grating sensors and several Kinect DK cameras. Preferably, there are five, four depth cameras are evenly deployed on both sides of the corridor, and the last one is placed in the middle of the top beam. The RFID reader is set at the midpoint of the gantry beam. Grating trigger sensors are set on both sides of the gantry, and all devices are connected to the industrial integrated computer via USB 3.0 cables and 485 data transmission cables.

[0034] The depth camera can use the Kinect DK sensor, which is equipped with a camera with a width of 120°*120°, an imaging depth range of 0.25-2.88m, and uses laser time-of-flight (TOF) technology to collect depth images. The uncertainty of ranging is 17mm, and the single-frame exposure time is 12.8ms.

[0035] The five Kinect DK camera systems start to collect animal depth image data, which is then converted into initial point cloud data through calculation based on the internal parameters of the camera, as shown in Equation 1.

[0036]

[0037] Where x and y are point cloud coordinates, x' and y' are depth image coordinates, D is the depth value, and fx and fy are the internal parameters of the DK camera.

[0038] In one embodiment, initial point cloud data of multiple different angles when the animal to be tested passes through is simultaneously obtained by depth cameras arranged on both sides and the top side of the channel, including: collecting the passing signal of the animal to be tested through the reflected gratings arranged on both sides of the channel; if the passing signal is detected, obtaining the initial point cloud data of multiple different angles when the animal to be tested passes through, and obtaining the animal identification through the radio frequency identification RFID reader and associating it with the point cloud data in real time.

[0039] The initial point cloud data at multiple different angles when the animal to be tested passes through is obtained, including synchronous acquisition by driving the depth camera through a synchronous trigger line.

[0040] By setting up a beam grating sensor, the information of animals such as cattle passing through can be obtained, and point cloud data can be collected in time. An RFID reader is set at the midpoint of the gantry where the animals pass, which can be used to collect the identification ID of the animal to be tested, such as the ear tag number of the cattle.

[0041] 102. Register the multiple initial point cloud data to generate overall point cloud data of the animal to be tested.

[0042] For example, through the registration by algorithms such as Super-4PCS, multiple point cloud data are registered to generate the overall point cloud data of the animal to be tested. Accordingly, a pre-processing step may also be included before the registration.

[0043] The animal point cloud multi-perspective real-time acquisition and 3D reconstruction method of the present invention simultaneously acquires animal point cloud data through a multi-perspective depth camera. Since each point cloud data is acquired instantly and synchronously, errors caused by interference from animal activities are avoided. The point cloud data is calculated from multi-perspective depth images on both sides and the top of the channel, which can effectively avoid the problem of inaccurate restoration of the animal 3D model due to the lack of local point clouds. After aligning multiple initial point cloud data, the shape characteristics of the animal can be accurately restored, which is conducive to the accurate 3D reconstruction of the animal.

[0044] In one embodiment, before the registration of the multiple initial point cloud data to generate the overall point cloud data of the animal to be tested, the process further includes: for each initial point cloud data, based on the three-dimensional vector of the initial point cloud, calculating the neighborhood of each point according to the three-dimensional Gaussian distribution, and removing abnormal points according to the size of the neighborhood.

[0045] Since the acquisition accuracy of the point cloud may be affected by dust and debris, the embodiment of the present invention uses radius filtering based on Gaussian local distance distribution to filter out these interference factors. The 3D Gaussian distribution formula is as shown in formula (2).

[0046]

[0047] in, is the point cloud 3D vector, is the mean of the vector, and A is the covariance matrix of all vectors.

[0048] Each point in the animal point cloud is recorded as a 3D vector. The neighborhood of each point is analyzed and the average distance to all neighboring points is calculated. By calculating the Gaussian distribution, points outside the sensitive neighborhood are considered outliers and are filtered out. The filtering process is as follows Figure 3 As shown, the left is the point cloud before removing abnormal points.

[0049] In one embodiment, before registering the multiple initial point cloud data to generate the overall point cloud data of the animal to be tested, the method further includes: for each initial point cloud data, extracting plane point cloud data based on the RANSAC algorithm to filter out the point cloud data on the ground.

[0050] First, an environmental filtering algorithm can be constructed based on the region growing Kdtree neighborhood clustering method to reduce other interferences in the animal channel environment, such as channel railings, ground manure, and sensor brackets. Figure 4 As shown, the horizontal bar part in the figure represents interference.

[0051] Due to the close contact between the ground and the animals, the neighborhood clustering method cannot remove the ground without losing the data points of interest. In this embodiment of the present invention, the random sample consensus (RANSAC) algorithm is selected for plane extraction and filtering. The extraction results are shown in Figure 2. Figure 5 As shown, the normal vector of the fitting plane is defined as vector M.

[0052] In one embodiment, the multiple initial point cloud data are registered to generate the overall point cloud data of the animal to be tested, including: for the point cloud data of each side of the left, right and top three sides, any camera point cloud data is used as the source point cloud, and the other camera point cloud data are used as the target point cloud in turn, and the Super-4PCS algorithm is used to register the source point cloud and the target point cloud based on the affine invariance of the point pair to obtain the preliminary registered point cloud data of the left, right and top three sides; the preliminary registered point cloud data is segmented by parallel planes at a preset height from the ground, and according to the point cloud data of the part above the segmented plane, the Super-4PCS algorithm is used to perform secondary registration on the preliminary registered point cloud data in three directions based on the affine invariance of the point pair to obtain the final registered point cloud data.

[0053] In one embodiment, the preset height is smaller than the highest point cloud of the preliminary registration point cloud data on both sides.

[0054] Take two Kinect DK cameras on the left and right sides and one Kinect DK camera on the top side as an example for explanation, and the five cameras are A, B, C, D and E respectively.

[0055] The point cloud data collected by the Kinect DK camera from different angles have a small overlap, which cannot be directly completed by traditional registration methods such as iterative closest point (ICP). In order to achieve 3D reconstruction of animals based on five different perspective point clouds (A, B, C, D and E), the embodiment of the present invention implements a step-by-step point cloud registration algorithm based on super4PCS. Taking cattle as an example, the cattle point cloud registration is divided into two steps.

[0056] The point clouds collected by the depth cameras on the same side of the animal usually show a large overlap. The Super-4PCS algorithm is used for registration, and the affine invariance of the point pairs is used to achieve registration. The point cloud collected by camera B is the source point cloud (denoted as S), and the target point cloud collected by camera A is denoted as W. Each time, 4 points are randomly selected from S according to the maximization principle to ensure that the distance between the points is large but does not exceed the threshold. These four points are represented as:

[0057] B=S a ,S b ,S c ,S d (3)

[0058] Assume that line S a ,S b intersects at point e, then two independent ratios and the distance between the two points are defined as:

[0059]

[0060] d1=||s a -s b ||d2=||s c -s d || (5)

[0061] Then select point sets P and Q in the target point cloud W

[0062] P={(t i ,t j )|t i ,t j ∈W,||s i -s j ||∈[(d1-μ,d1+μ)} (6)

[0063] Q={(t i ,t j )|t i ,t j ∈W,||s i -s j ||∈[(d2-μ,d2+μ)} (7)

[0064] For every point t in W iAs the center of the sphere, draw a sphere with d2 and d1 as the radius, t i The points falling within the range (d1-μ, d1+μ) satisfy the P set condition, t i The points falling within the range (d2-μ, d2+μ) satisfy the Q set condition.

[0065] For every pair of points in P (t i ,t j ), the intersection point e is calculated from r1, that is,

[0066] e1=t i +r1(t j -t i ) (8)

[0067] For every pair of points in Q (t i ,t j ), calculate the intersection point e from r2, that is

[0068] e2=t i +r2(t j -t i ) (9)

[0069] When e1 and e2 are approximately equal to the point pair and the angles are approximately equal, in S a ,S b ,S c ,S d and t a ,t b ,t c ,t d The transformation matrix T is calculated between them. T is used to transform the point cloud S into R and calculate the registration evaluation coefficient Pr (the proportion of points whose distance between the transformed point cloud R and the target point cloud W is less than the threshold). The relationship between the registration points is as follows: Figure 6 shown.

[0070] The above process is repeated multiple times until the registration coefficient is greater than a preset threshold, such as 0.7, and then the registration is stopped and the matrix T is saved.

[0071] Since the overlap between the vertical point cloud (i.e., top side power source) and the two side point clouds is small, the registration uncertainty of super 4-PCS direct measurement is relatively large, so the point cloud overlapping area should be located before registration.

[0072] In the embodiment of the present invention, the ground plane point cloud is regarded as the origin of the z-axis, and the normal vector M is located in the positive direction of the z-axis. The cutting plane perpendicular to the normal vector at z = 0.6n (i.e., the preset height is 0.6n) is represented as U, where n is the maximum coordinate value on the z-axis of the point cloud. All point clouds between the point cloud cutting plane and the ground are filtered out, and the remaining point cloud parts are used as the basis for alignment. Finally, the super 4-PCS of the remaining area is used to achieve alignment.

[0073] The animal point cloud multi-view real-time acquisition and 3D reconstruction method of the embodiment of the present invention realizes the precise registration of multiple point clouds through the Super-4PCS algorithm. At the same time, according to the point cloud data above the plane after segmentation, the Super-4PCS algorithm is used to perform registration based on the affine invariance of the point pairs to obtain the final registered point cloud data, thereby avoiding the uncertainty problem caused by direct registration of the point cloud data on both sides and the top side.

[0074] In one embodiment, after registering the multiple initial point cloud data to generate the overall point cloud data of the animal to be tested, it also includes: downsampling the overall point cloud data using Octree point cloud processing, extracting the main direction of the animal point cloud through the PCA algorithm, and performing coordinate calibration.

[0075] To ensure uniform density of the point cloud and facilitate subsequent processing, the embodiment of the present invention uses an octree to implement point cloud downsampling. After Octree voxelizes the point cloud, only the center point of each voxel is retained, and other points in the voxel are deleted to make the sampled point cloud reach a uniform state. Then, the principal component analysis method is used to extract the covariance matrix, eigenvalues ​​and eigenvectors of the point cloud. After determining the main direction of the animal, a rotation matrix is ​​generated to reconstruct the coordinate system of the point cloud.

[0076] Based on the methods of the above embodiments, six rounds of point cloud reconstruction experiments were conducted on cows, pigs and model cows in actual breeding environments to test the accuracy and speed of 3D reconstruction.

[0077] Point cloud reconstruction results:

[0078] Select a result as an example to demonstrate the point cloud reconstruction process. The point clouds collected by two cameras on the same side of the gantry are registered, such as Figure 7 In each picture, the two pictures on the left are point clouds collected by the camera from different angles, and the middle and right pictures are the registered point clouds.

[0079] After the point cloud registration on the same side is completed, we register the point cloud collected by the camera on the top of the gantry with the point cloud registered on both sides. The final registration result is as follows: Figure 8 shown.

[0080] In order to ensure uniform density of point cloud and facilitate subsequent processing, octree is used to implement point cloud downsampling. After Octree voxelizes the point cloud, only the center point of each voxel is retained, and other points in the voxel are deleted to make the sampled point cloud reach a uniform state.

[0081] like Fig. 9 As shown, the left side is the 3D scan result of the cow, and the right side is the downsampled point cloud.

[0082] Direction Calibration:

[0083] In order to reduce the impact of animal posture differences on point cloud analysis, the coordinates of the point cloud are converted to standard directions in the reconstructed coordinate system, where the x-axis points to the front and left, parallel to the body width, the y-axis points to the direction of the animal's head, parallel to the body length, and the z-axis points to the sky, parallel to the body height. Two steps are required to complete the reconstruction:

[0084] 1) The ground normal vector (M) is defined as the z-axis.

[0085] 2) The covariance matrix, eigenvalues ​​and eigenvectors of the point cloud are extracted by principal component analysis (PCA). After determining the main orientation of the animal, a rotation matrix is ​​generated to reconstruct the coordinate system of the point cloud.

[0086] Acquisition speed test:

[0087] The system conducted multiple point cloud acquisition and reconstruction experiments in the laboratory and ranch to test the system's operating speed.

[0088] Table 1

[0089]

[0090]

[0091] Among them, T1: depth image acquisition time; T2: point cloud generation and storage time; T3: point cloud preprocessing time; T4: point cloud registration time; T5: 3D reconstruction time; T6: total time; T7: total time excluding registration; the time unit is second (s).

[0092] The acquisition times described in Table 1 represent the system efficiency when acquiring the point cloud of a moving cow. During the data processing, the most time-consuming part is the point cloud registration process, which accounts for an average of about 87% of the total time. In order to improve the efficiency of point cloud acquisition, the registration algorithm is only executed when the device is successfully installed for the first time, and then the obtained registration matrix is ​​directly used for subsequent calculations. In this case, the point cloud reconstruction time takes about 11-16 seconds each time. And T3-T5 is put into the background and run through multi-threading, and the point cloud acquisition thread only retains the T1 / T2 part, which takes about 2-4 seconds.

[0093] Point cloud accuracy test:

[0094] The reconstructed cow point cloud is manually selected, and the height and width of the cow are obtained after marking the measurement points, and the calculated data are compared with the corresponding true values. However, when measuring live cows with a tape measure or ruler, the results will inevitably have errors. In order to effectively evaluate the theoretical error, this paper only selects cow and pig models in the laboratory as objects, and the length measurement unit is millimeter (mm).

[0095] Table 2

[0096] body height Body height error Body width Body width error Average Error True 912 0 469 0 0 1 898 0.015 476 0.015 0.015 2 915 0.003 472 0.006 0.0045 3 928 0.018 481 0.026 0.022 4 916 0.004 470 0.002 0.003 5 902 0.01 454 0.032 0.021 6 906 0.007 467 0.004 0.0055

[0097] As shown in Table 2, the error of measuring point cloud data currently does not exceed 3.2%. Compared with manual measurement with a tape measure, the absolute error is within 20 mm, which can meet the requirements of practical applications.

[0098] Disturbance caused by sunlight:

[0099] The acquisition speed test results (Table 1) show that sunlight interferes with the acquisition effect. Therefore, the same cow and pig models are placed in the same position but under different lighting conditions for point cloud acquisition. In this experiment, a Kinect DK sensor is used for point cloud collection, and the DK camera is placed at a height of 40 cm facing the cow.

[0100] 357 COW point cloud data points were collected under sunlight with light intensities ranging from 3251 lux to 85,743 lux. Fig.10 A-f show examples of different collection results, respectively.

[0101] By counting the number of points, the effect of light on point cloud collection can be evaluated. If the number of points in the point cloud collected at 3251 Lux is expressed as the complete number of points C. The point cloud quality Qc is defined as

[0102] Qc=Nu / C (10)

[0103] Where Nu is the number of data points in the evaluated point cloud and C is the number of complete points.

[0104] The fitting equation between the point cloud quality index (Qc) and the illumination intensity (Ia) can be described by a quartic polynomial as

[0105] Qc=-6.96*10 -20 Ia 4 +1.42*10 -14 Ia 3 -8.74*10 -10 Ia 2 +4.94*10 -6 Ia+0.9871 (11)

[0106] In order to study the acceptable limit of light intensity, the distribution of points collected under different light intensities was statistically analyzed. 500 data points were selected as the threshold to divide the number of points and define the effective collection interval. If the number of points under a certain coordinate reaches 500, it is marked as a valid position, otherwise it is marked as an invalid position.

[0107] When the light intensity increases, the integrity of the point cloud at the position parallel to the camera (near 40 cm) does not significantly decay, but is negatively correlated with the vertical distance between the camera and the animal. The animal point cloud is divided vertically into several intervals with a unit of 1 cm. The area with more than 500 points is recorded as valid, and the collection of all valid areas is defined as the valid vertical height range.

[0108] As the light intensity increases, attenuation occurs within the effective vertical height range, which can be considered to be equivalent to the attenuation of the maximum acquisition angle and distance between the depth camera and the subject. Increasing the number of cameras is an effective way to reduce the impact of light intensity. The attenuation coefficient (An) is defined as An = 1.098*10 -10 Ia 2 -2.11*10 -5 Ia+1.112 (12)

[0109] Wherein, Ia represents the light intensity.

[0110] The An coefficient ranges from 0 to 1. By estimating the attenuation coefficient, the attenuation of the camera field of view and distance can be directly calculated.

[0111] From the above results, we can see that when exposed to strong light, the point cloud quality will be attenuated due to the reflective characteristics of animal fur and the limitations of the working principle of TOF depth camera. The attenuation caused by light intensity can be overcome by adding a small number of cameras, but it will increase the cost and maintenance burden of the vision system. Otherwise, when used outdoors in strong sunlight, an effective sunshade needs to be added to ensure the quality of animal point cloud collection.

[0112] The following is a description of the animal point cloud multi-perspective real-time acquisition and 3D reconstruction device provided by the present invention. The animal point cloud multi-perspective real-time acquisition and 3D reconstruction device described below and the animal point cloud multi-perspective real-time acquisition and 3D reconstruction method described above can be referenced to each other.

[0113] Fig.12 : is a schematic diagram of the structure of the device for real-time multi-view point cloud acquisition and 3D reconstruction provided by the present invention. Fig.12As shown, the animal point cloud multi-view real-time acquisition and 3D reconstruction device includes: an acquisition module 1201 and a processing module 1203. The acquisition module 1201 is used to simultaneously obtain initial point cloud data of multiple different angles when the animal to be tested passes through the depth camera set on both sides and the top side of the channel; the processing module 1202 is used to pre-process and align the multiple initial point cloud data to generate the overall point cloud data of the animal to be tested.

[0114] The device embodiments provided in the embodiments of the present invention are for implementing the above-mentioned method embodiments. For specific processes and detailed contents, please refer to the above-mentioned method embodiments, which will not be repeated here.

[0115] The animal point cloud multi-perspective real-time acquisition and 3D reconstruction device provided by the embodiment of the present invention simultaneously acquires animal point cloud data through a multi-perspective depth camera. Since each point cloud data is acquired synchronously, errors caused by interference from animal activities are avoided. The point cloud data is calculated from multi-perspective depth images acquired on both sides and the top of the channel, which can effectively avoid the problem of inaccurate restoration of the animal 3D model due to local occlusion and other interferences. After aligning multiple initial point cloud data, the shape characteristics of the animal can be accurately restored, which is conducive to the accurate 3D reconstruction of the animal.

[0116] Fig.13 is a schematic diagram of the structure of the electronic device provided by the present invention, such as Fig.13 As shown, the electronic device may include: a processor 1301, a communication interface 1302, a memory 1303 and a communication bus 1304, wherein the processor 1301, the communication interface 1302 and the memory 1303 communicate with each other through the communication bus 1304. The processor 1301 may call the logic instructions in the memory 1303 to execute the animal point cloud multi-view real-time acquisition and 3D reconstruction method, the method comprising: by setting the depth cameras on both sides and the top side of the channel, simultaneously obtaining the initial point cloud data of multiple different angles when the animal to be tested passes; aligning the multiple initial point cloud data to generate the overall point cloud data of the animal to be tested.

[0117] In addition, the logic instructions in the above-mentioned memory 1303 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0118] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the animal point cloud multi-perspective real-time acquisition and 3D reconstruction method provided by the above-mentioned methods. The method includes: using depth cameras set on both sides and the top side of the channel to simultaneously obtain initial point cloud data at multiple different angles when the animal to be tested passes through; aligning the multiple initial point cloud data to generate overall point cloud data of the animal to be tested.

[0119] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it is implemented to execute the animal point cloud multi-perspective real-time acquisition and 3D reconstruction method provided in the above-mentioned embodiments. The method includes: by means of depth cameras arranged on both sides and the top side of the channel, initial point cloud data of multiple different angles when the animal to be tested passes through are simultaneously acquired; the multiple initial point cloud data are aligned to generate overall point cloud data of the animal to be tested.

[0120] On the other hand, the present invention also provides an animal point cloud multi-view real-time acquisition and 3D reconstruction system, which can be combined with Figure 2 Referring to, it includes: a gantry, multiple depth cameras, a grating trigger sensor, an RFID reader and the electronic device described in the above embodiment; the grating trigger sensor is arranged on both sides of the gantry to obtain animal passing signals; the RFID reader is used to read the unique identification of the animal; at least two depth cameras are arranged on both sides of the gantry, and at least one depth camera is arranged on the top; the depth camera is used to obtain depth images of the animal to be tested at multiple different angles when the animal passes, and convert the depth images at different angles into initial point cloud data corresponding to each depth camera.

[0121] The specific role of the electronic device has been described in the above embodiment, which can be used to implement the steps in the above method embodiments. For details, please refer to the above method embodiments. The system execution process of the entire system can be referred to. Fig.11 , I will not go into details here.

[0122] The animal point cloud multi-view real-time acquisition and 3D reconstruction system of the present invention can measure the body of active cattle in the pasture. The system can realize the sensing of cattle entry, the reading of cattle numbers, and the real-time acquisition of multi-view point clouds without any pressure or interference on the cattle.

[0123] The device or system embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

[0124] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0125] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for real-time multi-viewpoint acquisition and 3D reconstruction of animal point clouds, characterized in that: include: By setting up depth cameras on both sides and the top side of the channel, the initial point cloud data of multiple different angles when the animal to be tested passes through is simultaneously obtained, wherein four depth cameras are evenly deployed on both sides of the channel and one depth camera is deployed on the top side; Registering the multiple initial point cloud data to generate overall point cloud data of the animal to be tested; The depth cameras arranged on both sides and the top side of the channel simultaneously obtain the initial point cloud data of multiple different angles when the animal to be tested passes through, including: The passing signal of the animal to be tested is collected by means of the opposing light barriers set on both sides of the channel; If the passing signal is detected, initial point cloud data of multiple different angles when the animal to be tested passes through is obtained, and a unique identification of the passing animal is obtained through a radio frequency identification RFID reader and associated with the point cloud data in real time; Among them, obtaining initial point cloud data at multiple different angles when the animal to be tested passes by includes synchronously collecting data by driving the depth camera through a synchronous trigger line; After registering the plurality of initial point cloud data to generate the overall point cloud data of the animal to be tested, the method further includes: For the overall point cloud data, after point cloud downsampling using octree point cloud processing, the principal direction of the animal point cloud is extracted through the principal component analysis (PCA) algorithm, and coordinate calibration is performed.

2. The method for real-time multi-viewpoint acquisition and 3D reconstruction of animal point clouds according to claim 1, characterized in that: Before registering the plurality of initial point cloud data to generate the overall point cloud data of the animal to be tested, the method further includes: For each initial point cloud data, based on the three-dimensional vector of the initial point cloud, the neighborhood of each point is calculated according to the three-dimensional Gaussian distribution, and abnormal points are removed according to the size of the neighborhood.

3. The method for real-time multi-viewpoint acquisition and 3D reconstruction of animal point clouds according to claim 1, characterized in that: Before registering the plurality of initial point cloud data to generate the overall point cloud data of the animal to be tested, the method further includes: For each initial point cloud data, plane point cloud data extraction is performed based on the random sample consensus RANSAC algorithm to filter out the point cloud data on the ground.

4. The method for real-time multi-viewpoint acquisition and 3D reconstruction of animal point clouds according to claim 1, characterized in that: The registering of the plurality of initial point cloud data to generate the overall point cloud data of the animal to be tested comprises: For multiple point cloud data on each side, any camera point cloud data is used as the source point cloud, and the other camera point cloud data are used as the target point cloud in turn. The Super-4PCS algorithm is used to align the source point cloud and the target point cloud based on the affine invariance of the point pair, and the preliminary registration point cloud data of the left, right and top three sides are obtained respectively; The preliminary registration point cloud data is segmented by a parallel plane at a preset height from the ground. According to the point cloud data of the part above the segmented plane, the Super-4PCS algorithm is used to perform secondary registration on the preliminary registration point cloud data in three directions based on the affine invariance of point pairs to obtain the final registration point cloud data.

5. A device for real-time multi-view point cloud acquisition and 3D reconstruction of animals, characterized in that: include: An acquisition module is used to simultaneously acquire initial point cloud data of multiple different angles when the animal to be tested passes through the passage through depth cameras arranged on both sides and the top side of the passage, wherein four depth cameras are evenly arranged on both sides of the passage and one depth camera is arranged on the top side; A processing module, used for preprocessing and registering the plurality of initial point cloud data to generate overall point cloud data of the animal to be tested; The acquisition module is specifically used for: The passing signal of the animal to be tested is collected by means of the opposing light barriers set on both sides of the channel; If the passing signal is detected, initial point cloud data of multiple different angles when the animal to be tested passes through is obtained, and a unique identification of the passing animal is obtained through a radio frequency identification RFID reader and associated with the point cloud data in real time; Among them, obtaining initial point cloud data at multiple different angles when the animal to be tested passes by includes synchronously collecting data by driving the depth camera through a synchronous trigger line; The processing module is also used for: For the overall point cloud data, after point cloud downsampling using octree point cloud processing, the principal direction of the animal point cloud is extracted through the principal component analysis (PCA) algorithm, and coordinate calibration is performed.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the animal point cloud multi-view real-time acquisition and 3D reconstruction method as described in any one of claims 1 to 4 are implemented.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for multi-view real-time acquisition and 3D reconstruction of animal point clouds as described in any one of claims 1 to 4 are implemented.

8. A system for real-time multi-view point cloud acquisition and 3D reconstruction of animals, characterized in that: include: A gantry, a plurality of depth cameras, a grating trigger sensor, an RFID reader, and the electronic device of claim 6; The grating trigger sensors are arranged on both sides of the gantry to obtain animal passing signals; The RFID reader is used to read the unique identification of the animal; At least two depth cameras are respectively arranged on both sides of the gantry, and at least one depth camera is arranged on the top; The depth camera is used to obtain depth images of the animal to be detected at multiple different angles when the animal passes by, and convert the depth images at different angles into initial point cloud data corresponding to each depth camera.

Images