Cow non-contact weight estimation method based on neural radiation field three-dimensional reconstruction

Abstract

The invention relates to the technical field of cow body parameter measurement, in particular to a non-contact cow body weight estimation method based on neural radiation field three-dimensional reconstruction. The method comprises the following steps: after fixing a dairy cow, acquiring multi-angle video data by using video acquisition equipment; based on the multi-angle video data, training a neural radiation field to perform three-dimensional reconstruction on a scene so as to obtain a scene three-dimensional model; according to a clutter background removing algorithm, clutter background interference and discrete noise points of the scene three-dimensional model are removed, so that a cow rough point cloud surrounding body is obtained; holes of the cow rough point cloud bounding body are complemented according to a complementation algorithm to obtain a cow fine point cloud bounding body; and based on the dairy cow fine point cloud bounding body, establishing a volume-weight regression model according to a volume calculation algorithm to obtain the weight data of the dairy cow. According to the method, the problems of time-consuming and labor-consuming measurement, low measurement efficiency and relatively low measurement precision caused by a traditional cow body parameter method are solved.

Description

Technical Field

[0001] The present application relates to the technical field of dairy cow body parameter measurement, and in particular to a non-contact dairy cow weight estimation method based on three-dimensional reconstruction of nerve radiation fields. Background Art

[0002] The volume of dairy cows is an important indicator of their physiological condition and production capacity. By regularly measuring the volume of dairy cows, their growth can be monitored and possible health problems can be discovered in time so that appropriate measures can be taken to improve the production efficiency of dairy cows.

[0003] At present, most ranches still use manual labor to measure cattle weight data. The measurement process is complicated and requires close contact with cows, which poses a great safety hazard. During the manual measurement process, the measurement personnel need to contact the cows, which can easily cause stress reactions in the cows and affect their health. Stimulated cows will show stress behaviors and attack the measurement personnel. At the same time, the manual measurement method takes too long to measure, and it is difficult to carry out measurement work on large ranches.

[0004] In recent years, depth cameras have been widely used in non-contact measurement of dairy cow body parameters. However, depth cameras have the disadvantages of complex and expensive equipment, the need to drive cows to designated locations, which makes measurement time-consuming and labor-intensive, and low measurement efficiency.

[0005] On the other hand, color image 3D reconstruction algorithms are becoming increasingly mature, and various methods are emerging. Among them, the most commonly used method is the traditional machine vision stereo matching method. However, this method has problems such as low 3D model accuracy and slow 3D model generation speed. Summary of the invention

[0006] The embodiment of the present application provides a non-contact cow weight estimation method based on three-dimensional reconstruction of nerve radiation field, so as to improve the problems of time-consuming and labor-intensive measurement, low measurement efficiency and low measurement accuracy caused by traditional cow body parameter methods.

[0007] To achieve the above-mentioned purpose, an embodiment of the present application provides a non-contact weight estimation method for a dairy cow based on three-dimensional reconstruction of a neural radiation field, comprising fixing a dairy cow and collecting multi-angle video data using a video acquisition device. Based on the multi-angle video data, the neural radiation field is trained to perform three-dimensional reconstruction of the scene to obtain a three-dimensional model of the scene. The cluttered background interference and discrete noise points of the three-dimensional model of the scene are removed according to a cluttered background removal algorithm to obtain a rough point cloud bounding volume of the dairy cow. The holes of the rough point cloud bounding volume of the dairy cow are completed according to a completion algorithm to obtain a fine point cloud bounding volume of the dairy cow. Based on the fine point cloud bounding volume of the dairy cow, a volume-weight regression model is established according to a volume calculation algorithm to obtain the weight data of the dairy cow.

[0008] In one embodiment, the training neural radiation field to reconstruct a scene in three dimensions comprises: Based on the multi-angle video data, the neural radiation field network is trained to be initialized to obtain the density and color of the scene point cloud; based on the density and color of the scene point cloud, rendering rays are generated according to the postures of different views, and the world coordinate system, pixel coordinate system and camera coordinate system of the scene point cloud are transformed to obtain the information coordinate data of the scene point cloud; based on the information coordinate data of the scene point cloud, the scene point cloud is rendered according to the volume rendering algorithm and the SDF algorithm to obtain the three-dimensional model of the scene.

[0009] In one embodiment, the cluttered background removal algorithm includes filtering the point cloud of the three-dimensional model of the scene according to a through filtering algorithm to obtain a point cloud within a preset range.

[0010] In one embodiment, the algorithm for removing cluttered background further includes removing discrete noise points of the three-dimensional model of the scene according to a Euclidean clustering algorithm to obtain the rough point cloud bounding volume of the cow.

[0011] In one embodiment, the completion algorithm includes completing the holes of the rough point cloud bounding volume of the cow according to an alpha shape algorithm to obtain the fine point cloud bounding volume of the cow.

[0012] Compared with the prior art, the present invention has the following beneficial effects: 1. The method for measuring the volume of a cow based on neural radiation field can achieve non-contact measurement of the body parameters of the cow by acquiring 10-20 seconds of video data of the cow through the video acquisition device. The three-dimensional model of the scene is reconstructed by training the neural radiation field, and then the virtual image and rough point cloud model of the cow in the virtual environment are obtained through the algorithm for removing the cluttered background, thereby achieving a better reconstruction effect and a faster reconstruction speed, thereby achieving the purpose of replacing the depth camera with an industrial camera. The volume calculation algorithm is used to obtain the fine point cloud of the cow rendered by the neural radiation processing device, and calculate the volume of the cow, so as to achieve the purpose of measuring the volume of the cow; 2. In view of the shortcomings of most non-contact cow weight acquisition methods at present, which are complex and slow, this non-contact cow weight estimation method can improve the detection accuracy and speed, increase the practicality of the method, and meet the rapid response and precise measurement needs of dairy farms; 3. Most current methods for acquiring three-dimensional information of dairy cows have the disadvantages of high cost, large amount of computation and difficulty in integrating equipment. The non-contact dairy cow weight estimation method reconstructs the scene in three dimensions through the neural radiation field, while improving the reconstruction accuracy, avoiding the use of professional and expensive depth shooting equipment, and using handheld devices to shoot dairy cow images, increasing the practicality of the method, thereby meeting the measurement needs of dairy farms for batch, low cost and easy operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0014] Figure 1 A schematic flow chart of a non-contact weight estimation method for dairy cows based on three-dimensional reconstruction of neural radiation fields provided in an embodiment of the present application; Figure 2 A schematic diagram of a video acquisition device provided in an embodiment of the present application; Figure 3 A schematic diagram of a flow chart of a method for three-dimensional reconstruction of a scene based on a neural radiation field provided in an embodiment of the present application; Figure 4 A three-dimensional model diagram of a scene based on three-dimensional reconstruction of a neural radiation field provided in an embodiment of the present application; Figure 5 A schematic diagram of a flow chart of a cluttered background removal algorithm provided in an embodiment of the present application; Figure 6 The embodiment of the present application provides a three-dimensional point cloud model of the cow based on three-dimensional reconstruction of the neural radiation field.

[0015] Icons: 100- video acquisition equipment; 10- rack; 20- industrial camera. DETAILED DESCRIPTION

[0016] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0017] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0018] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application may be combined with each other.

[0019] The following is an introduction to the technical terms involved in this application.

[0020] 3D reconstruction: 3D reconstruction is the process of restoring a 3D scene from multiple 2D perspective images or depth data. NeRF achieves high-quality 3D reconstruction by training neural networks.

[0021] Neural Radiance Fields (NeRF) is a technique in the field of computer vision and computer graphics that is used for the reconstruction and rendering of 3D scenes. NeRF restores a dense representation of a 3D scene from a 2D image, and does so by learning the color and density of each point in the scene, enabling the generation of realistic 3D reconstructions.

[0022] Rendering: Rendering is the process of converting a 3D scene into a 2D image. In NeRF, rendering is used to convert the learned radiance field into an image to present a realistic 3D scene.

[0023] Point Cloud: A point cloud is a collection of three-dimensional points that usually represents the surface of an object or the three-dimensional information of a scene. NeRF restores the three-dimensional scene by learning the color and density in the point cloud.

[0024] Signed Distance Function (SDF): An SDF is a function that provides, for each point in space, the distance to the nearest surface of an object, which can be positive (outside the object) or negative (inside the object). SDFs are often used to represent and describe the geometry of three-dimensional objects.

[0025] Pass-through filtering: Pass-through filtering is a signal processing technique used to retain or pass signal components within a specific frequency range and filter out components of other frequencies. This means that only signal components within a certain frequency range will be retained, while signals in other frequency ranges will be filtered out.

[0026] Euclidean clustering is a common clustering algorithm used to divide samples in a dataset into groups with similar characteristics.

[0027] Alpha shape method: Alpha shape is a geometric structure based on parameter alpha, which is used to describe the spatial connectivity of a set of points. Alpha shape can be regarded as a polygon or multidimensional shape formed by connecting the points in the point cloud, where the degree of connectivity is controlled by the parameter alpha.

[0028] With the continuous development of technologies such as machine vision and 3D reconstruction, the automation and intelligence of agriculture and animal husbandry have also been inspired to a certain extent. However, there are few examples of the integration of high-precision cutting-edge technologies in the field of machine vision with practical agricultural applications. The requirements of agriculture and animal husbandry are increasing day by day, and the integration of technologies is imminent. Most of the existing methods for measuring cattle weight data on ranches are still to employ manual labor for measurement, which consumes a lot of manpower and material resources and does not meet animal welfare requirements. Non-contact measurement avoids the need for close contact with cows, thereby reducing the risk of cow stress response and protecting the health and safety of cows. However, the existing non-contact cow weight measurement method using active depth cameras requires the construction of complex depth acquisition facilities, which is unaffordable for small and medium-sized ranches; this facility requires driving cows to a designated location, and the measurement equipment, measurement environment, animal status, etc. are required to be high, resulting in a decrease in the actual application effect of the measurement method, and a lack of low-cost in-situ methods for generating weight data for cows. The traditional RGB motion recovery structure method algorithm takes a long time to use, and there is a lack of algorithms that can use quickly acquired cow images and quickly generate 3D models in combination with cow visualization. Neural radiation fields are known for their high-precision and high-efficiency 3D reconstruction capabilities. Compared with traditional depth cameras and other equipment, the 3D reconstruction method based on neural radiation fields can use relatively simple equipment, reducing the measurement cost, and is especially suitable for small and medium-sized farms. At present, the trend of smart agriculture is gradually emerging, and various modern technologies are being applied to agriculture and animal husbandry. Applying 3D reconstruction based on neural radiation fields to the measurement of dairy cow weight is in line with the development direction of intelligent agriculture. It not only has the advantages of high precision, non-contact, and fast measurement, but also meets the market needs of improving breeding efficiency, reducing costs, and reducing stress responses.

[0029] In view of this, this embodiment integrates computer vision, agricultural animal husbandry, machine learning, statistical analysis and other knowledge disciplines, and builds a non-contact cow weight estimation system based on three-dimensional reconstruction of neural radiation field on the basis of fully investigating the needs of smart agriculture. While improving the reconstruction accuracy, it avoids the use of professional and expensive depth shooting equipment, improves the detection speed, and increases the practicality of the non-contact cow body parameter measurement method, providing real-time and accurate data support for the physiological condition of dairy cows, promoting the rapid development of modern agriculture, and providing technical support and auxiliary tools for animal husbandry production management decisions.

[0030] See also Figure 1 , Figure 1 A schematic flow chart of a non-contact weight estimation method for dairy cows based on three-dimensional reconstruction of neural radiation fields provided in an embodiment of the present application. The method comprises: S100: Fixed cows.

[0031] Specifically, a cow collar is used to immobilize the cow.

[0032] S200: Using a video acquisition device to acquire multi-angle video data.

[0033] See also Figure 2 , Figure 2 The schematic diagram of a video acquisition device provided in an embodiment of the present application is as follows: Specifically, the video acquisition device 100 includes a frame 10 and an industrial camera 20 .

[0034] There are two industrial cameras 20, and the two industrial cameras 20 are respectively arranged at both ends of the frame 10 in the vertical direction. The industrial camera is a CCD camera, which is low in cost and can effectively reduce the construction cost of the video acquisition equipment. The operator holds the frame 10 and shoots around the cow to be tested, and obtains a color video of the cow with a duration of 10-20 seconds.

[0035] S300: Based on multi-angle video data, train the neural radiation field to perform three-dimensional reconstruction of the scene to obtain a three-dimensional model of the scene.

[0036] See also Figure 3 , Figure 3 A schematic diagram of a process for three-dimensional reconstruction of a scene based on a neural radiation field provided in an embodiment of the present application.

[0037] Specifically, the main steps of the scene 3D model based on neural radiation field 3D reconstruction are: S310: Based on the multi-angle video data, train the neural radiation field network to initialize to obtain the density and color of the scene point cloud. Specifically, the neural radiation field is recorded as a function (c, σ) = F Θ (x, d), where F Θ It is a fully connected network, x is the three-dimensional spatial coordinate to be analyzed, d is the two-dimensional viewing direction, c is the three-channel color output at the x position, and σ is the volume density at the x position; the neural radiation field uses a multilayer perceptron (MLP) to reconstruct the three-dimensional scene and fit the color distribution of the spatial point and the function of the light density distribution.

[0038] S320: Based on the density and color of the scene point cloud, rendering rays are generated according to the postures of different views, and the world coordinate system, pixel coordinate system and camera coordinate system of the scene point cloud are transformed to obtain information coordinate data of the scene point cloud.

[0039] S330: Based on the information coordinate data of the scene point cloud, the scene point cloud is rendered according to the volume rendering algorithm and the SDF algorithm to obtain a three-dimensional model of the scene. Specifically, the volume rendering introduces the SDF algorithm, which corresponds to the volume density in the traditional volume rendering formula, and obtains better surface geometry quality.

[0040] After the above steps, we finally get the 3D model of the scene. Figure 4 , Figure 4 A three-dimensional model diagram of a scene based on three-dimensional reconstruction of a neural radiation field provided in an embodiment of the present application.

[0041] S400: removing cluttered background interference and discrete noise points of the scene three-dimensional model according to a cluttered background removal algorithm to obtain a rough point cloud bounding volume of the cow.

[0042] Specifically, see Figure 5 , Figure 5 A flow chart of a clutter removal algorithm provided in an embodiment of the present application. The clutter removal algorithm includes the following steps: S410: Filter the point cloud of the three-dimensional model of the scene according to the straight-through filtering algorithm to obtain a point cloud within a preset range. Specifically, the straight-through filtering method is used to set an expected range on the attributes of the point according to the attributes of the point cloud, that is, the three-dimensional coordinate values ​​of the points, filter the points, and remove the points if they are within the retained range or outside the retained range. It should be noted that the expected range is specifically determined by the operator.

[0043] S420: Remove the discrete noise points of the scene three-dimensional model according to the Euclidean clustering algorithm to obtain a rough point cloud bounding volume of the cow. Specifically, the Euclidean clustering algorithm is mainly used to remove outlier noise points and can effectively remove messy outlier noise.

[0044] S500: Completing the holes of the rough point cloud bounding volume of the cow according to the completion algorithm to obtain the fine point cloud bounding volume of the cow. Specifically, the completion algorithm is the alpha shape algorithm. Compared with other completion algorithms, such as the convex hull method, the alpha shape algorithm can understand the concave hull situation and can outline multiple boundary lines for multiple point clouds, which meets the requirements of fast processing of irregular fine point clouds of cows. Therefore, the alpha shape algorithm is used to complete the holes of the rough point cloud bounding volume of the cow and obtain the fine point cloud bounding volume of the cow.

[0045] S600: Based on the cow's fine point cloud bounding volume, a volume-weight regression model is established according to a volume calculation algorithm to obtain the cow's weight data. Specifically, the volume calculation is performed using the VTK library.

[0046] The above is only a preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A non-contact weight estimation method for dairy cows based on three-dimensional reconstruction of neural radiation field, used to measure the volume of dairy cows, characterized in that: The method comprises: Fixed cows; Use video acquisition equipment to collect multi-angle video data; Based on the multi-angle video data, training the neural radiation field to perform three-dimensional reconstruction of the scene to obtain a three-dimensional model of the scene; Removing cluttered background interference and discrete noise points from the three-dimensional model of the scene according to a cluttered background removal algorithm to obtain a rough point cloud bounding volume of the cow; Completing the holes of the rough point cloud bounding volume of the cow according to the completion algorithm to obtain a fine point cloud bounding volume of the cow; Based on the fine point cloud bounding volume of the dairy cow, a volume-weight regression model is established according to a volume calculation algorithm to obtain the weight data of the dairy cow.

2. The non-contact weight estimation method for dairy cows based on three-dimensional reconstruction of neural radiation field according to claim 1 is characterized in that: The training neural radiation field for three-dimensional reconstruction of the scene includes: Based on the multi-angle video data, training the neural radiance field network initialization to obtain the density and color of the scene point cloud; Based on the density and color of the scene point cloud, rendering rays are generated according to the poses of different views, and the world coordinate system, the pixel coordinate system and the camera coordinate system of the scene point cloud are converted to obtain information coordinate data of the scene point cloud; Based on the information coordinate data of the scene point cloud, the scene point cloud is rendered according to a volume rendering algorithm and an SDF algorithm to obtain the three-dimensional model of the scene.

3. The non-contact weight estimation method for dairy cows based on three-dimensional reconstruction of neural radiation field according to claim 1 is characterized in that: The algorithm for removing cluttered background includes: The point cloud of the three-dimensional model of the scene is filtered according to a through filtering algorithm to obtain a point cloud within a preset range.

4. The method for measuring the volume of dairy cows based on the neural radiation field according to claim 3, characterized in that: The algorithm for removing cluttered background includes: The discrete noise points of the three-dimensional model of the scene are removed according to the Euclidean clustering algorithm to obtain the rough point cloud bounding volume of the cow.

5. The method for measuring the volume of dairy cows based on nerve radiation field according to claim 1, characterized in that: The completion algorithm comprises: The holes of the rough point cloud bounding volume of the cow are completed according to the alpha shape algorithm to obtain the fine point cloud bounding volume of the cow.

Images