A refined residual material monitoring method and system for cage breeding areas

Abstract

The present invention discloses a refined leftover feed monitoring method and system for caged areas. The present invention specifically includes: first, installing an image acquisition device at the feeder, and using the image acquisition device to obtain the original RGB image of the feed trough; then, based on the acquired original RGB image of the feed trough, accurately monitor the distribution of leftover feed inside the trough by combining deep learning with a 3D point cloud processing method, and accurately calculate the volume and weight data of leftover feed in different areas. In addition, data analysis can be performed based on the weight of the leftover food at the location and corresponding warnings can be made. The present invention does not require additional hardware equipment, and the cost is greatly reduced. The present invention can accurately provide a more accurate leftover feed situation for each location, improve feeding efficiency, and reduce economic costs; since the present invention uses a cloud server, the storage space is not limited by hardware and has an extremely fast processing speed.

Description

Technical Field

[0001] The invention belongs to a residual feed monitoring method in the technical field of animal husbandry, and in particular relates to a refined residual feed monitoring method and system for cage breeding areas. Background Art

[0002] With the development of modern agricultural breeding technology, cage breeding has occupied a major position in livestock and poultry breeding due to its high efficiency and density. However, in this compact breeding space, the daily feeding amount and the amount of remaining feed are closely related to the growth rate and overall health status of laying hens. Therefore, implementing precise feeding control and obtaining the weight of remaining feed in the cage not only ensures that the laying hens in each area get balanced nutrition and maximizes economic benefits, but also provides real-time feeding data for breeding personnel to analyze, so that they can quickly identify and deal with laying hens in poor health in the area, thereby further optimizing the management of cage areas and playing a key role in the sustainable development of the breeding industry.

[0003] Most of the current methods of feeding control and residual feed weight measurement are manual monitoring. For example, researchers use sensors to measure the weight of food, but facing large cage farms, this method is costly and time-consuming. With the development of deep learning technology, residual feed surface detection methods based on machine learning are also being used in cage systems. Patent CN115526880B, a method for distinguishing residual feed in caged pigeon feed troughs, uses machine learning technology to judge the residual feed situation in the feed trough through the feed trough image and ultrasonic ranging sensor data, but cannot obtain the specific weight data of the residual feed, and the ultrasonic ranging sensor only obtains points on three planes for curve fitting, including the distance from the left point inside the feed trough, the distance from the ultrasonic wave to the midpoint inside the feed trough, and the distance from the ultrasonic wave to the right point inside the feed trough. This discrimination method will produce problems such as inaccurate results when facing a residual feed surface with multiple curves. Summary of the invention

[0004] In order to solve the problems and needs in the background technology, the present invention proposes a refined leftover feed management method and system for caged areas. The present invention combines 3D point cloud technology, deep learning technology and cloud service technology to achieve feeding control and leftover feed weight measurement in caged areas, which can help large-scale farms to achieve low-cost and high-efficiency refined management of regional food and reduce potential economic losses.

[0005] The technical solution of the present invention is as follows:

[0006] 1. A refined method for monitoring residual feed in caged areas

[0007] S1: Use a binocular camera to obtain the original RGB image of the trough, and then obtain the left / right initial images of different locations;

[0008] S2: After preprocessing, stereo matching and 3D reconstruction of the left / right initial images of each location, the target 3D point cloud of the current location is obtained;

[0009] S3: after processing the target three-dimensional point cloud of the current location in combination with the three-dimensional model of the empty material point cloud, the three-dimensional point cloud of the remaining material at the current location is obtained;

[0010] S4: After calculating the volume of the leftover material in the current location using the dynamic leftover material image volume measurement method, the volume of the leftover material is obtained; and then the weight of the leftover material in the current location is calculated by combining the feed density model;

[0011] S5: Repeat S2-S4, calculate the weight of the remaining material for the left / right original RGB images of the trough in the remaining area, obtain the weight of the remaining material in different areas, and complete the remaining material monitoring.

[0012] In S1, the original RGB image of the trough taken by the left camera and the original RGB image of the trough taken by the right camera are automatically segmented by using the location feature division method in combination with the trough edge image, so as to obtain a left initial image and a right initial image of a location.

[0013] The S2 is specifically:

[0014] S21: Preprocessing the left initial image and the right initial image of each location respectively to obtain a left preprocessed image and a right preprocessed image;

[0015] S22: After stereo matching of the left pre-processed image and the right pre-processed image, a dense disparity map of the trough is obtained;

[0016] S23: Convert the trough dense disparity map into a depth map, and then perform three-dimensional reconstruction on the depth map to obtain a target three-dimensional point cloud.

[0017] In S21, for the left initial image of each location, the preprocessing process is specifically as follows:

[0018] S211: using a bilateral filtering method to eliminate isolated noise points in the left initial image of each location to obtain a filtered image, and then using a logarithmic function to enhance a low-light portion in the filtered image to obtain an enhanced low-contrast image;

[0019] S212: combining the enhanced low-contrast image and the filtered image using an adaptive logarithmic image processing method to obtain a target uniform illumination map;

[0020] S213: After processing the target uniform illumination map using the CLAHE function, a left preprocessed image is obtained.

[0021] The S22 is specifically:

[0022] S221: extracting feature points corresponding to the left pre-processed image and the right pre-processed image respectively;

[0023] S222: extracting feature descriptors corresponding to the left preprocessed image and the right preprocessed image respectively;

[0024] S223: According to the feature descriptors of the left preprocessed image and the right preprocessed image, the feature points of the left preprocessed image and the right preprocessed image are matched to obtain a set of matched feature points; then, the set of matched feature points is screened using a random sampling consistency algorithm to remove abnormal maximum values ​​in the matching points; then, a method based on a triangular irregular network is used to remove incorrect matching points in the set of matched feature points from which the abnormal maximum values ​​have been removed to obtain a set of correct matching points, thereby obtaining left and right views after feature matching;

[0025] S224: After stereo matching of the left and right views after feature matching is performed using the SGBM algorithm, an initial dense disparity map is obtained, and then the holes in the initial dense disparity map are filled using the nearest neighbor interpolation method to obtain a trough dense disparity map.

[0026] The S3 is specifically:

[0027] The ICP algorithm is used to align the target 3D point cloud to the coordinate system of the empty material point cloud 3D model to obtain the aligned 3D point cloud. The overlapping point clouds are then subtracted from the aligned 3D point cloud to obtain the point cloud data of the non-overlapping part and use it as the 3D point cloud of the remaining material in the current location.

[0028] In S4, the volume of the leftover material is calculated for the leftover material three-dimensional point cloud at the current location using the dynamic leftover material image volume measurement method, and the volume of the leftover material is obtained, specifically:

[0029] First, the 3D point cloud of the leftover material in the current location is horizontally sliced ​​from top to bottom to obtain multiple slice models, and a point cloud belt is formed between two adjacent slice models;

[0030] Next, after performing boundary search on each of the slice models, boundary contour information of each slice model is obtained;

[0031] Then, the cross-sectional area of ​​each slice model is calculated according to the boundary contour information of each slice model; the volume of each point cloud band is calculated in combination with the slice height; and the volume of the remaining material is obtained by accumulating the volumes of all point cloud bands.

[0032] In S4, the weight of the remaining feed in the current location is calculated in combination with the feed density model, specifically:

[0033] After matching the feed density of laying hen feed in the current location using the feed density model, the feed density is obtained, and the weight of the remaining feed is obtained by multiplying the volume of the remaining feed by the feed density.

[0034] 2. A refined residual feed monitoring system for cage breeding areas

[0035] Equipment control module, used to control the feeder and camera;

[0036] A feed density model generating unit is used to calculate the volume and weight of the feed according to the feed data of laying hens constructed according to the feed formula of laying hens of different ages, and obtain the density data of the feed of different ages, so as to generate a feed density model and store it in the cloud storage unit;

[0037] The empty material point cloud three-dimensional model generating unit is used to generate an empty material point cloud three-dimensional model after cutting the trough part of the location empty material trough point cloud image and store it in the cloud storage unit;

[0038] The residual material weight detection unit is used to calculate the residual material weight at different locations according to the original RGB image of the trough acquired by the camera, and store the residual material weight at different locations in the cloud storage unit;

[0039] The residual material warning unit is used to conduct early warning analysis based on the residual material weight results in different locations;

[0040] The cloud storage unit is used to store the original RGB image of the feed trough, the feed density model, the feed density model, and the residual feed weight results at different locations.

[0041] The residual material weight detection unit comprises:

[0042] A location segmentation unit is used to segment the original RGB image of the trough into left / right initial images corresponding to different locations;

[0043] A preprocessing unit, used for preprocessing the left / right initial image of each location to obtain a left preprocessed image and a right preprocessed image;

[0044] A stereo matching unit, used for stereo matching the left pre-processed image and the right pre-processed image of each location to obtain a dense disparity map of the trough;

[0045] A 3D reconstruction unit is used to convert the dense disparity map of the trough into a depth map and perform 3D reconstruction to obtain a target 3D point cloud;

[0046] The leftover material 3D point cloud generation unit is used to combine the empty material point cloud 3D model, process the target 3D point cloud of each location, and obtain the corresponding leftover material 3D point cloud;

[0047] The residual material weight calculation unit is used to calculate the residual material volume of the residual material three-dimensional point cloud of each location, and then calculate the residual material weight.

[0048] Compared with the prior art, the beneficial effects achieved by the refined residual material monitoring method and system for caged areas proposed by the present invention are:

[0049] 1. The present invention proposes a non-contact method for estimating the weight of residual materials. By combining deep learning with 3D point cloud technology, it can not only more accurately monitor the distribution of residual materials inside the discharge chute, but also accurately obtain the volume and weight data of the residual materials.

[0050] 2. The present invention only needs to install the collection equipment in the feeding unit, configure the parameters of each device and the communication relationship, and then automatically extract and process the image information through the program code to accurately obtain the food situation in the trough. No additional equipment is required, and the cost is greatly reduced.

[0051] 3. The present invention also proposes a set of regional residual feed monitoring system based on cloud communication, cloud storage and cloud computing technology. The storage space is not limited by hardware and has an extremely fast processing speed. It can also make vertical and horizontal comparisons based on the weight of residual feed to determine the location of laying hens with poor health, and provide accurate data support for the breeding personnel in the next decision-making behavior. In addition, the system also provides more detailed distribution of residual feed in the feed trough, which lays a strong foundation for the next step of accurate feeding, avoids underfeeding and leakage during feeding, improves feeding efficiency and reduces costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It is the overall flow chart of the present invention.

[0053] Figure 2 This is an experimental scene diagram of collecting RGB images of a trough according to the present invention.

[0054] Figure 3 This is a flow chart of the stereo matching unit of the present invention.

[0055] Figure 4 This is a flow chart of the residual material three-dimensional point cloud generation unit of the present invention.

[0056] Figure 5 It is a flow chart of the residual material weight detection unit of the present invention. DETAILED DESCRIPTION

[0057] The embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by ordinary technical managers without creativity are within the protection scope of the present invention.

[0058] like Figure 1 As shown, the embodiment of the present invention includes the following processes:

[0059] S1: Use a binocular camera to obtain the original RGB image of the trough or the video frame of the trough, and then obtain the left / right initial images of different locations;

[0060] In S1, combined with the trough edge image, the original RGB image of the trough taken by the left camera and the original image containing the trough taken by the right camera are automatically segmented by instance using the location feature division method to obtain a left initial image and a right initial image of a location.

[0061] Specifically:

[0062] First, collect the trough edge images and the original RGB images of the trough. Then annotate the bounding boxes and category labels for the number and location of the troughs in each image. Then divide the image dataset into training set, validation set and test set in a ratio of 8:1:1. Next, build a MaskTrack-RCNN model based on Pytorch, set hyperparameters such as learning rate, batch size, and number of iterations, and train the model to obtain the trough image segmentation model. Then, use the trough image segmentation model to identify the trough edge of each image in the trough video frame. When two trough edges are detected, extract the detection box information of the two trough edges and calculate the median of the two trough edge detection boxes. Take the area between the medians as the position line area, and use the trough image segmentation model to automatically segment the trough image in the position line area to obtain the initial image of the trough with the background removed.

[0063] MaskTrack-RCNN is a deep learning model specifically designed for object detection, instance segmentation, and object tracking in video sequences. It is an extended version of the Mask R-CNN model that can not only identify and segment objects in static images, but also track objects consistently in consecutive video frames. In this embodiment, MaskTrack-RCNN can be used to detect and segment the edges of the trough from the video frames, and then accurately segment the location line area in the video by tracking and segmenting the edges of the trough, and finally obtain the initial image of the trough in the area. This method can effectively improve the accuracy and efficiency of the automated detection system.

[0064] S2: After preprocessing, stereo matching and 3D reconstruction of the left / right initial images of each location, the target 3D point cloud of the current location is obtained;

[0065] S2 is specifically:

[0066] S21: Preprocessing the left initial image and the right initial image of each location respectively to obtain a left preprocessed image and a right preprocessed image;

[0067] Due to the complexity of the cage system environment, the collected images are easily affected by environmental interference such as lighting and electromagnetics, and there will be low regional brightness and contrast, making the collected images non-uniform brightness images. The overall feature information of the image is inconsistent, and it is easy to have scenes where key points cannot be identified, which will affect the accuracy of subsequent volume calculations. Therefore, image preprocessing is a key part. The preprocessing process in S21 includes filtering processing to remove excess noise. For the left initial image of each location, the preprocessing process is specifically as follows:

[0068] S211: Use the bilateral filtering method to eliminate isolated noise points in the left initial image of each location to obtain a filtered image, and then use a logarithmic function to enhance the low-light part of the filtered image to obtain an enhanced low-contrast image. The formula is as follows:

[0069] W = exp(((ik) 2 +(i- l ) 2 ) / (2σ s 2 ))×exp(|I(i,j)+I(k, l )| 2 / (2σ r 2 ))

[0070] W Low =log(max(x)+1)×log(1+W)

[0071] Where W is the filtered image, i and j are the two coordinate values ​​of pixel (i, j), k, l is the pixel (k, l ) are two coordinate values, I(i, j) is the grayscale value of pixel (i, j), I(k, l ) is the pixel (k, l ) gray value, σ s and σ r is the standard deviation of the Gaussian function, and are the weights of the control space and range, respectively. Low It is to enhance low contrast images.

[0072] S212: After combining the enhanced low-contrast image and the filtered image (ie, the normal-contrast image) using an adaptive logarithmic image processing method, a target uniform illumination map is obtained; the formula is as follows:

[0073] W Fused =αW Low+ (1-α) W

[0074] Among them, W fused is the target uniform illumination map; W low is the enhanced low-contrast image. W is the filtered image. α is the fusion weight, which is used to control the contribution of the two images in the fusion result.

[0075] S213: After processing the target uniform illumination map using the CLAHE function, the overall brightness of the image is improved to obtain a left preprocessed image. The preprocessing process of the right initial image of each location is the same as the preprocessing process of the left initial image. The preprocessing method proposed in the present invention can better balance the contrast of high and low brightness in the image and enhance the image quality.

[0076] S22: After stereo matching of the left pre-processed image and the right pre-processed image, a dense disparity map of the trough is obtained;

[0077] like Figure 3 As shown, S22 is specifically:

[0078] S221: The feature points corresponding to the left pre-processed image and the right pre-processed image are respectively extracted using the Harris corner point detection method with added scale space processing; for the feature points at each pixel position of the left pre-processed image, the specific calculation steps are as follows:

[0079] First, multi-scale Gaussian smoothing is performed on each pixel position of the left preprocessed image, and the first-order gradient and second-order partial derivative of the pixel position at each scale are calculated. In order to reduce the influence of noise and ensure scale invariance, the second-order partial derivative of the pixel position is Gaussian filtered to obtain the filtered results; the filtered results are used to construct the feature matrix M at different scales, and then the corner point response function is calculated based on the feature matrix M at different scales of the pixel position. The corner point response function is then arranged in descending order by the non-maximum suppression method to calculate the preliminary corner point position. Finally, based on the preliminary corner point position, the pixel grayscale value of the sub-region of the pixel position is interpolated to obtain the accurate feature point.

[0080] S222: extract feature descriptors corresponding to the left preprocessed image and the right preprocessed image respectively; for each feature point, the calculation steps of its feature descriptor are as follows:

[0081] First, with each feature point as the center point (i.e., the center of the circle), determine a circular area with a radius of 2. Within the circular area, calculate the gradient direction and modulus of each pixel. Then, divide the circular area into multiple sub-rings, and the size of the sub-ring is 2×2. Count the gradient modulus values ​​of each pixel in each sub-ring in different gradient directions (specifically 8 gradient directions), and summarize the gradient modulus values ​​in each gradient direction into a histogram. After sorting all histograms according to the size of the gradient modulus value, a 32-dimensional feature vector is formed and used as a feature descriptor. In the present invention, the feature descriptor is simplified from the original 128 dimensions to 32 dimensions, which can significantly reduce the calculation time.

[0082] S223: According to the feature descriptors of the left preprocessed image and the right preprocessed image, the feature points of the left preprocessed image and the right preprocessed image are matched to obtain a set of matching feature points. Among them, for the two feature point sets A and B to be matched, the Euclidean distance between the feature descriptor corresponding to each feature point a in the feature point set A and the feature descriptors corresponding to all feature points b in the feature point set B is calculated. For each feature point a from the feature point set A, the feature point b1 (nearest neighbor) closest to it and the feature point b2 (second nearest neighbor) with the second closest distance are found in the feature point set B. The ratio of the Euclidean distances of these two pairs of feature points is calculated. If it is less than the threshold, it is considered that this is a pair of matching points, and a set of matching feature points is obtained; then the random sampling consistency algorithm is used to screen the set of matching feature points to remove abnormal maximum values ​​in the matching points; then a method based on a triangular irregular network is used to remove the wrong matching points in the matching feature point set from which the abnormal maximum values ​​have been removed, and the correct matching point set is obtained, thereby obtaining the left and right views after feature matching;

[0083] S224: After stereo matching of the left and right views after feature matching using the SGBM algorithm, an initial dense disparity map is obtained. In order to perform more accurate volume calculation, the nearest neighbor interpolation method is used to fill the remaining larger neighborhood holes in the initial dense disparity map to obtain a dense disparity map of the trough.

[0084] S23: Convert the trough dense disparity map into a depth map, and then perform three-dimensional reconstruction on the depth map to obtain the target three-dimensional point cloud. The specific formula is as follows:

[0085] Z=Bf / d

[0086] w x =(i x -u0)z / f x , w y =(i y -v0)z / f y , w z =z×f / d

[0087] Among them, B is the distance between the two cameras, f is the focal length of the camera, d is the parallax, Z is the depth map, and i x , i y is the coordinate of the trough image boundary point in the pixel coordinate system, w x , w y , w z is the coordinate (i x , i y ) is the transformed three-dimensional coordinate point, z is the distance from the camera to the object, f x , f y are the horizontal and vertical intrinsic parameter matrix values ​​of the camera respectively, and u0 and v0 are the coordinates of the center point of the image in the pixel coordinate system respectively.

[0088] S3: after processing the target three-dimensional point cloud of the current location in combination with the three-dimensional model of the empty material point cloud, the three-dimensional point cloud of the remaining material at the current location is obtained;

[0089] like Figure 4 As shown, S3 is specifically:

[0090] The empty material point cloud 3D model is extracted from the cloud database, and the target 3D point cloud is registered to the coordinate system of the empty material point cloud 3D model using the ICP algorithm to obtain the registered 3D point cloud. The overlapping point clouds are then subtracted from the registered 3D point cloud to obtain the point cloud data of the non-overlapping part and use it as the 3D point cloud of the remaining material in the current location, which is then transmitted to the cloud system.

[0091] Among them, the ICP algorithm is used to register the target 3D point cloud to the coordinate system of the blank point cloud 3D model to obtain the registered 3D point cloud, specifically:

[0092] Define the initial transformation matrix, for each point P in the source point cloud (ie, the target 3D point cloud) i , search for the nearest point Q in the template point cloud (i.e. the blank point cloud 3D model) j As corresponding points; store all corresponding point pairs (P i , Q j ), calculate the sum of the squared Euclidean distances between all corresponding point pairs as the error metric for the current iteration. Use the least squares method and other methods to calculate the optimal rotation matrix R and translation vector t based on the corresponding point pairs. Apply the rotation matrix R and translation vector t to the source point cloud to generate a new transformed point cloud. If the error metric is less than the threshold, stop the iteration; otherwise, continue to iterate the above calculation process to obtain two point cloud images in the same coordinate system;

[0093] Subtract the overlapping point clouds from the registered 3D point clouds to obtain the point cloud data of the non-overlapping part and use it as the 3D point cloud of the remaining material at the current location. Specifically:

[0094] First, traverse each point in the target 3D point cloud. For each point, calculate the Euclidean distance between it and each point in the target 3D point cloud. If there is a point whose distance is less than a preset threshold, this point in the target 3D point cloud is considered to be a background point, and it is removed from the target 3D point cloud or marked as an invalid point. Finally, the point cloud data of the non-overlapping part is obtained, that is, the 3D point cloud of the remaining material in the current location.

[0095] S4: After calculating the volume of the leftover material in the current location using the dynamic leftover material image volume measurement method, the volume of the leftover material is obtained; and then the weight of the leftover material in the current location is calculated by combining the feed density model;

[0096] In S4, the volume of the remaining material is calculated for the remaining material three-dimensional point cloud at the current location using the dynamic remaining material image volume measurement method, and the volume of the remaining material is obtained, specifically:

[0097] First, the 3D point cloud of the leftover material in the current location is horizontally segmented from top to bottom to obtain multiple slice models. Each slice model represents a cross section of the object at a certain height, and a point cloud belt is formed between two adjacent slice models.

[0098] Next, after performing boundary search on each slice model, that is, transforming the points that are in a disordered state in the three-dimensional coordinate system into points that are in an orderly state in the two-dimensional space, clear boundary contour information of each slice model is obtained;

[0099] Then, based on the clear boundary contour information of each slice model, the cross-sectional area of ​​each slice model is calculated; combined with the slice height, the volume of each point cloud band is calculated; after accumulating the volumes of all point cloud bands, the volume of the remaining material is obtained. The details are as follows:

[0100] Input the point cloud set P, and cut the point cloud in sequence from top to bottom using a set of equal horizontal planes with a spacing of H / n between the point cloud elevation values ​​of height H to obtain the slice model T of the horizontal point cloud i (i=0, 1, 2, ..., n-1), there is a point cloud strip between every two slice models, and n horizontal strips are obtained; finally, each horizontal point cloud slice T is calculated i Area x , the calculation process of the residual volume V is as follows:

[0101]

[0102] V=∑ x=0 n-1 ( (S x ×N) / n)

[0103] Among them, x1, y1, x2, y2 and x3, y3 represent the coordinates of the three vertices that constitute the polygon slice, V represents the total volume of the point cloud, S x Represents the slice cross-sectional volume, N represents the interpolation of the maximum and minimum elevation values, and n represents the number of point cloud bands.

[0104] Finally, after discrete extraction of the trough video frame, the volume calculated for every 30 frames of the trough image is accumulated and averaged to obtain the final location trough image volume corresponding to each frame. In S4, combined with the feed density model, the weight of the remaining feed in the current location is calculated, specifically:

[0105] The feed density of different periods downloaded from the database is the feed density model. After matching the feed density of the laying hen feed in the current location with the feed density model, the feed density is obtained, and the weight of the leftovers is obtained by multiplying the volume of the leftovers by the feed density.

[0106] The cloud-based leftover food monitoring system analyzes the weight of leftover food in each location and issues corresponding warnings.

[0107] S5: Repeat S2-S4, calculate the weight of the remaining material for the left / right original images of the trough in the remaining area, obtain the weight of the remaining material in different areas, and complete the remaining material monitoring.

[0108] like Figure 5 As shown, the present invention also proposes a refined residual material monitoring system for cage breeding areas, comprising:

[0109] The equipment control module is used to control the feeder and the binocular camera. The control parameters of the feeder include the start / stop status and the moving speed, and the control parameters of the camera include the shooting angle and distance. The binocular camera is installed directly below the feeder. The binocular camera can obtain RGB images of targets at two different horizontal positions, and the binocular camera can capture color images. The camera needs to be calibrated according to its internal and external parameters to correct the distortion of the lens. In addition, due to insufficient light in some laying hen areas, auxiliary light sources are required to increase the stability of the acquisition environment. Figure 2 As shown, the equipment control module controls the feeder and the binocular camera to work together, and uses the acquired RGB image combined with the location feature division method to collect the trough conditions in each location to obtain the left and right RGB images of two horizontal positions.

[0110] The feed density model generation unit is used to construct laying hen feed data based on laying hen feed formulas of different ages, calculate the volume and weight of the feed, and obtain the density data of the feed at different ages, thereby generating a feed density model and storing it in a cloud storage unit; wherein, there are 50 portions of laying hen feed formulas of different ages.

[0111] The empty material point cloud three-dimensional model generating unit is used to generate an empty material point cloud three-dimensional model after cutting the trough part of the location empty material trough point cloud image and store it in the cloud storage unit;

[0112] The residual material weight detection unit is used to calculate the residual material weight at different locations according to the original RGB image of the trough acquired by the camera, and store the residual material weight at different locations in the cloud storage unit;

[0113] The residual material warning unit is used to conduct early warning analysis based on the residual material weight results in different locations; including: horizontal comparison and vertical comparison.

[0114] Horizontal comparison refers to comparing the weight of the remaining material today in the area with the weight of the remaining material in the previous three days. If it is less than the average weight of the remaining material in the previous three days, it is an abnormal state and an early warning is issued;

[0115] The vertical comparison focuses on the relative position of the current location's residual weight among all locations. By comparing the current location's residual weight W i Compared with the standard deviation σ of the residual weight of all the number of zones N, if W i <μ– 3×σ, where μ is the average weight of all the remaining materials in all locations, it is considered abnormal and an early warning is issued to determine whether the current location of the remaining material weight is far below the normal level. The expression of the standard deviation σ is:

[0116] σ=((∑ i=1 N (w i -μ) 2 ) / N) 0.5

[0117] In addition to the above comparative analysis, the cloud-based residual material monitoring system can also provide detailed residual material distribution. The system can record the residual material weight of each location trough and display the residual material distribution in a visual way, providing data support for accurate feeding.

[0118] The cloud storage unit is used to store the original RGB image of the trough, the feed density model, the feed density model and the residual weight results of different locations. The equipment control module, the feed density model generation unit, the empty material point cloud 3D model generation unit, the residual weight detection unit and the residual early warning unit are all deployed on the online cloud server, and there is a communication relationship between the online cloud server and the equipment.

[0119] The online cloud server can choose high-performance servers from Amazon AWS, Microsoft Azure, and Alibaba Cloud and configure them; the monitoring system is developed based on qt, including the user end, algorithm end, control end, and database; the control end is used to send decision signals to the feeder and binocular camera for control; the algorithm layer is the key part of the leftover material management of the input data and is the basis for leftover material monitoring; the user end is used to visualize the location leftover material data to help administrators make corresponding strategies; the data layer is responsible for the storage, organization, and analysis management of various types of leftover material data, providing effective and reliable data support for the system. During the image acquisition process, the decision instructions issued by the monitoring system deployed in the cloud control the feeder and camera to work together, and the data transmission between the two devices and the monitoring system is connected through the 5G+ network.

[0120] Residual material weight detection unit, including:

[0121] A location segmentation unit is used to segment the original RGB image of the trough into left / right initial images corresponding to different locations;

[0122] A preprocessing unit, used for preprocessing the left / right initial image of each location to obtain a left preprocessed image and a right preprocessed image;

[0123] A stereo matching unit, used for stereo matching the left pre-processed image and the right pre-processed image of each location to obtain a dense disparity map of the trough;

[0124] A 3D reconstruction unit is used to convert the dense disparity map of the trough into a depth map and perform 3D reconstruction to obtain a target 3D point cloud;

[0125] The leftover material 3D point cloud generation unit is used to combine the empty material point cloud 3D model, process the target 3D point cloud of each location, and obtain the corresponding leftover material 3D point cloud;

[0126] The residual material weight calculation unit is used to calculate the residual material volume of the residual material three-dimensional point cloud of each location, and then calculate the residual material weight.

[0127] Finally, it should be noted that the above embodiments and explanations are only used to illustrate the technical solution of the present invention rather than to limit it. Those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope disclosed in the technical solution of the present invention, which should be included in the scope of protection of the claims of the present invention.

Claims

1. A refined method for monitoring leftover materials in caged areas, characterized in that: The following steps are involved: S1: Use a binocular camera to obtain the original RGB image of the trough, and then obtain the left / right initial images of different locations; S2: After preprocessing, stereo matching and 3D reconstruction of the left / right initial images of each location, the target 3D point cloud of the current location is obtained; S3: after processing the target three-dimensional point cloud of the current location in combination with the three-dimensional model of the empty material point cloud, the three-dimensional point cloud of the remaining material at the current location is obtained; S4: After calculating the volume of the leftover material in the current location using the dynamic leftover material image volume measurement method, the volume of the leftover material is obtained; and then the weight of the leftover material in the current location is calculated by combining the feed density model; S5: Repeat S2-S4, calculate the weight of the remaining material for the left / right original RGB images of the trough in the remaining area, obtain the weight of the remaining material in different areas, and complete the remaining material monitoring.

2. A refined leftover material monitoring method for caged areas according to claim 1, characterized in that: In S1, the original RGB image of the trough taken by the left camera and the original RGB image of the trough taken by the right camera are automatically segmented by using the location feature division method in combination with the trough edge image, so as to obtain a left initial image and a right initial image of a location.

3. A refined leftover material monitoring method for caged areas according to claim 1, characterized in that: The S2 is specifically: S21: Preprocessing the left initial image and the right initial image of each location respectively to obtain a left preprocessed image and a right preprocessed image; S22: After stereo matching of the left pre-processed image and the right pre-processed image, a dense disparity map of the trough is obtained; S23: Convert the trough dense disparity map into a depth map, and then perform three-dimensional reconstruction on the depth map to obtain a target three-dimensional point cloud.

4. A refined leftover material monitoring method for caged areas according to claim 3, characterized in that: In S21, for the left initial image of each location, the preprocessing process is specifically as follows: S211: using a bilateral filtering method to eliminate isolated noise points in the left initial image of each location to obtain a filtered image, and then using a logarithmic function to enhance a low-light portion in the filtered image to obtain an enhanced low-contrast image; S212: combining the enhanced low-contrast image and the filtered image using an adaptive logarithmic image processing method to obtain a target uniform illumination map; S213: After processing the target uniform illumination map using the CLAHE function, a left preprocessed image is obtained.

5. A refined leftover material monitoring method for caged areas according to claim 3, characterized in that: The S22 is specifically: S221: extracting feature points corresponding to the left pre-processed image and the right pre-processed image respectively; S222: extracting feature descriptors corresponding to the left preprocessed image and the right preprocessed image respectively; S223: According to the feature descriptors of the left preprocessed image and the right preprocessed image, the feature points of the left preprocessed image and the right preprocessed image are matched to obtain a set of matched feature points; then, the set of matched feature points is screened using a random sampling consistency algorithm to remove abnormal maximum values ​​in the matching points; then, a method based on a triangular irregular network is used to remove incorrect matching points in the set of matched feature points from which the abnormal maximum values ​​have been removed to obtain a set of correct matching points, thereby obtaining left and right views after feature matching; S224: After stereo matching of the left and right views after feature matching is performed using the SGBM algorithm, an initial dense disparity map is obtained, and then the holes in the initial dense disparity map are filled using the nearest neighbor interpolation method to obtain a trough dense disparity map.

6. A refined leftover material monitoring method for caged areas according to claim 1, characterized in that: The S3 is specifically: The ICP algorithm is used to align the target 3D point cloud to the coordinate system of the empty material point cloud 3D model to obtain the aligned 3D point cloud. The overlapping point clouds are then subtracted from the aligned 3D point cloud to obtain the point cloud data of the non-overlapping part and use it as the 3D point cloud of the remaining material in the current location.

7. A refined leftover material monitoring method for caged areas according to claim 1, characterized in that: In S4, the volume of the leftover material is calculated for the leftover material three-dimensional point cloud at the current location using the dynamic leftover material image volume measurement method, and the volume of the leftover material is obtained, specifically: First, the 3D point cloud of the leftover material in the current location is horizontally sliced ​​from top to bottom to obtain multiple slice models, and a point cloud belt is formed between two adjacent slice models; Next, after performing boundary search on each of the slice models, boundary contour information of each slice model is obtained; Then, the cross-sectional area of ​​each slice model is calculated according to the boundary contour information of each slice model; the volume of each point cloud band is calculated in combination with the slice height; and the volume of the remaining material is obtained by accumulating the volumes of all point cloud bands.

8. A refined residual material monitoring method for caged areas according to claim 1, characterized in that: In S4, the weight of the remaining feed in the current location is calculated in combination with the feed density model, specifically: After matching the feed density of laying hen feed in the current location using the feed density model, the feed density is obtained, and the weight of the remaining feed is obtained by multiplying the volume of the remaining feed by the feed density.

9. A refined residual material monitoring system for caged areas, characterized in that: include: Equipment control module, used to control the feeder and camera; A feed density model generating unit is used to calculate the volume and weight of the feed according to the feed data of laying hens constructed according to the feed formula of laying hens of different ages, and obtain the density data of the feed of different ages, so as to generate a feed density model and store it in the cloud storage unit; The empty material point cloud three-dimensional model generating unit is used to generate an empty material point cloud three-dimensional model after cutting the trough part of the location empty material trough point cloud image and store it in the cloud storage unit; The residual material weight detection unit is used to calculate the residual material weight at different locations according to the original RGB image of the trough acquired by the camera, and store the residual material weight at different locations in the cloud storage unit; The residual material warning unit is used to conduct early warning analysis based on the residual material weight results in different locations; The cloud storage unit is used to store the original RGB image of the feed trough, the feed density model, the feed density model, and the residual feed weight results at different locations.

10. A refined residual material monitoring system for cage breeding areas according to claim 9, characterized in that: The residual material weight detection unit comprises: A location segmentation unit is used to segment the original RGB image of the trough into left / right initial images corresponding to different locations; A preprocessing unit, used for preprocessing the left / right initial image of each location to obtain a left preprocessed image and a right preprocessed image; A stereo matching unit, used for stereo matching the left pre-processed image and the right pre-processed image of each location to obtain a dense disparity map of the trough; A 3D reconstruction unit is used to convert the dense disparity map of the trough into a depth map and perform 3D reconstruction to obtain a target 3D point cloud; The leftover material 3D point cloud generation unit is used to combine the empty material point cloud 3D model, process the target 3D point cloud of each location, and obtain the corresponding leftover material 3D point cloud; The residual material weight calculation unit is used to calculate the residual material volume of the residual material three-dimensional point cloud of each location, and then calculate the residual material weight.

Images