A method and system for determining livestock presence in pens based on non-contact measurement

Abstract

This invention discloses a method and system for determining livestock presence in a pen based on non-contact measurement, relating to the field of livestock management technology. First, depth and visible light images of the grazing herd are acquired, and then image fusion and segmentation are performed sequentially to obtain several segmented regions. The images of these segmented regions are input into a livestock recognition model to determine whether the target within the segmented region is livestock. The segmented regions where livestock are identified are designated as the livestock's location. For each livestock location region, midpoint measurement, distance calculation, and triangulation are performed sequentially to obtain the geographical coordinates of the midpoint within each region. Based on the geographical coordinates of the midpoint of each livestock location region and the location of a virtual electronic fence, the relationship between points and lines is used to determine whether the livestock is within the virtual electronic fence's range. This invention employs non-contact measurement technology, improving the free movement of livestock and effectively reducing the cost and labor intensity of pasture management, while accurately determining the livestock's pen presence status.

Description

Technical Field

[0001] This invention relates to the field of livestock management technology, and more specifically to a method and system for determining livestock presence in pens based on non-contact measurement. Background Technology

[0002] With the development of agricultural production and the expansion of pasture size, the challenges of managing herds are also increasing. Under conditions of grazing in the wild, livestock may face various natural and environmental problems, such as disease, threats from predators, and difficulties in accessing water and food. Therefore, timely acquisition of livestock location information and assessment of their pen status are of great significance for herd management under wild conditions.

[0003] Traditional methods for real-time livestock location tracking and pen-keeping assessment typically involve constructing artificial fences. As long as the fences remain intact, the herd is considered to be within the fenced area. However, this method restricts livestock movement, impacting their natural behavior and adaptability to the ecological environment. Furthermore, it leads to overgrazing, resource imbalances, and accelerated grassland degradation.

[0004] With the continuous development of positioning technology, wearable positioning devices such as collars and electronic ear tags have gradually replaced traditional physical fences. However, wearable positioning devices suffer from problems such as easy damage, cumbersome maintenance, and impact on the comfort of the herd. In recent years, the accelerated integration of sensor technology and intelligent grazing components has enabled contactless and intelligent livestock presence detection methods to solve many problems associated with traditional fences and wearable fences, providing ranches with a more intelligent and efficient livestock presence detection solution.

[0005] Therefore, how to combine sensor technology with the determination of livestock pen status to conduct accurate non-contact livestock position measurement is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] In view of this, the present invention provides a method and system for determining livestock presence in a pen based on non-contact measurement.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] A method for determining livestock presence in a pen based on non-contact measurement includes the following steps:

[0009] Step 1: Acquire depth and visible light images of the grazing herd;

[0010] Step 2: Perform image fusion and image segmentation on the depth image and the visible light image in sequence to obtain several segmented regions;

[0011] Step 3: Input the image of the segmented region into the livestock recognition model, determine whether the target in the segmented region is livestock, and take the segmented region where the determination result is livestock as the livestock location region;

[0012] Step 4: Perform midpoint measurement, distance calculation, and triangulation on the area where each livestock is located to obtain the geographical coordinates of the midpoint of the area where each livestock is located, which will be used as the geographical coordinates of the livestock.

[0013] Step 5: Based on the geographical coordinates of the points in the area where each livestock is located and the location of the virtual electronic fence, determine whether the livestock is within the range of the virtual electronic fence by using the relationship between points and lines.

[0014] Optionally, in step 2, a CNN convolutional neural network is used to perform image fusion on the depth image and the visible light image. The specific method is as follows:

[0015] For depth images and visible light images, two branches are designed to extract features from the depth images and visible light images respectively, obtaining the depth features F in the depth images. de Visible light features F in visible light images vi ;

[0016] For the depth feature F de and visible light characteristics F vi The images are then fused to generate a fused feature image.

[0017] Optionally, in step 2, image segmentation is performed on the fused feature image after image fusion. The specific method is as follows:

[0018] Preprocessing: Denoising, grayscale conversion, and size standardization are performed on the fused feature image;

[0019] Feature extraction: Local binary mode is used for feature extraction.

[0020]

[0021] Among them, LBP R,R Features extracted for local binary patterns, where P is the number of pixels in the neighborhood, R is the radius of the neighborhood, and g p It is the gray value of the p-th pixel in the neighborhood, g c s(x) is the gray value of the center pixel, and s(x) is the step function, which is 1 when x≥0 and 0 otherwise.

[0022] Algorithm segmentation: Based on the extracted features, the improved Canny edge detection algorithm is used to perform foreground-background segmentation on the fused feature image to obtain several segmented regions;

[0023] Post-processing: Morphological operations are used to dilate and erode the segmentation results to fill holes and connect broken edges; Canny edge detection is performed separately for each segmented region using a multi-scale edge detection method.

[0024] Optionally, in step 3, the process of establishing the livestock identification model is as follows:

[0025] Based on the YOLOv8m model, the modified backbone feature extraction network Swin Transformer was used. The number of output channels in the detection head was adjusted, and the size and ratio of the Anchor Box were optimized to obtain the basic framework of the livestock recognition model, which served as the initial model.

[0026] Images of grazing herds in the wild were captured, cropped, labeled, and converted into an image format suitable for YOLOv8 to serve as a dataset; the labeled information included the target category and bounding box location.

[0027] Using the dataset, with the cropped images as input features and the corresponding annotation information as output features, the initial model is trained to obtain a livestock recognition model.

[0028] Optionally, in step 4, the method for measuring the midpoint of the area where the livestock are located is as follows:

[0029] For each livestock area, iterate through all pixels within it, summing the x and y coordinates of each pixel. Divide the sum of the x coordinates by the number of pixels to get the average x-coordinate, and divide the sum of the y coordinates by the number of pixels to get the average y-coordinate. These two averages are the coordinates of the midpoint of the livestock area.

[0030]

[0031] Where, x i and y i is the coordinate of each pixel within the area where the livestock is located, and n is the total number of pixels within the area where the livestock is located.

[0032] Optionally, in step 4, the distance to the midpoint is calculated based on the midpoint measurement results of the area where the livestock are located. The specific method is as follows:

[0033] After obtaining the midpoint coordinates, it is necessary to obtain the midpoint coordinates in the camera coordinate system. The calculation formula is:

[0034]

[0035]

[0036] Z = depth;

[0037] Among them, (x avy y avy () represents the average coordinates of all pixels within the area where the livestock is located, i.e., the midpoint coordinates; depth represents the depth value corresponding to the midpoint coordinates; X, Y, and Z are the three-dimensional coordinates in the camera coordinate system; f is the focal length of the camera; (c x c y () represents the principal point coordinates of the camera;

[0038] The distance between the midpoint and the camera can be obtained using the Euclidean distance formula, as follows:

[0039]

[0040] The camera is mounted on the robot dog, and an inertial measurement unit, consisting of an accelerometer and a gyroscope, is also mounted on the robot dog to acquire the robot dog's motion state and direction.

[0041] Optionally, in step 4, triangulation is performed based on the depth calculation results to obtain the geographic coordinates of the midpoint. The specific method is as follows:

[0042] The geographical coordinates of the midpoint, i.e., latitude and longitude, are calculated using triangulation methods, as follows:

[0043]

[0044]

[0045] Among them, lat mid lom mid The coordinates of the midpoint are latitude and longitude; d represents the distance between the midpoint and the camera; heading indicates the robot dog's orientation; lat dog log dog This indicates the robot dog's latitude and longitude; ER represents the Earth's radius.

[0046] Optionally, in step 5, the method for determining whether livestock are within the virtual electronic fence area based on the relationship between points and lines is as follows:

[0047] The ray method is used to determine whether livestock are in the pen. A meridian is drawn from the livestock's position, i.e., the midpoint, in any direction. The number of intersections between this meridian and the virtual electronic fence is calculated. If the number of intersections is odd, it means that the livestock is inside the virtual electronic fence. If the number of intersections is even, it means that the livestock is outside the virtual electronic fence.

[0048] Optionally, when the intersection of the meridian and the virtual electronic fence is at the vertex, the vertex is raised above the meridian and the intersection of the meridian and the virtual electronic fence is recalculated; when the livestock is close to the boundary or the vertex, a second boundary is set at a preset distance in the inner circle of the virtual electronic fence. If the livestock is between the second boundary and the virtual electronic fence, it is determined that the livestock is about to leave the range of the virtual electronic fence.

[0049] A livestock pen presence detection system based on non-contact measurement includes:

[0050] The image acquisition module is used to acquire depth and visible light images of the grazing herd.

[0051] The image segmentation module is used to sequentially perform image fusion and image segmentation on the depth image and the visible light image to obtain several segmented regions;

[0052] The target recognition module is used to input the image of the segmented region into the livestock recognition model, determine whether the target in the segmented region is livestock, and take the segmented region where the determination result is livestock as the livestock location region;

[0053] The coordinate calculation module is used to perform midpoint measurement, distance calculation, and triangulation on the area where each livestock is located to obtain the geographic coordinates of the midpoint of the area where each livestock is located, which are used as the geographic coordinates of the livestock.

[0054] The fence determination module is used to determine whether livestock are within the virtual electronic fence area based on the geographical coordinates of the points in the area where each livestock is located and the location of the virtual electronic fence, by using the relationship between points and lines.

[0055] As can be seen from the above technical solution, the present invention provides a method and system for determining livestock presence in the pen based on non-contact measurement, which has the following advantages compared with the prior art:

[0056] (1) This invention employs non-contact measurement technology, overcoming the limitations of traditional physical fences and wearable devices. By integrating advanced sensor technology, this invention incorporates depth cameras and other equipment onto a robotic dog, enabling non-contact detection of livestock within a virtual electronic fence. This invention can monitor the livestock's location and behavior in real time, thus determining their presence without direct contact or carrying equipment on them. This invention not only improves the livestock's freedom of movement and reduces the impact on their comfort, but also effectively lowers the cost and labor intensity of ranch management.

[0057] (2) In view of the fact that the depth image has low resolution and the visible light image is easily affected by lighting conditions, occlusion and specular reflection, resulting in unstable image quality, blurred edges and color distortion, which limits its application in complex environments, this invention adopts the fusion of visible light image and depth image based on convolutional network. By comprehensively utilizing the three-dimensional perception capability of the depth image and the high-resolution information of the visible light image, it makes up for the defects of low resolution of depth image and visible light image being affected by lighting conditions, occlusion and specular reflection, thereby improving the stability, clarity and accuracy of the image, and improving the accuracy of image segmentation and livestock individual position judgment.

[0058] (3) To address the uncertainty and time-consuming nature of conventional pen-keeping judgment, this invention employs the ray method for pen-keeping judgment and provides special handling for special locations. A threshold distance is preset within the boundary range. If the distance of the livestock is less than the threshold range, it is determined that the livestock is about to leave the pen range. If the distance of the livestock is greater than the threshold range, the ray method is used to determine the position of the livestock, thereby improving the accuracy of pen-keeping judgment.

[0059] (4) This invention is based on the YOLOv8m network model and employs an adjusted backbone feature extraction network, SwinTransformer, to better capture various targets that may exist in a herd grazing scenario. Furthermore, the detection head has been improved, and the number of output channels has been adjusted to accommodate different categories and states. The size and proportion of the AnchorBox have been optimized to better adapt to targets of different sizes in the herd grazing scenario. Data augmentation techniques specifically designed for herd scenarios have been employed to simulate different lighting, occlusion, and long-distance observation conditions, thereby improving the model's generalization ability. Attached Figure Description

[0060] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0061] Figure 1 This is a schematic diagram illustrating the principle of the ray method.

[0062] Figure 2 A schematic diagram showing the intersection of the meridian of the target point and the virtual electronic fence at the vertex;

[0063] Figure 3 A schematic diagram showing different distances between the target point and the virtual electronic fence;

[0064] Figure 4 This is a schematic diagram of the method steps of the present invention. Detailed Implementation

[0065] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0066] This invention discloses a method for determining livestock presence in a pen based on non-contact measurement. See [link to relevant documentation]. Figure 4 This includes the following steps:

[0067] Step 1: Use a depth camera to acquire depth and visible light images of the grazing herd.

[0068] In the specific implementation process, the target herds / livestock can be cattle, sheep, horses, etc., and this invention does not limit them.

[0069] Step 2: Perform image fusion and image segmentation on the depth image and the visible light image in sequence to obtain several segmented regions.

[0070] (1) Image Fusion

[0071] Depth images possess efficient 3D perception capabilities, enabling accurate object localization and scene understanding, providing strong support for computer vision and machine perception applications. However, their limited resolution and sensitivity to lighting variations can affect accuracy in complex environments, and depth perception of transparent or reflective surfaces presents challenges. Visible light images, on the other hand, offer higher resolution and realism in representing the color, shape, and texture of objects, providing more natural visual information within the human eye's perceptible spectral range. Visible light images are also less affected by lighting variations, making them suitable for various environmental conditions. Therefore, fusing depth and visible light images is chosen to comprehensively utilize color, shape, and 3D depth information, improving the accuracy and comprehensiveness of scene understanding. Since occlusion occurs in grazing environments, image segmentation techniques are employed to segment the fused image, accurately obtaining the midpoint position of each animal in the fused image.

[0072] The fusion method employs a Convolutional Neural Network (CNN). Using CNNs for image fusion allows the system to automatically learn abstract features from images, gradually extracting and understanding hierarchical information through convolution and pooling operations. Specifically, for a pair of depth and visible light images, two branches are designed to process the depth and visible light images, allowing the outputs of the two branches to be fused at higher levels of the network to obtain feature representations. The feature representations are as follows:

[0073] F de =CNN(I de );

[0074] F vi =CNN(I vi );

[0075] Where I de For depth images, I vi This is a visible light image. F de F is represented as the depth features obtained by feature extraction from a depth image using a CNN. vi This represents the visible light features obtained by feature extraction from a visible light image using a CNN. CNN(·,·) represents the operation of a convolutional neural network.

[0076] Then, generate the fused feature image:

[0077] F fu =FusionLayer(F de ,F vi );

[0078] FusionLayer(·,·) is the operation of the fusion layer, which aims to fuse the features of the depth image and the visible light image to generate the final fused features.

[0079] (2) Image segmentation

[0080] Because the filming environment was a grazing field, the actual grazing of livestock would result in overlapping of herds, leading to overlapping in the merged images. Therefore, image segmentation is necessary to obtain the midpoint position of each animal. The image segmentation steps include preprocessing, feature extraction, algorithmic segmentation, and post-processing. The output of each step serves as the input for the next step. Specifically:

[0081] 1) Preprocessing: Denoising, grayscale conversion, and size standardization are performed on the fused feature image to optimize the original image and improve the stability and performance of the segmentation algorithm;

[0082] 2) Feature Extraction: Feature extraction involves extracting features from each fused image. Since the color, texture, and edge features of individual herd members are relatively obvious, Local Binary Patterns (LBP) are used for feature extraction to effectively distinguish the fused image from the environmental background. The formula for calculating LBP is:

[0083]

[0084]

[0085] Among them, LBP R,R Features extracted for local binary patterns, where P is the number of pixels in the neighborhood, R is the radius of the neighborhood, and g p It is the gray value of the p-th pixel in the neighborhood, g c s(x) is the gray value of the center pixel, and s(x) is the step function, which is 1 when x≥0 and 0 otherwise.

[0086] 3) Segmentation Algorithm: The segmentation algorithm employs an improved Canny edge detection algorithm. First, the visible light image, depth image, and fused image of the livestock are combined to form a multi-channel input. Then, an adaptive thresholding strategy is used to dynamically adjust the Canny edge detection threshold based on local contrast to adapt to the image characteristics of different regions. Foreground-background segmentation is performed on the fused image, dividing it into different regions to obtain several segmented regions.

[0087] 4) Post-processing: Morphological operations are used to dilate and erode the segmentation results to fill holes and connect broken edges, enhancing the continuity of the segmentation; combined with multi-scale edge detection methods, Canny edge detection is performed separately for each segmented region to obtain more comprehensive edge information.

[0088] Step 3: Input the image of the segmented region into the livestock recognition model, determine whether the target in the segmented region is livestock, and take the segmented region where the determination result is livestock as the livestock location region.

[0089] The specific process for establishing the livestock identification model is as follows:

[0090] (1) Building the model framework

[0091] To identify grazing herds in a field environment and ensure accurate pen detection, the YOLOv8m model was customized and optimized. First, a deep feature extraction network better suited to complex scenes and varying lighting conditions was chosen. A modified backbone feature extraction network, the Swin Transformer, was employed. The Swin Transformer effectively captures multi-scale features in images through an innovative hierarchical window self-attention mechanism and a staged design, better capturing various targets that may exist in grazing scenes. In the detection head, the number of output channels was adjusted to accommodate different categories and states, and the size and proportion of the anchor boxes were optimized to better accommodate targets of different sizes in grazing scenes. Furthermore, data augmentation techniques specifically designed for grazing scenes were employed to simulate different lighting, occlusion, and long-distance observation conditions, thereby improving the model's generalization ability.

[0092] (2) Data preparation

[0093] Data acquisition: Images of grazing herds in the wild were captured, cropped, labeled, and converted into an image format suitable for YOLOv8 to serve as a dataset; the labeled information included the target category and bounding box location.

[0094] Data Cleaning: The dynamic and complex nature of outdoor environments makes sensors susceptible to various uncontrollable factors, resulting in missing and outlier values. To effectively address these issues, appropriate measures need to be taken during data acquisition and processing. One method for handling missing values ​​is imputation using the mean. First, identify the missing values ​​in the data and mark their locations using statistical methods or visualization tools. Next, for features containing missing values, calculate the mean of their non-missing values ​​using the following formula:

[0095]

[0096] Where X i represents the non-missing feature values, and n is the number of non-missing values. Finally, we use the calculated mean X. mean The specific formula for filling missing values ​​is X. imp =X mean This effectively fills in missing values, making the dataset more complete in terms of missing values.

[0097] The approach used to handle outliers is based on unsupervised learning using autoencoders. First, an autoencoder model is built, consisting of an encoder and a decoder. The encoder compresses the input image into a compact representation, and the decoder attempts to reconstruct the original image. During training, the model is trained using normal images. During testing, the original image X is reconstructed from the input image, and the results are compared with the reconstructed image. Mean squared error (MSE) between:

[0098]

[0099] Where X represents the original input image. X represents the reconstructed image obtained by the autoencoder, where n represents the number of pixels or elements in the image, and X... i This shows the values ​​of the i pixels in the original image X. Reconstructed Image The value MSE of the i-th pixel represents the mean squared difference between the original image and the reconstructed image. A threshold is set; if the reconstruction error exceeds this threshold, the image is marked as abnormal.

[0100] Data standardization: Standardization is performed using the Min-Max method, with the following formula:

[0101]

[0102] Where X is the original data, X min It is the minimum value of the parameter, X max It is the maximum value of the parameter, while X norm This is the normalized data. Min-Max normalization helps handle different feature scales, making them have similar ranges, thus improving the stability and performance of the model.

[0103] (3) Model Training

[0104] The dataset is divided into a training set and a validation set. The training set is used to train the initial model with the cropped images as input features and the corresponding annotation information as output features, after configuring the model parameters (such as image size, number of categories, learning rate, etc.).

[0105] The model's output features include: 1. Target category prediction: Predicting the category of each detected target. 2. Bounding box location prediction: Predicting the location of the bounding box for each detected target, represented by the center coordinates, width, and height of the bounding box. 3. Target confidence: Representing the model's confidence in each detected target, i.e., the probability that the target is contained within the bounding box.

[0106] (4) Model evaluation and parameter optimization

[0107] To verify the performance of the developed model, a validation set was used to compare the model's results with the actual results. Since a single metric is unlikely to comprehensively reflect all aspects of the model's performance, mean squared error (MSE) and goodness of fit (R²) were employed. 2 The model is evaluated comprehensively using three metrics: mean relative error (ARE) and mean relative error (ARE).

[0108] Mean square error:

[0109]

[0110] Where n is the number of samples, y i For the i-th actual value, Let be the i-th predicted value.

[0111] Goodness of fit:

[0112]

[0113] in, The mean is the goodness of fit R. 2 The value of R ranges from 0 to 1. 2 The closer the value is to 1, the better the model fits.

[0114] Mean relative error:

[0115]

[0116] Among them, ARE≥0 and the smaller the value of ARE, the better the model fit.

[0117] After validating the model's performance using the above metrics, the model parameters are adjusted based on the performance results to obtain the final model.

[0118] Step 4: Perform midpoint measurement, distance calculation, and triangulation on the area where each livestock is located to obtain the geographical coordinates of the midpoint of the area where each livestock is located, which will be used as the geographical coordinates of the livestock.

[0119] (1) Midpoint measurement

[0120] For each livestock area, iterate through all pixels within it, summing the x and y coordinates of each pixel. Divide the sum of the x coordinates by the number of pixels to get the average x-coordinate, and divide the sum of the y coordinates by the number of pixels to get the average y-coordinate. These two averages are the coordinates of the midpoint of the livestock area.

[0121]

[0122] Where, x i and y i is the coordinate of each pixel within the area where the livestock is located, and n is the total number of pixels within the area where the livestock is located.

[0123] (2) Depth calculation

[0124] After obtaining the midpoint coordinates, it is necessary to obtain the midpoint coordinates in the camera coordinate system. The calculation formula is:

[0125]

[0126]

[0127] Z = depth;

[0128] Among them, (x avy y avy () represents the average coordinates of all pixels within the area where the livestock is located, i.e., the midpoint coordinates; depth represents the depth value corresponding to the midpoint coordinates; X, Y, Z are the three-dimensional coordinates in the camera coordinate system; f is the focal length of the camera; (c x c y () represents the principal point coordinates of the camera;

[0129] The distance between the midpoint and the camera can be obtained using the Euclidean distance formula, as follows:

[0130]

[0131] A camera is mounted on the robot dog, which uses the equipment on the robot dog to perform herding work. Since the acquired distance can be represented as all circular areas with the robot dog as the center and the distance as the radius, an inertial measurement unit is installed on the robot dog to address this issue. This unit consists of an accelerometer and a gyroscope, which can accurately obtain the robot dog's motion state and direction, avoiding directional errors and other situations.

[0132] (3) Triangulation

[0133] The geographical coordinates of the midpoint, i.e., latitude and longitude, are calculated using triangulation methods, as follows:

[0134]

[0135]

[0136] Among them, lat mid lom mid The coordinates of the midpoint are latitude and longitude; d represents the distance between the midpoint and the camera; heading indicates the robot dog's orientation; lat dog lon dog This indicates the robot dog's latitude and longitude; ER represents the Earth's radius.

[0137] Step 5: Based on the geographical coordinates of the points in the area where each livestock is located and the location of the virtual electronic fence, determine whether the livestock is within the range of the virtual electronic fence by using the relationship between points and lines.

[0138] An improved Pnpoly algorithm, particularly the ray casting method, is used to determine if livestock are in the pen. The principle is to draw a meridian in any direction from the livestock's position (midpoint), and calculate the number of intersections between this meridian and the virtual electronic fence. If the number of intersections is odd, the livestock is inside the virtual electronic fence; if the number is even, the livestock is outside. Figure 1 As shown, for ease of standardization and statistics, both target point A and target point B emit meridians in a due east direction. The meridian of target point A intersects the boundary of the fence at two points, so target point A is not within the fence area. The meridian of target point B intersects the fence at one point, so target point B is within the fence area.

[0139] Next, we'll handle special positions:

[0140] 1) When the intersection of the meridian and the virtual electronic fence is at the vertex, move the vertex above the meridian and then recalculate the intersection of the meridian and the virtual electronic fence (e.g., Figure 2 When denoted as A, B, D, or E in the diagram, the vertex should be raised above the horizontal plane of the ray (e.g., ...). Figure 2As shown, vertex A is raised above the ray of the target point x, becoming vertex A', and then the intersection of the ray and the polygon boundary is recalculated. Figure 2 Taking the target points x, y, and z as an example, the number of intersection points after the vertices A, B, D, and E are lifted are 2, 1, and 0 respectively.

[0141] 2) When livestock are near the boundary or apex, a threshold needs to be set because errors at the apex or boundary could lead to misjudgments. Therefore, a second boundary is set at a preset distance within the inner circle of the virtual electronic fence; this preset distance can be 0.5m-1m. If livestock are between the second boundary and the virtual electronic fence, it is determined that the livestock is about to leave the virtual electronic fence area; otherwise, the ray casting method is used to determine the livestock's position. Figure 3 As shown, target point A is located between the second boundary and the virtual electronic fence, indicating that it is about to leave the virtual electronic fence area. The distance between target point B and the virtual electronic fence is greater than the preset distance, so the ray method is used to determine its position.

[0142] Another embodiment of the present invention provides a livestock pen presence determination system based on non-contact measurement, comprising:

[0143] The image acquisition module is used to acquire depth and visible light images of the grazing herd.

[0144] The image segmentation module is used to sequentially perform image fusion and image segmentation on the depth image and the visible light image to obtain several segmented regions;

[0145] The target recognition module is used to input the image of the segmented region into the livestock recognition model, determine whether the target in the segmented region is livestock, and take the segmented region where the determination result is livestock as the livestock location region;

[0146] The coordinate calculation module is used to perform midpoint measurement, distance calculation, and triangulation on the area where each livestock is located to obtain the geographic coordinates of the midpoint of the area where each livestock is located, which are used as the geographic coordinates of the livestock.

[0147] The fence determination module is used to determine whether livestock are within the virtual electronic fence area based on the geographical coordinates of the points in the area where each livestock is located and the location of the virtual electronic fence, by using the relationship between points and lines.

[0148] The system modules disclosed in the embodiments are described simply because they correspond to the methods disclosed in the embodiments. For relevant details, please refer to the method section.

[0149] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0150] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for determining livestock presence in a pen based on non-contact measurement, characterized in that, Includes the following steps: Step 1: Acquire depth and visible light images of the grazing herd; Step 2: Perform image fusion and image segmentation on the depth image and the visible light image in sequence to obtain several segmented regions; Step 3: Input the image of the segmented region into the livestock recognition model, determine whether the target in the segmented region is livestock, and take the segmented region where the determination result is livestock as the livestock location region; Step 4: Perform midpoint measurement, distance calculation, and triangulation on the area where each livestock is located to obtain the geographical coordinates of the midpoint of the area where each livestock is located, which will be used as the geographical coordinates of the livestock. Step 5: Based on the geographical coordinates of the points in the area where each livestock is located and the location of the virtual electronic fence, determine whether the livestock is within the range of the virtual electronic fence by using the relationship between points and lines; In step 2, a CNN convolutional neural network is used to fuse the depth image and the visible light image. The specific method is as follows: For depth images and visible light images, two branches are designed to extract features from the depth images and visible light images respectively, obtaining the depth features from the depth images. Visible light features in visible light images ; For the depth features and visible light characteristics The images are then fused to generate a fused feature image.

2. The method for determining livestock presence in a pen based on non-contact measurement according to claim 1, characterized in that, In step 2, image segmentation is performed on the fused feature image after image fusion. The specific method is as follows: Preprocessing: Denoising, grayscale conversion, and size standardization are performed on the fused feature image; Feature extraction: Local binary mode is used for feature extraction. ； in, Features extracted from local binary patterns It is the number of pixels in the neighborhood. It is the radius of the domain. It is the gray value of the p-th pixel in the neighborhood. s(x) is the gray value of the center pixel, and s(x) is the step function, which is 1 when x≥0 and 0 otherwise. Algorithm segmentation: Based on the extracted features, the improved Canny edge detection algorithm is used to perform foreground-background segmentation on the fused feature image to obtain several segmented regions; Post-processing: Morphological operations are used to dilate and erode the segmentation results to fill holes and connect broken edges; Canny edge detection is performed separately for each segmented region using a multi-scale edge detection method.

3. The method for determining livestock presence in a pen based on non-contact measurement according to claim 1, characterized in that, In step 3, the process of establishing the livestock identification model is as follows: Based on the YOLOv8m model, the modified backbone feature extraction network Swin Transformer was used. The number of output channels in the detection head was adjusted, and the size and ratio of the Anchor Box were optimized to obtain the basic framework of the livestock recognition model, which served as the initial model. Images of grazing herds in the wild were captured, cropped, labeled, and converted into an image format suitable for YOLOv8 to serve as a dataset; the labeled information included the target category and bounding box location. Using the dataset, with the cropped images as input features and the corresponding annotation information as output features, the initial model is trained to obtain a livestock recognition model.

4. The method for determining livestock presence in a pen based on non-contact measurement according to claim 1, characterized in that, In step 4, the method for measuring the midpoint of the area where the livestock are located is as follows: For each livestock area, iterate through all pixels within it, summing the x and y coordinates of each pixel. Divide the sum of the x coordinates by the number of pixels to obtain the average x-coordinate, and divide the sum of the y coordinates by the number of pixels to obtain the average y-coordinate. These two averages are the coordinates of the midpoint of the livestock area. ； in, and is the coordinate of each pixel within the area where the livestock is located, and n is the total number of pixels within the area where the livestock is located.

5. The method for determining livestock presence in a pen based on non-contact measurement according to claim 4, characterized in that, In step 4, the distance to the midpoint is calculated based on the midpoint measurement results of the area where the livestock are located. The specific method is as follows: After obtaining the midpoint coordinates, it is necessary to obtain the midpoint coordinates in the camera coordinate system. The calculation formula is as follows: ； ； ； in, The midpoint coordinates represent the average coordinates of all pixels within the area where the livestock are located; depth represents the depth value corresponding to the midpoint coordinates; X, Y, and Z are the three-dimensional coordinates in the camera coordinate system. It is the camera's focal length; () represents the principal point coordinates of the camera; The distance between the midpoint and the camera can be obtained using the Euclidean distance formula, as follows: ； The camera is mounted on the robot dog, and an inertial measurement unit, consisting of an accelerometer and a gyroscope, is also mounted on the robot dog to acquire the robot dog's motion state and direction.

6. The method for determining livestock presence in a pen based on non-contact measurement according to claim 5, characterized in that, In step 4, triangulation is performed based on the depth calculation results to obtain the geographic coordinates of the midpoint. The specific method is as follows: The geographical coordinates of the midpoint, i.e., latitude and longitude, are calculated using triangulation methods, as follows: ； ； in, , Indicates the latitude and longitude of the midpoint; This indicates the distance between the midpoint and the camera; Indicates the robot dog's orientation; , This indicates the latitude and longitude information of the robot dog; This represents the Earth's radius.

7. The method for determining livestock presence in a pen based on non-contact measurement according to claim 1, characterized in that, In step 5, the method for determining whether livestock are within the virtual electronic fence area based on the relationship between points and lines is as follows: The ray method is used to determine whether livestock are in the pen. A meridian is drawn from the livestock's position, i.e., the midpoint, in any direction. The number of intersections between this meridian and the virtual electronic fence is calculated. If the number of intersections is odd, it means that the livestock is inside the virtual electronic fence. If the number of intersections is even, it means that the livestock is outside the virtual electronic fence.

8. The method for determining livestock presence in a pen based on non-contact measurement according to claim 7, characterized in that, When the intersection of the meridian and the virtual electronic fence is at the vertex, the vertex is raised above the meridian and the intersection of the meridian and the virtual electronic fence is recalculated. When the livestock is close to the boundary or the vertex, a second boundary is set at a preset distance in the inner circle of the virtual electronic fence. If the livestock is between the second boundary and the virtual electronic fence, it is determined that the livestock will leave the range of the virtual electronic fence.

9. A livestock pen presence detection system based on non-contact measurement, characterized in that, The livestock presence determination method based on non-contact measurement according to any one of claims 1 to 8, wherein the livestock presence determination system comprises: The image acquisition module is used to acquire depth and visible light images of the grazing herd. The image segmentation module is used to sequentially perform image fusion and image segmentation on the depth image and the visible light image to obtain several segmented regions; The target recognition module is used to input the image of the segmented region into the livestock recognition model, determine whether the target in the segmented region is livestock, and take the segmented region where the determination result is livestock as the livestock location region; The coordinate calculation module is used to perform midpoint measurement, distance calculation, and triangulation on the area where each livestock is located to obtain the geographic coordinates of the midpoint of the area where each livestock is located, which are used as the geographic coordinates of the livestock. The fence determination module is used to determine whether livestock are within the virtual electronic fence area based on the geographical coordinates of the points in the area where each livestock is located and the location of the virtual electronic fence, by using the relationship between points and lines.

Images