Feeding method with scene perception and space positioning functions and feeding trolley

Abstract

The invention belongs to the technical field of calf breeding, and provides a feeding method and a feeding trolley with scene perception and spatial localization, and the method comprises the steps: obtaining calf feeding scene image information through a depth camera; judging whether a calf, a calf island, a milk bucket and a milk bucket opening exist in the image or not through a YOLOv8 network structure, and if yes, calculating three-dimensional space coordinates of the calf, the calf island, the milk bucket and the milk bucket opening by taking the depth camera as an original point; according to the three-dimensional space coordinates, the relative position of a milk adding pipe and a milk barrel opening and the relative positions of the calf, the calf island and the milk barrel are judged; according to the relative position of a milk adding pipe on the feeding trolley and a milk barrel opening, the three-dimensional module on the feeding trolley controls the milk adding pipe to move to the milk barrel opening, and milk is added into the milk barrel; judging whether the calf drinks milk or not according to the relative positions of the calf, the calf island and the milk barrel. The three-dimensional space information is constructed by detecting the information of the calf feeding scene to solve the positioning problem, milk is added through the three-dimensional module, and whether the calf drinks milk or not is judged.

Description

Technical Field

[0001] The invention belongs to the technical field of calf breeding and provides a feeding method and a feeding trolley with scene perception and spatial positioning. Background Art

[0002] Calf feeding is a crucial and basic link in the calf breeding process, and its feeding status is directly related to the healthy development, growth rate and overall survival rate of the calf. With the expansion of modern animal husbandry and the improvement of production efficiency requirements, the traditional semi-automatic feeding method based on manual labor has gradually exposed problems such as high labor intensity and high management costs. In order to achieve intelligent and unmanned feeding of calves, it is urgent to use advanced intelligent technology to detect and locate multiple targets in the calf feeding environment, develop a feeding device to replace manual feeding, and continuously track calves. However, due to the complex targets and severe background interference in the calf feeding environment, traditional computer vision, tracking methods, and feeding devices still have obvious limitations, which are specifically manifested in the following aspects:

[0003] Complex background interference: Detection in the calf feeding environment faces complex background interference, including complex background factors such as ground texture and fences, which can easily interfere with target feature extraction and affect detection accuracy.

[0004] Large variations in target size: Detection in a calf feeding environment faces the problem of large target variations. Calf feeding scenarios involve multiple targets, including calves, calf hutches, milk buckets (leglens), and milk bucket mouths. These targets vary in size, shape, and material. For example, calves have diverse shapes, the calf hutch is relatively large, and the milk bucket mouth is relatively small, making traditional detection methods difficult to apply.

[0005] Traditional feeding equipment has a limited degree of automation and is mainly manual, with problems such as high labor intensity and high management costs. Summary of the Invention

[0006] In order to solve the above technical problems, the present invention provides a feeding method and a feeding cart with scene perception and spatial positioning. The method can detect the information of the calf feeding scene, construct the three-dimensional spatial information of the calf, the calf island, the milk bucket and the milk bucket mouth, solve the positioning problem of the calf, the calf island, the milk bucket and the milk bucket mouth, and then add milk through the three-dimensional module, and determine whether the calf drinks the milk after adding milk.

[0007] The technical solution of the present invention comprises the following steps: S100: Acquire image information of a calf feeding scene.

[0008] S200: Analyze the image information of the calf feeding scene obtained through the YOLOv8 network structure to determine whether there are calves, calf islands, milk buckets, and milk bucket openings in the image. If not, the feeding cart continues to move. If so, calculate the three-dimensional spatial coordinates of the calf, calf island, milk bucket, and milk bucket opening with the depth camera as the origin.

[0009] S300: Determine the relative position of the milking tube and the milk bucket opening according to the three-dimensional spatial coordinates, determine the relative position of the calf, the calf island, and the milk bucket, and fuse the three-dimensional spatial coordinate information of the calf with the Kalman filter algorithm to track the calf in real time.

[0010] S400: According to the relative position of the milk adding tube on the feeding trolley and the opening of the milk bucket, the three-dimensional module on the feeding trolley controls the milk adding tube to move to the opening of the milk bucket to add milk into the milk bucket.

[0011] S500: Determine whether the calf drinks milk based on the relative positions of the calf, the calf island, and the milk bucket. If the calf does not drink milk, the feeding cart issues an early warning and records the information.

[0012] Furthermore, the BiFPN network is introduced into the image analysis process of the YOLOv8 network structure. The BiFPN network processing includes the following steps: S201: Inputting different levels of feature maps generated by the backbone network during the YOLOv8 network structure image processing process into the BiFPN network.

[0013] S202: High-level features are fused with adjacent low-level features through upsampling.

[0014] S203: The low-level features are fused with the adjacent high-level features through downsampling.

[0015] S204: Cross-scale skip connection, retaining the direct connection of the original input feature map to avoid information loss.

[0016] S205: Assign a learnable weight to each input feature map to balance the contributions of features of different resolutions and obtain an output feature map.

[0017] Furthermore, in step S200, SCAtention is introduced into the image analysis process of the YOLOv8 network structure, including the following steps: S211: Input the output feature map in the BiFPN network into the SCAttention module as the starting point for subsequent attention calculations.

[0018] S212: Calculate self-attention through CSMM to calculate the attention score.

[0019] S213: After the multi-head attention calculation, the output of the multi-head is weighted averaged through Average.

[0020] S214: Use Channel exp to adjust the dimension of the output or further process it to improve the expressiveness of the model.

[0021] S215: Improve target positioning accuracy through Spatial Attention, allowing the model to focus on the target area and reduce false detections.

[0022] Furthermore, in step S200, when calculating the three-dimensional coordinates of the calf island, the calf, the milk bucket, and the milk bucket mouth, the three-dimensional coordinates are calculated by using an eight-area average weighted method, including the following steps: S231: Calculate the two-dimensional coordinates of the center points of the detection frames of the calf, the calf island, the milk bucket, and the milk bucket mouth.

[0023] S232: Based on the two-dimensional coordinates of the center points of the detection frames of the calf, the calf island, the milk bucket, and the milk bucket mouth, a preliminary depth distance is obtained through the depth camera, and then valid area points are screened out based on the preliminary depth distance.

[0024] S233: Assign a weight to each selected field point, and then perform weighted average to calculate the accurate depth distance.

[0025] S234: Calculate the distance from the camera to the center point of the detection frame based on the precise depth distance.

[0026] S235: Obtain the three-dimensional coordinates of the center point of the detection frame of the calf, the calf island, the milk bucket, and the milk bucket mouth according to the distance from the detection camera to the center point of the detection frame.

[0027] Furthermore, in step S300, when determining the relative position, the three-dimensional spatial coordinates of the calf are integrated with the Kalman filter algorithm to track the calf in real time, including the following steps: S341: Initialize the state vector of the Kalman filter using the three-dimensional coordinates of the center point of the calf detection frame.

[0028] S342: The Kalman filter predicts the position and speed of the calf in the current frame based on the updated state vector of the previous frame.

[0029] Furthermore, in step S500: When the distance between the calf and the milk bucket is greater than 1 meter and the distance between the calf and the calf island is less than 1.5 meters, it is judged that the calf is not drinking milk; When the distance between the calf and the milk bucket is less than or equal to 1 meter and the distance between the calf and the calf island is greater than 1.5 meters, the calf is judged to be feeding.

[0030] The present invention also provides a feeding trolley, comprising: Car body.

[0031] A milk tank is fixed to the upper end of the vehicle body.

[0032] The three-dimensional module is fixed on the upper end of the vehicle body.

[0033] The milking assembly includes a pump and a milk pipe. The pump is fixed to the upper end of the vehicle body. The input end of the pump is connected to the milk tank. The output end of the pump is connected to the milk pipe. The milk tank is fixed to the three-dimensional module.

[0034] A depth camera is fixed on the upper end of the vehicle body.

[0035] The laser radar is fixed on the upper end of the vehicle body.

[0036] The technical solution provided by the embodiment of the present invention has the following advantages compared with the existing technology: Obtain image information of the calf feeding scene, analyze the obtained calf feeding scene image information through the YOLOv8 network structure, and determine whether there are calves, calf islands, milk buckets and milk bucket openings in the image. If not, the feeding cart continues to move. If so, the calf, calf island, milk bucket and milk bucket opening are calculated with the depth camera as the origin in three-dimensional space coordinates. The relative position of the milk adding pipe and the milk bucket opening, the relative position of the calf, the calf island and the milk bucket are determined based on the three-dimensional space coordinates. Based on the relative position of the milk adding pipe on the feeding cart and the milk bucket opening, the three-dimensional module on the feeding cart controls the milk adding pipe to move to the milk bucket opening and add milk into the milk bucket. Based on the relative position of the calf, the calf island and the milk bucket, determine whether the calf is drinking milk. If there is no calf and it is not drinking milk, the feeding cart issues an early warning and records the information. Compared with the existing technology, the present invention can construct three-dimensional spatial information of the calf, calf island, milk bucket and milk bucket mouth by detecting information about the calf feeding scene, solve the positioning problem of the calf, calf island, milk bucket and milk bucket mouth, and then add milk through the three-dimensional module, and judge whether the calf drinks milk after adding milk.

[0037] Other advantages, objectives and features of the present invention will be reflected in part from the following description and will be understood by those skilled in the art through study and practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0039] Figure 1 This is the overall network architecture diagram of the improved YOLOv8 in this invention.

[0040] Figure 2 This is a network structure diagram of the SCAtention model structure of the present invention; Figure 3 This is a diagram of the bidirectional feature pyramid network structure of the present invention; Figure 4 This is the overall flow chart of calf feeding of the present invention; Figure 5 This is a statistical diagram of examples of calves, calf islands, milk barrels, and milk barrel openings of the present invention; Figure 6 A size distribution diagram of the bounding box predicted by the model of the present invention; Figure 7 This is a frequency information distribution diagram of the target appearing at the horizontal and vertical coordinate positions of the present invention; Figure 8 This is the target frame width and height distribution diagram of the present invention; Figure 9 This is a diagram showing the effect of concentrated detection in the test of the present invention; Figure 10 The actual detection effect of the present invention in different scene models Figure 1 ; Figure 11 The actual detection effect of the present invention in different scene models Figure 2 ; Figure 12 This is a schematic diagram of the feeding trolley structure of one embodiment of the present invention. Figure 1 ; Figure 13 This is a schematic diagram of the feeding trolley structure of one embodiment of the present invention. Figure 2 .

[0041] Reference numerals: 1. Vehicle body; 2. Milk tank; 3. 3D module; 4. Pump; 5. Milk tube; 6. Depth camera; 7. LiDAR. DETAILED DESCRIPTION

[0042] A specific embodiment of the present invention is described in detail below with reference to the accompanying drawings, but it should be understood that the protection scope of the present invention is not limited by the specific embodiment.

[0043] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "axial", "radial", "circumferential" and the like to indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the technical solutions of the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0044] In the description of the embodiments of the present invention, unless otherwise specified, “a plurality of” means two or more.

[0045] like Figures 1 to 13 As shown, the present invention provides a feeding method with scene perception and spatial positioning, and the technical solution of the present invention includes the following steps: S100: The feeding trolley is moved in the calf feeding passage by means of navigation by the laser radar 7 on the feeding trolley, while the depth camera 6 on the feeding trolley acquires image information of the feeding scene.

[0046] S200: Analyze the image information of the calf feeding scene obtained through the YOLOv8 network structure to determine whether there are calves, calf islands, milk buckets and milk bucket openings in the image. If not, the feeding cart continues to move. If so, calculate the three-dimensional spatial coordinates of the calf, calf island, milk bucket and milk bucket opening with the depth camera 6 as the origin.

[0047] S300: Determine the relative position of the milk feeding pipe 5 and the milk bucket opening according to the three-dimensional spatial coordinates, and determine the relative position of the calf, the calf island, and the milk bucket.

[0048] S400: According to the relative position of the milk adding tube 5 on the feeding trolley and the milk bucket opening, the three-dimensional module 3 on the feeding trolley controls the milk adding tube 5 to move to the milk bucket opening to add milk into the milk bucket.

[0049] S500: Determine whether the calf is drinking milk based on the relative positions of the calf, the calf island, and the milk bucket. If the calf is not drinking milk, the feeding cart will issue an early warning and record the information.

[0050] The feeding cart is navigated by the laser radar 7 on the feeding cart to move in the calf feeding channel. At the same time, the depth camera 6 on the feeding cart obtains image information of the calf feeding scene, and the obtained image information of the calf feeding scene is analyzed by the YOLOv8 network structure to determine whether there is a calf, a calf island, a milk bucket and a milk bucket mouth in the image. If not, the feeding cart continues to move. If so, the calf, the calf island, the milk bucket and the milk bucket mouth are calculated with the depth camera 6 as the origin in three-dimensional space coordinates. The relative position of the milk adding pipe 5 and the milk bucket mouth, the relative position of the calf, the calf, the calf island and the milk bucket are determined according to the three-dimensional space coordinates. According to the relative position of the milk adding pipe 5 on the feeding cart and the milk bucket mouth, the three-dimensional module 3 on the feeding cart controls the milk adding pipe 5 to move to the milk bucket mouth and add milk into the milk bucket. According to the relative position of the calf, the calf island and the milk bucket, it is determined whether the calf is drinking milk. If there is no calf and the milk is not drinking, the feeding cart issues an early warning and records the information. Compared with the existing technology, the present invention can construct the three-dimensional coordinate information of the calf, calf island, milk bucket, and milk bucket mouth by detecting the image information of the calf feeding scene, solve the positioning problem of the calf, calf island, milk bucket, and milk bucket mouth, and then add milk through the three-dimensional module 3, and determine whether the calf drinks milk after adding milk.

[0051] Specifically, the YOLOv8 network structure mainly consists of three parts: Backbone (backbone network), Neck (neck network), and Head (detection head).

[0052] The original YOLOv8 network structure includes Conv (classic convolution module), C2f (cross-stage feature fusion module), SPPF (spatial pyramid pooling module), Upsample module, and Detect (detection head module). Among them:

[0053] The Conv module is a standard convolution operation used to extract and process spatial information in feature maps.

[0054] The C2f module is a feature extraction module that can improve feature extraction efficiency and expression capabilities.

[0055] The SPPF module is an improved version of spatial pyramid pooling, which enhances the receptive field and robustness of features through multi-scale pooling.

[0056] The Upsample module is used to upsample the feature map to increase the spatial resolution of the feature map so that it can be fused with the high-resolution feature map.

[0057] The Detect module is the output layer of the object detection network, which is used to predict the bounding box, category and confidence of the object.

[0058] In the embodiment provided by the present invention, in step S200, a BiFPN network (bidirectional feature pyramid network) is introduced in the image analysis process of the YOLOv8 network structure. BiFPN includes the following steps: multi-scale input feature map operation, top-down path operation, bottom-up path operation, cross-scale jump connection operation and weighted feature fusion operation. Through the bidirectional feature pyramid network, the detection capability of targets of different scales such as calf island, calf, milk bucket and milk bucket mouth can be improved, while simplifying the calculation process and ensuring the speed and accuracy of target detection. Specifically:

[0059] S201: Multi-scale input feature maps, inputting different levels of feature maps generated by the backbone network during the YOLOv8 network structure image processing process into the BiFPN network. The input order is from high to low resolution and from shallow to deep semantic information.

[0060] S202: Top-down path, high-level features are fused with adjacent low-level features through upsampling.

[0061] S203: Bottom-up path, low-level features are fused with adjacent high-level features through downsampling.

[0062] S204: Cross-scale skip connection, retaining the direct connection of the original input feature map to avoid information loss.

[0063] S205: Weighted feature fusion, assigning a learnable weight to each input feature map, balancing the contribution of features of different resolutions, and obtaining an output feature map.

[0064] Specifically: Bidirectional Feature Pyramid Network is based on the traditional FPN (Feature Pyramid Network), and introduces a bidirectional feature fusion and weighting mechanism to improve the expression ability of multi-scale features. In response to background interference such as fences and ground textures, BiFPN (Bidirectional Feature Pyramid Network) enhances target recognition by fusing low-level details with high-level semantic features. In the face of large differences between targets such as the mouth of the milk bucket and the calf island, and the calf being obscured by the bucket cover, BiFPN can capture rich contextual information across scales, supplement missing features, and improve the detection accuracy of small targets and occluded targets. At the same time, it optimizes the calculation process to ensure detection speed and accuracy. The feature fusion network structure adopted in the embodiment of the present invention is as follows: Figure 3 As shown, P3, P4, P5, P6, and P7 represent feature map levels of different scales.

[0065] In the embodiment provided by the present invention, in step S200, as Figure 2As shown in the figure, the attention mechanism SCAttention is introduced in the image analysis process of the YOLOv8 network structure. The SCAttention attention mechanism includes Input (input data) operation, CSMM (computational self-attention) operation, Average (weighted average) operation, Channel exp (channel expansion) operation and Spatial Attention (spatial attention) operation. The attention mechanism can improve the detection accuracy of calves and calf islands when they are blocked. Specifically:

[0066] S211: Input means inputting the output feature map of the BiFPN network into the SCAttention module as the starting point for subsequent attention calculations.

[0067] S212: CSMM stands for Computational Self-Attention, which is used to calculate attention scores. This means that a multi-head self-attention mechanism calculates attention scores on feature maps and uses multi-scale region partitioning to capture global dependencies.

[0068] S213: After the multi-head attention calculation, the output of the multi-head is weighted averaged through Average. That is, Average performs average pooling on the output of multiple branches of the CSMM, fusing multi-scale features to generate a unified feature representation.

[0069] S214: Channel exp is used to adjust the output dimension or further process it to improve the model's expressiveness. Specifically, Channel exp uses 1×1 convolution to adjust the output channel dimension and enhance feature expression.

[0070] S215: Spatial Attention improves target positioning accuracy, allowing the model to focus on the target area and reduce false detections. Spatial Attention calculates the average and maximum values ​​along the channel dimension and uses 7×7 convolution to generate a spatial attention map. This feature map is weighted pixel by pixel based on the attention weights, enhancing the target area features and suppressing background noise, thereby improving target positioning accuracy and reducing false detections.

[0071] Specifically: The SCAttention module is an attention mechanism used to improve recognition performance in occluded situations. Traditional attention mechanisms may significantly affect feature extraction when the calf is occluded. SCAttention combines spatial attention and feature enhancement mechanisms to improve overall feature representation by enhancing the feature response of unoccluded areas, thereby improving detection performance when the calf is occluded. This not only improves recognition accuracy in occluded situations, but also improves the ability to understand features in complex scenes.

[0072] In the embodiment provided by the present invention, in order to train the above-mentioned improved YOLOv8 model, the present invention creates a data set of calf feeding scene target objects for training the detection model, which mainly includes calves, calf islands, milk buckets and milk bucket mouths. Since there are few public data sets and the targets are single, multiple images of calf feeding scene targets are collected. In order to ensure excellent detection effects in actual scenes, the images are all taken in real scenes. Since some extreme situations may be encountered during detection, images are collected at different angles and distances, and data enhancement operations are performed on the collected data to ensure the generalization of the data set. The data set contains 3000 images, which are randomly divided into training set, verification set and test set in a ratio of 8:1:1, resulting in 2400 training images, 300 verification images and 300 test images.

[0073] In the embodiment provided by the present invention, in step S200, when calculating the three-dimensional coordinates of the calf island, the calf, the milk bucket, and the milk bucket mouth, the three-dimensional coordinates are calculated by using the eight-area average weighted method, including the following steps: S231: Calculate the two-dimensional coordinates of the center points of the detection frames of the calf, the calf island, the milk bucket, and the milk bucket mouth.

[0074] S232: Based on the two-dimensional coordinates of the center points of the detection frames of the calf, the calf island, the milk bucket, and the milk bucket mouth, a preliminary depth distance is obtained through the depth camera, and then valid area points are screened out based on the preliminary depth distance.

[0075] S233: Assign a weight to each selected field point, and then perform weighted average to calculate the accurate depth distance.

[0076] S234: Calculate the distance from the camera to the center point of the detection frame based on the precise depth distance.

[0077] S235: Obtain the three-dimensional coordinates of the center point of the detection frame of the calf, the calf island, the milk bucket, and the milk bucket mouth according to the distance from the detection camera to the center point of the detection frame.

[0078] Specifically, first calculate the two-dimensional coordinates of the center points of the calf, calf island, milk bucket, and milk bucket mouth detection frame. The calculation formula is as follows: Where, 、 They represent the horizontal and vertical coordinates of the center point to be sought, respectively. 、 Represents the coordinates of the upper left vertex and the lower right vertex respectively.

[0079] Based on the two-dimensional coordinates of the center points of the calf, calf island, milk bucket, and milk bucket mouth detection frames, the depth camera obtains the depth distance and preliminarily screens out valid points based on the depth distance. The screening conditions are as follows: in, is the depth value of the center point of the calf, calf island, milk bucket, and milk bucket mouth detection frame, is the depth value of the eight field points, is the depth threshold.

[0080] Based on the filtered area points, a weight is assigned to the depth value of each neighboring point, and then a weighted average is performed to calculate a more accurate depth distance. The calculation formula is as follows:

[0081] Where, is the Euclidean distance from the surrounding area to the center point, and .

[0082] Calculate the depth distance of the center point using the following formula: Where, is the depth value of the eight field points, is the corresponding weight, The depth distance to the center point.

[0083] Combined with the center point depth distance, the three-dimensional coordinates of the center points of the calf, calf island, milk bucket, and milk bucket mouth detection frame are obtained ( , , ).

[0084] According to the three-dimensional coordinates of the mouth of the milk barrel, the three-dimensional module controls the pipeline to move to the mouth of the milk barrel and performs the milk adding operation through the tank body.

[0085] After adding milk, the depth camera captures the relative position of the calf, the calf island, and the milk bucket. When the distance between the calf and the milk bucket is greater than 1 meter and the distance between the calf and the calf island is less than 1.5 meters, it is determined that the calf is not feeding. The breeding nursing calf feeding cart will issue an alarm and record the information.

[0086] When the distance between the calf and the milk bucket is greater than 1 meter and the distance between the calf and the calf island is less than 1.5 meters, it is judged that the calf is not drinking milk; When the distance between the calf and the milk bucket is less than or equal to 1 meter, and the distance between the calf and the calf island is greater than 1.5 meters, the calf is judged to be feeding.

[0087] In the embodiment provided by the present invention, in step S300, when determining relative position, a Kalman filter algorithm is introduced to further improve the continuity and robustness of calf tracking in the calf feeding scene. The calf's three-dimensional spatial information is integrated with the Kalman filter algorithm to achieve real-time tracking of the calf target. The specific operation process is as follows:

[0088] When the improved YOLOv8 network detects a calf, the state vector of the Kalman filter is initialized with the three-dimensional coordinates of the center point of the calf detection box: in: is the 3D coordinate of the center point of the calf detection frame. is the velocity component of the calf in the image plane, T is the transpose sign in linear algebra, and the initial frame is usually set to 0, or calculated based on the position difference between the first two frames:

[0089] in, and is the spatial position of the calf in the current frame and the previous frame, is the time interval between two consecutive frames.

[0090] Status prediction: The Kalman filter uses the updated state vector of the previous frame to predict the position and speed of the calf in the current frame. The calculation formula is as follows: in: is the state vector predicted for the current frame, is the updated state vector of the previous frame, is the process noise covariance matrix, which represents the uncertainty of the model in the state prediction process.

[0091] State transition matrix Expressed as: Where, is the time interval between two consecutive frames.

[0092] Actual measurement: When a calf is detected, the coordinates of the center point of the calf detection frame in the current frame are obtained and depth information , combined to obtain the spatial position of the target , as the measurement value of the Kalman filter: Status Update: The actual measurement value and the predicted state are fused to calculate the Kalman gain matrix Then, update the state vector: in: is the updated state vector, which contains the updated center point position and velocity; Measurement Matrix is defined as: Kalman gain matrix The calculation formula is: in: is the state prediction error covariance matrix; To measure the noise covariance matrix, characterize the uncertainty in the YOLOv8 detection process.

[0093] The present invention proposes a precise positioning method based on the depth information obtained by the depth camera and the eight-neighborhood weighted averaging method, which provides high-quality spatial position information for the Kalman filter, making the Kalman filter prediction more accurate, thereby effectively improving the tracking accuracy of calves in real environments.

[0094] Based on YOLOv8 target detection and depth information positioning, this paper further proposes a real-time CALIB target tracking method based on Kalman filtering. Compared with traditional target tracking methods, the Kalman filtering algorithm of this invention has the following advantages in CALIB target tracking:

[0095] The stability of target state prediction is enhanced. Traditional target tracking algorithms are prone to tracking interruptions due to temporary occlusion of the target or missing target features. The present invention uses a Kalman filter to predict the target position and motion trajectory, so that even when the target is temporarily occluded, lost, or the detection features are insufficient, it can still maintain continuous and stable tracking of the target, thereby improving the stability of target tracking.

[0096] like Figure 12 and Figure 13 As shown, the present invention also provides a feeding trolley, comprising: Vehicle body 1.

[0097] The milk tank 2 is fixed on the upper end of the vehicle body 1 .

[0098] The three-dimensional module 3 is fixed on the upper end of the vehicle body 1 .

[0099] The milking assembly includes a pump 4 and a milk pipe 5. The pump 4 is fixed to the upper end of the vehicle body 1. The input end of the pump 4 is connected to the milk tank 2. The output end of the pump 4 is connected to the milk pipe 5. The milk tank 2 is fixed on the three-dimensional module 3.

[0100] The depth camera 6 is fixed on the upper end of the vehicle body 1 .

[0101] The laser radar 7 is fixed on the upper end of the vehicle body 1 .

[0102] When feeding, the intelligent feeding cart navigates to the side of the calf island through the laser radar 7, and then performs target detection and positioning through the depth camera 6 to construct the feeding scene. After obtaining the three-dimensional spatial information of the milk bucket mouth, the three-dimensional module 3 controls the milk adding pipe 5 to move to the center of the milk bucket mouth to implement the milk adding operation, and the depth camera 6 is used to monitor the milking status of the good-breed lactating calves in real time.

[0103] Three-dimensional spatial information fusion strategy. The Kalman filter algorithm of the present invention further combines depth information and proposes to use the three-dimensional coordinates of the center point of the calf detection frame as the state vector of the Kalman filter. It improves the traditional Kalman filter algorithm that only uses two-dimensional position information, adds the calf's depth information, and significantly improves the accuracy of target tracking. It overcomes the problems of large spatial position error and lack of depth information in traditional two-dimensional tracking methods when processing the spatial position between the calf and the milk bucket.

[0104] The first is the statistics and distribution of dataset labels, such as Figure 5 As shown in the figure, instances (number of instances) intuitively shows the proportion of different categories in the dataset, which can quickly understand the distribution of calf, calf hutch, milk bucket (leglen), and milk bucket mouth (mouth) in the dataset. Figure 6 As shown in the figure, the distribution of bounding boxes predicted by the model is shown. The distribution of bounding boxes is relatively concentrated, indicating that the model has a good positioning effect in a specific area. Figure 7 As shown in the figure, the x (horizontal coordinate) and y (vertical coordinate) in the image show the frequency of target appearance at different horizontal and vertical coordinate positions in the image. It can be seen that the objects are evenly distributed in the entire image, indicating that the YOLOv8 network model can effectively detect all parts of the image instead of concentrating on a small area. Figure 8 As shown in the figure, the width and height in the image reflect the relationship between the width and height of the target bounding box. It can be seen that the YOLOv8 network model can overcome the problem of large size differences between the calf, the calf island, the milk bucket, and the milk bucket mouth. Figures 5 to 8 It can be seen that the dataset constructed by the present invention includes four types of targets: calves, calf islands, milk buckets, and milk bucket mouths. The number of targets in each type is counted to intuitively reflect the distribution characteristics of each type of target in the dataset.

[0105] In order to reflect the effectiveness of the improved algorithm of this patent, the present invention selects mean average precision (mAP), precision (Precision), recall rate (Recall), inference speed (GFLOPs), frame rate per second (FPS), and model size (Modelsize) as evaluation indicators. The experimental results are shown in Table 1.

[0106] Table 1: Comparison of various indicators between the improved model and the original model During the process of improving the model, a large number of ablation experiments were conducted to explore the optimal structure of the model. The experimental results are shown in Table 2.

[0107] Table 2: Comparison of various indicators of ablation experiments In the subsequent model test, the detection algorithm proposed by the present invention showed excellent results. Figure 9 As shown in Figure 2, the actual detection effects in different scenarios are as follows: Figure 10 and Figure 11 shown.

[0108] The present invention has the following advantages: Powerful learning ability: YOLOv8 uses deep learning technology. It can train its own network model through a large amount of data sets, especially convolutional neural networks, which can obtain a large number of features from the data.

[0109] Optimization of network architecture: YOLOv8 improves the speed and accuracy of target detection through an improved model architecture and feature extractor, making the model more efficient and reliable.

[0110] Multi-scale object detection: YOLOv8 uses multi-scale feature maps to detect objects of different scales and improve the accuracy of object detection.

[0111] The threshold-based eight-area weighted averaging method can more accurately locate and detect targets, thereby obtaining more accurate depth values.

[0112] The present invention uses laser radar navigation and Zed2i depth camera for real-time detection and positioning, and a three-dimensional module to move the milking tube for milking operations, thereby realizing unmanned feeding and improving calf feeding efficiency.

[0113] The Kalman filter algorithm is introduced based on the YOLOv8 network. The calf's three-dimensional information is integrated with the Kalman filter algorithm to make tracking more stable.

[0114] In summary, the present invention can solve the problems of target detection and tracking technology in calf feeding scenes, such as background interference such as fences and ground textures, large target differences between the milk bucket mouth and the calf island, calf being blocked by the milk bucket cover resulting in unstable calf tracking, and low feeding efficiency and high cost of traditional fine breed lactating calves, through its optimized network architecture, target positioning method, three-dimensional spatial information fusion, and intelligent feeding.

[0115] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0116] Although the embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and exemplary embodiments. They can be applied to a variety of fields suitable for the present invention. Further modifications will be readily apparent to those skilled in the art. Therefore, the present invention is not limited to the specific details and illustrations shown and described herein without departing from the general concept defined by the claims and their equivalents.

Claims

1. A feeding method with scene perception and spatial positioning, characterized in that: The following steps are involved: S100: Acquire image information of a calf feeding scene; S200: Analyze the acquired calf feeding scene image information through the YOLOv8 network structure to determine whether there are calves, calf islands, milk buckets, and milk bucket openings in the image. If not, the feeding cart continues to move. If so, calculate the three-dimensional space coordinates of the calf island, calf, milk bucket, and milk bucket opening with the depth camera as the origin; S300: Determining the relative position of the milking tube and the milk bucket opening according to the three-dimensional spatial coordinates, and determining the relative position of the calf, the calf island, and the milk bucket; S400: Based on the relative position of the milk adding tube on the feeding trolley and the opening of the milk bucket, the three-dimensional module on the feeding trolley controls the milk adding tube to move to the opening of the milk bucket to add milk into the milk bucket; S500: Determine whether the calf drinks milk based on the relative positions of the calf, the calf island, and the milk bucket. If the calf does not drink milk, the feeding cart issues an early warning and records the information.

2. A feeding method with scene perception and spatial positioning according to claim 1, characterized in that: In step S200, the BiFPN network is introduced into the image analysis process of the YOLOv8 network structure. The BiFPN network processing includes the following steps: S201: Inputting different levels of feature maps generated by the backbone network during the YOLOv8 network structure image processing into the BiFPN network; S202: High-level features are fused with adjacent low-level features through upsampling; S203: low-level features are fused with adjacent high-level features through downsampling; S204: Cross-scale skip connection, retaining the direct connection of the original input feature map to avoid information loss; S205: Assign a learnable weight to each input feature map to balance the contributions of features of different resolutions and obtain an output feature map.

3. A feeding method with scene perception and spatial positioning according to claim 2, characterized in that: In step S200, SCAtention is introduced into the image analysis process of the YOLOv8 network structure, including the following steps: S211: Input the output feature map of the BiFPN network into the SCAttention module as the starting point for subsequent attention calculations; S212: Calculate self-attention through CSMM to calculate the attention score; S213: After the multi-head attention calculation, the output of the multi-head is weighted averaged through Average; S214: Use Channel exp to adjust the output dimension or further process it to improve the model's expressiveness; S215: Improve target positioning accuracy through Spatial Attention, allowing the model to focus on the target area and reduce false detections.

4. A feeding method with scene perception and spatial positioning according to claim 1, characterized in that: In step S200, when calculating the three-dimensional coordinates of the calf island, the calf, the milk bucket, and the milk bucket mouth, the three-dimensional coordinates are calculated by using the eight-area average weighted method, including the following steps: S231: Calculate the two-dimensional coordinates of the center points of the detection frames of the calf, the calf island, the milk bucket, and the milk bucket mouth; S232: Based on the two-dimensional coordinates of the center points of the detection frames of the calf, the calf island, the milk bucket, and the milk bucket mouth, a preliminary depth distance is obtained using a depth camera, and then valid area points are screened based on the preliminary depth distance. S233: assigning a weight to each selected area point, and then performing weighted average to calculate the accurate depth distance; S234: Calculate the distance from the camera to the center of the detection frame based on the precise depth distance; S235: Obtain the three-dimensional coordinates of the center point of the detection frame of the calf, the calf island, the milk bucket, and the milk bucket mouth according to the distance from the detection camera to the center point of the detection frame.

5. A feeding method with scene perception and spatial positioning according to claim 1, characterized in that: In step S300, when determining the relative position, the three-dimensional spatial coordinate information of the calf is integrated with the Kalman filter algorithm to track the calf in real time, including the following steps: S341: Initialize the state vector of the Kalman filter using the three-dimensional coordinates of the center point of the calf detection frame; S342: The Kalman filter predicts the position and speed of the calf in the current frame based on the updated state vector of the previous frame.

6. A feeding method with scene perception and spatial positioning according to claim 1, characterized in that: In step S500: When the distance between the calf and the milk bucket is greater than 1 meter and the distance between the calf and the calf island is less than 1.5 meters, it is judged that the calf is not drinking milk; When the distance between the calf and the milk bucket is less than or equal to 1 meter and the distance between the calf and the calf island is greater than 1.5 meters, the calf is judged to be feeding.

7. A feeding trolley with scene perception and spatial positioning, characterized in that: include: vehicle body; a milk tank fixed to the upper end of the vehicle body; A three-dimensional module is fixed to the upper end of the vehicle body; A milking assembly, comprising a pump and a milk pipe, wherein the pump is fixed to the upper end of the vehicle body, the input end of the pump is connected to the milk tank, the output end of the pump is connected to the milk pipe, and the milk tank is fixed to the three-dimensional module; a depth camera, fixed to the upper end of the vehicle body; The laser radar is fixed on the upper end of the vehicle body.

Images