A multimodal beef cattle behavior recognition method and system

Abstract

The present application discloses a multimodal beef cattle behavior recognition method and system, belonging to the field of artificial intelligence technology, comprising: using an improved YOLOv8 target detection algorithm to perform visual analysis on cattle, obtaining cattle behaviors including eating, lying, standing, moving and estrus as a first behavior; collecting cattle movement data, and identifying walking or running behaviors through feature analysis as a second behavior; obtaining cattle position data, and identifying cattle feeding, drinking or resting behaviors as a third behavior by comparing the cattle position with the coordinates of preset functional areas; fusing the first behavior, the second behavior and the third behavior to obtain the final behavior type of the cattle; in view of the fact that a single visual model in the existing technology is difficult to adapt to the recognition of similar posture behaviors of cattle in complex environments, the present application improves the recognition accuracy of cattle behavior types.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and more specifically, to a multimodal beef cattle behavior recognition method and system. Background Art

[0002] Traditional beef cattle management relies on regular inspections of the barn by breeders, who visually observe the cattle's appearance, feeding habits, and behavior to assess their health and growth status. This approach has numerous drawbacks, including low efficiency and significant labor and time costs. Furthermore, manual observation struggles with real-time, continuous monitoring, making it prone to oversights and inability to detect abnormal behavior or sudden health issues in cattle and implement appropriate measures.

[0003] With the advancement of computer vision technology, YOLOv8, as an advanced object detection algorithm, can efficiently process image or video data in beef cattle farming scenarios, identifying individual cattle, their posture, and various behaviors. This provides farmers with intuitive and accurate information on the daily activity patterns and health status of cattle, facilitating refined livestock management. However, YOLOv8 introduces additional technical techniques and network structures, such as feature pyramid networks and attention mechanisms, resulting in a relatively large model size. This requires more storage space to store model parameters and consumes significant memory resources during model loading and inference. This can put significant operational strain on resource-constrained devices, and the detection of fine details is relatively poor, making missed or false detections more likely. Furthermore, computer vision technology relies solely on visual information, which can lead to misjudgments in complex situations due to the limited information available, or fail to fully and accurately assess the true condition of cattle.

[0004] Therefore, in the current beef cattle breeding environment, the internal scene of the cowshed is complex and there are many interference factors. The lighting conditions in the cowshed are changeable, ranging from brightly lit areas to dark corners, and the light intensity varies greatly at different times of the day. There are a large number of facilities and equipment in the cowshed, such as railings, feeding troughs, and water troughs. These objects may block or confuse the beef cattle visually, resulting in a significant challenge to the recognition accuracy of YOLOv8, increasing the difficulty of target detection, and easily resulting in misjudgment or missed judgment, leading to a series of wrong measures and greater cost losses.

[0005] Chinese patent application number CN202411190892.6, published on December 6, 2024, discloses a new method and system for digital twin behavioral perception modeling of cattle based on the fusion of sensor data and video data. Step 1: Acquire sensor data and video data of the cattle. Step 2: Build a dataset based on the cattle data acquired in Step 1, unifying the time length of the cattle sensor data and the cattle video data in Step 1, assuming the same start time. Step 3: Fusion the cattle sensor data and cattle video data after unification in Step 2. Step 4: Train the fused data from Step 3. Step 5: Evaluate the metrics of the trained data from Step 4. However, this solution only uses video data for visual analysis, resulting in limited recognition accuracy in complex lighting and occlusion conditions. Summary of the Invention

[0006] In view of the fact that the single visual model in the existing technology is difficult to adapt to the recognition of similar posture behaviors of cattle in complex environments, the present application provides a multimodal beef cattle behavior recognition method and system, improves the YOLOv8 algorithm, reconstructs the backbone network using the channel reduction unequal grouping method, combines the C-NBottleneck module and the conical multi-scale dimensionality reduction feature extraction algorithm, enhances the visual recognition ability of cattle behavior characteristics, and combines motion sensing analysis and position positioning analysis to improve the recognition accuracy of cattle behavior types.

[0007] One aspect of an embodiment of the present specification provides a method for multimodal management of cattle, including: using an improved YOLOv8 target detection algorithm to perform visual analysis on cattle, obtaining cattle behaviors including eating, lying, standing, moving and estrus as a first behavior; collecting cattle movement data, identifying walking or running behaviors through feature analysis as a second behavior; obtaining cattle position data, and identifying cattle feeding, drinking or stationary behaviors as a third behavior by comparing the cattle position with the coordinates of preset functional areas; fusing the first behavior, the second behavior and the third behavior to obtain the final behavior type of the cattle; adjusting the cattle shed environmental parameters according to the final behavior type of the cattle; the cattle shed environmental parameters include light brightness, fan operation parameters and ultraviolet disinfection parameters.

[0008] Furthermore, an improved YOLOv8 target detection algorithm is used to perform visual analysis of cattle, including: optimizing the backbone network of the YOLOv8 target detection algorithm using a channel reduction unequal grouping method; constructing a C-NBottleneck network based on the optimized backbone network, and using the C-NBottleneck network to replace the Bottleneck component in the C2F module in the backbone network of the YOLOv8 target detection algorithm; adding a conical multi-scale dimensionality reduction feature extraction algorithm to the replaced YOLOv8 target detection algorithm, and obtaining an enhanced feature map through a channel attention mechanism and a spatial attention mechanism; obtaining a final improved YOLOv8 target detection algorithm; and using the final improved YOLOv8 target detection algorithm to analyze cattle images and identify cattle behavior.

[0009] Furthermore, a channel reduction unequal grouping method is used to optimize the backbone network of the YOLOv8 target detection algorithm, including: defining a channel reduction unequal grouping convolution operation; replacing all convolution operations except the first convolution layer in the backbone network with the defined channel reduction unequal grouping convolution operation to obtain an optimized backbone network;

[0010] Furthermore, a channel reduction unequal grouping convolution operation is defined, including: performing a preliminary convolution operation on the input feature map to obtain an intermediate feature map; dividing the channel dimension of the intermediate feature map into n groups, and when the channel dimension cannot be divided by n, setting the number of channels in each of the first n-1 groups to be equal, and the last group contains the remaining channels; dividing the convolution kernels into n groups according to the same grouping strategy as the intermediate feature map; each group of feature maps uses the convolution kernel of the corresponding group to perform a convolution operation; splicing the convolution results of each group with the intermediate feature map along the channel dimension to obtain the convolution output feature map.

[0011] Furthermore, a C-NBottleneck network is constructed based on the optimized backbone network, including: obtaining a feature map output by the optimized backbone network as a first feature map; performing a nonlinear transformation on the first feature map using a SiLU activation function; performing convolution processing on the first feature map after the nonlinear transformation using the optimized backbone network to obtain a second feature map; normalizing the second feature map; and performing a residual connection between the normalized second feature map and the first feature map to generate a C-NBottleneck network.

[0012] Furthermore, a conical multi-scale dimensionality reduction feature extraction algorithm is added to the replaced YOLOv8 target detection algorithm, and an enhanced feature map is obtained through a channel attention mechanism and a spatial attention mechanism, including: obtaining a feature map F output by the replaced YOLOv8 target detection algorithm; according to the feature map F, a channel-weighted feature map is obtained through a channel-reduced unequal grouped convolution based on a channel attention mechanism; according to the channel-weighted feature map, a spatial-weighted feature map is obtained through a channel-reduced unequal grouped convolution based on a spatial attention mechanism; according to the spatial-weighted feature map, an enhanced feature map is obtained through a depth-separated convolution operation on the spatial-weighted feature map and the feature map F.

[0013] Furthermore, according to the feature map F, a channel-weighted feature map is obtained through channel reduction unequal grouping convolution based on the channel attention mechanism, including: performing convolution processing on the feature map F using a channel reduction unequal grouping method with a convolution kernel size of N1*N1 to obtain an intermediate feature map; performing a global average pooling operation on the intermediate feature map to obtain a channel descriptor; performing a nonlinear transformation on the channel descriptor through a fully connected layer to obtain a channel weight; and obtaining a channel-weighted feature map based on the channel weight and the feature map F.

[0014] Furthermore, according to the channel-weighted feature map, a spatial weighted feature map is obtained by performing channel reduction unequal grouping convolution based on a spatial attention mechanism, including: performing convolution processing on the channel-weighted feature map using a channel reduction unequal grouping method with a convolution kernel size of N2*N2 to obtain a spatial attention feature map; performing a global average pooling operation on the spatial attention feature map to obtain a spatial feature descriptor; performing a nonlinear transformation on the spatial feature descriptor through a fully connected layer to obtain a spatial attention weight;

[0015] According to the spatial attention weight and the channel weighted feature map, a spatial weighted feature map is obtained.

[0016] Another aspect of the embodiments of this specification also provides a multimodal beef cattle behavior recognition system, which collects cattle movement data and identifies walking or running behavior as a second behavior through feature analysis, including: using the acceleration data of the cattle through an accelerometer; collecting the magnetic field strength data of the cattle through a magnetometer; processing the acceleration data using fast Fourier transform to obtain frequency domain features; performing time domain analysis and spectrum analysis on the magnetic field strength data to obtain magnetic field features; using a local linear embedding algorithm to perform nonlinear dimensionality reduction fusion on the frequency domain features and the magnetic field features to obtain a fused feature vector; and using a support vector machine (SVM) to classify the fused feature vector to obtain the second behavior.

[0017] Furthermore, the cattle location data is obtained, and the cattle's feeding, drinking or resting behavior is identified as the third behavior by comparing the cattle's location with the coordinates of the preset functional areas, including: arranging multiple UWB base stations in the cowshed and equipping each cow with a UWB tag; using a two-way time-of-flight algorithm strategy to calculate the distance from the UWB tag to the UWB base station; using the distance data of at least three base stations, using the triangulation method to calculate the two-dimensional coordinates of the cattle as the cattle location data; judging whether the cattle is in the preset functional area based on the cattle location data and the functional area division of the cowshed, the preset functional area includes a feeding area and a drinking area.

[0018] Compared with the existing technology, the advantages of this application are:

[0019] The improved YOLOv8 algorithm improves the ability to distinguish between similar posture behaviors such as eating and standing, lying down and resting through channel reduction unequal grouping and C-NBottleneck modules. At the same time, the conical multi-scale dimensionality reduction feature extraction algorithm combined with the attention mechanism strengthens the model's ability to extract the core behavioral characteristics of cattle, enabling the system to maintain stable recognition performance under complex environmental conditions such as light changes and partial occlusion. In addition, visual analysis provides morphological features but is greatly affected by the environment, motion sensing provides dynamic features but has difficulty distinguishing static behavioral details, and position positioning provides spatial semantics but lacks posture information. The three modalities work together to achieve information complementarity and cross-validation. Therefore, this application significantly improves the recognition accuracy of similar posture behaviors of cattle in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a schematic diagram of the overall structure of the multimodal cattle management system of this application;

[0021] Figure 2 Flowchart of the channel reduction unequal grouping algorithm implemented for this application;

[0022] Figure 3 Flowchart of C-NBottleneck implemented for this application;

[0023] Figure 4 Flowchart of CN-C2f implemented for this application;

[0024] Figure 5 Flowchart of the tapered multi-scale dimensionality reduction feature extraction implemented in this application;

[0025] Figure 6 IoU is used to measure the degree of overlap between the predicted box and the real box in the target detection task.

[0026] Figure 7 The structure diagram of YOLOv8 implemented in this application;

[0027] Figure 8 The five levels of bulb brightness for this application;

[0028] Figure 9 A flowchart of behavior-based measures generated for this application;

[0029] Figure 10 A model diagram for generating measures based on behavior for this application. DETAILED DESCRIPTION

[0030] The present application is described in detail below with reference to the accompanying drawings and specific embodiments.

[0031] Example 1

[0032] like Figure 1 As shown in the figure, cattle images are collected and their behaviors are annotated, including eating, lying down, standing, moving, and estrus. The dataset, validation set, and test set are labeled. The identities of the cattle are identified by ear tags, and the improved YOLOv8 algorithm is used for training and behavior recognition.

[0033] Accelerometers were placed on the legs of cattle and magnetometers on their backs. Manual observation was conducted and the parameters of the accelerometers and magnetometers were repeatedly recorded during the cattle's eating, lying down, standing, walking, running, and estrus behaviors. After normalization, the data was processed to obtain the nonlinear feature vectors of the accelerometers and magnetometers, respectively. A nonlinear dimensionality reduction method was used to fuse and train the feature layers and identify the corresponding behaviors.

[0034] Use ultra-wideband technology to locate the position of cattle and analyze their walking, running, drinking, feeding, and resting behaviors based on their location.

[0035] The obtained behaviors are fused at the decision level using the majority voting decision fusion method, and the bulb brightness, fan, and ultraviolet light are adjusted according to the accurately identified cattle behaviors, and warnings are issued to farmers.

[0036] In order to extract and fuse the spatial and channel information of the input original feature map, a 3x3 standard convolution operation is usually used to obtain the output feature map The formula is: , where F' represents the output feature map; represents the input feature map, is a set of real numbers, indicating that the elements of the feature graph F are all real numbers, is the number of input channels, Represents the width and height of the input feature map before the convolution operation, Represents the width and height of the output feature map after the convolution operation, is the number of output channels, , represents the learned filter kernel set, where Represents the filter kernel corresponding to the 1st to cth channel number, , is the size of the convolution kernel, is the bias term, represents the convolution operation. The number of floating point operations required by F ( ) is calculated as:

[0037] ;

[0038] ;in, is the number of input channels, Represents the width and height of the input feature map before the convolution operation, is the size of the convolution kernel, For filling, is the step length, Represents the width and height of the output feature map after the convolution operation, is the number of output channels.

[0039] As can be seen, the number of floating-point operations is high, especially for images larger than 256x256 pixels, which means greater computational power and time consumption. It can also be seen that the number of parameters to optimize is clearly determined by the input dimensions and the output feature maps. The number of output feature maps from convolutional layers often contains a lot of redundancy, with some being very similar. Therefore, using a large number of FLOPs and parameters to generate redundant feature maps one by one is unnecessary.

[0040] So, if Figure 2 As shown in Figure 2, the channel dimension reduction and non-uniform grouping (CDR–NUG) method is used to optimize the ordinary convolution operation and the formula is introduced: ,in, , represents the learned filter kernel set, where Indicates the 1st to The filter kernel corresponding to the number of channels, , , the default value is n=4, and it is necessary to ensure that n is divisible In order to simplify the symbols and improve the model speed, the bias term is omitted. , we can conclude , in order to solve the problem of less output feature maps, The channels are divided into Group, the characteristic graph of group g is If the groups cannot be divided evenly, unequal grouping is used in the last group. The method is: the number of channels in each of the first n-1 groups is , the last set of channels is , where / / represents integer division and % represents remainder. is the feature map of the mth group, , size is , and its corresponding convolution kernel , then the number of input channels is , the number of output channels is , is also divided into n groups. The last group uses unequal grouping. The unequal grouping method is the same as above and is recorded as , is the convolution kernel of the mth group, , size is . Perform convolution operations separately and take the step size ,filling , then we can make:

[0041] , , to ensure that the width w and height h obtained before and after the convolution operation are equal.

[0042] In the YOLOv8 structure, a 3x3 convolution kernel is used, so p=1. At this time, the output feature map of each group is , , is the feature map of the mth group, is the mth group of feature maps after pooling; then the channel dimension of the feature map obtained by the first step of standard convolution is used as the starting part, and then the feature maps after each group of convolution operations are stacked in sequence along the channel dimension to complete the splicing operation, so that the final output feature map after splicing is obtained. The width and height are and , the number of channels is , the same as the original output channel number.

[0043] You can use the formula By verifying the ratio of the amount of computation, it can be seen that, except for the case where the number of output channels >> the number of input channels, the optimized amount of computation is significantly more efficient than that of ordinary convolution operations:

[0044] .

[0045] like Figure 3As shown, the feature map is subjected to the convolution operation improved by CDR-NUG to extract features. In order to reduce the problem of gradient disappearance or explosion and make the network more robust to different initializations, BN normalization is performed on it. In order to enable the network to learn and fit more complex data patterns, after the SiLU activation function, the expression is ,in It is a Sigmoid activation function, x represents the input feature tensor, and then uses the CDR-NUG improved convolution operation and BN normalization, and adds it to the feature map to obtain C-NBottleneck.

[0046] like Figure 4 As shown in Figure 3, its CN-C2f module can also replace all Bottleneck components in the original network C2f module with C-NBottleneck units. This structure adopts a cross-stage local connection method.

[0047] like Figure 5 As shown in the figure, the feature information of the detection image of cattle behavior is enhanced by the pyramid-shaped multi-scale dimensionality reduction feature extraction (PMDR) algorithm.

[0048] First, the convolution operation of the feature map F is processed using the channel reduction unequal grouping method, using a 3x3 convolution kernel and the same number of channels to extract features and reduce the number of parameters. Among them, CN is the CDR-NUG method, 3x3 is the convolution kernel size, and C is the number of channels.

[0049] In order to stabilize data distribution and accelerate training convergence, Batch Normalization (BN) operation is added. The calculation formula of BN is:

[0050] ;

[0051] ,in is the scaling factor of the c-th channel, which can restore the feature expression ability. is the offset factor of the cth channel, which can adjust the feature offset. for The nth sample, the cth channel, the value at position (i, j), For The process of normalizing the value of the nth sample, the cth channel, and the (i, j) position, is the mean of the cth channel, is the variance of the c-th channel, is a very small constant to prevent the denominator from being zero. Therefore, the overall formula is: ,in is the scaling factor, is the offset factor. is the result of the normalization step. And using the activation function ,get , the formula is: , where x represents Elements in .

[0052] This makes the calculation efficient and enhances the sparsity and interpretability of the model. Its expression is: , in order to show that the C The numerical distribution of The weights of the C feature maps in, Each channel Perform global average pooling operations separately, becoming , the formula is: ,in, is a 1×1×c eigenvector, is the output feature map A two-dimensional matrix with row i, column j, and channel c, yes Each channel Perform global average pooling operations separately.

[0053] Then, the spatial information is compressed into channel descriptors to obtain the global receptive field and reduce the computational complexity. A fully connected layer (FC layer) is used to convert the vector Compress to , after experimental verification, r is 16, and then the activation function is used , perform nonlinear transformation on the result after dimensionality reduction and avoid the gradient vanishing problem, so that the model can converge faster. The layer Linear (C / r, C) restores the features to the C channel. Because it is necessary to multiply the channels of the original feature map channel by channel to achieve recalibration of the feature map, the channel weights to be obtained are In the range of 0-1. Therefore, the activation function Sigmoid is used, and its formula is , x represents the value of each element on the feature map. The formula for this step is:

[0054] ,in It is an intermediate function that represents the operation of applying a fully connected layer and an activation function (ReLU). is the weight matrix of the fully connected layer, Used to map from c-dimensional space to c / r-dimensional space, Used to map it from c / r dimensional space back to c dimensional space.

[0055] Then, starting from the first channel in the feature map F (channel index c=0), traverse in order to the last channel (channel index ). For the current channel, consider all elements of channel c in the original feature map F. This is a two-dimensional sub-tensor with a shape of , expressed as (in The range is 0 to , The range is 0 to ). At the same time, get the channel weight vector Elements of channel c ,because The shape is ,so that is . Each element of channel c in the original feature map F and channel weights Multiply to get a new two-dimensional sub-tensor , and its calculation formula is: The overall formula is: This operation can enhance or weaken the characteristics of a specific channel. If the value of an element in the channel weight tensor is large (close to 1), the corresponding channel feature will be enhanced; if the element value is small (close to 0), the corresponding channel feature will be weakened. This allows the network to pay more attention to the channel features that are more important to the task, thereby improving model performance.

[0056] In order to reduce the amount of calculation and speed up the operation efficiency, Perform dimensionality reduction operation. From the above CDR-NUG improved convolution, we can see that when the number of output channels is less than the number of channels, the computational complexity of the convolution operation is greatly reduced. Therefore, the result is The convolution operation using the channel reduction unequal grouping method is performed with a 7x7 convolution kernel and the number of channels is reduced to ,get .in It is an integer that can divide the number of channels. The default is 4. And use BN batch normalization and ReLU activation function, then perform average pooling operation on the channel to generate a size of Tensor of .

[0057] At the same time, the convolution feature map is passed through the Sigmoid activation function to limit its value between 0 and 1 to obtain the spatial attention feature map. , its specific expression is: ,in, Yes Average pooling of channels.

[0058] exist On each channel c, the spatial position The eigenvalue at , The spatial position weight in is , then the eigenvalue of the multiplied result at this channel and position becomes: .

[0059] The overall formula is:

[0060] , if the weight of a spatial position If the weight is close to 1, the feature of this position remains basically unchanged after multiplication, because it is considered to be a more important spatial position; if the weight If it is close to 0, the feature of this position will be greatly weakened after multiplication, thereby achieving the effect of suppressing unimportant spatial positions.

[0061] Finally, yes Performing dimensionality increase operation, using depthwise separable convolution operation (DepthwiseSeparable Convolution) can reduce the amount of calculation, the convolution kernel size is 7x7, the number of output channels is C, and batch normalization is used. Sigmoid activation function is used again to obtain the weight of this module , whose expression is:

[0062] .

[0063] Finally, for each position elements, and Multiply the corresponding elements in :

[0064] ,in The height, width, channel, and batch position of the feature map are used to obtain the final output result. Complete the spatial attention operation, The expression is: .

[0065] Its complete expression is:

[0066] ;

[0067] Among them, DSC is the Depthwise Separable Convolution method.

[0068] like Figure 6 As shown, this is It is used to measure the degree of overlap between the predicted box and the real box in the target detection task. The calculation formula is: ; Its symbol is as shown in the figure, but, There is a fatal flaw, when there is no overlap between the predicted box and the true box, that is, or hour, , then and The partial derivative of , it can be concluded that the gradient of back propagation disappears, which will lead to or Updates cannot be performed while training.

[0069] The training data inevitably contains low-quality examples, so this study uses Wise-IoU. In the first step, the penalty term is defined as the normalized length of the center point connection. , whose formula is ,because May produce gradients that hinder convergence. In order to effectively eliminate the factors that hinder convergence, we put and Separate from the computation graph, using To express it, we get: .

[0070] According to the distance metric, the distance attention is constructed and a two-layer attention mechanism is obtained. Let the attention function , which can better amplify the normal quality anchor box , and constructed based on the distance metric The formula is: .

[0071] In the second step, to address the imbalance in sample quality, the Focal-EIOU loss function develops a monotonic focusing strategy specifically for cross-entropy loss, significantly reducing the impact of easy-to-classify samples on the overall loss. This improvement allows the model to focus more on solving challenging samples, thereby improving the accuracy of the classification task. Similarly, we can construct Monotonic focusing coefficient ,in, is the exponential power, is the base, Indicates separation from the computational graph, then ; During the training process, the monotonic focusing coefficient along with This will cause the convergence speed to gradually slow down in the later stages of training. Therefore, we first set a momentum m and introduce The sliding average As a normalization factor:

[0072] ,

[0073] ;

[0074] This method ensures The overall performance is always maintained at a relatively good state, effectively addressing the problem of slowed convergence of the model in the later stages of training.

[0075] Step 3: We define an outlier To describe the quality of the anchor box: , smaller outliers indicate high-quality anchor boxes, for which we assign lower gradient gains to them to promote bounding box regression to focus on anchor boxes of average quality. For anchor boxes with large outliers, by assigning smaller gradient gains, we can prevent them from having too large a negative impact on the model, thereby avoiding excessive error propagation caused by low-quality samples. We set the hyperparameters and , we get the formula: ; When the parameter , making When the outlier degree of the anchor box reaches a certain threshold in is a predefined constant), the anchor box obtains the highest gradient gain. is dynamically changing, and the quality evaluation criteria of the anchor box are also dynamically adjusted. This dynamism makes Able to continuously optimize the distribution of gradient gains to adapt to the current training status.

[0076] like Figure 7As shown in the figure, the backbone network is the basic feature extraction component of YOLOv8. As can be seen from the figure, it begins with a simple convolutional layer (Conv), which initially captures the features of the input image. This is followed by multiple convolution operations using the channel reduction unequal grouping method, followed by a C2f module, an improvement on the convolution operation based on the channel reduction unequal grouping, denoted as C-N-C2f. These modules efficiently extract features at different levels through specific grouping and convolution methods. C-N-C2f plays a key role in the backbone network, mining features at different scales, enabling the backbone network to obtain more representative and discriminative features. At the end of the backbone network, there is a Spatial Pyramid Pooling - Fast (SPPF) module, which pools features at different scales, helping to fuse local features and ensure that the model can obtain sufficient information when processing objects of different sizes.

[0077] The neck network is responsible for feature fusion and transfer between the backbone network and the detection head. The figure shows that the neck network includes multiple concatenation and upsampling operations. Through upsampling operations, low-resolution feature maps can be restored to higher resolutions, allowing them to be spliced ​​with other high-resolution feature maps. These splicing operations can effectively fuse features at different levels. For example, feature maps of different depths in the backbone network can be fused to provide the detection head with more comprehensive and richer feature information. A conical multi-scale dimensionality reduction feature extraction algorithm is also incorporated to enrich feature extraction. In addition, the neck network also has convolution operations and C-N-C2f processed by the channel reduction unequal grouping method. These operations continue to process the fused features, further optimizing the feature representation and ensuring higher quality features transferred to the detection head.

[0078] The accelerometer and magnetometer are combined at the data level and fused at the LLE feature level. This method is named AMLM (AccMag–LLE's Method). The accelerometer and magnetometer can be installed simultaneously on the object to be monitored (such as cattle) and collect data at the same time. Compared with YOLOv8, AMLM can clearly distinguish whether the cattle are walking or running, unlike YOLOv8 which can only identify movement.

[0079] An accelerometer operates based on Newton's second law (F=ma). It typically consists of a mass and a sensor that detects the force acting on the mass. When an accelerometer is subjected to acceleration, the mass will move due to inertia. The sensor detects this displacement and measures the magnitude and direction of the acceleration.

[0080] An accelerometer is worn on the leg of a cow. Assume that the accelerometer can measure the acceleration of the cow in three-dimensional space (x, y, z) with a sampling frequency of f (unit: Hz). Sampling interval time At the moment Collect acceleration data.

[0081] For each sampling moment The accelerometer records the acceleration of the cattle on the x, y, and z axes

[0082] Check whether there are any abnormal values ​​in the collected data. Assume that the reasonable range of acceleration is , for each sampling point n, if ( Similarly), the point is considered an outlier and can be corrected using linear interpolation. The formula is: ,in for The two most recent non-outlier values ​​can be obtained by the same logic. .

[0083] Remove the noise in the data. Moving average filtering can be used. Assuming the moving average window size is M, the x-axis acceleration value after filtering is , similarly we can get and .

[0084] Since the data units and ranges of the accelerometer and magnetometer may be different, the data needs to be , , For acceleration data, the formula , similarly we can get , .

[0085] According to the characteristics of cattle behavior, the collected data is divided into several segments, each segment corresponds to a behavior. Let the behavior category be For each piece of data, behavior annotation is performed through field observation or other auxiliary means (such as video recording), and the annotated data set consists of L data pairs. in is the acceleration data of the i-th segment, Indicates the corresponding behavior.

[0086] In order to reflect the overall motion amplitude and stability, the time domain characteristics of the accelerometer are calculated: the average x-axis acceleration is ,in is the number of sampling points of the i-th segment of data. Similarly, the mean value of the y-axis acceleration can be obtained and the mean z-axis acceleration .

[0087] Because acceleration itself is related to force and energy (according to Newton's second law F=ma), the variance can well describe the degree of energy dispersion of acceleration over a period of time. The variance of the acceleration along the x-axis is Similarly, we can get the y-axis acceleration variance and the z-axis acceleration variance .

[0088] X-axis peak acceleration Similarly, the peak acceleration of the y-axis can be obtained and z-axis acceleration peak value.

[0089] In order to highlight the periodic and rhythmic behavior characteristics, Fast Fourier Transform (FFT) was used to perform fast Fourier transform on the x-axis to obtain the frequency domain sequence , , is the imaginary unit, and similarly we get , .

[0090] Calculate frequency domain energy ,here Indicates plural and find the frequency with the maximum energy , To find Get the maximum value of k. Similarly, we can get the y-axis and the frequency domain characteristics of the z-axis .

[0091] Thus, feature extraction is completed, and the feature vector is extracted using the obtained data for:

[0092] .

[0093] The magnetometer is based on the principle of electromagnetic induction. When a magnetic field passes through the coil, according to Faraday's law of electromagnetic induction ( ), an induced electromotive force will be generated in the coil, and the magnetic field strength can be calculated by measuring this electromotive force. Where E is the induced electromotive force, N is the number of coil turns, is the rate of change of magnetic flux.

[0094] Similar to the data collection of cattle by accelerometer, it is assumed that the data accuracy of the magnetometer measuring the magnetic field intensity in the three axes of x, y, and z is (Unit: Tesla, T), the number of samples collected is N, respectively ,in , check whether the collected data has abnormal values ​​and use moving average filtering to obtain .

[0095] For feature extraction, since the computational power required by the magnetometer is more complex, in order to save costs, we choose to calculate the magnetic field intensity amplitude instead of the xyz axis as the feature vector: .

[0096] Normalize the amplitude to get ; Calculate the amplitude change rate: ,in is the sampling interval (in seconds).

[0097] Extract time domain features: Calculate the mean value in the same way as the accelerometer , maximum and minimum values The unit of magnetic field strength measured by the magnetometer is Tesla (T), and the unit of standard deviation is also Tesla, which makes the standard deviation more intuitive in physical terms to represent the fluctuation range of magnetic field strength relative to the average value. , calculate, and perform fast Fourier transform FFT to obtain the frequency domain sequence . Calculate the power spectral density: , and find the one that makes The largest frequency k is the main frequency, which is recorded as . Set 6 frequency ranges , band energy .

[0098] Thus, feature extraction is completed, and the feature vector is extracted using the obtained data for .

[0099] In feature fusion, local linear embedding (LLE) is selected to obtain , LLE is a nonlinear dimensionality reduction method. Its basic idea is that each data point can be approximately reconstructed by a linear combination of its nearest neighbors. First, for each data point , using Euclidean distance , where m is the original dimension of the feature vector, to find its k nearest neighbors Then, a weight matrix W is solved so that each data point can be approximately represented by a linear combination of its neighboring points, that is, And satisfy the constraints , by minimizing the reconstruction error To solve the weight matrix. Use the weight matrix W obtained by solving to map the data points to the low-dimensional space. Let the data points in the low-dimensional space be , a low-dimensional embedding is obtained by solving the eigenvalue problem of a low-rank matrix M, that is, minimizing the objective To solve low-dimensional data points ,in yes The corresponding neighboring points in the low-dimensional space are selected as the eigenvector corresponding to the smallest non-zero eigenvalue as the low-dimensional embedding result, thereby removing redundant information in the data and being able to handle nonlinear features well.

[0100] Using the minimization objective To solve low-dimensional data points ,in yes Corresponding neighbor points in low-dimensional space;

[0101] make , where I is the identity matrix, solve the eigenvalue problem of matrix M , take the eigenvector corresponding to the smallest d+1 non-zero eigenvalues ​​(remove the smallest eigenvalue corresponding to the full vector) to obtain the low-dimensional embedding result ,in are data points in the low-dimensional space.

[0102] Use support vector machine (SVM) to select and build the model: 6 SVM classifiers need to be built. For the i-th SVM classifier (i=1, 2, 3, 4, 5, 6), The samples of the class are marked as positive (y=+1), and the samples of the other five classes are marked as negative (y=-1). The radial basis function (RBF) is selected as the kernel function. ,in is the kernel parameter, which determines the width of the kernel function. K-fold cross validation is used to optimize the penalty parameters C and . and is the eigenvector (the eigenvector extracted from the acceleration data ).

[0103] For the i-th SVM classifier, use the labeled data for training. The training data set is , among which hour, , hour, The optimization objective function of each SVM classifier is , the constraints are , where C is the penalty coefficient, is the slack variable.

[0104] The optimization problem of each SVM classifier is solved by the sequential minimum optimization algorithm SMO to obtain the optimal weight vector and bias .

[0105] For a new sample , input them into the 6 SVM classifiers respectively, and get 6 decision function values: Compare the 6 decision function values ​​and select the category corresponding to the classifier with the largest decision function value as The predicted category, that is: if , then the prediction belong kind.

[0106] UWB: A radio technology used in indoor positioning systems, it is used to identify and locate cattle groups and, combined with other sensor data, enhance the detection of cattle behaviors such as feeding, drinking, running, walking, and resting. It differs from YOLOv8 and AMLM in that it effectively reduces confusion between two similar behavior patterns (such as feeding and drinking).

[0107] Multiple UWB base stations are deployed throughout the cattle barn. Their locations are known and form the infrastructure of the positioning system, receiving signals from UWB tags worn by the cattle. Each cow is equipped with a UWB tag that periodically transmits UWB signals. These signals are transmitted in the form of narrow, non-sinusoidal pulses in the nanosecond to picosecond range, propagating throughout the barn. These signals are received by pre-deployed UWB base stations within the barn. Due to the ultra-wideband nature of UWB signals, they can effectively propagate in complex indoor environments and are highly resistant to interference.

[0108] Distance measurement: When the base station receives the signal sent by the tag, the round-trip flight time of the signal from the tag to the base station is measured using the two-way time of flight method (TW-TOF). ). Based on the speed of light (c) and the measured round-trip flight time ( ), calculate the distance (d) between the base station and the tag, the calculation formula is: The reason for dividing by 2 is that the round-trip distance is measured, and the one-way distance is needed.

[0109] After measuring the distance, the positioning calculation is performed. At least three base stations need to receive the signal of the same tag in order to calculate the location of the cattle through triangulation or other positioning algorithms. Assume that the coordinates of the three base stations are , their distances to the labels are , the coordinates of the labels (cows) can be solved by the following system of equations: By solving this set of equations, we can obtain the two-dimensional coordinate position of the cattle in the cowshed, thereby realizing the positioning of the cattle.

[0110] The final step in multimedia feature fusion is decision-level fusion. Because the three sensors can detect different behaviors, most commercially available algorithms are not applicable. To reduce computing power, a majority voting decision fusion method is used.

[0111] The identified results are eating, foraging, drinking, lying down, standing, moving, walking, running, estrus and abnormalities.

[0112] Eating: When YOLOv8 and AMLM jointly identify eating, and UWB does not identify grass or water, the displayed behavior is eating.

[0113] Forage: When at least one of YOLOv8 and AMLM identifies eating and UWB identifies forage, the displayed behavior is forage.

[0114] Drinking: When at least one of YOLOv8 and AMLM identifies eating, and UWB identifies drinking, the displayed behavior is drinking.

[0115] Prone: When YOLOv8 and AMLM both identify the subject as prone, or UWB identifies the subject as stationary, and at least one of YOLOv8 and AMLM identifies the subject as prone, the displayed behavior is prone.

[0116] Standing: When YOLOv8 and AMLM both identify standing, or UWB identifies stillness, and at least one of YOLOv8 and AMLM identifies standing, the displayed behavior is standing.

[0117] Motion: When YOLOv8 and UWB jointly identify motion, and AMLM does not identify walking or running, the behavior is displayed as motion.

[0118] Walking: When at least one of YOLOv8 and UWB detects movement, and AMLM detects walking, the displayed behavior is walking.

[0119] Running: When at least one of YOLOv8 and UWB identifies movement, and AMLM identifies running, the displayed behavior is running.

[0120] Estrus: When at least two of YOLOv8, AMLM, and UWB identify estrus, the displayed behavior is estrus.

[0121] Abnormal: When none of the above conditions occur, it is considered abnormal.

[0122] like Figure 9 , Figure 10 As shown, when the system does not detect any behavior or abnormality of cattle in the cowshed, that is, when no cattle appear, the system turns on the ultraviolet lamp for sterilization and disinfection for 20 minutes, turns on the sound of appropriate volume to drive the cattle out of the cowshed, and turns on the fan at the same time to allow air circulation to prevent ozone accumulation from causing physical discomfort to farmers and cattle. This is only done once a day. When the disinfection is completed or cattle are detected to enter the cowshed, the above measures are stopped immediately.

[0123] Use a BH1750 digital light sensor and connect it to a computer via the I2C interface. Connect the sensor's VCC to power, GND to ground, and SCL and SDA to the corresponding I2C clock and data pins on the development board or adapter, respectively. Use OpenCV and the Python-BH1750 library to write code to set the light intensity range to 0-100.

[0124] like Figure 8 As shown, according to the identified light intensity, PWM is used to control the brightness of the bulb, which is divided into five levels, and the brightness is increased or decreased according to the corresponding measures taken according to the identified cow behavior.

[0125] When detected When the light bulb is brightened appropriately according to the current brightness, the light bulb automatically adjusts the brightness to 20 minutes of stall.

[0126] in , To detect the number of cattle eating or grazing behaviors, is the sum of all behaviors of cows detected at the same time, , is the detected brightness, Indicates divisibility.

[0127] When detected If the time exceeds 5 minutes, the bulb will automatically adjust the brightness to The gear is turned off 2 minutes after the end of the behavior, and the fan is turned on and turned off 2 minutes after the end of the behavior. At the same time, the situation of frequent cattle movement is reported to the farmer.

[0128] in , To detect the number of cattle movement or walking behaviors, it is determined that agitation occurs. , and when , the power is turned off.

[0129] When the running behavior of cattle is detected, it is determined that there is agitation, and the light bulb automatically adjusts the brightness to The light is turned on and off 2 minutes after the restless behavior ends, and the fan is turned on and off 2 minutes after the restless behavior ends. At the same time, the identity of the cow is recorded through the ear tag information, and the fact that the cow is running is reported to the farmer.

[0130] When detected , and the time exceeds 5 minutes, the bulb automatically adjusts the brightness to The file will be closed 2 minutes after the end of the behavior.

[0131] in, , The number of cows detected lying down.

[0132] In addition, when a cow lies down for a long time, the identity of the cow is recorded through the ear tag information, and the fact that the cow has been lying down for a long time is reported back to the farmer.

[0133] When estrus behavior of a cow is detected, the video of the behavior is automatically saved, and the automatic saving stops 1 minute after the estrus behavior ends. At the same time, the identity of the cow is recorded through the ear tag information, and the fact that the cow is running is reported back to the farmer.

[0134] Ultraviolet light can damage the DNA or RNA structures of microorganisms such as bacteria, viruses, and fungi. When exposed to ultraviolet light of a certain intensity and duration, the nucleic acids in these microorganisms absorb the energy, causing molecular chain breakage or the formation of pyrimidine dimers. This prevents the microorganisms from normal reproduction and metabolism, ultimately leading to their death. For example, common cattle barn pathogens such as Escherichia coli, Salmonella, and cowpox virus can be effectively killed by ultraviolet light. This broad-spectrum bactericidal property can significantly reduce the number of pathogenic microorganisms in cattle barns, reducing the risk of disease in cattle.

[0135] Careful light management of cattle's feeding environment can significantly impact their feeding behavior, digestive efficiency, and overall health. When cattle are foraging on grass, appropriately increasing light intensity has been shown to increase feed intake. This is likely because moderate light mimics daytime conditions in a natural environment, stimulating foraging behavior. Furthermore, good light conditions help cattle better see and select their feed, which in turn promotes digestion and nutrient absorption.

[0136] When cattle move frequently or exhibit restless behavior, they may be responding to a change in their environment or some form of stress. This may include noise, extreme temperatures, uncomfortable housing conditions, or disturbances from other animals. In these situations, dimming the lighting and air flow appropriately can be an effective management strategy to reduce stress responses in cattle. A darker environment can help cattle feel safer and less nervous, encouraging them to remain calm, reduce unnecessary energy expenditure, and activate fans for ventilation.

[0137] The above schematically describes the invention and its implementation methods. This description is not restrictive. Without departing from the spirit or basic features of the invention, the invention can be implemented in other specific forms. What is shown in the accompanying drawings is only one of the implementation methods of the invention. The actual structure is not limited to this. Any figure mark in the claims should not limit the claims involved. Therefore, if a person of ordinary skill in the art is inspired by it and designs a structural method and embodiment similar to the technical solution without creativity without departing from the purpose of the invention, they should all fall within the scope of protection of this application. In addition, the word "including" does not exclude other elements or steps, and the word "one" before an element does not exclude the inclusion of "multiple" elements. The multiple elements stated in the product claim can also be implemented by one element through software or hardware. Words such as first and second are used to indicate names and do not indicate any specific order.

Claims

1. A multimodal beef cattle behavior recognition method, characterized in that: include: Improve the YOLOv8 target detection algorithm and use it to perform visual analysis on cattle to obtain cattle behaviors including eating, lying down, standing, moving, and estrus as the first behavior; Collect cattle movement data and identify, through feature analysis, behaviors including but not limited to walking or running as a second behavior; Obtain cattle location data and identify cattle feeding, drinking, or resting behaviors as the third behavior by comparing cattle locations with preset functional area coordinates; The first, second and third behaviors are integrated to obtain the final behavior type of the cattle; Adjust the environmental parameters of the barn according to the final behavior type of the cattle; Cow house environmental parameters include light brightness, fan operation parameters, and ultraviolet disinfection parameters; Improved YOLOv8 target detection algorithm, including: The channel reduction unequal grouping method is used to optimize the backbone network of the YOLOv8 target detection algorithm; Based on the optimized backbone network, a C-NBottleneck network is constructed, and the C-NBottleneck network is used to replace the Bottleneck component in the C2F module in the backbone network of the YOLOv8 target detection algorithm, including: obtaining a feature map output by the optimized backbone network as a first feature map; performing a nonlinear transformation on the first feature map using the SiLU activation function; performing convolution processing on the first feature map after the nonlinear transformation using the optimized backbone network to obtain a second feature map; normalizing the second feature map; and performing a residual connection between the normalized second feature map and the first feature map to generate a C-NBottleneck network; A conical multi-scale dimensionality reduction feature extraction algorithm is added to the replaced YOLOv8 target detection algorithm, and an enhanced feature map is obtained through a channel attention mechanism and a spatial attention mechanism, thereby obtaining an improved YOLOv8 target detection algorithm, including: obtaining a feature map F output by the replaced YOLOv8 target detection algorithm; obtaining a channel-weighted feature map based on the feature map F through channel-reduced unequal grouped convolution based on the channel attention mechanism; obtaining a spatially weighted feature map based on the channel-weighted feature map through channel-reduced unequal grouped convolution based on the spatial attention mechanism; and obtaining an enhanced feature map based on the spatially weighted feature map and the feature map F through a depthwise separable convolution operation. The channel reduction unequal grouping method is used to optimize the backbone network of the YOLOv8 target detection algorithm, including: Define the channel reduction unequal grouping convolution operation, including: performing a preliminary convolution operation on the input feature map to obtain an intermediate feature map; dividing the channel dimension of the intermediate feature map into n groups. When the channel dimension is not divisible by n, set the number of channels in each of the first n-1 groups to be equal, and the last group contains the remaining channels; dividing the convolution kernels into n groups according to the same grouping strategy as the intermediate feature map; performing a convolution operation on each group of feature maps using the convolution kernel of the corresponding group; concatenating the convolution results of each group with the intermediate feature map along the channel dimension to obtain the convolution output feature map; All convolution operations except the first layer of convolution in the backbone network are replaced by the defined channel reduction unequal group convolution operation to obtain the optimized backbone network.

2. The multimodal beef cattle behavior recognition method according to claim 1, characterized in that: Get the channel weighted feature map, including: The feature map F is convolved using the channel reduction unequal grouping method with a convolution kernel size of N1*N1 to obtain the intermediate feature map; Perform global average pooling on the intermediate feature map to obtain the channel descriptor; Perform nonlinear transformation on the channel descriptor through the fully connected layer to obtain the channel weight; According to the channel weight and feature map F, the channel weighted feature map is obtained.

3. The multimodal beef cattle behavior recognition method according to claim 1, characterized in that: Get the spatial weighted feature map, including: The channel weighted feature map is convolved using the channel reduction unequal grouping method with a convolution kernel size of N2*N2 to obtain the spatial attention feature map; Perform global average pooling on the spatial attention feature map to obtain the spatial feature descriptor; The spatial feature descriptor is nonlinearly transformed through the fully connected layer to obtain the spatial attention weight; According to the spatial attention weight and the channel weighted feature map, a spatial weighted feature map is obtained.

4. The multimodal beef cattle behavior recognition method according to any one of claims 2 or 3, characterized in that: Collect cattle movement data and identify walking or running behaviors through feature analysis as the second behavior, including: The acceleration data of cattle is collected through accelerometers; the magnetic field strength data of cattle is collected through magnetometers; Fast Fourier transform is used to process the acceleration data to obtain frequency domain features; Perform time domain analysis and spectrum analysis on magnetic field intensity data to obtain magnetic field characteristics; The local linear embedding algorithm is used to perform nonlinear dimensionality reduction fusion on the frequency domain features and magnetic field features to obtain the fused feature vector; The fused feature vector is classified using support vector machine (SVM) to obtain the second behavior.

5. The multimodal beef cattle behavior recognition method according to claim 4, characterized in that: Identify cattle feeding, drinking, or resting behaviors as the third behavior, including: Multiple UWB base stations are deployed in the cowshed, and each cow is equipped with a UWB tag; The distance from the UWB tag to the UWB base station is determined using a two-way time-of-flight algorithm. The two-dimensional coordinates of the cattle are calculated using the distance data of at least three base stations using the triangulation method as the cattle position data; According to the cattle location data and the functional area division of the cattle house, it is judged whether the cattle are in the preset functional area, and the preset functional area includes the forage area and the drinking water area.

Images