Pig attack behavior identification method, system, computer device and storage medium

Abstract

The present invention provides a method, system, computer device and storage medium for identifying pig attack behaviors. The method comprises the following steps: extracting a sequence of key frames to be identified from a video of a group of pigs to be identified, inputting the video of the group of pigs to be identified into a lightweight 3D convolutional neural network to extract corresponding spatiotemporal features to be fused, and inputting the sequence of key frames to be identified into a 2D convolutional neural network to extract corresponding spatial features to be fused, then fusing the spatiotemporal features to be fused with the spatial features to be fused to obtain spatiotemporal fusion features, and using a hole convolution layer to perform bounding box prediction and action classification on the spatiotemporal fusion features to obtain an identification result. The method solves the application limitations of the prior art, such as excessive reliance on human experience, time-consuming and labor-intensive, and insufficient intelligence, while achieving accurate and efficient identification and classification of pig actions, providing real-time and effective monitoring of pig attack behaviors, and providing reliable guarantees for the healthy growth of pigs and the economic income of the pig farming industry.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and behavior recognition, and in particular to a method, system, computer equipment and storage medium for pig attack behavior recognition based on real-time video stream spatiotemporal action positioning. Background Art

[0002] Pig farming is an important industry in my country's agriculture, providing an effective guarantee for the supply of meat products. Since the pig farming industry promotes high-density pig rearing, group pig farming will face problems such as limited living space and competition for food resources, resulting in intra-group aggression. Pig aggression can easily lead to physical injuries and infections on both sides, affecting their normal growth in the later stages. In severe cases, it can lead to death, which in turn affects pork production and causes economic losses to the pig farming industry. Therefore, with the development of intelligent pig farming, the study of pig aggression has attracted much attention. It is expected that video surveillance can be used to timely and accurately identify pig aggression and avoid its adverse effects.

[0003] At present, many scholars have proposed corresponding automatic monitoring methods for pig aggressive behavior, such as the use of linear discriminant analysis of the motion history graph algorithm of interactive aggressive behavior of pigs based on the mean value and duration of the pig's motion pixel intensity over a period of time; the use of SVM classification to identify and classify aggressive behaviors and give the types of aggressive behaviors based on the maximum, minimum, average, speed standard deviation and distance between pigs; the use of hierarchical clustering to calculate the acceleration threshold based on the overall motion acceleration characteristics of pigs with aggressive behaviors between adjacent frames; the use of ALR-GMM algorithm to extract the activity index characteristics of fattening pig behavior video units, construct the activity index, and use the linear kernel function support vector machine to establish a classifier for aggressive behavior detection methods. Although the existing pig aggressive behavior recognition methods can effectively detect pig aggressive behaviors to a certain extent, they are all based on motion features to identify pig aggressive behaviors. These methods need to rely on manually designed features and are too dependent on human experience. The determination of the best feature combination requires repeated experiments, which is not only time-consuming and labor-intensive, but also not intelligent enough and has limited application. Summary of the invention

[0004] The purpose of the present invention is to propose a pig attack behavior recognition method based on spatiotemporal action positioning of real-time video streams from the perspective of computer vision. After extracting the spatiotemporal information of the pig behavior video using a lightweight 3D+2D real-time action positioning network, an improved self-attention mechanism is used to fuse the spatiotemporal information, and then a void convolution layer is used to predict the bounding box and classify the action. The spatiotemporal features of the pigs in the real-time monitoring video are quickly extracted and fused, and the accurate and efficient recognition and classification of the pig actions are achieved, providing real-time and effective monitoring of the pig attack behavior, thereby effectively ensuring the healthy growth of the pigs and providing reliable protection for the economic income of the pig farming industry.

[0005] In order to achieve the above objectives, it is necessary to provide a method, system, computer device and storage medium for identifying pig attack behavior in response to the above technical problems.

[0006] In a first aspect, an embodiment of the present invention provides a method for identifying pig attack behavior, the method comprising the following steps:

[0007] Obtain a video of a group of pigs to be identified, and extract a key frame sequence to be identified from the video of the group of pigs to be identified;

[0008] The pig farming video to be identified is input into a lightweight 3D convolutional neural network to extract spatiotemporal information, and obtain corresponding spatiotemporal features to be fused; the lightweight 3D convolutional neural network is an XSepConvMobileNetv3 network improved based on the MobileNetv3-small network;

[0009] Inputting the key frame sequence to be identified into a 2D convolutional neural network to extract spatial information and obtain corresponding spatial features to be fused;

[0010] Fusing the spatiotemporal features to be fused with the spatial features to be fused to obtain spatiotemporal fusion features;

[0011] A hole convolution layer is used to perform bounding box prediction and action classification on the spatiotemporal fusion features to obtain a recognition result.

[0012] Furthermore, the step of extracting a key frame sequence to be identified from the video of the pig farming group to be identified includes:

[0013] Performing image preprocessing on the video frame of the to-be-identified group pig-raising video to obtain the to-be-identified video frame image;

[0014] According to the connected domain area and adhesion index of each video frame image to be identified, the key frame sequence to be identified is extracted from the video frame image to be identified.

[0015] Furthermore, the steps of obtaining the XSepConvMobileNetv3 network based on the improvement of the MobileNetv3-small network include:

[0016] The inverse residual linear bottleneck layer module with a convolution kernel size of 5×5 in the MobileNetv3-small network is replaced with an XSepCovbneck module; the XSepCovbneck module includes a 1×1 dimensionality increase convolution, an XSepConv extreme separation convolution, a channel attention module and a 1×1 dimensionality reduction convolution connected in sequence.

[0017] Furthermore, the step of inputting the to-be-identified group pig farming video into a lightweight 3D convolutional neural network to extract spatiotemporal information to obtain corresponding spatiotemporal features to be fused includes:

[0018] Input the pig farming video of the group to be identified into a lightweight 3D convolutional neural network to extract spatiotemporal information and obtain corresponding spatiotemporal features;

[0019] The spatiotemporal features are compressed according to a preset fusion size to obtain the spatiotemporal features to be fused.

[0020] Furthermore, the 2D convolutional neural network includes a CSPDarknet53 network, a residual module and an SPPNet network connected in sequence; the residual module includes a 1×1 convolutional layer, a 3×3 convolutional layer and a 1×1 convolutional layer connected in sequence;

[0021] The step of inputting the key frame sequence to be identified into a 2D convolutional neural network to extract spatial information and obtain corresponding spatial features to be fused comprises:

[0022] Performing feature extraction on the key frame sequence to be identified through the CSPDarknet53 network to obtain a first spatial feature;

[0023] Inputting the first spatial feature into the residual module for convolution processing to obtain a second spatial feature;

[0024] The second spatial feature is subjected to spatial pyramid pooling through the SPPNet network to obtain the to-be-fused spatial feature of the preset fusion size.

[0025] Furthermore, the feature fusion includes channel fusion and attention mechanism fusion;

[0026] The step of fusing the spatiotemporal features to be fused with the spatial features to be fused to obtain spatiotemporal fusion features comprises:

[0027] Perform channel fusion on the spatiotemporal feature to be fused and the spatial feature to be fused to obtain a first spatiotemporal fusion feature;

[0028] Inputting the first spatiotemporal fusion feature into a convolution module to obtain a second spatiotemporal fusion feature; the convolution module includes two 3×3 convolution layers;

[0029] Using the Gram matrix method to perform attention mechanism fusion on the second spatiotemporal fusion feature to obtain a third spatiotemporal fusion feature;

[0030] The third spatiotemporal fusion feature is input into the convolution module to obtain the spatiotemporal fusion feature.

[0031] Furthermore, the step of using the Gram matrix method to perform attention mechanism fusion on the second spatiotemporal fusion feature to obtain the third spatiotemporal fusion feature includes:

[0032] Performing two-dimensional vectorization processing on the second spatiotemporal fusion feature to obtain a first processing matrix;

[0033] Taking a product of the first processing matrix and the transpose of the first processing matrix, to obtain a corresponding Gram matrix;

[0034] The Gram matrix is ​​input into the softmax module for classification processing to obtain channel attention features;

[0035] The second spatiotemporal fusion feature is corrected by using the channel attention feature, and the corrected spatiotemporal fusion feature is dimensionally transformed to obtain a corrected spatiotemporal fusion feature;

[0036] The second spatiotemporal fusion feature is added to the modified spatiotemporal fusion feature to obtain the third spatiotemporal fusion feature.

[0037] In a second aspect, an embodiment of the present invention provides a pig attack behavior recognition system, the system comprising:

[0038] A video acquisition module, used to acquire a video of a group of pigs to be identified, and extract a key frame sequence to be identified from the video of the group of pigs to be identified;

[0039] The spatiotemporal feature module is used to input the pig farming video of the group to be identified into a lightweight 3D convolutional neural network to extract spatiotemporal information and obtain the corresponding spatiotemporal features to be fused; the lightweight 3D convolutional neural network is an XSepConvMobileNetv3 network improved based on the MobileNetv3-small network;

[0040] A spatial feature module is used to input the key frame sequence to be identified into a 2D convolutional neural network to extract spatial information and obtain corresponding spatial features to be fused;

[0041] A feature fusion module, used to fuse the spatiotemporal features to be fused with the spatial features to be fused to obtain spatiotemporal fusion features;

[0042] The result recognition module is used to use a hole convolution layer to perform bounding box prediction and action classification on the spatiotemporal fusion features to obtain a recognition result.

[0043] In a third aspect, an embodiment of the present invention further provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0044] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above method when executed by a processor.

[0045] The above-mentioned application provides a method, system, computer device and storage medium for identifying pig attack behavior. Through the method, after extracting the key frame sequence to be identified from the acquired pig farming video to be identified, the pig farming video to be identified is input into a lightweight 3D convolutional neural network to extract the corresponding spatiotemporal features to be fused, and the key frame sequence to be identified is input into a 2D convolutional neural network to extract the corresponding spatial features to be fused, and then the spatiotemporal features to be fused are fused with the spatial features to be fused to obtain spatiotemporal fusion features, and a void convolution layer is used to perform bounding box prediction and action classification on the spatiotemporal fusion features to obtain the recognition result. Compared with the existing technology, the pig attack behavior recognition method adopts a lightweight 3D+2D real-time action localization network to quickly extract the spatiotemporal information of pigs in pig behavior videos, and uses an improved self-attention mechanism for effective fusion, and then combines the bounding box prediction and action classification of the void convolution layer. It effectively solves the application limitations of the existing technology such as over-reliance on human experience, time-consuming and labor-intensive, and lack of intelligence, while achieving accurate and efficient recognition and classification of pig actions, providing real-time and effective monitoring of pig attack behavior, and providing reliable protection for the healthy growth of pigs and the economic income of the pig farming industry. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 Schematic diagram of an application scenario of a method for identifying pig attack behaviors in an embodiment of the present invention;

[0047] Figure 2 Schematic diagram of the network structure of the method for identifying pig attack behavior in an embodiment of the present invention;

[0048] Figure 3 is a flow chart of a method for identifying pig attack behavior in an embodiment of the present invention;

[0049] Figure 41 is a schematic diagram of the XSepConvbneck structure of the XSepConvMobileNetv3 network in an embodiment of the present invention;

[0050] Figure 5 yes Figure 3 Step S12 is a flow chart of obtaining the spatiotemporal features to be fused;

[0051] Figure 6 yes Figure 3 Step S13 is a flow chart of obtaining the spatial features to be fused;

[0052] Figure 7 yes Figure 3 Step S14 is a flow chart of fusing the spatiotemporal features to be fused and the spatial features to be fused;

[0053] Figure 8 It is a schematic diagram of the feature fusion structure of the spatiotemporal features to be fused and the spatial features to be fused;

[0054] Fig. 9 is a structural diagram of a pig attack behavior recognition system according to an embodiment of the present invention;

[0055] Fig.10 It is a diagram of the internal structure of a computer device in an embodiment of the present invention. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical scheme and beneficial effects of the present application clearer, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. Obviously, the embodiments described below are part of the embodiments of the present invention and are only used to illustrate the present invention, but are not used to limit the scope of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0057] The pig attack behavior recognition method provided by the present invention can be applied to Figure 1 The terminal or server shown. The terminal may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, and portable wearable devices, and the server may be implemented as an independent server or a server cluster composed of multiple servers. The server may adopt the pig attack behavior recognition method provided by the present invention based on the real-time acquired video of the group of pigs to be identified. Figure 2 The network architecture shown completes the recognition of the pig attack behavior in each video, and applies the final recognition result to other learning tasks on the server, or transmits it to the terminal for the terminal user to receive and use.

[0058] In one embodiment, Figure 3As shown, a method for identifying pig attack behavior is provided, comprising the following steps:

[0059] S11. Obtain a video of a group of pigs to be identified, and extract a key frame sequence to be identified from the video of the group of pigs to be identified; wherein the video of the group of pigs to be identified is a surveillance video acquired in real time for attack behavior recognition and analysis, which is composed of a continuous frame sequence arranged in chronological order and has a shape of [C×D×H×W], wherein C represents the number of channels C=3, D represents the number of input frames, and H and W represent the height and width of the input image, respectively; since the 2D convolutional neural network can only process images, it is necessary to extract a key frame sequence to be identified from the video of the group of pigs to be identified for analysis and identification. In principle, the specific method for extracting the key frame sequence to be identified can adopt any achievable existing technology. In order to ensure the rationality and effectiveness of the identification of the key frame sequence to be identified, in this embodiment, the connected domain area and adhesion index of the video frame image in the video of the group of pigs to be identified are analyzed and extracted, and the following steps are specifically included:

[0060] Performing image preprocessing on the video frame of the to-be-identified group pig-raising video to obtain the to-be-identified video frame image;

[0061] According to the connected domain area and adhesion index of each video frame image to be identified, the key frame sequence to be identified is extracted from the video frame image to be identified.

[0062] S12. Input the pig farming video of the group to be identified into a lightweight 3D convolutional neural network to extract spatiotemporal information and obtain the corresponding spatiotemporal features to be fused; the lightweight 3D convolutional neural network is the XSepConvMobileNetv3 network improved based on the MobileNetv3-small network; wherein the MobileNetv3-small network is an existing lightweight network model, which uses 11 inverse residual linear bottleneck layer modules (bottleneck, bneck) to perform convolution operations on feature maps, wherein the first three bnecks have a depthwise separable convolution kernel size of 3×3, and the next 8 bnecks use a depthwise separable convolution with a convolution kernel of 5×5 to increase the receptive field. In order to further reduce the amount of calculation and improve the reasoning speed of the model, this embodiment uses XSepConv instead of the depthwise separable convolution to obtain the following: Figure 4The XSepConvbneck structure shown in the figure is then used to replace the inverse residual linear bottleneck layer module bneck structure with a convolution kernel of 5×5 in MobileNetv3-small by using the XSepConvbneck structure including the sequentially connected 1×1 dimensionality increase convolution, XSepConv extreme separation convolution (i.e., K×KDWConv=2×2DWConv (improved symmetric padding strategy) + 1×K DWConv + K×1DWConv, which uses spatial separation convolution to reduce the parameter size and computational complexity of large depth convolution kernels, and uses an additional 2x2 depth convolution and an improved symmetric padding strategy to compensate for the side effects of spatially separable convolution), channel attention module (SE (Squeeze and Excitation) module) and 1×1 dimensionality reduction convolution. The new XSepConvMobileNetv3 network is obtained as a 3D CNN to extract spatiotemporal information from the pig farming video to be identified. It should be noted that, if Figure 4 In the XSepConvbneck module shown, BN represents batch normalization, NL represents the activation function, and the hard-swish activation function is preferentially selected in this embodiment. By piecewise linearizing the swish function, the amount of calculation is effectively reduced without sacrificing accuracy, thereby improving the learning efficiency of the model.

[0063] Specifically, Figure 5 As shown, step S12 of the XSepConvMobileNetv3 network obtained based on the improvement of the MobileNetv3-small network includes:

[0064] S121. Input the to-be-identified group pig-raising video into a lightweight 3D convolutional neural network to extract spatiotemporal information and obtain corresponding spatiotemporal features; wherein, after the feature map corresponding to the to-be-identified group pig-raising video is input into the XSepConvbneck module of the lightweight 3D convolutional neural network, it first undergoes a 1×1 dimensionality increase convolution, then an XSepConv convolution operation, and then a 1×1 dimensionality reduction convolution, and then the result is added to the residual edge for output, and the lightweight channel attention module SE added between the output of the XSepConv convolution and the 1×1 dimensionality reduction convolution can effectively improve the accuracy of the model without increasing the amount of calculation.

[0065] S122. Compress the spatiotemporal features according to the preset fusion size to obtain the spatiotemporal features to be fused. The preset fusion size is the feature size defined by channel fusion and attention mechanism fusion, which is the size that both the spatiotemporal features to be fused and the spatial features to be fused must meet. Specifically, the output of the last transformation layer of XSepConvMobileNetv3 is a feature map with a shape of [C1×D1×H′×W′], where C1 is the number of output channels, D1=1, H'=H / 32 and W'=W / 32, and the depth dimension of the output feature map is reduced to 1, so that the output volume is compressed to [C1×H′×W′] to match the output feature map of the 2D convolutional neural network.

[0066] S13, input the key frame sequence to be identified into a 2D convolutional neural network for spatial information extraction to obtain the corresponding spatial features to be fused; wherein the 2D convolutional neural network is used to solve the spatial positioning problem. While using a lightweight 3D convolutional neural network to extract spatiotemporal information, it is necessary to use the 2D convolutional neural network in parallel to extract the spatial information of the key frame sequence to be identified. The 2D convolutional neural network of this embodiment is as follows Figure 2 As shown, a 2D network of a CSPDarknet53 network, a residual module and an SPPNet network connected in sequence is preferably used, that is, CSPDarknet53 is used as the basic architecture of the 2D convolutional neural network. In order to further expand the receptive field, a residual module is added after the last feature extraction layer of the CSPDarknet53 network, and then the residual processing result is input into the SPPNet network for maximum pooling operation. Feature extraction is performed through maximum pooling with different pooling kernel sizes to obtain the corresponding spatial features to be fused, wherein the residual module includes a 1×1 convolutional layer, a 3×3 convolutional layer and a 1×1 convolutional layer connected in sequence.

[0067] Specifically, Figure 6 As shown, the step S13 of inputting the key frame sequence to be identified into a 2D convolutional neural network to extract spatial information and obtain the corresponding spatial features to be fused includes:

[0068] S131. Perform feature extraction on the key frame sequence to be identified through the CSPDarknet53 network to obtain the first spatial feature; wherein, the shape of each key frame sequence to be identified is [C×H×W], and the shape of the output feature map after processing by CSPDarknet53 is [C2×H′×W′], wherein C=3, C2 is the number of output channels, H′=H / 32 and W′=W / 32.

[0069] S132, inputting the first spatial feature into the residual module for convolution processing to obtain a second spatial feature;

[0070] S133. Perform spatial pyramid pooling on the second spatial features through the SPPNet network to obtain the to-be-fused spatial features of the preset fusion size.

[0071] S14, fusing the spatiotemporal features to be fused with the spatial features to be fused to obtain spatiotemporal fusion features; wherein the feature fusion is as follows: Figure 7 As shown in the figure, it includes channel fusion and attention mechanism fusion, that is, after extracting the 2D spatial features to be fused and the 3D spatiotemporal features to be fused, the concat connection is first used to fuse the two feature maps. It can simply superimpose features along the channel to achieve the first step of channel fusion. The fused feature map encodes motion and appearance information, and then passes it as input to the CFAM module for attention mechanism fusion. The attention mechanism fusion is based on the Gram matrix to map the dependencies between channels, which can reasonably fuse features from different sources, ensure the rationality and effectiveness of feature fusion, and significantly improve the overall performance. Specifically, as Figure 8 As shown, the step S14 of fusing the spatiotemporal features to be fused with the spatial features to be fused to obtain spatiotemporal fusion features includes:

[0072] S141, performing channel fusion on the spatiotemporal feature to be fused and the spatial feature to be fused to obtain a first spatiotemporal fusion feature;

[0073] S142, inputting the first spatiotemporal fusion feature into a convolution module to obtain a second spatiotemporal fusion feature; the convolution module includes two 3×3 convolution layers;

[0074] S143, using the Gram matrix method to perform an attention mechanism fusion on the second space-time fusion feature to obtain a third space-time fusion feature; wherein, the step of performing an attention mechanism fusion on the second space-time fusion feature to obtain the third space-time fusion feature using the Gram matrix method includes: performing two-dimensional vectorization processing on the second space-time fusion feature to obtain a first processing matrix; performing a product of the first processing matrix and the transpose of the first processing matrix to obtain a corresponding Gram matrix; inputting the Gram matrix into a softmax module for classification processing to obtain a channel attention feature; using the channel attention feature to correct the second space-time fusion feature, and performing dimensionality conversion on the corrected space-time fusion feature to obtain a corrected space-time fusion feature; adding the second space-time fusion feature to the corrected space-time fusion feature to obtain the third space-time fusion feature.

[0075] S144. Input the third spatiotemporal fusion feature into the convolution module to obtain the spatiotemporal fusion feature.

[0076] Specifically, Figure 7As shown in Figure 1, the 2D spatial features to be fused [C2×H1×W1] and the 3D spatiotemporal features to be fused [C1×H1×W1] are stacked up through channel fusion and splicing to obtain the first spatiotemporal fusion feature. The corresponding preliminary fusion feature matrix is Then A is input into the convolution module including two 3×3 convolutional layers for convolution processing to generate a new feature map, namely the second spatiotemporal fusion feature map. Then B is subjected to a variety of operations: First, B is vectorized into two dimensions to obtain the first processing matrix Where N = H × W; Next, multiply the first processing matrix F by the transpose of F to obtain the Gram matrix matrix By increasing the feature response value and suppressing the non-feature response value, the mutual correlation between features is obtained; then, the Gram matrix G is softmaxed to generate the channel attention map channel attention feature; and the obtained channel attention feature is used to modify the original feature map (the second spatiotemporal fusion feature), that is, F1=M·F is calculated. After correction, it is reshaped (dimensional conversion) to C×H×W to obtain the corrected spatiotemporal fusion feature F2; finally, the original second spatiotemporal fusion feature B is added to the corrected spatiotemporal fusion feature F2 to obtain the third spatiotemporal fusion feature C=α·F2+B, where α is a learnable parameter that starts from 0 during training. Thereafter, the third spatiotemporal fusion feature C is input into a convolution module including two 3×3 convolutional layers for convolution processing to output the feature map D of the CFAM module, that is, the above-mentioned spatiotemporal fusion feature is obtained.

[0077] S15. Use a dilated convolution layer to perform bounding box prediction and action classification on the spatiotemporal fusion features to obtain recognition results. Among them, the dilated convolution layer, that is, fills 0 in the middle of the standard convolution kernel, and its actual convolution kernel size is K=k+(k-1)(r-1), k is the original convolution kernel size, and r is the dilated convolution parameter dilation rate. This embodiment uses a dilated convolution layer to further process the above-mentioned spatiotemporal fusion features, which can expand the receptive field without losing resolution, and then achieve the final bounding box and action classification to obtain the recognition result. In addition, in order to further improve the accuracy of the recognition results, score sorting and non-maximum suppression screening can also be performed based on the obtained recognition results.

[0078] The embodiment of the present application designs the XSepConvMobileNetv3 network by introducing XSepConv, and extracts the spatiotemporal features of the pig farming video to be identified through the 3D-CNN based on the XSepConvMobileNetv3 architecture, and uses the XSepConv extreme separation convolution to mix the deep separation convolution with the spatially separable convolution to form a spatially separable deep convolution, which further reduces the parameter size and calculation amount of the large deep convolution kernel, effectively improves the learning efficiency of the model, and at the same time includes the CSPDarknet53 network, the residual network connected in sequence, and the 3D-CNN based on the XSepConvMobileNetv3 architecture. The 2D-CNN of the difference module and SPPNet network extracts the spatial features of the key frames, effectively expanding the receptive field of the model, and uses the channel fusion and attention mechanism (CFAM) to aggregate the spatiotemporal features. The spatiotemporal fusion features are then predicted by the void convolution layer and the action classification is performed to obtain the final recognition result, which effectively improves the accuracy of the model. It effectively solves the application limitations of the existing technology such as over-reliance on human experience, time-consuming and labor-intensive, and not intelligent enough, while achieving accurate and efficient recognition and classification of pig actions, providing real-time and effective monitoring of pig attack behavior, and providing reliable guarantee for the healthy growth of pigs and the economic income of the pig farming industry.

[0079] It should be noted that although the steps in the above flowchart are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders.

[0080] In one embodiment, Fig. 9 As shown, a pig attack behavior recognition system is provided, the system comprising:

[0081] Video acquisition module 1, used to acquire a video of a group of pigs to be identified, and extract a key frame sequence to be identified from the video of the group of pigs to be identified;

[0082] The spatiotemporal feature module 2 is used to input the pig farming video to be identified into a lightweight 3D convolutional neural network to extract spatiotemporal information and obtain corresponding spatiotemporal features to be fused; the lightweight 3D convolutional neural network is an XSepConvMobileNetv3 network improved based on the MobileNetv3-small network;

[0083] The spatial feature module 3 is used to input the key frame sequence to be identified into a 2D convolutional neural network to extract spatial information and obtain corresponding spatial features to be fused;

[0084] A feature fusion module 4 is used to fuse the spatiotemporal features to be fused with the spatial features to be fused to obtain spatiotemporal fusion features;

[0085] The result recognition module 5 is used to use a hole convolution layer to perform bounding box prediction and action classification on the spatiotemporal fusion features to obtain a recognition result.

[0086] For the specific definition of a pig attack behavior identification system, please refer to the definition of a pig attack behavior identification method mentioned above, which will not be repeated here. Each module in the above-mentioned pig attack behavior identification system can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0087] Fig.10 FIG. 1 shows an internal structure diagram of a computer device in an embodiment, and the computer device may specifically be a terminal or a server. Fig.10 As shown, the computer device includes a processor, a memory, a network interface, a display and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for identifying pig attack behavior is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0088] It can be understood by those skilled in the art that Fig.10 The structure shown in the figure is only a block diagram of a part of the structure related to the present application scheme, and does not constitute a limitation on the computer device to which the present application scheme is applied. The specific computing device may include more or fewer components than shown in the figure, or combine certain components, or have the same component arrangement.

[0089] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the steps of the above method are implemented when the processor executes the computer program.

[0090] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0091] In summary, the embodiments of the present invention provide a method, system, computer device and storage medium for identifying pig attack behaviors. The pig attack behavior identification method adopts a lightweight 3D (XSepConvMobileNetv3) + 2D (CSPDarknet53 network + residual module + SPPNet network) real-time action positioning network to quickly extract the spatiotemporal information of pigs in pig behavior videos, and adopts an improved self-attention mechanism for effective fusion, and then combines the bounding box prediction and action classification of the void convolution layer. It effectively solves the application limitations of the prior art such as excessive reliance on human experience, time-consuming and labor-intensive, and not intelligent enough, while achieving accurate and efficient identification and classification of pig actions, providing real-time and effective monitoring of pig attack behaviors, and providing reliable guarantees for the healthy growth of pigs and the economic income of the pig farming industry.

[0092] Each embodiment in this specification is described in a progressive manner, and the same or similar parts of each embodiment can be directly referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. It should be noted that the technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above-mentioned embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0093] The above-mentioned embodiments only express several preferred implementation modes of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in the technical field, several improvements and substitutions can be made without departing from the technical principles of the present invention, and these improvements and substitutions should also be regarded as the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be based on the protection scope of the claims.

Claims

1. A method for identifying pig attack behavior, characterized in that: The method comprises the following steps: Obtain a video of a group of pigs to be identified, and extract a key frame sequence to be identified from the video of the group of pigs to be identified; The pig farming video to be identified is input into a lightweight 3D convolutional neural network to extract spatiotemporal information, and obtain corresponding spatiotemporal features to be fused; the lightweight 3D convolutional neural network is an XSepConvMobileNetv3 network improved based on the MobileNetv3-small network; Inputting the key frame sequence to be identified into a 2D convolutional neural network to extract spatial information and obtain corresponding spatial features to be fused; Fusing the spatiotemporal features to be fused with the spatial features to be fused to obtain spatiotemporal fusion features; Using a hole convolution layer to perform bounding box prediction and action classification on the spatiotemporal fusion features to obtain a recognition result; The steps of obtaining the XSepConvMobileNetv3 network based on the MobileNetv3-small network improvement include: The inverse residual linear bottleneck layer module with a convolution kernel size of 5×5 in the MobileNetv3-small network is replaced with an XSepCovbneck module; the XSepCovbneck module includes a 1×1 dimensionality increase convolution, an XSepConv extreme separation convolution, a channel attention module and a 1×1 dimensionality reduction convolution connected in sequence; the channel attention module includes an FC fully connected layer and a hard-swish activation function; The step of inputting the to-be-identified group pig farming video into a lightweight 3D convolutional neural network to extract spatiotemporal information to obtain corresponding spatiotemporal features to be fused comprises: Input the pig farming video of the group to be identified into a lightweight 3D convolutional neural network to extract spatiotemporal information and obtain corresponding spatiotemporal features; Compressing the spatiotemporal features according to a preset fusion size to obtain the spatiotemporal features to be fused; The 2D convolutional neural network includes a CSPDarknet53 network, a residual module and an SPPNet network connected in sequence; the residual module includes a 1×1 convolutional layer, a 3×3 convolutional layer and a 1×1 convolutional layer connected in sequence; the step of inputting the key frame sequence to be identified into the 2D convolutional neural network for spatial information extraction to obtain the corresponding spatial features to be fused includes: Performing feature extraction on the key frame sequence to be identified through the CSPDarknet53 network to obtain a first spatial feature; Inputting the first spatial feature into the residual module for convolution processing to obtain a second spatial feature; The second spatial feature is subjected to spatial pyramid pooling through the SPPNet network to obtain the to-be-fused spatial feature of the preset fusion size.

2. The pig attack behavior identification method according to claim 1, characterized in that: The step of extracting a key frame sequence to be identified from the pig farming video to be identified comprises: Performing image preprocessing on the video frame of the to-be-identified group pig-raising video to obtain the to-be-identified video frame image; According to the connected domain area and adhesion index of each video frame image to be identified, the key frame sequence to be identified is extracted from the video frame image to be identified.

3. The pig attack behavior identification method according to claim 1, characterized in that: The feature fusion includes channel fusion and attention mechanism fusion; The step of fusing the spatiotemporal features to be fused with the spatial features to be fused to obtain spatiotemporal fusion features comprises: Perform channel fusion on the spatiotemporal feature to be fused and the spatial feature to be fused to obtain a first spatiotemporal fusion feature; Inputting the first spatiotemporal fusion feature into a convolution module to obtain a second spatiotemporal fusion feature; the convolution module includes two 3×3 convolution layers; Using the Gram matrix method to perform attention mechanism fusion on the second spatiotemporal fusion feature to obtain a third spatiotemporal fusion feature; The third spatiotemporal fusion feature is input into the convolution module to obtain the spatiotemporal fusion feature.

4. The pig attack behavior identification method according to claim 3, characterized in that: The step of using the Gram matrix method to perform attention mechanism fusion on the second spatiotemporal fusion features to obtain the third spatiotemporal fusion features comprises: Performing two-dimensional vectorization processing on the second spatiotemporal fusion feature to obtain a first processing matrix; Taking a product of the first processing matrix and the transpose of the first processing matrix, to obtain a corresponding Gram matrix; The Gram matrix is ​​input into the softmax module for classification processing to obtain channel attention features; The second spatiotemporal fusion feature is corrected by using the channel attention feature, and the corrected spatiotemporal fusion feature is dimensionally transformed to obtain a corrected spatiotemporal fusion feature; The second spatiotemporal fusion feature is added to the modified spatiotemporal fusion feature to obtain the third spatiotemporal fusion feature.

5. A pig attack behavior recognition system, characterized in that: The system comprises: A video acquisition module, used to acquire a video of a group of pigs to be identified, and extract a key frame sequence to be identified from the video of the group of pigs to be identified; The spatiotemporal feature module is used to input the pig farming video of the group to be identified into a lightweight 3D convolutional neural network to extract spatiotemporal information and obtain the corresponding spatiotemporal features to be fused; the lightweight 3D convolutional neural network is an XSepConvMobileNetv3 network improved based on the MobileNetv3-small network; A spatial feature module is used to input the key frame sequence to be identified into a 2D convolutional neural network to extract spatial information and obtain corresponding spatial features to be fused; A feature fusion module, used to fuse the spatiotemporal features to be fused with the spatial features to be fused to obtain spatiotemporal fusion features; A result recognition module, used for performing bounding box prediction and action classification on the spatiotemporal fusion features by using a hole convolution layer to obtain a recognition result; The XSepConvMobileNetv3 network improved based on the MobileNetv3-small network includes: The inverse residual linear bottleneck layer module with a convolution kernel size of 5×5 in the MobileNetv3-small network is replaced with an XSepCovbneck module; the XSepCovbneck module includes a 1×1 dimensionality increase convolution, an XSepConv extreme separation convolution, a channel attention module and a 1×1 dimensionality reduction convolution connected in sequence; the channel attention module includes an FC fully connected layer and a hard-swish activation function; The step of inputting the pig farming video to be identified into a lightweight 3D convolutional neural network to extract spatiotemporal information and obtain corresponding spatiotemporal features to be fused includes: Input the pig farming video of the group to be identified into a lightweight 3D convolutional neural network to extract spatiotemporal information and obtain corresponding spatiotemporal features; Compressing the spatiotemporal features according to a preset fusion size to obtain the spatiotemporal features to be fused; The 2D convolutional neural network includes a CSPDarknet53 network, a residual module and an SPPNet network connected in sequence; the residual module includes a 1×1 convolutional layer, a 3×3 convolutional layer and a 1×1 convolutional layer connected in sequence; the key frame sequence to be identified is input into the 2D convolutional neural network for spatial information extraction to obtain the corresponding spatial features to be fused, including: Performing feature extraction on the key frame sequence to be identified through the CSPDarknet53 network to obtain a first spatial feature; Inputting the first spatial feature into the residual module for convolution processing to obtain a second spatial feature; The second spatial feature is subjected to spatial pyramid pooling through the SPPNet network to obtain the to-be-fused spatial feature of the preset fusion size.

6. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images