A Pig Behavior Recognition Method, Device and Storage Medium

Abstract

The present invention discloses a method, device and storage medium for pig behavior recognition. The method includes: obtaining a feature image to be recognized, recognizing the feature image according to a preset pig behavior recognition model, so that the pig behavior recognition model performs dimensionality folding extraction on the feature image to obtain a one-dimensional image feature vector, extracting feature information from the image feature vector according to a preset coding network to obtain feature information, extracting behavior information from the feature information according to a preset decoding network to obtain behavior information, and performing operations on the behavior information according to an FFN network to obtain a plurality of prediction boxes. By calculating a bipartite matching loss function, an optimal bipartite matching is generated between the prediction boxes and the ground truth boxes, and recognition boxes are generated on the feature image, and the feature image with the recognition boxes is output. The machine automatically recognizes the behavior of pigs, improving the accuracy of pig behavior recognition.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and particularly to a method, device, and storage medium for identifying pig behaviors. Background Art

[0002] By observing and analyzing pig behaviors, diseases that occur during the growth process of pigs can be intervened. Traditional pig behavior observation relies on a large amount of manpower and cannot achieve 24-hour continuous detection. With the advent of urbanization, there is a significant shortage of labor. Therefore, it has become increasingly impractical to continue to develop the breeding industry with a large amount of labor. Thus, it is necessary to automatically identify pig behaviors through machines, and computer vision is a key technology for developing machines for automatic behavior identification.

[0003] Existing pig behavior recognition technologies use sensors installed on individual pigs to record parameters for pig behavior recognition. Although installing sensors can record multiple parameters, each sensor can only record the behavior data of a single pig and cannot record the behavior information of multiple pigs simultaneously. Therefore, for large-scale farms, a large number of sensors are required, resulting in a significant increase in breeding costs. Secondly, installing sensors on pigs may cause certain harm to the pigs, resulting in abnormal behaviors that affect the observation of pig behaviors. Moreover, pigs also have certain aggressive behaviors, and the contact between pigs can cause sensor damage, which will also correspondingly increase breeding costs. Summary of the Invention

[0004] The present invention provides a method, device, and storage medium for identifying pig behaviors to solve the technical problem of improving the accuracy of pig behavior recognition.

[0005] To solve the above technical problem, an embodiment of the present invention provides a method for identifying pig behaviors, including obtaining a feature image to be recognized; wherein, the feature image to be recognized is extracted from a pig image acquisition device.

[0006] Identify the feature image according to a preset pig behavior recognition model, so that the pig behavior recognition model performs dimensionality folding extraction on the feature image to obtain a one-dimensional image feature vector. Then, according to a preset encoding network, a set position encoding, and the image feature vector, extract feature information from the image feature vector to obtain feature information. Next, according to a preset decoding network, a set target query vector, and the position encoding, extract behavior information from the feature information to obtain behavior information. Finally, perform operations on the behavior information through an FFN network to obtain a number of prediction boxes. By calculating the bipartite matching loss function, an optimal bipartite matching is generated between the prediction boxes and the ground truth boxes, and an identification box is generated on the feature image, and the feature image with the identification box is output; wherein, the identification box includes a rectangular box, pig prediction behavior information, and pig prediction position information.

[0007] As a preferred solution, in the present invention, a one-dimensional image feature vector is input into a preset encoding and decoding network in a preset pig behavior recognition model to extract feature and behavior information. The encoding and decoding network has the characteristic of fast training convergence speed, which improves the accuracy of image feature recognition. At the same time, the output of the encoding and decoding network passes through an FFN to obtain prediction boxes. This method avoids the huge computational amount generated by presetting a large number of anchor boxes in the traditional method and finally filtering out inaccurate detection boxes through non-maximum suppression, and improves the accuracy of image feature recognition. This makes the accuracy of image feature recognition of the entire pig behavior recognition model improved.

[0008] As a preferred solution, the pig behavior recognition model performs dimensionality folding extraction on the feature image to obtain a one-dimensional image feature vector, specifically:

[0009] Input the feature image into an extraction network to obtain image features, and input the image features into a 1x1 convolutional layer to output a one-dimensional image feature vector.

[0010] As a preferred solution, in the present invention, the 1x1 convolutional layer reduces the channel dimension of the feature map, thereby reducing the computational amount of the network.

[0011] As a preferred solution, the method of extracting feature information from the image feature vector according to a preset encoding network, a set position encoding, and the image feature vector to obtain feature information is specifically:

[0012] Input the image feature vector into the Norm layer of the first encoding sub-module, and input the output result of the Norm layer into the fully connected layer of the first encoding sub-module and the second Add&Norm module;

[0013] Add the set position encoding and the image feature vector to obtain a first weighted sum vector. Input the first weighted sum vector and the image feature vector into the Multi-Head Self-Attention module as the input vector of the first encoding sub-module. Input the input vector and the image feature vector into the first Add&Norm module, output through the fully connected layer, and then input into the second Add&Norm module to obtain the output of the first encoding sub-module.

[0014] The output of the previous encoding sub-module is input into the next encoding sub-module as the image feature vector of the next encoding sub-module. The output feature information of the last encoding sub-module is used to construct an encoding network with several encoding sub-modules.

[0015] As a preferred solution, through the improvement of the encoding sub-module structure in the present invention, the input of each encoding sub-module is added to the output of the FFN after passing through a Norm layer, and then input into the Add&Norm module. Add the input of the first encoding sub-module to the output of the last encoding sub-module, and then perform the Norm layer. The feature information extracted by this encoding network is further fused, retaining more original feature information, avoiding the loss of original features to a greater extent. This encoding network has the characteristics of fast training convergence speed and improves the accuracy of image feature recognition.

[0016] As a preferred solution, according to the preset decoding network, the set target query vector, and the position encoding, extract behavior information from the feature information to obtain behavior information, specifically:

[0017] Add the target query vector to itself to obtain a second weighted sum vector. Input the target query vector and the second weighted sum vector into the first Multi-Head Self-Attention module of the first decoding sub-module to form the first input vector of the first decoding sub-module. Input the first input vector and the target query vector into the first Add&Norm module, and add the output result of the first Add&Norm module to the target query vector to obtain a third weighted sum vector.

[0018] The fourth weighted sum vector obtained by adding the third weighted sum vector to the feature information and adding the feature information to the position encoding is input into the second Multi-Head Self-Attention module of the first decoding sub-module to form the second input vector of the first decoding sub-module. The second input vector and the output result of the first Add&Norm module are input into the second Add&Norm module. The output result of the second Add&Norm module passes through the fully connected layer of the first decoding sub-module and then is input into the third Add&Norm module of the first decoding sub-module, and the output of the first decoding sub-module is obtained.

[0019] The output of the previous decoding sub-module is used as the target query vector of the next decoding sub-module and is input into the next decoding sub-module. The output of the last decoding sub-module is the behavior information, and several decoding sub-modules are constructed into a decoding network.

[0020] As a preferred solution, through the improvement of the decoding sub-module structure, the decoding network of the present invention has the characteristics of fast training convergence speed and improves the accuracy of image feature recognition.

[0021] As a preferred solution, by calculating the bipartite matching loss function, an optimal bipartite matching is generated between the predicted box and the ground truth box, and a recognition box is generated on the feature image, specifically:

[0022] A number of images are taken frame by frame from the pig image acquisition device, the area where the pigs are located in the images is marked with a rectangular box, and corresponding behavior labels are set according to the behaviors of the pigs to obtain the ground truth boxes; wherein, the ground truth boxes include behavior labels and position information.

[0023] Calculate the bipartite matching loss function between the ground truth box and the predicted box. When the bipartite matching loss function converges, determine the pig prediction behavior information and pig prediction position information of the predicted box, and generate a recognition box on the feature image. The recognition box includes a rectangular box, pig prediction behavior information, and pig prediction position information.

[0024] As a preferred solution, by calculating the bipartite matching loss function between the ground truth box and the predicted box, the bounding box loss is optimized, and the accuracy of image feature recognition is improved.

[0025] As a preferred solution, the pig behavior recognition model is obtained by training the model loss with a training sample set, and the training method is specifically:

[0026] Obtain a number of the ground truth boxes as the training sample set. The training sample set includes a number of images and the corresponding behavior labels and position information for each image.

[0027] Input the training sample set into the model loss function for training to obtain a trained pig behavior recognition model.

[0028] As a preferred solution, the present invention trains the model loss with the ground truth box as the training sample set to improve the recognition accuracy of the pig behavior recognition model.

[0029] Correspondingly, the present invention also proposes a pig behavior recognition device, including: an image acquisition module and an image recognition module;

[0030] The image acquisition module is used to acquire the feature image to be recognized; wherein, the feature image to be recognized is extracted from the pig image acquisition device;

[0031] The image recognition module is used to recognize the feature image according to the preset pig behavior recognition model, so that the pig behavior recognition model performs dimensionality folding extraction on the feature image to obtain a one-dimensional image feature vector, and according to the preset encoding network, the set position encoding and the image feature vector, extracts feature information from the image feature vector to obtain feature information, and according to the preset decoding network, the set target query vector and the position encoding, extracts behavior information from the feature information to obtain behavior information, and according to the FFN network, operates on the behavior information to obtain a number of prediction boxes, and by calculating the bipartite matching loss function, an optimal bipartite matching is generated between the prediction boxes and the ground truth boxes, and a recognition box is generated on the feature image, and the feature image with the recognition box is output; wherein, the recognition box includes a rectangular box, pig predicted behavior information and pig predicted position information.

[0032] As a preferred solution, the present invention extracts feature and behavior information by inputting the one-dimensional image feature vector into the preset encoding and decoding networks in the preset pig behavior recognition model of the image recognition module. The encoding and decoding networks have the characteristics of fast training convergence speed and improve the recognition accuracy of image features.

[0033] As a preferred solution, the image recognition module includes: an information extraction unit and an information matching unit;

[0034] Among them, the information extraction unit is used to input the image feature vector and the position encoding into the encoding network and output feature information;

[0035] Input the target query vector, the position encoding and the feature information into the decoding network and output behavior information;

[0036] The information matching unit is used to operate on the behavior information according to the FFN network to obtain a number of prediction boxes;

[0037] Take a number of images frame by frame from the pig image acquisition device, label the area where the pigs are located in the image with a rectangular box, and set corresponding behavior labels according to the behavior of the pigs to obtain ground truth boxes; wherein, the ground truth boxes include behavior labels and position information;

[0038] Calculate the bipartite matching loss function between the ground truth boxes and the predicted boxes. When the bipartite matching loss function converges, determine the pig prediction behavior information and pig prediction position information of the predicted boxes, and generate recognition boxes on the feature image. The recognition boxes include rectangular boxes, pig prediction behavior information and pig prediction position information.

[0039] As a preferred solution, the information matching unit of the present invention obtains the predicted boxes through the FFN network. This method avoids the huge computational amount generated by presetting a large number of anchor boxes in the traditional method and finally filtering out the boxes with inaccurate detection through non-maximum suppression, and improves the accuracy of image feature recognition.

[0040] Correspondingly, the present invention also proposes a computer-readable storage medium, which includes a stored computer program; wherein, the computer program controls the device where the computer-readable storage medium is located to execute the pig behavior recognition method described in the content of the present invention when running. Description of the Drawings

[0041] Figure 1 is a schematic flowchart of an embodiment of the pig behavior recognition method provided by the present invention;

[0042] Figure 2 is a schematic structural diagram of an embodiment of the pig behavior recognition model of the pig behavior recognition method provided by the present invention;

[0043] Figure 3 is a schematic structural diagram of an embodiment of the Transformer model of the pig behavior recognition method provided by the present invention;

[0044] Figure 4 is a schematic flowchart of an embodiment of the training of the pig behavior recognition model of the pig behavior recognition method provided by the present invention;

[0045] Figure 5 is a schematic structural diagram of an embodiment of the pig behavior recognition device provided by the present invention. Detailed Embodiments

[0046] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0047] Embodiment 1

[0048] Please refer to Figure 1 , a pig behavior recognition method provided by an embodiment of the present invention, including the following steps S101-S102:

[0049] Step S101: Obtain a feature image to be recognized; wherein, the feature image to be recognized is extracted from a pig image acquisition device;

[0050] In this embodiment, the image acquisition device is installed above the pigsty at a preset overlooking angle, and then the image acquisition device is connected to an Intel microcomputer or other micro embedded devices; the required pig images are taken out frame by frame from the collected video; and the collected pig image data is transmitted to the Intel microcomputer.

[0051] Step S102: Recognize the feature image according to a preset pig behavior recognition model, so that the pig behavior recognition model performs dimensionality folding extraction on the feature image to obtain a one-dimensional image feature vector, and according to a preset encoding network, a set position encoding, and the image feature vector, extract feature information from the image feature vector to obtain feature information, and according to a preset decoding network, a set target query vector, and the position encoding, extract behavior information from the feature information to obtain behavior information, and perform operations on the behavior information according to the FFN network to obtain a plurality of prediction boxes. By calculating the bipartite matching loss function, an optimal bipartite matching is generated between the prediction box and the ground truth box, and a recognition box is generated on the feature image, and a feature image with the recognition box is output; wherein, the recognition box includes a rectangular box, pig prediction behavior information, and pig prediction position information.

[0052] In this embodiment, the pig behavior recognition model uses a ResNet50 extraction network to extract features in the image.

[0053] In this embodiment, the encoding network is composed of 6 encoder sub-modules, and each encoding sub-module is composed of a Multi-Head Self-Attention module, two Add&Norm modules, a fully connected layer, and a Norm layer.

[0054] Input the image feature vector into the Norm layer of the first encoding sub-module, and input the output result of the Norm layer into the fully connected layer of the first encoding sub-module and the second Add&Norm module;

[0055] Add the set position encoding and the image feature vector to obtain a first weighted sum vector. Input the first weighted sum vector as the K vector and Q vector into the Multi-Head Self-Attention module, and input the image feature vector as the V vector into the Multi-Head Self-Attention module. The Multi-Head Self-Attention module outputs the input vector of the first encoding sub-module. Input the input vector and the image feature vector into the first Add&Norm module, output through the fully connected layer, and then input into the second Add&Norm module to obtain the output of the first encoding sub-module, thus constituting an encoding sub-module.

[0056] The output of the previous encoding sub-module is input into the next encoding sub-module as the image feature vector of the next encoding sub-module. The output feature information of the last encoding sub-module is used to construct an encoding network with 6 encoding sub-modules.

[0057] The decoding network is composed of 6 decoder sub-modules. Each decoder sub-module consists of two Multi-Head Self-Attention modules, three Add&Norm modules, one Multi-Head Attention module, and one fully connected layer.

[0058] Add the target query vector to itself to obtain a second weighted sum vector. Input the second weighted sum vector as the K vector and Q vector into the first Multi-Head Self-Attention module, and input the target query vector as the V vector into the first Multi-Head Self-Attention module. The first Multi-Head Self-Attention module outputs the first input vector of the first decoding sub-module. Input the first input vector and the target query vector into the first Add&Norm module of the first decoding sub-module, and add the output result of the first Add&Norm module to the target query vector to obtain a third weighted sum vector;

[0059] Use the third weighted sum vector as the Q vector, use the feature information as the V vector, and use the fourth weighted sum vector obtained by adding the feature information and the position encoding as the K vector to input into the second Multi-Head Attention module of the first decoding sub-module to form a second input vector. Input the second input vector and the output result of the first Add&Norm module into the second Add&Norm module. The output result of the second Add&Norm module passes through the fully connected layer of the first decoding sub-module, and then is input into the third Add&Norm module of the first decoding sub-module to output the output of the first decoding sub-module;

[0060] Use the output of the previous decoding sub-module as the target query vector of the next decoding sub-module and input it into the next decoding sub-module. The output of the last decoding sub-module is the behavior information, and six decoding sub-modules are constructed into a decoding network.

[0061] Reference Figure 3 , and construct a Transformer model according to the encoding network and the decoding network.

[0062] In this embodiment, a number of images are taken frame by frame from the pig image acquisition device, the area where the pig is located in the image is marked with a rectangular frame, and corresponding behavior labels are set according to the behavior of the pig to obtain a ground truth box; wherein, the ground truth box includes a behavior label and position information; the label types include but are not limited to: stand, sit, lie, side_lie, climb, which respectively represent standing, sitting, lying on the back, lying on the side, and climbing the railing.

[0063] Calculate the bipartite matching loss function between the ground truth box and the predicted box. When the bipartite matching loss function converges, determine the pig predicted behavior information and pig predicted position information of the predicted box, and generate a recognition box on the feature image. The recognition box includes a rectangular frame, pig predicted behavior information, and pig predicted position information.

[0064] In this embodiment, the bipartite matching loss function is:

[0065] ;

[0066] where y represents the ground truth target, , represents N predicted targets. represents the permutation of the N elements with the lowest loss. represents the loss between the ground truth target and the predicted target with an index ( ).

[0067] ;

[0068] where, , represents the target class label, which defines the center coordinates, width, and height of the ground truth box relative to the image. represents the probability of the class with the predicted label being , and it is the localization loss.

[0069] Reference Figure 2 is the structural schematic diagram of the pig behavior recognition model in this embodiment.

[0070] In this embodiment, a number of the ground truth boxes are obtained as the training sample set, and the training sample set includes a number of images and the corresponding behavior labels and location information for each image; the training sample set is divided into a training set and a validation set in a ratio of 8:2 for training and validation.

[0071] The training sample set is input into the model loss function for training to obtain the trained pig behavior recognition model.

[0072] In this embodiment, the model loss includes a classification loss and a localization loss. Among them, the classification loss and the localization loss functions are:

[0073] ;

[0074] Among them, represents the classification loss, is the optimal value calculated in the bipartite matching loss function. is the localization loss, that is, the bounding box loss,

[0075] The localization loss function is:

[0076] ;

[0077] Among them, =2, =5, is the GIOU loss,

[0078] is expressed as:

[0079] ;

[0080] Among them, |.| represents the area, represents the maximum rectangle containing .

[0081] Reference Figure 4 is the flow schematic diagram of the training of the pig behavior recognition model in this embodiment.

[0082] Implementing the embodiments of the present invention has the following effects:

[0083] By inputting a one-dimensional image feature vector into a preset encoding and decoding network in a preset pig behavior recognition model to extract feature and behavior information, the encoding and decoding network has the characteristic of fast training convergence speed, and improves the accuracy of image feature recognition. At the same time, the output of the encoding and decoding network passes through the FFN to obtain prediction boxes. This method avoids the huge computational amount generated by presetting a large number of anchor boxes in the traditional method and finally filtering out inaccurate detection boxes through non-maximum suppression, and improves the accuracy of image feature recognition. This makes the entire pig behavior recognition model improve the accuracy of image feature recognition.

[0084] Embodiment 2

[0085] Please refer to Figure 5 , the present invention also provides a pig behavior recognition device, including: an image acquisition module 201 and an image recognition module 202.

[0086] Among them, the image acquisition module 201 is used to acquire a feature image to be recognized; among them, the feature image to be recognized is extracted from a pig image acquisition device;

[0087] The image recognition module 202 is used to recognize the feature image according to a preset pig behavior recognition model, so that the pig behavior recognition model performs dimensional folding extraction on the feature image to obtain a one-dimensional image feature vector, and according to a preset encoding network, a set position encoding and the image feature vector, extract feature information from the image feature vector to obtain feature information, and according to a preset decoding network, a set target query vector and the position encoding, extract behavior information from the feature information to obtain behavior information, and according to the FFN network, perform operations on the behavior information to obtain a plurality of prediction boxes, and by calculating the bipartite matching loss function, generate an optimal bipartite matching between the prediction boxes and the true boxes, generate a recognition box on the feature image, and output the feature image with the recognition box; among them, the recognition box includes a rectangular box, pig prediction behavior information and pig prediction position information.

[0088] In this embodiment, the image recognition module includes: an information extraction unit and an information matching unit;

[0089] Among them, the information extraction unit is used to input the image feature vector and the position encoding into the encoding network and output feature information;

[0090] Input the target query vector, the position encoding and the feature information into the decoding network, and the output is behavior information;

[0091] The information matching unit is used to perform operations on the behavior information according to the FFN network to obtain a plurality of prediction boxes;

[0092] Take a number of images frame by frame from the pig image acquisition device, label the area where the pigs are located in the image with a rectangular box, and set corresponding behavior labels according to the behaviors of the pigs to obtain ground truth boxes; wherein, the ground truth boxes include behavior labels and position information;

[0093] Calculate the bipartite matching loss function between the ground truth box and the predicted box. When the bipartite matching loss function converges, determine the pig prediction behavior information and pig prediction position information of the predicted box, and generate recognition boxes on the feature image. The recognition boxes include rectangular boxes, pig prediction behavior information, and pig prediction position information.

[0094] The above-mentioned pig behavior recognition device can implement the pig behavior recognition method of the above method embodiment. The optional items in the above method embodiment are also applicable to this embodiment and will not be elaborated here. The remaining content of the embodiments of the present application can refer to the content of the above method embodiment and will not be repeated in this embodiment.

[0095] Implementing the embodiments of the present invention has the following effects:

[0096] By inputting one-dimensional image feature vectors into a preset encoding and decoding network in the preset pig behavior recognition model of the image recognition module to extract feature and behavior information, the encoding and decoding network has the characteristics of fast training convergence speed and improves the accuracy of image feature recognition.

[0097] The information matching unit obtains the predicted box through the FFN network. This method avoids the huge computational amount generated by presetting a large number of anchor boxes in the traditional method and finally filtering out the boxes with inaccurate detection through non-maximum suppression, and improves the accuracy of image feature recognition.

[0098] Embodiment 4

[0099] Correspondingly, the present invention also provides a computer-readable storage medium, which includes a stored computer program. When the computer program runs, it controls the device where the computer-readable storage medium is located to execute the pig behavior recognition method described in any one of the above embodiments.

[0100] Exemplarily, the computer program can be divided into one or more modules / units. The one or more modules / units are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program in the terminal device.

[0101] The terminal device may be a computing device such as a desktop computer, a notebook, a palm computer, and a cloud server. The terminal device may include, but is not limited to, a processor and a memory.

[0102] The so-called processor may be a Central Processing Unit (CPU), or may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The processor is the control center of the terminal device, connecting various parts of the entire terminal device through various interfaces and lines.

[0103] The memory may be used to store the computer programs and / or modules. The processor realizes various functions of the terminal device by running or executing the computer programs and / or modules stored in the memory, and by calling the data stored in the memory. The memory may mainly include a program storage area and a data storage area. Among them, the program storage area may store an operating system, application programs required for at least one function, etc.; the data storage area may store data created according to the use of the mobile terminal, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, at least one magnetic disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0104] Among them, if the modules / units integrated in the terminal device are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, to implement all or part of the processes in the above-described embodiment methods of the present invention, it can also be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above-described various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file, or some intermediate form, etc. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disc, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal, and software distribution medium, etc.

[0105] The specific embodiments described above have further elaborated on the purpose, technical solutions, and beneficial effects of the present invention. It should be understood that the above description is only for the specific embodiments of the present invention and is not used to limit the protection scope of the present invention. It is particularly pointed out that for those skilled in the art, any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for identifying pig behavior, characterized in that, Including: Obtain a feature image to be recognized; wherein, the feature image to be recognized is extracted from a pig image acquisition device; Recognize the feature image according to a preset pig behavior recognition model, so that the pig behavior recognition model performs dimensionality folding extraction on the feature image to obtain a one-dimensional image feature vector, and according to a preset encoding network, a set position encoding, and the image feature vector, extract feature information from the image feature vector to obtain feature information, and according to a preset decoding network, a set target query vector, and the position encoding, extract behavior information from the feature information to obtain behavior information, and according to an FFN network, perform operations on the behavior information to obtain a number of prediction boxes, and by calculating a bipartite matching loss function, generate an optimal bipartite matching between the prediction boxes and the ground truth boxes, generate a recognition box on the feature image, and output the feature image with the recognition box; wherein, the recognition box includes a rectangular box, pig prediction behavior information, and pig prediction position information; The step of extracting behavior information from the feature information according to a preset decoding network, a set target query vector, and the position encoding to obtain behavior information is specifically: Add the target query vector to itself to obtain a second weighted sum vector, input the target query vector and the second weighted sum vector into the first Multi-Head Self-Attention module of the first decoding sub-module to form the first input vector of the first decoding sub-module, input the first input vector and the target query vector into the first Add&Norm module, and add the output result of the first Add&Norm module to the target query vector to obtain a third weighted sum vector; Input the third weighted sum vector, the fourth weighted sum vector obtained by adding the feature information and the sum of the feature information and the position encoding into the second Multi-Head Self-Attention module of the first decoding sub-module to form the second input vector of the first decoding sub-module, input the second input vector and the output result of the first Add&Norm module into the second Add&Norm module, the output result of the second Add&Norm module passes through the fully connected layer of the first decoding sub-module, and then is input into the third Add&Norm module of the first decoding sub-module to output the output of the first decoding sub-module; Take the output of the previous decoding sub-module as the target query vector of the next decoding sub-module and input it into the next decoding sub-module, and the output of the last decoding sub-module is the behavior information, and build a decoding network with a number of decoding sub-modules.

2. The pig behavior recognition method according to claim 1, characterized in that, The specific step of the pig behavior recognition model performing dimensionality folding extraction on the feature image to obtain a one-dimensional image feature vector is: Input the feature image into an extraction network to obtain image features, and input the image features into a 1x1 convolutional layer to output a one-dimensional image feature vector.

3. The pig behavior recognition method according to claim 1, characterized in that, Performing feature information extraction on the image feature vector according to the preset coding network, the set position coding, and the image feature vector to obtain feature information, specifically: Inputting the image feature vector into the Norm layer of the first coding sub-module, and inputting the output result of the Norm layer into the fully connected layer of the first coding sub-module and the second Add&Norm module; Adding the set position coding and the image feature vector to obtain a first weighted sum vector, inputting the first weighted sum vector and the image feature vector into the Multi-Head Self-Attention module as the input vector of the first coding sub-module, inputting the input vector and the image feature vector into the first Add&Norm module, outputting through the fully connected layer, and then inputting into the second Add&Norm module to obtain the output of the first coding sub-module; The output of the previous coding sub-module is input into the next coding sub-module as the image feature vector of the next coding sub-module, and the output feature information of the last coding sub-module is used to construct a coding network with several coding sub-modules.

4. The pig behavior recognition method according to claim 1, wherein, Generating a recognition box on the feature image by calculating a bipartite matching loss function to produce an optimal bipartite matching between the predicted box and the ground truth box, specifically: Taking out several images frame by frame from the pig image acquisition device, annotating the area where the pigs are located in the image with a rectangular box, and setting corresponding behavior labels according to the behaviors of the pigs to obtain the ground truth box; wherein, the ground truth box includes behavior labels and position information; Calculating the bipartite matching loss function between the ground truth box and the predicted box, and when the bipartite matching loss function converges, determining the pig prediction behavior information and pig prediction position information of the predicted box, and generating a recognition box on the feature image, where the recognition box includes a rectangular box, pig prediction behavior information, and pig prediction position information.

5. The pig behavior recognition method according to claim 4, characterized in that, The pig behavior recognition model is obtained by training the model loss using a training sample set, and the training method is specifically: Obtaining several ground truth boxes as the training sample set, where the training sample set includes several images and the corresponding behavior labels and position information for each image; Inputting the training sample set into the model loss function for training to obtain a trained pig behavior recognition model.

6. A pig behavior recognition device, characterized in that, Including: An image acquisition module and an image recognition module; The image acquisition module is used to acquire the feature image to be recognized; wherein, the feature image to be recognized is extracted from the pig image acquisition device; The image recognition module is used to recognize the feature image according to a preset pig behavior recognition model, so that the pig behavior recognition model performs dimensionality folding extraction on the feature image to obtain a one-dimensional image feature vector, and according to a preset encoding network, set position encoding, and the image feature vector, extracts feature information from the image feature vector to obtain feature information, and according to a preset decoding network, set target query vector, and the position encoding, extracts behavior information from the feature information to obtain behavior information, and according to the FFN network, operates on the behavior information to obtain a number of prediction boxes. By calculating the bipartite matching loss function, an optimal bipartite matching is generated between the prediction boxes and the ground truth boxes, and recognition boxes are generated on the feature image, and a feature image with the recognition boxes is output; wherein, the recognition boxes include rectangular boxes, pig prediction behavior information, and pig prediction position information; The specific process of extracting behavior information from the feature information according to the preset decoding network, set target query vector, and the position encoding is as follows: Add the target query vector to itself to obtain a second weighted sum vector. Input the target query vector and the second weighted sum vector into the first Multi-Head Self-Attention module of the first decoding sub-module to form the first input vector of the first decoding sub-module. Input the first input vector and the target query vector into the first Add&Norm module, and add the output result of the first Add&Norm module to the target query vector to obtain a third weighted sum vector; Input the third weighted sum vector, the fourth weighted sum vector obtained by adding the feature information and the sum of the feature information and the position encoding into the second Multi-Head Self-Attention module of the first decoding sub-module to form the second input vector of the first decoding sub-module. Input the second input vector and the output result of the first Add&Norm module into the second Add&Norm module. The output result of the second Add&Norm module passes through the fully connected layer of the first decoding sub-module and then is input into the third Add&Norm module of the first decoding sub-module to output the output of the first decoding sub-module; Take the output of the previous decoding sub-module as the target query vector of the next decoding sub-module and input it into the next decoding sub-module. The output of the last decoding sub-module is the behavior information, and a number of decoding sub-modules are constructed into a decoding network.

7. The pig behavior recognition device according to claim 6, characterized in that, The image recognition module includes: an information extraction unit and an information matching unit; Among them, the information extraction unit is used to input the image feature vector and the position encoding into the encoding network and output feature information; Input the target query vector, the position encoding, and the feature information into the decoding network, and the output is behavior information; The information matching unit is used to operate on the behavior information according to the FFN network to obtain a number of prediction boxes; Take a number of images frame by frame from the pig image acquisition device, label the area where the pigs are located in the image with a rectangular box, and set corresponding behavior labels according to the behaviors of the pigs to obtain ground truth boxes; wherein, the ground truth boxes include behavior labels and location information; Calculate the bipartite matching loss function between the ground truth boxes and the predicted boxes. When the bipartite matching loss function converges, determine the pig predicted behavior information and pig predicted location information of the predicted boxes, and generate recognition boxes on the feature images. The recognition boxes include rectangular boxes, pig predicted behavior information and pig predicted location information.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program; wherein, the computer program controls the device where the computer-readable storage medium is located to execute the pig behavior recognition method according to any one of claims 1 to 5 when running.

Images