Live pig behavior intelligent sensing method and system based on machine vision

Abstract

The invention relates to the technical field of intelligent perception, and particularly discloses a machine vision-based intelligent perception method and system for live pig behaviors, and the method comprises the steps: carrying out the semantic understanding and context semantic association analysis of a live pig behavior monitoring video collected by a camera based on a computer vision technology and an artificial intelligence technology; and whether the monitored live pig object has an estrus behavior or not is identified based on the context semantic association features of the live pig behaviors. Therefore, the estrus behavior of the live pigs can be found in time, so that breeding personnel can reasonably arrange mating time, and the breeding efficiency of the live pigs is improved.

Description

Technical Field

[0001] This application relates to the field of intelligent perception technologies, and more specifically, to an intelligent perception method and system for pig behavior based on machine vision. Background Art

[0002] Pig behavior is an important information source during pig farming. Intelligent perception of pig behavior can enable farmers to accurately grasp the growth and health conditions during pig farming. During pig farming, the identification of the estrus period of pigs is crucial. The estrus period refers to a specific time period when sows enter the estrus state and have the ability to reproduce. By accurately detecting and identifying the estrus behavior of pigs, it can help farmers reasonably arrange the breeding time, thereby improving the breeding efficiency and production benefits of pigs.

[0003] In traditional pig farming, the estrus behavior of pigs is usually obtained through manual records, and the estrus of pigs is indirectly judged according to the activity rules of pigs over a period of time. This method of manual monitoring and recording requires the coordination of the eyes and hands of farmers, which is not only inefficient, but also highly subjective and prone to errors. Moreover, with the expansion of the scale of the farm and the increase in the number of pigs, this method is increasingly unable to meet the requirement of timely detecting the estrus behavior of pigs.

[0004] Therefore, there is an expectation for an intelligent perception method and system for pig behavior based on machine vision. Summary of the Invention

[0005] To solve the above technical problems, this application is proposed. Embodiments of this application provide an intelligent perception method and system for pig behavior based on machine vision, which, based on computer vision technology and artificial intelligence technology, perform semantic understanding and context semantic association analysis on the pig behavior monitoring video collected by a camera, and identify whether the monitored pig object has estrus behavior based on the context semantic association characteristics of pig behavior. In this way, the estrus behavior of pigs can be detected in a timely manner, enabling farmers to reasonably arrange the breeding time, thereby improving the breeding efficiency of pigs.

[0006] Correspondingly, according to one aspect of this application, there is provided an intelligent perception method for pig behavior based on machine vision, which includes:

[0007] Obtain the behavior monitoring video of the monitored pig object collected by the camera;

[0008] Extract the semantic features of the behavior monitoring video to obtain a sequence of pig behavior detail enhanced semantic expression feature vectors;

[0009] Extract the pig behavior association features of the sequence of pig behavior detail enhanced semantic expression feature vectors to obtain a pig behavior context semantic association feature vector;

[0010] Based on the semantic correlation feature vector of the behavior context of the live pigs, determine whether the monitored live pig object has estrus behavior.

[0011] In the above-mentioned intelligent perception method of live pig behavior based on machine vision, extracting the semantic features of the behavior monitoring video to obtain a sequence of semantic expression feature vectors for enhancing the details of live pig behavior, including: segmenting the behavior monitoring video to obtain a sequence of behavior monitoring video segments; extracting the behavior features of live pigs from the sequence of behavior monitoring video segments to obtain a sequence of semantic expression feature maps of live pig behavior; and enhancing the behavior details of the sequence of semantic expression feature maps of live pig behavior to obtain the sequence of semantic expression feature vectors for enhancing the details of live pig behavior.

[0012] In the above-mentioned intelligent perception method of live pig behavior based on machine vision, extracting the behavior features of live pigs from the sequence of behavior monitoring video segments to obtain a sequence of semantic expression feature maps of live pig behavior, including: passing the sequence of behavior monitoring video segments through a live pig behavior understanding device based on a three-dimensional convolutional neural network model to obtain the sequence of semantic expression feature maps of live pig behavior.

[0013] In the above-mentioned intelligent perception method of live pig behavior based on machine vision, enhancing the behavior details of the sequence of semantic expression feature maps of live pig behavior to obtain the sequence of semantic expression feature vectors for enhancing the details of live pig behavior, including: passing the sequence of semantic expression feature maps of live pig behavior through a behavior detail enhancer based on a significant feature extractor to obtain the sequence of semantic expression feature vectors for enhancing the details of live pig behavior.

[0014] In the above-mentioned intelligent perception method of live pig behavior based on machine vision, passing the sequence of semantic expression feature maps of live pig behavior through a behavior detail enhancer based on a significant feature extractor to obtain the sequence of semantic expression feature vectors for enhancing the details of live pig behavior, including: enhancing the behavior details of the sequence of semantic expression feature maps of live pig behavior with the following behavior detail enhancement formula to obtain the sequence of semantic expression feature vectors for enhancing the details of live pig behavior; where the behavior detail enhancement formula is:

[0015] V output = GAP{ReLU(Conv 1×1 [Conv 3×3 (F input )])}

[0016] where, V output is the semantic expression feature vector for enhancing the details of live pig behavior, F input is the semantic expression feature map of live pig behavior, GAP represents performing a pooling operation, R eLU represents performing ReLU activation processing, Conv 1×1 (·) represents performing convolution operation based on a 1×1 convolution kernel, Conv 3×3 (·) represents performing convolution operation based on a 3×3 convolution kernel.

[0017] In the above-mentioned intelligent perception method of pig behavior based on machine vision, to extract the pig behavior association features of the sequence of pig behavior detail-enhanced semantic expression feature vectors to obtain the pig behavior context semantic association feature vector, it includes: passing the sequence of pig behavior detail-enhanced semantic expression feature vectors through a pig growth behavior context encoder based on a transformer to obtain the pig behavior context semantic association feature vector.

[0018] In the above-mentioned intelligent perception method of pig behavior based on machine vision, based on the pig behavior context semantic association feature vector, to determine whether the monitored pig object has estrus behavior, it includes: passing the pig behavior context semantic association feature vector through a classifier to obtain a classification result, and the classification result is used to indicate whether the monitored pig object has estrus behavior.

[0019] In the above-mentioned intelligent perception method of pig behavior based on machine vision, it also includes a training step: training the pig behavior understanding device based on the three-dimensional convolutional neural network model, the behavior detail enhancer based on the significant feature extractor, the pig growth behavior context encoder based on the transformer, and the classifier.

[0020] In the above-mentioned intelligent perception method for pig behavior based on machine vision, the training step includes: obtaining training data, where the training data includes the training behavior monitoring video of the monitored pig object collected by the camera, and the true value of whether the monitored pig object has estrus behavior; performing video segmentation on the training behavior monitoring video to obtain a sequence of training behavior monitoring video segments; passing the sequence of training behavior monitoring video segments through the pig behavior understanding device based on the three-dimensional convolutional neural network model to obtain a sequence of training pig behavior semantic expression feature maps; passing the sequence of training pig behavior semantic expression feature maps through the behavior detail enhancer based on the significant feature extractor to obtain a sequence of training pig behavior detail enhanced semantic expression feature vectors; passing the sequence of training pig behavior detail enhanced semantic expression feature vectors through the growth behavior context encoder based on the transformer to obtain a training pig behavior context semantic association feature vector; passing the training pig behavior context semantic association feature vector through the classifier to obtain a classification loss function value; calculating a specific loss function value for the sequence of the training pig behavior context semantic association feature vector and the sequence of the training pig behavior detail enhanced semantic expression feature vectors; using the weighted sum of the classification loss function value and the specific loss function value as the loss function value to train the pig behavior understanding device based on the three-dimensional convolutional neural network model, the behavior detail enhancer based on the significant feature extractor, the growth behavior context encoder based on the transformer, and the classifier.

[0021] According to another aspect of the present application, there is provided an intelligent perception system for pig behavior based on machine vision, which includes:

[0022] A pig object behavior monitoring module, configured to obtain the behavior monitoring video of the monitored pig object collected by the camera;

[0023] A pig behavior semantic understanding module, configured to extract the semantic features of the behavior monitoring video to obtain a sequence of pig behavior detail enhanced semantic expression feature vectors;

[0024] A pig behavior context semantic association module, configured to extract the pig behavior association features of the sequence of the pig behavior detail enhanced semantic expression feature vectors to obtain a pig behavior context semantic association feature vector;

[0025] An estrus behavior judgment module, configured to determine whether the monitored pig object has estrus behavior based on the pig behavior context semantic association feature vector.

[0026] Compared with the prior art, the method and system for intelligent perception of pig behavior based on machine vision provided by this application are based on computer vision technology and artificial intelligence technology, perform semantic understanding and context semantic association analysis on the pig behavior monitoring video collected by the camera, and identify whether the monitored pig object has estrus behavior based on the context semantic association characteristics of the pig behavior. In this way, the estrus behavior of pigs can be detected in a timely manner, enabling breeders to reasonably arrange the breeding time, thereby improving the breeding efficiency of pigs. Brief Description of the Drawings

[0027] By describing the embodiments of this application in more detail in conjunction with the accompanying drawings, the above and other objects, features, and advantages of this application will become more obvious. The accompanying drawings are used to provide a further understanding of the embodiments of this application, and constitute a part of the specification, and are used to explain this application together with the embodiments of this application, and do not constitute a limitation to this application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0028] Figure 1 It is a flowchart of the method for intelligent perception of pig behavior based on machine vision according to an embodiment of this application.

[0029] Figure 2 It is a schematic diagram of the architecture of the method for intelligent perception of pig behavior based on machine vision according to an embodiment of this application.

[0030] Figure 3 It is a flowchart of extracting the semantic features of the behavior monitoring video to obtain a sequence of semantic expression feature vectors with enhanced pig behavior details in the method for intelligent perception of pig behavior based on machine vision according to an embodiment of this application.

[0031] Figure 4 It is a flowchart of the training step in the method for intelligent perception of pig behavior based on machine vision according to an embodiment of this application.

[0032] Figure 5 It is a block diagram of the system for intelligent perception of pig behavior based on machine vision according to an embodiment of this application. Detailed Description of the Embodiments

[0033] For the convenience of understanding the embodiments of the present invention, the specific embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the specific embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0034] Figure 1 It is a flowchart of the method for intelligent perception of pig behavior based on machine vision according to an embodiment of this application.Figure 2 This is a schematic architecture diagram of an intelligent perception method for pig behavior based on machine vision according to an embodiment of the present application. As Figure 1 and Figure 2 shown, the intelligent perception method for pig behavior based on machine vision according to an embodiment of the present application includes the steps of: S110, obtaining a behavior monitoring video of a monitored pig object collected by a camera; S120, extracting semantic features of the behavior monitoring video to obtain a sequence of pig behavior detail-enhanced semantic expression feature vectors; S130, extracting pig behavior association features of the sequence of pig behavior detail-enhanced semantic expression feature vectors to obtain a pig behavior context semantic association feature vector; S140, based on the pig behavior context semantic association feature vector, determining whether the monitored pig object has estrus behavior.

[0035] As mentioned in the above background art, in the process of pig breeding, the identification of the estrus period of pigs is crucial. The estrus period is the most suitable time for sows to be bred. By accurately judging the estrus period of pigs, breeders can reasonably arrange the breeding time according to the estrus situation of sows, avoid missing the best breeding opportunity, thereby maximizing the conception rate and litter size and improving the breeding efficiency.

[0036] However, for the judgment of the estrus period of pigs, the observation in the manual mode can no longer meet the requirements of real-time dynamic data collection in intelligent pig breeding. Although early related innovative research can be used to monitor the movement of pigs, many of these studies require pigs to be equipped with sensors or tags, such as spiked ear tags. This invasive method increases the biosafety risk and brings inconvenience to management. In addition, the development of wearable devices for pig behavior monitoring is relatively slow and the cost is high, which limits large-scale applications.

[0037] To address the above technical problems, the technical concept of the present application is based on computer vision technology and artificial intelligence technology, performing semantic understanding and context semantic association analysis on the pig behavior monitoring video collected by the camera, and identifying whether the monitored pig object has estrus behavior based on the context semantic association features of pig behavior. In this way, the estrus behavior of pigs can be detected in a timely manner, enabling breeders to reasonably arrange the breeding time, thereby improving the breeding efficiency of pigs.

[0038] In the above-mentioned intelligent perception method for pig behavior based on machine vision, in step S110, obtain the behavior monitoring video of the monitored pig object collected by the camera. It should be understood that the behavior monitoring video of the monitored pig object collected by the camera records the behavior activities of the pig within a specific time period, including behavior characteristics such as the movement trajectory, posture, and facial expression of the pig. By further analyzing and processing the video through computer vision technology, the semantic feature information of the pig behavior can be extracted, such as the communication behavior and courtship behavior of the pig, and then it can be determined whether the monitored pig object has estrus behavior. Moreover, this method of using the camera to monitor the pig behavior can effectively avoid the drawbacks of traditional invasive methods and better achieve real-time behavior monitoring of pigs.

[0039] In the above-mentioned intelligent perception method for pig behavior based on machine vision, in step S120, extract the semantic features of the behavior monitoring video to obtain a sequence of semantic expression feature vectors with enhanced pig behavior details. That is, convert the pig behavior in the behavior monitoring video into the corresponding semantic feature expression, so as to convert the pig behavior into a form that can be understood and processed by the computer, and capture the evolution and changes of the pig behavior over time. Then, use the behavior context information to analyze and judge the behavior of the monitored pig object, and improve the accuracy of estrus behavior recognition.

[0040] Figure 3 It is a flowchart for extracting the semantic features of the behavior monitoring video to obtain a sequence of semantic expression feature vectors with enhanced pig behavior details in the intelligent perception method for pig behavior based on machine vision according to an embodiment of the present application. As Figure 3 shown, step S120 includes: S121, perform video segmentation on the behavior monitoring video to obtain a sequence of behavior monitoring video segments; S122, extract pig behavior features from the sequence of behavior monitoring video segments to obtain a sequence of pig behavior semantic expression feature maps; S123, perform behavior detail enhancement on the sequence of pig behavior semantic expression feature maps to obtain the sequence of semantic expression feature vectors with enhanced pig behavior details.

[0041] Specifically, in step S121, perform video segmentation on the behavior monitoring video to obtain a sequence of behavior monitoring video segments. It should be understood that the behavior monitoring video is a continuous video stream. By segmenting the video into multiple segments, the granularity of behavior analysis can be adjusted to a smaller time period to more accurately capture the details and changes of pig behavior. Moreover, directly performing behavior analysis and recognition on the entire behavior monitoring video may lead to excessive computational complexity and affect the processing efficiency of the algorithm. By segmenting the behavior monitoring video into multiple segments, the behavior analysis task can be decomposed into multiple subtasks and each segment can be processed separately, thereby improving the processing efficiency and real-time performance of the algorithm.

[0042] Specifically, in step S122, the sequence of the behavior monitoring video segments is subjected to pig behavior feature extraction to obtain a sequence of pig behavior semantic expression feature maps. In a specific example of the present application, the encoding method for subjecting the sequence of the behavior monitoring video segments to pig behavior feature extraction to obtain a sequence of pig behavior semantic expression feature maps is to pass the sequence of the behavior monitoring video segments through a pig behavior understanding device based on a three-dimensional convolutional neural network model to obtain the sequence of the pig behavior semantic expression feature maps. Those of ordinary skill in the art should know that a three-dimensional convolutional neural network (3DCNN) is a neural network model, which is usually used to process data with a time dimension such as videos. In a traditional two-dimensional convolutional neural network (2D CNN), the convolution operation is performed on a two-dimensional image, and the spatial features of the image are extracted by sliding a convolutional kernel. Compared with the traditional convolutional neural network, the three-dimensional convolutional neural network can consider the feature information of both the time dimension and the spatial dimension at the same time, and is more suitable for processing temporal data such as videos and action sequences. In the technical solution of the present application, each of the behavior monitoring video segments contains the behavior change information of the pig in the time dimension. Therefore, a pig behavior understanding device based on a three-dimensional convolutional neural network model is further used to separately mine the behavior change features of each of the behavior monitoring video segments. Specifically, by applying a three-dimensional convolutional kernel to perform a sliding convolution operation on the behavior monitoring video segment, the pig behavior understanding device can extract the semantic feature information of the pig behavior in the time dimension and the spatial dimension from the behavior monitoring video segment, such as features such as movement patterns and posture changes, which reflect the key information of the pig behavior and provide a data basis for the subsequent recognition of the pig estrus behavior.

[0043] Specifically, in step S123, the sequence of the semantic expression feature maps of the pig behaviors is enhanced in terms of behavior details to obtain the sequence of the semantic expression feature vectors of the enhanced pig behavior details. In a specific example of the present application, the encoding method for enhancing the sequence of the semantic expression feature maps of the pig behaviors in terms of behavior details to obtain the sequence of the semantic expression feature vectors of the enhanced pig behavior details is to pass the sequence of the semantic expression feature maps of the pig behaviors through a behavior detail enhancer based on a significant feature extractor to obtain the sequence of the semantic expression feature vectors of the enhanced pig behavior details. It should be understood that the behavior detail enhancer based on the significant feature extractor is composed of a convolutional layer and a pooling layer, which can learn the significant features of local regions in the input data and adjust the input data to highlight and manifest the significant region features. In the technical solution of the present application, the significant feature extractor is applied to process the sequence of the semantic expression feature maps of the pig behaviors to extract the significant behavior details therein. The sequence of the semantic expression feature maps of the pig behaviors is input into the behavior detail enhancer based on the significant feature extractor for processing. The behavior detail enhancer can enhance and highlight the information of specific regions, making the information of these specific regions more prominent in the feature map, so as to enhance the behavior detail information of the pigs, thereby improving the discrimination and representativeness of the features, enabling the semantic expression feature vectors of the enhanced pig behavior details to better express the key information of the pig behaviors, and reducing the interference of noise and redundant information, and improving the accuracy of subsequent behavior recognition tasks. Moreover, through the processing of the behavior detail enhancer, the semantic expression feature maps of the pig behaviors are further transformed into the semantic expression feature vectors of the enhanced pig behavior details with lower dimensions and more compactness, which helps to reduce the storage space and computational complexity of the data and is more convenient for subsequent behavior recognition tasks.

[0044] Specifically, passing the sequence of the semantic expression feature maps of the pig behaviors through a behavior detail enhancer based on a significant feature extractor to obtain the sequence of the semantic expression feature vectors of the enhanced pig behavior details includes: enhancing the sequence of the semantic expression feature maps of the pig behaviors in terms of behavior details with the following behavior detail enhancement formula to obtain the sequence of the semantic expression feature vectors of the enhanced pig behavior details; wherein, the behavior detail enhancement formula is:

[0045] V output = GAP{ReLU(Conv 1×1 [Conv 3×3 (F input )])}

[0046] wherein, V output is the semantic expression feature vector of the enhanced pig behavior details, and F inputis the semantic expression feature map of the pig behavior. GAP represents performing pooling operation, ReLU represents performing ReLU activation processing, and Conv 1×1 (·) represents performing convolution operation based on a 1×1 convolution kernel, and Conv 3×3 (·) represents performing convolution operation based on a 3×3 convolution kernel.

[0047] In the above-mentioned intelligent perception method of pig behavior based on machine vision, in step S130, the pig behavior association features of the sequence of the pig behavior detail-enhanced semantic expression feature vectors are extracted to obtain the pig behavior context semantic association feature vectors. In a specific example of this application, the implementation manner of extracting the pig behavior association features of the sequence of the pig behavior detail-enhanced semantic expression feature vectors to obtain the pig behavior context semantic association feature vectors is to pass the sequence of the pig behavior detail-enhanced semantic expression feature vectors through a growth behavior context encoder based on a Transformer to obtain the pig behavior context semantic association feature vectors. Those of ordinary skill in the art should know that a Transformer is a neural network model for processing sequence data. When processing sequence data, traditional recurrent neural networks (RNNs) need to process each element of the input sequence step by step in chronological order. While the Transformer adopts a global attention mechanism, which can process the entire input sequence simultaneously without processing in order. The core idea of the Transformer is the self-attention mechanism. The self-attention mechanism determines the importance of each position to other positions by calculating attention weights, and applies these weights to the weighted sum of features, enabling the model to perform adaptive weighted aggregation on different positions in the input sequence during transformation encoding, thereby capturing long-range dependencies in the input sequence. In the technical solution of this application, by using a growth behavior context encoder based on a Transformer to perform context modeling on the sequence of the pig behavior detail-enhanced semantic expression feature vectors, the context semantic association information in the sequence of the pig behavior detail-enhanced semantic expression feature vectors can be captured, that is, the correlation and interaction between different behavior details. By analyzing the association relationships and changes between different behavior features, the overall semantic meaning and context association of pig behavior can be better understood, such as the changes and continuity before and after estrus behavior, which helps to further improve the accuracy of pig behavior analysis and judgment.

[0048] In the above intelligent perception method for pig behavior based on machine vision, in step S140, based on the semantic association feature vector of the pig behavior context, it is determined whether the monitored pig object has estrus behavior. In a specific example of the present application, the implementation manner of determining whether the monitored pig object has estrus behavior based on the semantic association feature vector of the pig behavior context is to pass the semantic association feature vector of the pig behavior context through a classifier to obtain a classification result, and the classification result is used to represent whether the monitored pig object has estrus behavior. It should be understood that the classifier can learn the association rule between the pig behavior characteristics and estrus behavior through a large amount of training data and output the corresponding classification judgment result. In the technical solution of the present application, the classifier receives the semantic association feature vector of the pig behavior context as input, and judges whether the pig is in estrus by learning the behavior characteristics in the semantic association feature vector of the pig behavior context, such as the characteristic pattern, frequency, duration, etc. of the pig behavior, and outputs the corresponding classification result, that is, "has estrus behavior" or "does not have estrus behavior", so as to convert the complex semantic association feature into a simple classification result. In this way, the estrus behavior of pigs can be monitored in real time and fed back to the breeding management personnel in time, so as to take corresponding management measures, such as breeding, etc.

[0049] It should be understood that before using the above neural network model, it is necessary to train the pig behavior understanding device based on the three-dimensional convolutional neural network model, the behavior detail enhancer based on the significant feature extractor, the growth behavior context encoder based on the transformer and the classifier. That is to say, in the intelligent perception method for pig behavior based on machine vision of the present application, it also includes a training step of training the pig behavior understanding device based on the three-dimensional convolutional neural network model, the behavior detail enhancer based on the significant feature extractor, the growth behavior context encoder based on the transformer and the classifier.

[0050] Figure 4 The flowchart of the training step in the intelligent perception method for pig behavior based on machine vision according to the embodiment of the present application. As Figure 4As shown, the training steps include: S210, obtaining training data, where the training data includes a training behavior monitoring video of the monitored live pig object collected by a camera, and the true value of whether the monitored live pig object has estrus behavior; S220, performing video segmentation on the training behavior monitoring video to obtain a sequence of training behavior monitoring video segments; S230, passing the sequence of training behavior monitoring video segments through the live pig behavior understanding device based on the three-dimensional convolutional neural network model to obtain a sequence of training live pig behavior semantic expression feature maps; S240, passing the sequence of training live pig behavior semantic expression feature maps through the behavior detail enhancer based on the significant feature extractor to obtain a sequence of training live pig behavior detail enhanced semantic expression feature vectors; S250, passing the sequence of training live pig behavior detail enhanced semantic expression feature vectors through the growth behavior context encoder based on the transformer to obtain a training live pig behavior context semantic association feature vector; S260, passing the training live pig behavior context semantic association feature vector through the classifier to obtain a classification loss function value; S270, calculating a specific loss function value of the sequence of the training live pig behavior context semantic association feature vector and the sequence of the training live pig behavior detail enhanced semantic expression feature vectors; S280, using the weighted sum of the classification loss function value and the specific loss function value as the loss function value to train the live pig behavior understanding device based on the three-dimensional convolutional neural network model, the behavior detail enhancer based on the significant feature extractor, the growth behavior context encoder based on the transformer, and the classifier.

[0051] In the technical solution of the present application, each training live pig behavior detail enhanced semantic expression feature vector in the sequence of training live pig behavior detail enhanced semantic expression feature vectors expresses the image semantic features of the training behavior monitoring video in the local feature distribution space strengthened by the temporal association in the local image semantic space domain determined by video segmentation in the global image semantic space domain. After passing the sequence of training live pig behavior detail enhanced semantic expression feature vectors through the growth behavior context encoder based on the transformer, the image semantic features in the local image semantic space domain can be contextually associated based on the semantic feature distribution context between local image semantic space domains. However, this will also result in different feature group density representations of the training live pig behavior context semantic association feature vector relative to the sequence of training live pig behavior detail enhanced semantic expression feature vectors, such that during the overall training of the model, there will be an iterative imbalance between the image semantic feature extraction of the three-dimensional convolutional neural network model and the significant feature extractor and the context association encoding of the growth behavior context encoder based on the transformer, affecting the overall training efficiency of the model.

[0052] Therefore, in the technical solution of the present application, in order to improve the consistency of the feature group density representation of the sequence of the training live pig behavior context semantic association feature vectors relative to the sequence of the training live pig behavior detail enhanced semantic expression feature vectors, a loss function for the sequence of the training live pig behavior context semantic association feature vectors and the training live pig behavior detail enhanced semantic expression feature vectors is further introduced.

[0053] Specifically, the step S270 includes: calculating a specific loss function value of the sequence of the training live pig behavior context semantic association feature vectors and the training live pig behavior detail enhanced semantic expression feature vectors by using the following specific loss function, where the specific loss function is expressed as:

[0054]

[0055] V1 is the training live pig behavior context semantic association feature vector, V2 is the concatenated feature vector obtained by concatenating the sequences of the training live pig behavior detail enhanced semantic expression feature vectors, v 1i and v 2i are the eigenvalues of the training live pig behavior context semantic association feature vector V1 and the concatenated feature vector V2 respectively, L is the length of the feature vector, and represents the square of the two-norm of the vector, is the specific loss function value.

[0056] Here, the specific loss function performs group counting attention based on the feature group density. It performs adaptive attention on different density representation patterns between the sequence of the training live pig behavior context semantic association feature vectors and the sequence of the training live pig behavior detail enhanced semantic expression feature vectors by taking the group counting as a recursive mapping for outputting the feature group density. By training the model with it as the specific loss function, the model can avoid overestimation and underestimation for different density patterns under the feature distributions of the sequence of the training live pig behavior context semantic association feature vectors and the sequence of the training live pig behavior detail enhanced semantic expression feature vectors, and learn the correspondence between the eigenvalue distribution and the group density distribution, so as to realize the optimization of the feature group density representation consistency between the sequence of the training live pig behavior context semantic association feature vectors and the sequence of the training live pig behavior detail enhanced semantic expression feature vectors with different feature densities, and improve the overall training efficiency of the model.

[0057] In summary, the intelligent perception method for pig behavior based on machine vision according to the embodiments of the present application is elucidated. Based on computer vision technology and artificial intelligence technology, it performs semantic understanding and context semantic association analysis on the pig behavior monitoring video collected by the camera, and identifies whether the monitored pig object has estrus behavior based on the context semantic association features of the pig behavior. In this way, the estrus behavior of pigs can be discovered in time, enabling breeders to reasonably arrange the breeding time, thereby improving the breeding efficiency of pigs.

[0058] Figure 5 FIG. is a block diagram of an intelligent perception system for pig behavior based on machine vision according to an embodiment of the present application. As Figure 5 shown, the intelligent perception system 100 for pig behavior based on machine vision according to an embodiment of the present application includes: a pig object behavior monitoring module 110, configured to obtain a behavior monitoring video of the monitored pig object collected by a camera; a pig behavior semantic understanding module 120, configured to extract semantic features of the behavior monitoring video to obtain a sequence of pig behavior detail enhanced semantic expression feature vectors; a pig behavior context semantic association module 130, configured to extract pig behavior association features of the sequence of pig behavior detail enhanced semantic expression feature vectors to obtain a pig behavior context semantic association feature vector; and an estrus behavior judgment module 140, configured to determine whether the monitored pig object has estrus behavior based on the pig behavior context semantic association feature vector.

[0059] Here, those skilled in the art can understand that the specific operations of the above-mentioned modules in the intelligent perception system for pig behavior based on machine vision have been described in detail above with reference to Figures 1 to 4 the description of the intelligent perception method for pig behavior based on machine vision, and therefore, the repeated description thereof will be omitted.

[0060] In the above embodiments, the descriptions of the various embodiments have their own focuses. For parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments. In the several embodiments provided by the present invention, it should be understood that the disclosed devices, systems, and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the module division is only a logical function division, and there may be other division methods in actual implementation. The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0061] In addition, in each of the embodiments of the present invention, the functional modules can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware, or in the form of a combination of hardware and software functional modules.

[0062] For those skilled in the art, it is obvious that the present invention is not limited to the details of the above-described exemplary embodiments, and without departing from the spirit or basic characteristics of the present invention, the present invention can be implemented in other specific forms. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be embraced by the present invention. Any reference signs in the claims should not be regarded as limiting the claims involved.

[0063] In addition, it is obvious that the word "including" does not exclude other units or steps, and the singular does not exclude the plural. The multiple units stated in the system claims can also be implemented by one unit through software or hardware.

[0064] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. An intelligent perception method for pig behavior based on machine vision, characterized in that, Including: Obtain the behavior monitoring video of the monitored live pig object collected by the camera; Extract the semantic features of the behavior monitoring video to obtain a sequence of pig behavior detail-enhanced semantic expression feature vectors; Extract the pig behavior correlation features of the sequence of pig behavior detail-enhanced semantic expression feature vectors to obtain a pig behavior context semantic correlation feature vector; Based on the pig behavior context semantic correlation feature vector, determine whether the monitored live pig object has estrus behavior.

2. The intelligent perception method for pig behavior based on machine vision according to claim 1, characterized in that Extract the semantic features of the behavior monitoring video to obtain a sequence of pig behavior detail-enhanced semantic expression feature vectors, including: Perform video segmentation on the behavior monitoring video to obtain a sequence of behavior monitoring video segments; Extract pig behavior features from the sequence of behavior monitoring video segments to obtain a sequence of pig behavior semantic expression feature maps; Perform behavior detail enhancement on the sequence of pig behavior semantic expression feature maps to obtain the sequence of pig behavior detail-enhanced semantic expression feature vectors.

3. The intelligent perception method for pig behavior based on machine vision according to claim 2, characterized in that, Extract pig behavior features from the sequence of behavior monitoring video segments to obtain a sequence of pig behavior semantic expression feature maps, including: Pass the sequence of behavior monitoring video segments through a pig behavior understanding device based on a three-dimensional convolutional neural network model to obtain the sequence of pig behavior semantic expression feature maps.

4. The intelligent perception method for pig behavior based on machine vision according to claim 3, wherein Perform behavior detail enhancement on the sequence of pig behavior semantic expression feature maps to obtain the sequence of pig behavior detail-enhanced semantic expression feature vectors, including: Pass the sequence of pig behavior semantic expression feature maps through a behavior detail enhancer based on a significant feature extractor to obtain the sequence of pig behavior detail-enhanced semantic expression feature vectors.

5. The intelligent perception method for pig behavior based on machine vision according to claim 4, characterized in that Pass the sequence of pig behavior semantic expression feature maps through a behavior detail enhancer based on a significant feature extractor to obtain the sequence of pig behavior detail-enhanced semantic expression feature vectors, including: Perform behavior detail enhancement on the sequence of pig behavior semantic expression feature maps with the following behavior detail enhancement formula to obtain the sequence of pig behavior detail-enhanced semantic expression feature vectors; where the behavior detail enhancement formula is: V output = GAP{ReLU(Conv 1×1 [Conv 3×3 (F input )} Among them, V output is the enhanced semantic expression feature vector of the pig behavior details, F input is the semantic expression feature map of the pig behavior, GAP represents performing pooling operation, ReLU represents performing ReLU activation processing, Conv 1×1 (·) represents performing convolution operation based on 1×1 convolution kernel, Conv 3×3 (·) represents performing convolution operation based on 3×3 convolution kernel.

6. The intelligent perception method for pig behavior based on machine vision according to claim 5, characterized in that, Extract the pig behavior correlation features of the sequence of pig behavior detail-enhanced semantic expression feature vectors to obtain a pig behavior context semantic correlation feature vector, including: Pass the sequence of pig behavior detail-enhanced semantic expression feature vectors through a growth behavior context encoder based on a transformer to obtain the pig behavior context semantic correlation feature vector.

7. The intelligent perception method for pig behavior based on machine vision according to claim 6, wherein Based on the pig behavior context semantic correlation feature vector, determine whether the monitored live pig object has estrus behavior, including: Pass the pig behavior context semantic correlation feature vector through a classifier to obtain a classification result, and the classification result is used to indicate whether the monitored live pig object has estrus behavior.

8. The intelligent perception method for pig behavior based on machine vision according to claim 7, characterized in that It also includes a training step: training the pig behavior understanding device based on the three-dimensional convolutional neural network model, the behavior detail enhancer based on the significant feature extractor, the growth behavior context encoder based on the transformer, and the classifier.

9. The intelligent perception method for pig behavior based on machine vision according to claim 8, wherein The training step includes: Obtain training data, where the training data includes training behavior monitoring videos of the monitored live pig objects collected by a camera, and the true values of whether the monitored live pig objects have estrus behaviors; Perform video segmentation on the training behavior monitoring videos to obtain a sequence of training behavior monitoring video segments; Pass the sequence of the training behavior monitoring video segments through the live pig behavior understanding device based on the three-dimensional convolutional neural network model to obtain a sequence of training live pig behavior semantic expression feature maps; Pass the sequence of the training live pig behavior semantic expression feature maps through the behavior detail enhancer based on the significant feature extractor to obtain a sequence of training live pig behavior detail enhanced semantic expression feature vectors; Pass the sequence of the training live pig behavior detail enhanced semantic expression feature vectors through the growth behavior context encoder based on the transformer to obtain a training live pig behavior context semantic association feature vector; Pass the training live pig behavior context semantic association feature vector through the classifier to obtain a classification loss function value; Calculate a specific loss function value of the sequence of the training live pig behavior context semantic association feature vector and the sequence of the training live pig behavior detail enhanced semantic expression feature vectors; Use the weighted sum of the classification loss function value and the specific loss function value as the loss function value to train the live pig behavior understanding device based on the three-dimensional convolutional neural network model, the behavior detail enhancer based on the significant feature extractor, the growth behavior context encoder based on the transformer, and the classifier.

10. A machine vision-based intelligent live pig behavior perception system, characterized in that A live pig object behavior monitoring module, configured to obtain behavior monitoring videos of the monitored live pig objects collected by a camera; A live pig behavior semantic understanding module, configured to extract semantic features of the behavior monitoring videos to obtain a sequence of live pig behavior detail enhanced semantic expression feature vectors; A live pig behavior context semantic association module, configured to extract live pig behavior association features of the sequence of the live pig behavior detail enhanced semantic expression feature vectors to obtain a live pig behavior context semantic association feature vector; An estrus behavior judgment module, configured to determine whether the monitored live pig objects have estrus behaviors based on the live pig behavior context semantic association feature vector.

Images