A method for recognizing lying patterns of pigs based on unsupervised clustering and deep learning

Abstract

The present invention discloses a method for recognizing lying patterns of pigs in groups based on unsupervised clustering and deep learning, comprising the following steps: collecting an initial data set and uniformly cropping the image; extracting a target image using the Grabcut image segmentation method; binarizing the target image through dynamic threshold segmentation, finding the cluster center using the unsupervised clustering algorithm Mean‑shift, fitting the distance from the center to the cluster center through each ellipse and outputting its variance and standard deviation; processing the standard deviation so that the value of its data is mapped to between [0,1], and recognizing the lying patterns of pigs in groups using the trained CNN‑SVM model. Beneficial effects of the present invention: Based on the CNN‑SVM classification model, 5 lying patterns of pigs in groups are classified and recognized with high precision, with a recognition accuracy of 97.09%, a balanced accuracy of 96.59%, and a weighted balanced accuracy of 97.07%.

Description

Technical Field

[0001] The present invention relates to the technical field of multi-posture recognition of a group of pigs, and in particular to a lying pattern recognition method of a group of pigs based on unsupervised clustering and deep learning. Background Art

[0002] In recent years, as the scale of pig farming has increased, people are not only interested in detecting the lying behavior of single pigs, but also in monitoring the lying behavior of large-scale pigs. The combination of machine vision and machine learning technology has increased the possibility of detecting the behavior, health and growth status of large-scale animals. By detecting their current conditions such as health status and social activities, managers can make timely adjustments. Video behavior detection of pigs is a typical way to apply machine vision technology and deep learning technology. This is a fast, low-cost and non-stressful method. This method is relatively easy and convenient for both breeders and animals. It is suitable for indoor and outdoor situations and avoids the tedious process of traditional manual observation and recording.

[0003] Traditionally, the study of pig lying patterns is done through video or manual observation and recording. This method is time-consuming, labor-intensive, and requires a lot of cost. At present, there are few studies on the spatiotemporal characteristics of pig lying patterns under commercial conditions, and most of them are studies on the specific lying postures of live pigs. So far, no unsupervised technology and deep learning have been used to study the lying patterns of pigs under commercial conditions. Summary of the invention

[0004] The purpose of this section is to summarize some aspects of embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the specification abstract and the invention title of this application to avoid blurring the purpose of this section, the specification abstract and the invention title, and such simplifications or omissions cannot be used to limit the scope of the present invention.

[0005] In view of the above existing problems, the present invention is proposed.

[0006] Therefore, the technical problem solved by the present invention is that the lying pattern of the pigs cannot be accurately identified.

[0007] In order to solve the above technical problems, the present invention provides the following technical solutions: a method for recognizing lying patterns of pigs in groups based on unsupervised clustering and deep learning, comprising the following steps: collecting an initial data set and uniformly cropping the image; extracting a target image using the Grabcut image segmentation method; binarizing the target image through dynamic threshold segmentation, and using a median fuzzy filter algorithm for multiple iterations to eliminate the interference of image noise and exclude moving pigs; marking the segmented area, then performing watershed segmentation, and performing ellipse fitting on the basis of the segmentation, while obtaining the center point of each fitted ellipse; based on The unsupervised clustering algorithm Mean-shift is used to find the cluster center of the pig group image after ellipse fitting, and the distance from each ellipse fitting center to the cluster center is used to output its variance and standard deviation; the standard deviation is processed to map its data value to [0,1], and the normalized data is defined as 5 equal parts, different numerical ranges represent different pig group aggregation patterns, and the normalized variance and standard deviation of the pig group aggregation in each frame image is input into the CNN-SVM model classifier as training data; the trained CNN-SVM model is used to identify the pig group lying pattern.

[0008] As a preferred solution of the method for recognizing lying patterns of pig groups based on unsupervised clustering and deep learning described in the present invention, the mapping process of the standard deviation is as follows:

[0009]

[0010] In the formula, x i Represents the coordinates of the center of the pig ellipse fitting in a certain frame of image, X i represents the distance from the pig ellipse fitting center to the cluster center, m represents the number of pigs in a frame image, x centroids-X Represents the coordinates of the cluster center in the frame image, S N-Fr(j) represents the standard deviation in the image frame, Represents the value after standardization of the standard deviation.

[0011] As a preferred solution of the method for recognizing lying patterns of pig groups based on unsupervised clustering and deep learning described in the present invention, the normalized data is divided into five equal parts:

[0012] As a preferred solution of the method for recognizing lying patterns of pig groups based on unsupervised clustering and deep learning described in the present invention, the input formula of the previous layer of the internal framework of the CNN-SVM model classifier is as follows:

[0013]

[0014] In the formula, xl-1 i and Represent the input and output of the convolutional layer respectively, k l-1 ij The weight kernel is from the i-th neuron in layer l-1 to the j-th neuron in layer l, dconv represents the convolution operation, b l j represents the bias of the jth neuron in layer l, M l-1 Represents the number of kernel filters in the l-1 layer.

[0015] As a preferred solution of the method for recognizing lying patterns of pig groups based on unsupervised clustering and deep learning described in the present invention, the ReLU activation function formula is as follows:

[0016]

[0017] x l j is the output of layer l before downsampling, and the output of the current layer is expressed as follows:

[0018]

[0019] In the formula, DS represents the maximum pooling function, y l j The output of layer l is also the input of layer l+1, and the output connection of the last pooling layer is the input of the full connection.

[0020] As a preferred solution of the method for recognizing lying patterns of pig groups based on unsupervised clustering and deep learning described in the present invention, after the CNN-SVM model obtains the prediction output, the prediction error E is calculated using the softmax function with cross entropy in the cost function, and the formula is as follows:

[0021]

[0022] Where Y ni ,y ni Represent the target label and predicted output respectively, i is the number of categories in the nth training set, and the role of model learning is to optimize the network-free parameters to minimize the prediction error E, where the weights and biases are updated with the learning rate (η).

[0023] As a preferred solution of the method for recognizing lying patterns of pigs based on unsupervised clustering and deep learning described in the present invention, stochastic gradient descent (SGD) is used as the optimization algorithm, and weight initialization is applied, the model learning rate (η) is 0.001, and the weight and bias update formulas in the convolutional layer and the fully connected layer are as follows:

[0024]

[0025] As a preferred solution of the method for recognizing the lying pattern of a group of pigs based on unsupervised clustering and deep learning described in the present invention, the lying pattern posture of the group of pigs recognized by the CNN-SVM model is evaluated using multiple classification evaluation indicators.

[0026] As a preferred solution of the method for recognizing lying patterns of groups of pigs based on unsupervised clustering and deep learning described in the present invention, the multi-classification evaluation indicators are standard precision (P), recall rate (R), F1 and accuracy rate (ACC).

[0027] The beneficial effects of the present invention are as follows: based on the CNN-SVM classification model, five lying modes of a group of pigs are classified and identified with high precision, with an identification accuracy of 97.09%, a balance accuracy of 96.59%, and a weighted balance accuracy of 97.07%. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them:

[0029] Figure 1 A schematic diagram of the light conditions of the experimental site of the present invention;

[0030] Figure 2 This is a flowchart of the image preprocessing of the present invention;

[0031] Figure 3 This is a schematic diagram of the principle of deleting and moving pigs according to the present invention;

[0032] Figure 4 It is a schematic diagram of the experimental results of K-Means described in the present invention;

[0033] Figure 5 This is a schematic diagram of the experimental results of Mean-shift described in the present invention;

[0034] Figure 6 Schematic diagram of the Mean Shift experimental process of the present invention;

[0035] Figure 7 This is an example diagram extracted from the lying mode of the present invention;

[0036] Figure 8 It is a schematic flow chart of the method for identifying the lying pattern of a group of pigs according to the present invention;

[0037] Fig. 9 This is a schematic diagram of the network structure of the CNN-SVM model of the present invention;

[0038] Fig.10 This is a schematic diagram of the visualization result of the pig group lying mode according to the present invention;

[0039] Fig.11 It is a schematic diagram of the ROC curve of the group pig lying mode described in the present invention. DETAILED DESCRIPTION

[0040] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0041] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0042] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0043] The present invention is described in detail with reference to schematic diagrams. When describing the embodiments of the present invention, for the sake of convenience, the cross-sectional diagrams showing the device structure will not be partially enlarged according to the general scale, and the schematic diagrams are only examples, which should not limit the scope of protection of the present invention. In addition, in actual production, the three-dimensional dimensions of length, width and depth should be included.

[0044] At the same time, in the description of the present invention, it should be noted that the directions or positional relationships indicated by the terms "upper, lower, inner and outer" are based on the directions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore cannot be understood as limiting the present invention. In addition, the terms "first, second or third" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0045] In the present invention, unless otherwise clearly specified and limited, the terms "install, connect, connect" should be understood in a broad sense, for example: it can be a fixed connection, a detachable connection or an integral connection; it can also be a mechanical connection, an electrical connection or a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0046] Example 1

[0047] This embodiment proposes a method for recognizing lying patterns of pigs in groups based on unsupervised clustering and deep learning, comprising the following steps:

[0048] S1: Collect the initial data set and uniformly crop the images;

[0049] S2: Extract the target image using Grabcut image segmentation method;

[0050] S3: binarizing the target image through dynamic threshold segmentation, and using the median fuzzy filtering algorithm for multiple iterations to eliminate the interference of image noise and exclude moving pigs;

[0051] S4: Mark the segmented area, perform watershed segmentation, and perform ellipse fitting based on the segmentation, and obtain the center point of each fitted ellipse;

[0052] S5: Based on the pig group images after ellipse fitting, the unsupervised clustering algorithm Mean-shift is used to find the cluster center, and the distance from each ellipse fitting center to the cluster center is calculated and its variance and standard deviation are output;

[0053] S6: Processing the standard deviation to map the data value to between [0, 1], and defining the normalized data as 5 equal parts, with different value ranges representing different pig group aggregation patterns, and inputting the normalized variance and standard deviation of the pig group aggregation in each frame image as training data into the CNN-SVM model classifier;

[0054] S7: Using the trained CNN-SVM model to identify the lying pattern of the pigs.

[0055] Specifically, in step S1, it is necessary to explain:

[0056] Various environmental factors in real scenes affect the results of the experiment to a certain extent, resulting in inaccurate analysis of the lying patterns of pigs. The main reasons affecting the experimental results include: (1) the pigs are too crowded together, resulting in inaccurate image segmentation; (2) the pigs have stains on their backs during daily activities, resulting in large errors in the contour extraction of the target pigs; (3) the changes in light in the experimental site (lighting, day and night changes, and reflections from water stains on the ground) make image processing more difficult.

[0057] This embodiment mainly studies the lying pattern of pigs in the entire pigpen, because pigs are generally raised in pens during production, and the crowded pigs will cause image adhesion, and the image processing difficulty is much greater than that of a single pig image. In recent years, traditional positioning pen farming has led to lack of exercise, long-term mental depression, and lack of nutrition for pigs, making the pen farming method adopted by more and more breeders. This embodiment identifies the lying pattern of a group of pigs according to the actual production environment. The difficulty of automatic detection in this environment is much higher than that of recognizing the lying posture of a single pig.

[0058] Due to their own behavior habits, pigs often have stains on their backs and sides during their daily activities, which can lead to inaccurate pig image extraction and interfere with subsequent research. In order to solve this problem, the experimental site staff regularly cleans the pig pens to keep the pig breeding environment clean and tidy, while further improving the quality of the collected images.

[0059] In order to reduce the impact of changes in lighting in the experimental field on image acquisition, changes in light caused by lighting and day-night cycles in the acquisition field must be considered. In order to capture 24-hour images of the target group of pigs, this embodiment uses an infrared dot matrix camera, which can obtain clear images during the day and at night.

[0060] In order to ensure the accuracy of the experimental results, a stable, clear and reliable data set must be constructed. From July to October 2021, the experimental data was collected at the local pig farming station "Taihuyuan Breeding and Testing Station" in Hangzhou, Zhejiang. In order to ensure the reliability of the experimental data, 110 days of data were collected continuously. This experiment used HIKVISION's infrared dot matrix network camera, the product model is Smart265, and the network hard disk recorder model is its DS-7800 series NVR, which supports H.265 video compression technology, HDMI / VGA high-definition output, and the switch is its Gigabit PoE (SP series).

[0061] There are a total of 24 pig pens in the breeding ground. The area of ​​the pig pens is 5.30m×3.8m, and the height is 2.3m~3m. The fences in the breeding ground are made of galvanized steel pipes. The use of this material not only ensures the stability of the pig pens, but also ensures the air circulation of the pig pens. The pig pens uniformly use flat-top LED energy-saving lamps and natural light sources to complement each other. The pig pens use uniform exhaust fans for ventilation, and wet curtains are uniformly installed in the breeding grounds for cooling. The age of the Landrace pigs observed in the experiment is about 60d to 150d, and the number of live pigs in each pig pen is 8-15. In order to observe the data as accurately as possible, an infrared camera is installed on the left and right sides of the pig pen wall in each experiment. The specific situation is as follows Figure 1 As shown, Figure 1 The figure shows the internal details of the experimental site, A and B are the camera positions; E, F and G represent the drinking water, exhaust fan and feed trough respectively; H1 and H2 = the height of the pig house; L = the length of the pig house; W = the width of the pig house.

[0062] In this embodiment, a HIKVISION camera is used to collect 24 hours of pig video, and an APP is used to remotely access the storage device for observation and extract relevant images from the video.

[0063] In order to analyze the lying pattern of pigs, it is necessary to obtain the position information of the pigs in the pigsty. First, the image size is uniformly cropped to 524*524 pixels. In order to obtain the target information, the background must be removed. Traditionally, the background removal of pigs is done by using the shadow difference method, that is, the subtraction between the target image and the background frame is used to achieve target extraction. However, it is difficult to obtain a single background image in actual production.

[0064] The target object is extracted using the Grabcut image segmentation method, which is based on the iterative energy optimization algorithm. It converts the image segmentation problem into a minimum cut problem of the graph. By inputting the foreground and background, the target is extracted without the background. The extraction effect is as follows: Figure 2 In the next step, the target image is binarized by dynamic threshold segmentation, and the median fuzzy filter algorithm is used for multiple iterations to eliminate the interference of image noise. This study mainly focuses on the lying pattern of pigs. Therefore, based on the above binary images, moving pigs are first excluded. Since the movement of pigs is relatively slow, we observe the pigs collected on site and select the corresponding pixels of the target frame F(s) and F(s+5s) and multiply them ( Figure 3 , where Fr represents image frames; S represents time seconds), delete the images of moving pigs (such as Figure 2 ).

[0065] Watershed transform is a commonly used segmentation method. This embodiment uses this method to segment overlapping (touching) pigs. Traditional watershed segmentation methods often have over-segmentation. In order to solve this over-segmentation phenomenon, this embodiment first marks the segmentation area and then performs watershed segmentation. In this way, by specifying the segmentation "valley" in advance, the image segmentation can be more effective. Ellipse fitting is performed on the basis of segmentation, and the center point of each fitted ellipse is calculated. The image processing flow is as follows: Figure 2 's hint.

[0066] Based on the above extraction of the center point of the pig ellipse fitting, under the premise of classifying the pigs in the pigpen into one category, this embodiment uses unsupervised clustering technology to find the cluster center (the lying center of the pig herd). Compared with the supervised technology, the unsupervised and weakly supervised technology can complete the model training without label data or a small amount of label data. The next step is to calculate the distance from the center of each pig ellipse fitting to the cluster center, and based on the above calculation, the standard deviation and variance are normalized.

[0067] In order to compare the characteristics and applicable objects or scopes of different clustering methods, this embodiment lists the application of common clustering algorithms and whether cluster centers are applied, etc., as shown in Table 1 below.

[0068] Table 1: Comparison of unsupervised clustering algorithms.

[0069]

[0070]

[0071] From Table 1, it can be seen that K-Means, Mean-shift and AP (Affinity Propagation) are all applied to cluster centers for data clustering, but the AP algorithm has high complexity and time, and the time for one iteration is about O(N3). The AP algorithm needs to calculate the similarity between each pair of data objects in advance. This embodiment is to quickly and conveniently identify the lying behavior of a group of pigs, so this method is not suitable. Mean-shift is a density-based non-parametric clustering algorithm. The final cluster center of this algorithm will be at the maximum local density, and generally speaking, the lying center of a group of pigs should also be closer to the maximum density of the lying behavior of the group of pigs. The cluster center of K-Means is the average of each pixel point, so the lying center of the group of pigs extracted by Mean-shift is more realistic than the lying center extracted by K-Means.

[0072] In order to more accurately compare the differences between the two algorithms, a virtual data sample is constructed and the cluster centers are extracted using the two algorithms. The experimental results are as follows: Figure 4 and Figure 5The ellipses in the figure represent the pigs after ellipse fitting; the purple diamonds represent the cluster centers.

[0073] Based on the above analysis, this embodiment selects Mean-Shift as a method to find the center of lying behavior of a group of pigs. The experimental process is as follows: first, since the number of pigs in a pigpen is limited, a dynamic function that adaptively adjusts the bandwidth of the Gaussian kernel function is designed in advance according to the number of samples. The bandwidth value is the value selected when the experimental data can be classified into one category. Secondly, based on the group pig image after the above ellipse fitting, Mean-Shift is used to find the cluster center (such as Figure 6 In order to investigate and study the lying behavior of the entire pig pen, we first calculated the distance from each ellipse fitting center to the cluster center and calculated its standard deviation. In order to determine the lying clustering of pigs in the entire pig pen, the standard deviation of the image frame of the experiment was processed by Min-MaxNormalization to map the data value to [0, 1]. The calculation formula is as shown in formula (1). The example of extracting the lying pattern of the group of pigs is as follows: Figure 7 As shown, cluster centroids represent cluster centers.

[0074]

[0075] where x i Represents the coordinates of the center of the pig ellipse fitting in a certain frame of image, X i represents the distance from the pig ellipse fitting center to the cluster center, m represents the number of pigs in a frame image, x centroids-X Represents the coordinates of the cluster center in the frame image, S N-Fr(j) represents the standard deviation in the image frame, Represents the value after standardization of standard deviation (the degree of aggregation of the pig herd).

[0076] The existing research on the lying pattern of pigs is to process the target pig images using image processing technology, construct the lying pattern of pigs using Delaunay triangulation technology, and divide the lying pattern into 3 categories. Although the lying pattern of pigs can be recognized to a certain extent, through the observation of 322,100 pictures collected on site, it is found that if the lying pattern is only divided into 3 categories, it cannot well represent the lying situation of pigs, because the lying pattern of pigs is diverse and complex. In order to facilitate a more comprehensive study of the lying pattern of pigs, this paper divides the normalized experimental data into 5 equal parts, and different numerical ranges represent different pig group aggregation patterns, which are specifically defined as shown in Table 2 below, and the variance and standard of the normalized pig aggregation of each frame image are used as the training data of the experiment and input into the model classifier.

[0077] Table 2: Definition of different pig aggregation patterns.

[0078]

[0079] The sternal position is lying on the sternum with the front and hind legs folded under the body; the chest is completely covered; the abdominal position is lying on the abdomen with the front legs folded under the body and the hind legs visible (right side, left side); the supine position is lying on both sides with all four legs visible (right side, left side); the breasts are completely visible.

[0080] Example 2

[0081] To study the lying patterns of pigs, Figure 8 The algorithm flow for detecting the lying posture of a group of pigs based on video stream data is described. In addition, this embodiment also classifies the data processed by the above process (the variance and standard deviation of the aggregation degree of the group of pigs in the image frame) in combination with the CNN-SVM classification model.

[0082] Different from general CNNs, in order to identify the lying posture patterns of a group of pigs, the variance and standard deviation of the group of pigs obtained above are used as the input of the model. The data for training the model is text data instead of traditional image data. This method can reduce the time of training the model.

[0083] The role of the convolution layer in the CNN structure is to use a learnable kernel filter to detect local features at different positions in the input feature map. The use of local connections and weight sharing reduces the number of free parameters and avoids over-fitting of data. CNN is usually composed of convolutional layers and activation layers of nonlinear operations. The ReLU=max(0,x) function is used in all activation layers and works in all activation layers. Because CNN has this nonlinear structure, the network training is faster and it solves the vanishing gradient problem. Each convolution layer is followed by a pooling layer. This structure reduces the complexity of the calculation. The pooling layer resamples the feature data of the previous convolution layer by calculating the average response or selecting the maximum response in a small overlapping or non-overlapping area. CNN is generally composed of pairs of convolutional layers and pooling layers, which are finally connected to the fully connected layer to generate the required output. The network structure is as follows: Fig. 9 's hint.

[0084] CNN-SVM internal framework diagram: The input formula of the previous layer of the kernel filter is as follows:

[0085]

[0086] where x l-1 i and z l j Represent the input and output of the convolutional layer, k l-1 ijThe weight kernel is from the i-th neuron in layer l-1 to the j-th neuron in layer l. dconv stands for convolution operation, b l j M represents the bias of the jth neuron in layer l. l-1 Represents the number of l-1 kernel filters, and the ReLU activation function formula is as follows: is the output of layer l before downsampling, and the output of the current layer can be expressed as follows:

[0087]

[0088] Among them, DS represents the maximum pooling function, y l j is the output of layer l and the input of layer l+1. The output connection of the last pooling layer is the input of the full connection. Once the predicted output is obtained, the prediction error E is calculated using the softmax function with cross entropy in the cost function, as follows:

[0089]

[0090] Among them, Y ni ,y ni Represent the target label and the predicted output respectively, i is the number of categories in the nth training set, and the role of model learning is to optimize the network-free parameters to minimize the prediction error E. The weights and biases are updated with the learning rate (η). At the same time, in order to speed up the convergence of the model, we use stochastic gradient descent (SGD) as the optimization algorithm (Heatal., 2015), and apply weight initialization. The model learning rate (η) is 0.001. The weight and bias update formulas in the convolutional layer and the fully connected layer are as follows:

[0091]

[0092] Based on the feature vectors obtained by the above CNN structure, the next step is to use the SVM classifier to achieve high-precision group pig lying pattern recognition. SVM has been proven to be a powerful learner for regression, pattern classification and prediction. It does not require a large number of samples to train and has strong generalization. We connect the SVM classifier after the CNN structure. Such a heterogeneous model can obtain better classification results than a single model, which can effectively improve the prediction performance. Secondly, this classification model combines the advantages of two models: (1) Due to the pooling layer and convolution layer of CNN, the extracted feature vectors are not easy to fit; (2) The feature vectors extracted by CNN are more scientific than simple projections, directions and centers of gravity; (3) Different convolutions, pooling and the size of the final output feature vector can be used to control the fitting ability of the actual overall model; (4) The powerful classification and generalization capabilities of the SVM classifier are utilized. High-precision group pig lying pattern recognition is achieved with a small sample size.

[0093] Example 3

[0094] To verify the recognition effect of the model of the present invention, this embodiment uses the CNN-SVM classification model to identify the lying pattern of pigs in groups. There are two main classification steps, namely training and testing. First, training is to initialize the weight parameters in each network layer. This experiment selected 322,100 images collected on site, and a total of 315,560 sets of experimental data were accurately extracted. The ratio of training set to test set is 8:2.

[0095] First, use the training data set to train the model. During the training, not only forward training is performed on the model, but also reverse training is performed using the softmax regression layer, and the trained model is saved. Secondly, the trained model is used for testing.

[0096] In order to test the accuracy of the pig lying pattern classification model in this paper, 322,100 images collected on-site were randomly selected and data were extracted according to the method in Section 3.2. A total of 315,660 image data were accurately extracted, and about 6,440 images were extracted incorrectly, with an accuracy rate of 98%. The reason is that the installed camera field of view cannot completely cover every corner of the site, there are blind spots in the field of view, or there is excessive overlap between pigs. These reasons make it impossible to accurately extract the outline of the pigs.

[0097] To evaluate the pig lying posture model, this embodiment uses the standard precision (P), recall (R), F1 and accuracy (ACC). In multi-classification, the classification precision (P) of each category is the fraction of correctly defining the category and the vector of all frames. Similarly, the recall rate is defined as the fraction of events for each category, in which all frames are of that class, and the correct detection is declared as that class. The accuracy (ACC) represents the ratio of the number of postures accurately recognized to the total postures, but it is not suitable for the case of asymmetric data.

[0098] In order to make up for this defect, this embodiment uses two indicators: balanced accuracy and balanced accuracy weight. The balanced accuracy starts from the confusion matrix, and the value is not affected by the size of the category. Balanced accuracy weight means that the weight of each category is weighted by the recall rate of that category, and finally the value is accumulated. It is suitable for the situation where the number of categories is uneven. F1 is an indicator that takes into account both precision (P) and recall (R), and its scope of application is wider than ACC.

[0099] Usually for multi-classification, a confusion matrix is ​​used to represent the classification results. The diagonal represents the number of accurate tests. Generally speaking, we hope that the larger the number on the diagonal, the better. Fig.10 The darker the diagonal line, the more accurate the result. The data of the confusion matrix in this experiment is as follows Fig.10As shown in Table 3, the overall recognition accuracy of the lying mode identified by the model is 97.09%, the balance accuracy is 96.59%, and the weighted balance accuracy is 97.07%. The recognition accuracy of the two lying modes "Crp" and "Fap" is relatively high, which are 99.57% and 98.64% respectively. The recognition accuracy of the "Dip" lying mode is relatively low, which is only 93.05%.

[0100] Table 3: Confusion matrix of group pig lying patterns.

[0101]

[0102]

[0103] At the same time, ROC and AUC have also been used in image classification in machine learning. The ROC curve is a curve with the false positive rate as the horizontal axis and the true positive rate as the vertical axis. Generally speaking, the closer the ROC curve is to the upper left corner (0, 1), the better the classification performance of the classifier. However, the ROC curve often cannot indicate which classifier is more effective. AUC, as a numerical value, represents the area under the ROC curve. Its value range is (0.5~1). The larger the AUC value, the better the classifier effect. This value can more quantitatively indicate which classifier has better performance. Fig.11 (AUC value represents the area under the curve, and categories 0 to 4 represent "Crp", "Clp", "Nop", "Dip" and "Fap" respectively) It can be seen that the five classifiers all have good performance, among which the "Crp" and "Fap" classifiers have the best performance, with values ​​of 0.9966 and 0.9989 respectively, and the classifiers of the three lying modes "Clp", "Nop" and "Dip" are slightly worse, with values ​​of 0.9871, 0.9803 and 0.9923 respectively.

[0104] It should be appreciated that embodiments of the present invention may be implemented or enforced by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer-readable memory. The method may be implemented in a computer program using standard programming techniques - including a non-transitory computer-readable storage medium configured with a computer program, wherein the storage medium so configured causes the computer to operate in a specific and predefined manner - according to the methods and drawings described in the specific embodiments. Each program may be implemented in a high-level procedural or object-oriented programming language to communicate with a computer system. However, if desired, the program may be implemented in an assembly or machine language. In any case, the language may be a compiled or interpreted language. In addition, the program may be run on a programmed ASIC for this purpose.

[0105] Furthermore, the operations of the processes described herein may be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The processes described herein (or variations and / or combinations thereof) may be performed under the control of one or more computer systems configured with executable instructions, and may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) that is executed collectively on one or more processors, by hardware, or a combination thereof. The computer program includes a plurality of instructions that may be executed by one or more processors.

[0106] Further, the method can be implemented in any type of computing platform that is operably connected to a suitable computer, including but not limited to a personal computer, a minicomputer, a mainframe, a workstation, a network or distributed computing environment, a separate or integrated computer platform, or in communication with a charged particle tool or other imaging device, etc. Aspects of the present invention can be implemented in machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated into a computing platform, such as a hard disk, an optical read and / or write storage medium, a RAM, a ROM, etc., so that it can be read by a programmable computer, and when the storage medium or device is read by the computer, it can be used to configure and operate the computer to perform the process described herein. In addition, the machine-readable code, or part thereof, can be transmitted via a wired or wireless network. When such media includes instructions or programs that implement the steps described above in conjunction with a microprocessor or other data processor, the invention described herein includes these and other different types of non-transitory computer-readable storage media. When programmed according to the methods and techniques described in the present invention, the present invention also includes the computer itself. The computer program can be applied to input data to perform the functions described herein, thereby converting the input data to generate output data stored in a non-volatile memory. The output information can also be applied to one or more output devices such as a display. In a preferred embodiment of the present invention, the converted data represents a physical and tangible object, including a specific visual depiction of the physical and tangible object produced on a display.

[0107] As used in this application, the terms "component", "module", "system", etc. are intended to refer to a computer-related entity, which can be hardware, firmware, a combination of hardware and software, software, or software in operation. For example, a component can be, but is not limited to: a process running on a processor, a processor, an object, an executable file, a thread in execution, a program, and / or a computer. As an example, an application running on a computing device and the computing device can both be components. One or more components can exist in a process and / or thread in execution, and a component can be located in a computer and / or distributed between two or more computers. In addition, these components can be executed from various computer-readable media having various data structures thereon. These components can communicate in a local and / or remote process manner, such as according to a signal having one or more data packets (e.g., data from a component that interacts with another component in a local system, a distributed system, and / or interacts with other systems in a signal manner through a network such as the Internet).

[0108] It should also be understood that the present invention is described through implementation methods, and the embodiments are only clear and complete descriptions of the technical solutions proposed in the claims of the present invention, that is, explanations of the claims. Therefore, when judging whether the technical solutions recorded in the specification of the present invention are sufficiently disclosed, full consideration should be given to the core essence of the solutions defined by the claims. In the specification, there must be other technical problems that are not related to the core technical problems proposed in this embodiment. The corresponding technical features and technical solutions do not belong to the essence of this embodiment and are non-essential technical features. Therefore, they can be referred to as implicit disclosures. Those skilled in the art can fully implement them in combination with existing technologies and common knowledge. Therefore, there is no need to describe them in detail.

[0109] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for recognizing lying patterns of pigs based on unsupervised clustering and deep learning, characterized in that: The following steps are included: Collect the initial data set and uniformly crop the images; The target image is extracted using the Grabcut image segmentation method; The target image is binarized by dynamic threshold segmentation, and the median fuzzy filtering algorithm is iterated multiple times to eliminate the interference of image noise and exclude moving pigs; Mark the segmented area, perform watershed segmentation, and perform ellipse fitting based on the segmentation, and obtain the center point of each fitted ellipse; Based on the pig group images after ellipse fitting, the unsupervised clustering algorithm Mean-shift is used to find the cluster center, and the distance from each ellipse fitting center to the cluster center is calculated and its variance and standard deviation are output; The standard deviation is processed so that the value of its data is mapped to between [0,1]. The mapping process of the standard deviation is as follows: In the formula, x i Represents the coordinates of the center of the pig ellipse fitting in a certain frame of image, X i represents the distance from the pig ellipse fitting center to the cluster center, m represents the number of pigs in a frame image, x centroids-X Represents the coordinates of the cluster center in the frame image, S N-Fr(j) Represents the standard deviation in the image frame, S x(j)* Represents the value after standardization of the standard deviation, and the normalized data is defined as 5 equal parts, and the defined 5 equal parts are 0≤S x(j)* ≤0.2, 0.2 x(j)* ≤0.4, 0.4 x(j)* ≤0.6, 0.6 x(j)* ≤0.8, 0.8 x(j)* ≤1.0, different numerical ranges represent different pig group aggregation patterns, and the normalized variance and standard deviation of the pig group aggregation in each frame image are input into the CNN-SVM model classifier as training data. The input formula of the previous layer of the internal framework of the CNN-SVM model classifier is as follows:​​​​ In the formula, x l-1 i and z l j Represent the input and output of the convolutional layer respectively, k l-1 ij The weight kernel is from the i-th neuron in layer l-1 to the j-th neuron in layer l, dconv represents the convolution operation, b l j represents the bias of the jth neuron in layer l, M l-1 represents the number of l-1 layer kernel filters; The trained CNN-SVM model is used to identify the lying pattern of pigs.

2. The method for recognizing pig lying patterns based on unsupervised clustering and deep learning as claimed in claim 1, characterized in that: The ReLU activation function formula is as follows: x l j is the output of layer l before downsampling, and the output of the current layer is expressed as follows: In the formula, DS represents the maximum pooling function, y l j The output of layer l is also the input of layer l+1, and the output connection of the last pooling layer is the input of the full connection.

3. The method for recognizing pig lying patterns based on unsupervised clustering and deep learning as claimed in claim 1, characterized in that: After the CNN-SVM model obtains the prediction output, the prediction error E is calculated using the softmax function with cross entropy in the cost function. The formula is as follows: Where Y ni ,y ni Represent the target label and predicted output respectively, i is the number of categories in the nth training set, and the role of model learning is to optimize the network-free parameters to minimize the prediction error E, where the weights and biases are updated with the learning rate (η).

4. The method for recognizing pig lying patterns based on unsupervised clustering and deep learning as claimed in claim 3, characterized in that: Stochastic gradient descent (SGD) is used as the optimization algorithm, and the weight initialization is applied. The model learning rate (η) is 0.

001. The weight and bias update formulas in the convolutional layer and the fully connected layer are as follows:

5. The method for recognizing pig lying patterns based on unsupervised clustering and deep learning as claimed in claim 1, characterized in that: Multi-classification evaluation indicators are used to evaluate the lying pattern postures of the pigs identified by the CNN-SVM model.

6. The method for recognizing pig lying patterns based on unsupervised clustering and deep learning as claimed in claim 1, characterized in that: The multi-classification evaluation indicators are standard precision (P), recall (R), F1 and accuracy (ACC).

Images