Breeding animal individual identity recognition method, device and system fusing local topology invariance and metric learning

Abstract

The invention discloses a local topology invariance and metric learning fused bred animal individual identity recognition method, device and system, and the method comprises the steps: (1) obtaining an animal image, and constructing a sample data set; (2) constructing a feature extraction deep network model, wherein the feature extraction deep network model comprises a basic feature extraction path, a local mode topology invariance extraction path and a feature fusion module; (3) training the model by adopting a sample data set; (4) extracting enhanced embedding vectors of a plurality of animal images with known identities by using the trained model, and storing the enhanced embedding vectors in a feature database; and (5) inputting a to-be-recognized animal image into the trained model, extracting the enhanced embedding vectors of the to-be-recognized animal image, calculating the similarity between the enhanced embedding vectors and the stored enhanced embedding vectors, and selecting the identity corresponding to the enhanced embedding vector with the highest similarity as the identity of the to-be-recognized animal individual. The method is higher in accuracy and robustness.

Description

TECHNICAL FIELD

[0001] The present application relates to computer vision technology, and in particular to a method, device and system for individual identity recognition of farmed animals by fusing local topological invariance and metric learning. BACKGROUND

[0002] The individual recognition technology of farmed animals is of great significance for the daily management of farms, such as the health monitoring of dairy cows, the milk yield recording, and the disease prevention and control. The traditional contact methods, such as ear tags and radio frequency identification technology, may be damaged or lost due to the activities of dairy cows, equipment failures and other factors, and have high installation and maintenance costs.

[0003] In recent years, the individual identity recognition method of dairy cows based on computer vision has been widely studied. Compared with the traditional contact method, the computer vision technology has the characteristics of non-contact, remote recognition, high efficiency and automation, which can significantly reduce the cost and improve the intelligent level of the farm.

[0004] Most of the existing methods use traditional artificial feature extraction algorithms or simple convolutional neural networks for feature extraction. These methods have limited ability to extract and express key features in dairy cow images, such as patterns, contours, textures, etc. Especially in complex environments, such as multi-view, light changes or background interference, it is difficult to ensure the accuracy and robustness of recognition. Therefore, it is necessary to improve the existing individual identity recognition method of farmed animals (such as dairy cows) based on computer vision. SUMMARY

[0005] In view of the problems existing in the prior art, the purpose of the present application is to provide an individual identity recognition method, device and system for farmed animals by fusing local topological invariance and metric learning, which have high accuracy and robustness.

[0006] In order to achieve the above-mentioned purpose of the application, the present application provides the following technical solutions:

[0007] An individual identity recognition method for farmed animals by fusing local topological invariance and metric learning, comprising the following steps:

[0008] (1) Obtain animal images of animals under different viewing angles and postures, and perform preprocessing to construct a sample data set;

[0009] (2) Construct a feature extraction deep network model, the feature extraction deep network model comprising:

[0010] a basic feature extraction path for extracting a convolution feature vector of an animal image by using a ResNet 50 network fusing an attention mechanism;

[0011] a local mode topology invariance extraction path for identifying key local mode points of the convolution feature vector, constructing a local graph between the key local mode points, and processing the local graph by using a graph neural network to extract a topology feature vector;

[0012] a feature fusion module for fusing the convolution feature vector and the topology feature vector to generate an enhanced embedding vector;

[0013] (3) training the feature extraction deep network model by using a sample data set;

[0014] (4) extracting enhanced embedding vectors of animal images of known identities by using the trained feature extraction deep network model, and storing the enhanced embedding vectors and the corresponding identities in a feature database;

[0015] (5) inputting an animal image to be identified into the trained feature extraction deep network model to extract an enhanced embedding vector thereof, calculating the similarity between the enhanced embedding vector and the enhanced embedding vectors stored in the feature database, and selecting the identity corresponding to the enhanced embedding vector with the highest similarity in the feature database as the identity of the animal individual to be identified.

[0016] Further, the ResNet 50 network with the fusion attention mechanism specifically adds a channel-spatial attention module at the end of each residual module of the ResNet 50 network, and the residual module is used to perform the following calculation:

[0017]

[0018] wherein X" represents the output of the residual module, X represents the input of the residual module, Conv 1×1 represents a 1×1 convolution, BN represents batch normalization, Conv 3×3 represents a 3×3 convolution, PReLU represents a Parametric ReLU activation function, CSA represents a channel-spatial attention module, + represents a residual connection, represents a function composition operation.

[0019] Further, the channel-spatial attention module includes a channel attention module and a spatial attention module, and the channel attention module is used to perform the following calculation:

[0020]

[0021] wherein X CA represents the output of the channel attention module, F in represents the input of the channel attention module, GAP represents a global average pooling operation, Conv 1×k represents a 1×k convolution operation, and k represents an adaptive selection of a convolution kernel step. C stands for F in The dimension of ,γ,b are adjustment parameters,| | odd Indicates finding the nearest odd number, Sigmoid indicates the Sigmoid activation function, ⊙ indicates element-wise multiplication, Represents function composition operation;

[0022] The spatial attention module is connected in series with the output of the channel attention module to perform the following calculations:

[0023]

[0024] Among them, X CSA Represents the output of the spatial attention module, Conv 3×3 Represents a 3×3 convolution operation, Concat represents feature concatenation, AvgPool and MaxPool represent average pooling and maximum pooling operations respectively.

[0025] Furthermore, the local pattern topological invariance extraction path includes:

[0026] A key feature point detection unit is used to find several points with the highest non-maximum suppression after the convolution feature vector is aggregated as key local pattern points;

[0027] A local graph construction unit is used to construct a local graph using key local pattern points as vertices and Euclidean distances between vertices as edges;

[0028] The graph neural network unit is used to take all key local pattern points and local graphs as input and obtain the node feature matrix through the K-layer graph attention network GAT learning;

[0029] The graph readout unit is used to aggregate the node feature matrix output by the graph neural network unit into a graph-level topological feature vector.

[0030] Furthermore, the key feature point detection unit is specifically configured to perform the following operations:

[0031]

[0032] Where H (0) =(h1,h2,...,h N ) represents the key local pattern point vector, h1,h2,...,h N represents the 1st, 2nd, ..., Nth key local pattern point, N is the number of key local pattern points, X cnn Represents the convolution feature vector, Agg represents channel aggregation, NMS represents non-maximum suppression, and TopN represents finding the N points with the highest values ​​as key local pattern points.

[0033] Further, the calculation formula of the feature fusion module is:

[0034] X out = W proj * Concat(X cnn , X topo ) + b proj

[0035] wherein X out represents an enhanced embedding vector, Concat represents concatenation, W proj represents a projection weight matrix, b proj represents a bias matrix, all of which are obtained through training, X cnn represents a convolution feature vector, and X topo represents a topological feature vector.

[0036] Further, when the feature extraction deep network model is trained in step (3), an improved metric learning loss function is used, and the specific calculation formula is:

[0037] L Total = L PC + ξ·L SR + β·L TC

[0038] wherein L Total is a total loss, L PC is a class proxy contrast loss, and the calculation formula thereof is:

[0039]

[0040] wherein N batch represents a batch training sample size, N c represents a class number, λ represents a projection hypersphere radius, δ represents a distance between a similarity of an enhanced embedding vector and a positive class proxy and a similarity of the enhanced embedding vector and a negative class proxy, X out represents an enhanced embedding vector, y i and y j represent the i-th and j-th classes respectively, P(y i ) and P(y j ) represent class proxies of the y i and y j intra-class multi-center representations respectively, and M represents a number of y i intra-class multi-centers, τ represents a temperature scaling factor, W FC (y i , m) and W FC (y i , n) represent the m-th and n-th intra-class centers in the y i class respectively.

[0041] L SR is the intra-class multi-center sparsity regularization term of the class, and the calculation formula is:

[0042]

[0043] wherein, represents the y i Intra-class multi-center sparse regularization of the class, the calculation formula is W FC (y i ,1), W FC (y i ,s), and W FC (y i ,t) represent the 1st, s-th, and t-th intra-class center in the y i class, respectively, and ξ represents the sparse regularization coefficient;

[0044] L TC is the intra-class consistency loss term of the topological feature, and the calculation formula is:

[0045]

[0046] wherein, is the index set of the training samples belonging to the y i class in the training sample batch, X topo (j) is the topological feature vector of the training sample j, μ topo (y i ) is the mean of the topological feature vector of the y i class in the training sample batch, and β is the weight coefficient of the topological consistency loss term, represents the square of the quadratic norm.

[0047] Further, the calculation formula for enhancing the similarity between the embedding vector and the enhanced embedding vector stored in the feature database in step (5) is:

[0048]

[0049] wherein, X′ out is the enhanced embedding vector of the animal image to be identified, represents the enhanced embedding vector of the k-th training sample of the y i class in the feature database, which has the same size as X′ out , and || || represents the norm.

[0050] The application discloses a farmed animal individual identity recognition device fusing local topological invariance and metric learning, and comprises a memory, a processor and a computer program stored in the memory and executable on the processor.

[0051] The application discloses a farmed animal individual identity recognition system fusing local topological invariance and metric learning, and comprises:

[0052] A data acquisition module is configured to acquire animal images of farmed animals.

[0053] An edge reasoning module is configured to execute the method.

[0054] A cloud storage module is configured to store and analyze the recognition result of the edge reasoning module.

[0055] A model optimization module is configured to optimize the model in the edge reasoning module according to the data of the cloud storage module.

[0056] A front-end display module is configured to display the recognition result of the edge reasoning module and the analysis result of the cloud storage module.

[0057] Compared with the prior art, the application has the following beneficial effects:

[0058] 1. The application not only adopts an improved convolutional neural network fusing attention mechanism (CSA) to extract fine visual appearance features of individual dairy cows, but also innovatively introduces a local pattern topology invariance (LPTI) extraction path in parallel, which is specially used for capturing structured topological features that are robust to non-rigid deformation caused by changes in posture and viewing angle. By effectively fusing these two features with complementary advantages, a more comprehensive enhanced individual representation is generated, which significantly improves the feature extraction robustness and accuracy of the model under severe changes in posture and viewing angle.

[0059] 2. The application adopts an improved metric learning loss function to train the model, and a multi-center class proxy contrast loss combined with a sparsification regular constraint acts on the final fused features, effectively enhancing the intra-class compactness and inter-class separability of the overall features; at the same time, a specially introduced intra-class consistency loss term directly optimizes the topological features output by the LPTI module, forcing them to remain stable and consistent among samples of the same class. The combination of the above loss functions makes the enhanced embedding vector learned finally more discriminative, further improving the accuracy and generalization ability of dairy cow individual identity recognition. BRIEF DESCRIPTION OF DRAWINGS

[0060] Figure 1is a flowchart of a breeding animal individual identity recognition method provided by the present application, which fuses local topological invariance and metric learning;

[0061] Figure 2 is a structural diagram of a feature extraction deep network model provided by the present application;

[0062] Figure 3 is a structural diagram of a basic feature extraction path provided by the present application;

[0063] Figure 4 is a structural diagram of a residual module in the basic feature extraction path provided by the present application;

[0064] Figure 5 is a structural diagram of a channel-attention module provided by the present application;

[0065] Figure 6 is a structural diagram of an LPTI extraction path provided by the present application;

[0066] Figure 7 is a structural diagram of a feature fusion module provided by the present application;

[0067] Figure 8 is a structural diagram of one embodiment of a breeding animal individual identity recognition system provided by the present application, which fuses local topological invariance and metric learning;

[0068] Figure 9 is a structural diagram of another embodiment of a breeding animal individual identity recognition system provided by the present application, which fuses local topological invariance and metric learning. DETAILED DESCRIPTION

[0069] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application.

[0070] Embodiment one

[0071] The present application provides a breeding animal individual identity recognition method which fuses local topological invariance and metric learning, as shown in the following steps: Figure 1

[0072] S1, acquire animal images of animals under different visual angles and postures, and perform preprocessing to construct a sample data set.

[0073] Data acquisition can be divided into the following four operations: image acquisition, image enhancement, normalization processing and data set making.

[0074] The four operations will be described in detail as follows:

[0075] ​Image acquisition: Use professional image acquisition equipment to take pictures of dairy cows from multiple different angles and various poses in the breeding environment of breeding animals (dairy cows), ensuring that the images collected can cover various common appearance situations of dairy cows in the actual breeding environment. During the shooting process, attention should be paid to the light conditions of the shooting environment, and factors such as shadows and direct sunlight should be avoided as much as possible, while maintaining the stability of the shooting equipment to prevent image blurring.

[0076] Image enhancement: A series of enhancement operations are performed on the collected images, such as randomly rotating the images within a range of ±30 degrees, randomly scaling the images within a range of 0.8 to 1.2 times, and horizontally flipping the images with a probability of 50%, etc. to increase the diversity of the data and the generalization ability of the model.

[0077] Normalization: First, divide each pixel value of the image by the maximum pixel value of the image, and normalize it to the range of 0-1, and then perform mean and standard deviation normalization on the image to ensure that the image data has a consistent distribution in the feature space, thereby improving the training efficiency and stability of the model.

[0078] Dataset creation: Divide the enhanced and normalized images into training set, validation set and test set according to a certain proportion, ensure that each subset contains images from different dairy cow individuals and various angles and poses, and label each image with its corresponding dairy cow individual identifier.

[0079] S2, constructing a feature extraction deep network model. As shown in Figure 2 , the feature extraction deep network model includes a basic feature extraction path, a local pattern topology invariance (LPTI) extraction path, and a feature fusion module.

[0080] The specific structure, parameters and mathematical expressions of the above modules are described in detail below.

[0081] (1) The basic feature extraction path is used to extract the convolutional feature vector of the animal image using a ResNet 50 network with a fusion attention mechanism. The ResNet 50 network with a fusion attention mechanism is an improved ResNet 50 network, as shown in Figure 3 , which includes an input convolutional layer, 16 bottleneck blocks (residual modules), and a global average pooling layer.

[0082] The input convolutional layer includes the following 3 modules: 7x7 convolution (Conv 7×7 ), batch normalization (BN), and activation function (ReLU). The data processed by the input convolutional layer can be represented as follows using the composite form of the function:

[0083]

[0084] In the above expression, I represents taking an animal image, which is a three-dimensional matrix with a size of DxDx3, corresponding to the data of the input image, and the image side length is D; X is a three-dimensional matrix with a size of D / 2xD / 2xC1, which is the output feature map of the input convolution layer, and C1 is the output channel number of the convolution layer, represents a function consistent operation, which is to execute the last one first, and then execute from back to front, for example, the above formula is to execute Conv 7×7 (I) first, then perform batch normalization on Conv 7×7 (I), and then perform the ReLU activation function on the batch normalized result.

[0085] The data processing process of the functions corresponding to the three modules in the above formula will be explained in detail below.

[0086] First, introduce the kxk convolution module function, and 7x7 convolution is a special case of k=7. The convolution operation formula for a convolution kernel space with a size of kxk is:

[0087]

[0088] Where X in is a three-dimensional input matrix of the convolution module function, with a size of HxWxC, H, W, and C represent the length, width, and channel number of the input image respectively, X in (h+m-1,w+n-1,c) is the value of X in at channel c and position (h+m-1,w+n-1); X' is a three-dimensional output matrix of the convolution module function, with a size of H'xW'xC'; the convolution kernel W k×k is a four-dimensional matrix with a size of kxkxCxC'; the bias b k×k is a vector with a size of C'. In the 7x7 convolution module of the application, the input is I; the output is X (1) with a size of D / 2xD / 2xC1; the parameters to be trained include the convolution kernel W 7×7 and the bias b 7×7 with sizes of 7x7x3xC1 and C1 respectively.

[0089] Secondly, the batch normalization operation is applied after each convolution layer to accelerate the training convergence speed of the model. For an input three-dimensional matrix X' with a size of H'xW'xC', the mean and variance of each channel are calculated respectively, and then the feature values of each channel are normalized to make the mean close to 0 and the variance close to 1. The specific calculation process is as follows:

[0090]

[0091]

[0092]

[0093]

[0094] where X(h,w,c) is the value of input feature map X' at channel c, position (h,w), μ(c) and σ 2 (c) are the mean and variance of X' at channel c respectively, ε is a very small constant to prevent the denominator from being zero, γ and β are the scaling and offset parameters to be trained, X BN is the output value after batch normalization, which has the same size as X. In the batch normalization module of the present application, the input is X (1) ; the output is The trained parameters include the scaling parameter γ (1) and the offset parameter β (1) , both of which have the size of C1

[0095] Finally, there is a ReLU activation function module, and the calculation formula of the ReLU activation function module is as follows:

[0096]

[0097] where X BN (h,w) represents the value of the feature map X BN after batch normalization at position (h,w), X ReLU is the output value after ReLU activation function processing, which has the same size as X BN . In the ReLU activation function module of the present application, the input is ; the output is There is no trained parameter.

[0098] The bottleneck block of the improved ResNet 50 network will be described in detail below, and the bottleneck block structure is shown in Figure 4 , which includes the following modules: 1x1 convolution (Conv 1×1 ), batch normalization (BN), activation function (PReLU), 3x3 convolution (Conv 3×3 ), channel-spatial attention (CSA) module. In the above modules, the three modules of "Conv 1×1 , BN, PReLU" appear twice, and the structures of the two times are the same, but the parameters are independent. The processing of the bottleneck block can be represented in the form of function composition as follows:

[0099]

[0100] In the above expression, + represents residual connection, X corresponds to the input feature map of the bottleneck block, X" is a three-dimensional matrix with a size of D / 2xD / 2xC2, is the output feature map of the bottleneck block, and C2 is the number of output channels of the second 1x1 convolution module in the bottleneck block.

[0101] 1x1 convolution and 3x3 convolution are kxk convolution special cases with k=1 and k=3, and batch normalization is described in detail above, which will not be repeated here. The data processing process of other modules involved in the above formula will be explained in detail below.

[0102] First, the PReLU activation function module, the calculation formula of the PReLU activation function module is:

[0103]

[0104] Compared with the ReLU activation function, the PReLU activation function introduces a learnable parameter a in the negative value domain, which alleviates the gradient disappearance problem of ReLU in the negative input part.

[0105] The CSA module is as shown in Figure 5 The CA module includes global average pooling (GAP), 1xk convolution (Conv 1×k ), and activation function (Sigmoid). The data processed by the CA module can be represented as follows using the composite form of the function:

[0106]

[0107] In the above expression, X CA represents the output of the channel attention module, which is a three-dimensional matrix with a size of D / 2xD / 2xC3, represents element-level multiplication, and F in represents the input of the channel attention module, which is a three-dimensional matrix with a size of D / 2xD / 2xC3, and corresponds to the output of the second PReLU activation function in the bottleneck block C3 is the number of output channels of the 3x3 convolution module in the bottleneck block.

[0108] The SA module includes average pooling (AvgPool), maximum pooling (MaxPool), feature concatenation (Concat), 3x3 convolution (Conv 3×3 ), and activation function (Sigmoid). The data processed by the SA module can be represented as follows using the composite form of the function:

[0109]

[0110] In the above expression, X CSA It is a three-dimensional matrix of size D / 2×D / 2×C3, which is the output of the CSA module and SA module.

[0111] The following is a detailed explanation of the data processing flow of the module corresponding function in the above formula.

[0112] The global average pooling module GAP in the CA module averages the pixel values ​​of the entire space of the feature map. The calculation process is as follows:

[0113]

[0114] in, yes The value of channel c, position (h, w), X GAP It is a three-dimensional matrix of size 1×1×C3, corresponding to the output after global average pooling.

[0115] The maximum pooling module in the SA module takes the maximum value on the channel of each feature point in the feature map. The calculation process is as follows:

[0116]

[0117] Among them, X CA (c,h,w) is X CA The value of channel c, position (h, w), X Maxpool It is a three-dimensional matrix of size D / 2×D / 2×, corresponding to the output of the maximum pooling module.

[0118] The average pooling module in the SA module takes the average value on the channel of each feature point in the feature map. The calculation process is as follows:

[0119]

[0120] Among them, X Avgpool is a three-dimensional matrix of size D / 2×D / 2×1, corresponding to the output of the average pooling module. The feature concatenation module in the SA module concatenates the two input feature maps in the channel dimension, as shown below:

[0121]

[0122] Among them, X Concat It is a three-dimensional matrix of size D / 2×D / 2×2, corresponding to the output of the feature concatenation module, X Concat (h,w,c) is X Concat The value at position (h,w) in channel c.

[0123] The global average pooling layer mainly contains one module, namely global average pooling. The data processing calculation formula is as follows:

[0124]

[0125] Where X′′′′ represents the input of the global average pooling layer, which corresponds to the output feature map after the 16th bottleneck block, with a size of H′′′×W′′×C4, where C4 is the number of output channels of the 16th bottleneck block; X qavg Represents the output of the global average pooling layer, that is, the convolutional feature vector, of size 1×1×C4; there are no parameters to be trained.

[0126] (2) The LPTI extraction path is used to identify the key local pattern points of the convolution feature vector, construct a local graph between the key local pattern points, and use a graph neural network to process the local graph to extract the topological feature vector.

[0127] like Figure 6 As shown, the LPTI extraction path includes:

[0128] The key feature point detection unit is used to find several points with the highest non-maximum suppression after the convolution feature vector is aggregated as key local pattern points; specifically, it is used to perform the following operations:

[0129]

[0130] Where H (0) =(h1,h2,...,h N ) represents the key local pattern point vector, h1,h2,...,h N represents the 1st, 2nd, ..., Nth key local pattern point, N is the number of key local pattern points, X cnn Represents the convolution feature vector, Agg represents channel aggregation, NMS represents non-maximum suppression, and TopN represents finding the N points with the highest values ​​as key local pattern points;

[0131] A local graph construction unit is used to use N key local pattern points as a vertex set V, calculate the Euclidean distances between all key local pattern points, use the Euclidean distances between vertices as an edge set E, determine adjacency information, and construct a local graph G = (V, E);

[0132] The graph neural network unit is used to take all key local pattern points and local graphs as input, and learn the node feature matrix through the K-layer graph attention network GAT; among them, for the k-th (k ranges from 0 to K-1) layer GAT, the attention value e is first calculated ij , the calculation formula is as follows:

[0133]

[0134] wherein, represents the input feature matrix H of the k-th layer GAT (k) , and W (k) represents the learnable linear transformation matrix of the layer, || represents vector splicing, a (k) represents the learnable attention parameter vector of the layer, and LeakyReLU represents an activation function.

[0135] Then, the normalized attention coefficient is calculated by a Softmax activation function The calculation formula is as follows:

[0136]

[0137] Finally, the neighbor node features are aggregated by using the attention coefficient for weighting, to obtain the node features of the next layer The calculation formula is as follows:

[0138]

[0139] wherein, σ represents an activation function. Until the node feature matrix H is calculated, the updated node feature matrix H

[0140] The graph reading unit is configured to aggregate the node feature matrix output by the graph neural network unit into a topology feature vector X topo .

[0141] (3) The feature fusion module is configured to fuse the convolution feature vector and the topology feature vector, to generate an enhanced embedding vector. As shown in Figure 7 , the feature fusion module includes feature splicing and feature projection, and the calculation formula is as follows:

[0142] X out =W proj *Concat(X cnn ,X topo )+b proj

[0143] wherein, X out represents the enhanced embedding vector, Concat represents splicing, W proj represents a projection weight matrix, b proj represents a bias matrix, and all are obtained by training, X cnn represents the convolution feature vector, and X topo represents the topology feature vector.

[0144] X outis the output of the model, but during training, the loss function calculation needs to be performed, so during training, the multi-center embedding layer also needs to be performed after the feature projection. The multi-center embedding layer mainly includes three modules: full connection (FC), activation function (Softmax), and weighted summation (WS). The processing of data through the multi-center embedding layer can be expressed in the form of a function as follows:

[0145]

[0146] In the above expression, P is a four-dimensional matrix with a size of 1×1×C4×N c , corresponding to the output class agent of the multi-center embedding layer, N c is the number of categories to be identified.

[0147] The full connection module in the multi-center embedding layer is to calculate the similarity between the input features and the intra-class multi-center, and the data processing calculation formula is as follows:

[0148] X FC (y i ,m)=X out ·W FC (y i ,m)

[0149] where W FC is a four-dimensional matrix with a size of 1×1×C4×M·N c , M is the number of intra-class centers of each class, W FC (y i ,m) represents the mth intra-class center in the y i class, with a size of 1×1×C4; X FC is a three-dimensional matrix with a size of 1×1×M·N c , and X FC (y i ,m) represents the similarity output of X out and the mth intra-class center in the y i class.

[0150] The Softmax activation function normalizes the output of the full connection layer and converts it into a probability distribution form, and the data processing calculation formula is as follows:

[0151]

[0152] where M p is a matrix with a size of 1×1×M·N c , M p (y i ,m) represents the mth intra-class center in the y i class in the y iwhere w represents the weight of the class, and represents the temperature scaling factor.

[0153] The weighted summation module generates a class agent for each class by weighting the probabilities output by the Softmax, and the data processing calculation formula is as follows:

[0154]

[0155] S3, training the feature extraction deep network model using a sample data set.

[0156] During training, the initialization network loss is calculated based on the improved metric learning loss function, the network parameters are optimized and adjusted through the back propagation algorithm, and continuous iteration is performed.

[0157] Before network training, the network needs to be initialized, and according to the above description of the feature extraction deep network training model and parameters, the network layers and parameters that need to be initialized are:

[0158] Input convolutional layer: 7x7 convolution kernel W 7×7 and bias b 7×7 ; scaling parameter g (1) and offset parameter b (1) of batch normalization;

[0159] Bottleneck block: 1x1 convolution kernel W 1×1 , W' 1×1 and bias b 1×1 , b' 1×1 ; scaling parameter g (2) , g (3) and offset parameter b (2) , b (3) of batch normalization; learnable parameters a, a' of PReLU activation function; 3x3 convolution kernel W 3×3 and bias b 3×3 ; 1xk convolution kernel W 1×k and bias b 1×k , convolution kernel W' 3×3 and bias b' 3×3 ;

[0160] K-layer graph attention network: linear transformation matrix W (k) and attention parameter vector a (k) ;

[0161] Feature fusion module: projection weight matrix W proj and bias matrix b proj ; weight matrix W FC of multi-center embedding layer.

[0162] The initialization method is as follows:

[0163] Different parameter initialization methods are used for different layer modules:

[0164] Convolution module: for the weights of each convolution module, if the activation function connected after is a non-symmetrical activation function such as ReLU, He initialization method is used, including the convolution kernel W of the input convolution layer 7×7 , the convolution kernel W of the 16 bottleneck blocks 1×1 , W′ 1×1 , W 3×3 , and the linear transformation matrix W of the K-layer graph attention network (k) ; if the activation function connected after is a symmetrical activation function such as Sigmoid, Xavier initialization method is used, including W of CSA in the 16 bottleneck blocks 1×k and W′ 3×3 .

[0165] The calculation process of He initialization of convolution weights is as follows: assuming that the input channel number of the convolution module is C in , the output channel number is C out , and the convolution kernel size is k x k. The initialization of the convolution weight W k×k obeys the normal distribution with mean 0 and standard deviation , that is, W k×k is randomly sampled and initialized in the normal distribution with mean 0 and standard deviation .

[0166] The calculation process of Xavier initialization of convolution weights is as follows: the initialization of the convolution weight W k×k obeys the normal distribution with mean 0 and standard deviation 2 / (C in +C out ), that is, W k×k is randomly sampled and initialized in the normal distribution with mean 0 and standard deviation 2 / (C in +C out ).

[0167] For the bias of each convolution module, it is usually uniformly initialized to 0, including the bias b of the input convolution layer 7×7 , the bias b of the 16 bottleneck blocks 1×1 , b′ 1×1 , b 3×3 , b 1×k , b′ 3×3 , the bias b of the feature fusion module proj , and the attention parameter vector a of the K-layer graph attention network (k) .

[0168] Fully connected module: for the weight W of the fully connected module FCand W proj , Xavier initialization method is adopted, W FC and W proj are initialized as 0. Random sampling initialization, wherein N in is the number of input neurons, and N out is the number of output neurons.

[0169] Batch normalization module: the scaling factor is usually initialized to 1, including the gamma (1) of the input convolutional layer and the gamma (2) of the 16 bottleneck blocks, gamma (3) ; the offset factor is initialized to 0, including the beta (1) of the input convolutional layer and the beta (2) of the 16 bottleneck blocks, beta (3) .

[0170] PReLU activation function module: for the slope parameters a, a' in the PReLU activation function module, it is usually initialized to 0.25.

[0171] After initialization, network training can be divided into the following four operations: feature acquisition, loss calculation, parameter optimization, and iterative training. The following will introduce the four operations in detail:

[0172] Feature acquisition: the improved ResNet 50 network after parameter initialization is used as the basic feature extraction path, and the K-layer graph attention network is used as part of the LPTI extraction path. The pretreated cow image I is input into the basic feature extraction path, the LPTI extraction path and the feature fusion module in turn, and finally a feature vector X out with a size of 1x1xC4 is obtained.

[0173] Loss calculation: the obtained feature vector X out is input into the multi-center embedding layer after parameter initialization to obtain the class agent P(y i ) of each class, and then the class agent contrast loss is calculated, and the calculation formula is:

[0174]

[0175] The class agent contrast loss improves the SoftMax loss by introducing multiple centers for each class, which helps to reduce the intra-class variance, and does not need to perform triple sampling like traditional triple loss.

[0176] In order to adaptively adjust the number of class centers, the intra-class multi-center is sparsely regularized, and the calculation formula is:

[0177]

[0178] wherein, denotes y i Intra-class multi-center sparsity regularization of the class, the calculation formula is W FC (y i ,1), W FC (y i ,s), W FC (y i ,t) respectively denote the 1st, s, t intra-class center in the y i class, and ξ denotes the sparsity regularization coefficient;

[0179] An intra-class consistency loss term L TC is added to the topological feature class, and the calculation formula is:

[0180]

[0181] wherein, is the index set of training samples belonging to the y i class in the training sample batch, X topo (j) is the topological feature vector of the training sample j, μ topo (y i ) is the mean of the topological feature vector of the y i class in the training sample batch, and β is the weight coefficient of the topological consistency loss term, denotes the square of the quadratic norm.

[0182] The improved metric learning loss function, the specific calculation formula is:

[0183] L Total =L PC +ξ·L SR +β·L TC

[0184] Parameter optimization: parameter optimization adjusts the network parameters according to the loss value by the back propagation algorithm. The back propagation algorithm is based on the chain rule, and calculates the gradient of the loss function with respect to each parameter in the network. The core idea is to start from the loss function, and gradually propagate the gradient backward, calculate the contribution of each parameter to the loss function, so as to determine how to adjust the parameters to reduce the loss. Parameter optimization can be divided into the following two operations: calculating the gradient and updating the parameters. The following will introduce these two operations in detail:

[0185] Calculate the gradient: first calculate the gradient of the loss function with respect to the output feature vector. For the joint loss function composed of the class proxy contrast loss, its related sparsity regularization constraint and the intra-class consistency loss of the topological feature class, according to its expression and derivation rule, the gradient of the output feature X outthe gradient of and about intra-class multi-center W FC the gradient of

[0186] According to the chain rule, the gradient is propagated back to each module of the network step by step, and finally the gradient of the loss function with respect to all network parameters θ in the network is calculated

[0187] Parameter update: use the optimizer to update the network parameters θ according to the calculated gradient, and use the Adam optimizer in this step, and the parameter update process is as follows:

[0188] Initialize the first moment estimation vector m0 and the second moment estimation vector v0 related to the parameters to all zero vectors, which have the same dimension as the dimension of the parameters θ to be optimized. In each iteration:

[0189] Calculate the first moment estimation, and the calculation formula is as follows:

[0190] m t = β1m t-1 + (1-β1)g t

[0191] Where g t represents the gradient of the loss function with respect to the parameter θ, t represents the current training iteration step number, β1 is a hyperparameter, usually taking a value in the interval (0, 1), and the common value is 0.9.

[0192] Calculate the second moment estimation, and the calculation formula is as follows:

[0193]

[0194] Where β2 is another hyperparameter, usually taking a value in the interval (0, 1), and the common value is 0.999.

[0195] Calculate the bias correction of the first moment estimation, and the calculation formula is as follows:

[0196]

[0197] As the iteration step t increases, will gradually tend to 0, and when the training is carried out to a certain extent, the influence of this bias correction will gradually become smaller, so that can more accurately reflect the actual average gradient.

[0198] Calculate the bias correction of the second moment estimation, and the calculation formula is as follows:

[0199]

[0200] Similarly, with the increase of the iteration step t, this bias correction can make the actual gradient variance more accurately reflect the situation.

[0201] Finally, according to the first moment estimate and the second moment estimate after the bias correction, the parameters are updated, and the calculation formula is as follows:

[0202]

[0203] Where η is the learning rate, which is a hyperparameter that controls the size of each parameter update step, and ε is a very small constant, which is added to prevent numerical instability in the case of a zero denominator.

[0204] Through the above calculation formula, according to the current parameter value θ t , the learning rate η and the average gradient and the gradient variance after the correction are updated to adjust the parameters towards the direction of the loss function.

[0205] Iterative training: repeat the above feature acquisition, loss calculation and parameter optimization process, and continue to iterate until the loss converges to a set threshold. In each iteration, a batch of data is randomly selected from the training data set for training, so that the model can learn more general feature representation and avoid overfitting. With the increase of the iteration step, the parameters of the network will be gradually adjusted according to the selected optimizer, so that the value of the loss function will continue to decrease. When the value of the loss function changes less than the set threshold in continuous multiple iterations, it is considered that the model converges, and the training is completed.

[0206] S4, using the trained feature extraction deep network model, extracting the enhanced embedding vectors of a plurality of known identity animal images, and storing the enhanced embedding vectors and the corresponding identities in the feature database.

[0207] In the database, a record is created for each individual cow, and its corresponding feature vector and related identification information (such as cow individual number, collection time, etc.) are stored in the corresponding field, so as to facilitate subsequent quick query and retrieval.

[0208] S5, inputting the animal image to be identified into the trained feature extraction deep network model to extract its enhanced embedding vector, calculating the similarity between the enhanced embedding vector and the enhanced embedding vector stored in the feature database, and selecting the identity corresponding to the enhanced embedding vector with the highest similarity in the feature database as the identity of the animal individual to be identified.

[0209] The calculation formula of the similarity is:

[0210]

[0211] In the formula, X'out is an enhanced embedding vector of the animal image to be identified, represents the y i enhanced embedding vector of the k-th training sample, with the same size as X' out , || || represents the norm. The value of the similarity ranges from -1 to 1, and the closer the value is to 1, the more similar the two vectors are.

[0212] Embodiment Two

[0213] The embodiment of the present application provides a farmed animal individual identity recognition device fusing local topological invariance and metric learning. The embodiment of the present application provides services for the implementation of the method of the above-mentioned embodiment one of the present application. The device can include a memory storing a computer executable program; a processor coupled with the memory; the processor invokes the computer executable program stored in the memory, and is used for executing the steps in the method described in embodiment one.

[0214] Embodiment Three

[0215] The embodiment of the present application provides a farmed animal individual identity recognition system fusing local topological invariance and metric learning. The system is implemented on a separate end device and belongs to an end scene. As shown in the figure, Figure 8 the system includes a data acquisition module, a model inference module and a front-end display module. The data acquisition module, the model inference module and the front-end display module are integrated in one device, for example, an intelligent camera with certain computing power. These end devices have image acquisition functions and can acquire image data of farmed animals such as cows in real time, and locally run the method of the above-mentioned embodiment one. The end device inputs the acquired image data into the locally trained model for real-time inference, and quickly obtains the recognition result of the individual of the farmed animal such as the cow.

[0216] The data acquisition module: In each key area of the pasture, such as the feeding area, the resting area, the activity area, etc., carefully deploy intelligent cameras with high resolution, low illumination and automatic focusing and automatic aperture adjustment functions. These cameras use advanced optical sensors that can clearly capture images of cows under different lighting conditions, ensuring that the clarity and detail richness of the images meet the needs of subsequent model inference. The installation angle of the camera is accurately calculated and field debugged to ensure that it can fully cover the positions and postures that the cow may appear, reduce the dead angle of shooting, and avoid image distortion or obstruction problems caused by improper installation. The image acquisition strategy is to set reasonable acquisition time interval and trigger condition according to the activity law of the cow and the daily operation process of the pasture, and the acquired image data is transmitted to the model inference module of the end device.

[0217] Model inference module: The cow feature extraction model is pre-stored on the end device and is trained to mature the parameters. During the model loading process, the hardware acceleration resources of the end device (such as a dedicated AI accelerator chip) are used for acceleration. Through fast decompression of model parameters and memory optimization allocation, the model can be loaded and put into inference state in a short time. The received image data is input into the loaded model to start the real-time inference process. The model quickly extracts and analyzes the cow features in the image, accurately identifies the individual identity of the cow, and outputs the corresponding identification result. The inference process fully utilizes the hardware acceleration capability of the end device to ensure that the inference task is completed in a short time, meeting the real-time requirement on site.

[0218] Front-end display module: The front-end display module is connected to the model inference module and can obtain the latest identification results and related data in real time and dynamically update and display on the interface. The interface is adapted to the screen size and resolution of the end device. When the model completes the inference on an image, the identification result is immediately displayed on the corresponding interface. At the same time, according to the individual information and historical data of the cow, the development of functions such as health condition monitoring and behavior anomaly monitoring can be increased.

[0219] Embodiment four

[0220] The embodiment of the present application provides a kind of breeding animal individual identity identification system fusing local topological invariance and metric learning, which is realized on edge-cloud scene, as shown in Figure 9 It includes data acquisition module, edge inference module, front-end display module, cloud storage module and model optimization module.

[0221] The following will be described in detail 5 modules.

[0222] Data acquisition module: Specifically, it is a camera or other acquisition device. The acquisition device deployed in edge-cloud scene and end scene is the same, but in edge-cloud scene, the image data collected is transmitted to edge inference module in real time through high-speed wireless network (such as 5G or high-performance Wi-Fi).

[0223] Edge inference module: Deploy edge inference modules on edge computing nodes at the edge of the pasture, such as small servers or gateway devices near the breeding area, to implement the individual identification method described in Embodiment I. Edge computing nodes have more powerful computing power and storage resources than end devices, but have lower network latency advantages compared to cloud servers. The edge inference module is responsible for receiving animal image data of dairy cows and other breeding animals collected from multiple end devices and centrally processing and analyzing these data. On the edge node, the preprocessed image data is input into the model for fast inference calculation, and the preliminary individual identification result of the dairy cow and other breeding animals is output and pushed to the front-end display module. The edge inference module connected to the cloud storage module can further upload the collected data and inference results to the cloud storage module.

[0224] Cloud storage module: The cloud server builds a large-scale distributed storage system based on advanced distributed file systems and distributed databases to provide reliable and efficient storage services for data uploaded from various edge nodes of the pasture. After receiving the data uploaded by the edge computing device, the cloud storage module classifies and stores the dairy cow image data, identification results, feature vectors, and related breeding information according to multiple dimensions such as pasture, time, and individual dairy cow, and establishes a perfect indexing mechanism for fast query and retrieval. In addition, the cloud storage module is also responsible for regularly backing up and archiving the stored data, backing up important data to off-site data centers or storage media to ensure data security and reliability.

[0225] Model optimization module: Compared to the model in the end scenario, which is fixed to hardware resources (such as chips), the model in the edge-cloud scenario can be dynamically optimized and updated. The cloud server obtains a large amount of dairy cow image data and related information from the cloud storage module, and further preprocesses and labels these data to meet the requirements of model training. Then, according to the previous training network steps, the parameters are further optimized and adjusted. The trained model needs to be comprehensively evaluated. According to the evaluation results, if the performance of the model meets or exceeds the expected target, the model is marked as a usable version and released and pushed through the version management system, sending model update notifications to each edge node to guide the edge node to update and deploy the model, ensuring that the edge inference module uses the most optimized model version. If the model performance does not meet the requirements, the reasons are analyzed in depth, and improvements and optimizations are made accordingly, and then the training and evaluation are performed again until a model version that meets the performance requirements is obtained.

[0226] The front-end display module: the front-end display module of the edge-cloud scene is more functional than that of the end scene. In addition to being able to obtain the latest recognition result in real time, it can also interact with the cloud storage module and the model optimization module, receive data analysis reports, model update notifications and other information from the cloud, and present them in a visual manner, providing strong support for making scientific and reasonable breeding decisions, thereby realizing a complete closed loop from data collection to decision execution.

[0227] It should be understood that the above embodiments and descriptions described in the specification are only the principles, main features and advantages of the present application. Without departing from the spirit and scope of the present application, various changes and improvements can be made to the present application, and these changes and improvements all fall within the scope of the present application.

Claims

1. A method for identifying individual farmed animals by integrating local topological invariance and metric learning, characterized in that: The following steps are involved: (1) Obtain animal images at different viewing angles and postures, perform preprocessing, and construct a sample dataset; (2) Constructing a feature extraction deep network model, wherein the feature extraction deep network model includes: The basic feature extraction path is used to extract convolutional feature vectors of animal images using a ResNet 50 network with a fused attention mechanism; A local pattern topology invariance extraction path is used to identify key local pattern points of the convolution feature vector, construct a local graph between the key local pattern points, and process the local graph using a graph neural network to extract a topological feature vector; A feature fusion module, configured to fuse the convolution feature vector with the topological feature vector to generate an enhanced embedding vector; (3) using a sample data set to train the feature extraction deep network model; (4) using the trained feature extraction deep network model to extract enhanced embedding vectors of several animal images with known identities, and storing the enhanced embedding vectors and corresponding identities in a feature database; (5) The image of the animal to be identified is input into the trained feature extraction deep network model to extract its enhanced embedding vector, and the similarity between the enhanced embedding vector and the enhanced embedding vector stored in the feature database is calculated. The identity corresponding to the enhanced embedding vector with the highest similarity in the feature database is selected as the individual identity of the animal to be identified.

2. The method for identifying individual farmed animals by integrating local topological invariance and metric learning according to claim 1, characterized in that: The ResNet 50 network with the integrated attention mechanism specifically adds a channel-spatial attention module at the end of each residual module of the ResNet 50 network. The residual module is used to perform the following calculations: Among them, X″ represents the output of the residual module, X represents the input of the residual module, Conv 1×1 represents 1×1 convolution, BN represents batch normalization, Conv 3×3 represents 3×3 convolution, PReLU represents Parametric ReLU activation function, CSA represents channel-spatial attention module, + represents residual connection, Represents function composition operation.

3. The method for identifying individual farmed animals by integrating local topological invariance and metric learning according to claim 2, characterized in that: The channel-spatial attention module includes a channel attention module and a spatial attention module. The channel attention module is used to perform the following calculations: Among them, X CA represents the output of the channel attention module, F in represents the input of the channel attention module, GAP represents the global average pooling operation, Conv 1×k Represents a 1×k convolution operation, k represents the adaptive selection of the convolution kernel step size, C stands for F in The dimension of ,γ,b are adjustment parameters, || odd Indicates finding the nearest odd number, Sigmoid indicates the Sigmoid activation function, ⊙ indicates element-wise multiplication, Represents function composition operation; The spatial attention module is connected in series with the output of the channel attention module to perform the following calculations: Among them, X CSA Represents the output of the spatial attention module, Conv 3×3 Represents a 3×3 convolution operation, Concat represents feature concatenation, AvgPool and MaxPool represent average pooling and maximum pooling operations respectively.

4. The method for identifying individual farmed animals by integrating local topological invariance and metric learning according to claim 1, wherein: The local pattern topological invariance extraction path includes: A key feature point detection unit is used to find several points with the highest non-maximum suppression after the convolution feature vector is aggregated as key local pattern points; A local graph construction unit is used to construct a local graph using key local pattern points as vertices and Euclidean distances between vertices as edges; The graph neural network unit is used to take all key local pattern points and local graphs as input and obtain the node feature matrix through the K-layer graph attention network GAT learning; The graph readout unit is used to aggregate the node feature matrix output by the graph neural network unit into a graph-level topological feature vector.

5. The method for identifying individual farmed animals by integrating local topological invariance and metric learning according to claim 4, characterized in that: The key feature point detection unit is specifically configured to perform the following operations: Where H (0) =(h1,h2,...,h N ) represents the key local pattern point vector, h1,h2,...,h N represents the 1st, 2nd, ..., Nth key local pattern point, N is the number of key local pattern points, X cnn Represents the convolution feature vector, Agg represents channel aggregation, NMS represents non-maximum suppression, TopN represents finding the N points with the highest values ​​as key local pattern points, and represents function compound operation.

6. The method for identifying individual farmed animals by integrating local topological invariance and metric learning according to claim 1, characterized in that: The calculation formula performed by the feature fusion module is: X out =W proj *Concat(X cnn ,X topo )+b proj Among them, X out represents the enhanced embedding vector, Concat represents concatenation, and W proj represents the projection weight matrix, b proj Represents the bias matrix, which is obtained through training, X cnn represents the convolution feature vector, X topo represents the topological eigenvector.

7. The method for identifying individual farmed animals by integrating local topological invariance and metric learning according to claim 1, characterized in that: When step (3) trains the feature extraction deep network model, an improved metric learning loss function is used, and the specific calculation formula is: L Total =L PC +ξ·L SR +β·L TC Where: L Total is the total loss, L PC is the class proxy contrast loss, which is calculated as: Where N batch Indicates the batch training sample size, N c represents the number of categories, λ represents the radius of the projected hypersphere, δ represents the distance between the similarity between the enhanced embedding vector and the positive agent and the similarity between the enhanced embedding vector and the negative agent, X out represents the enhanced embedding vector, y i 、y j Represents the i-th and j-th categories respectively, P(y i ), P(y j ) represent y i 、y j Class agents with multiple center representations within the class, and M stands for y i The number of multi-centers in the class, τ represents the temperature scaling factor, W FC (y i ,m),W FC (y i ,n) represent y i The mth and nth intra-class centers in a class; L SR is the total sparsification regularization constraint term for multiple centers within the class, and its calculation formula is: In the formula, R{W FC (y i ,1),...,W FC (y i ,m)} represents y i The intra-class multi-center sparse regularization of the class is calculated as follows: W FC (y i ,1),W FC (y i ,s),W FC (y i ,t) represent y i The 1st, sth, and tth intra-class centers in the class, ξ represents the sparse regularization coefficient; L TC is the intra-class consistency loss term of topological features, and its calculation formula is: in, is the number of training samples belonging to y i The training sample index set of the class, X topo (j) is the topological feature vector of training sample j, μ topo (y i ) is y i The mean of the topological feature vector of the class in the training sample batch, β is the weight coefficient of the topological consistency loss term, Represents the square of the quadratic norm.

8. The method for identifying individual farmed animals by integrating local topological invariance and metric learning according to claim 1, wherein: The calculation formula for the similarity between the enhanced embedding vector in step (5) and the enhanced embedding vector stored in the feature database is: Where X′ out is the enhanced embedding vector of the animal image to be identified, Represents y in the feature database i The enhanced embedding vector of the kth training sample of the class is of the same size as X′ out Same, || || represents the norm.

9. A device for identifying individual farmed animals that integrates local topological invariance and metric learning, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: The processor executes the computer program to implement the method according to any one of claims 1 to 8.

10. A farmed animal individual identification system integrating local topological invariance and metric learning, characterized in that: include: A data acquisition module for collecting animal images of farmed animals; An edge reasoning module, configured to execute the method according to any one of claims 1 to 8; A cloud storage module, used to store and analyze the recognition results of the edge reasoning module; The model optimization module is used to optimize the model in the edge inference module based on the data of the cloud storage module; The front-end display module is used to display the recognition results of the edge reasoning module and the analysis results of the cloud storage module.

Images