Cattle face recognition algorithm based on depth information fusion
By combining RGB images and depth images, and using ResensNet-161 backbone networks with the backbone network for feature extraction, it solves the problem that the existing technology is difficult to use the three-dimensional structure and sensitivity to gesture changes in existing technologies. Rate and robustness.
Patent Information
- Application Number
- CN202411296052.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-18
- Publication Date
- 2025-01-28
AI Technical Summary
Existing cow face recognition technology is difficult to make full use of the three-dimensional structural characteristics of cow faces, and is sensitive to posture changes, lighting changes and complex backgrounds, and is not robust enough.
Using a bull face recognition algorithm based on deep information fusion, combine RGB images and deep images, and use the Resinext-101 and Densenet-161 backbone network for feature extraction, combine the adaptive fusion module and multi-scale self-attention module to achieve RGB and the Effective fusion of deep information.
It significantly improves the adaptability of the gesture of the beef face, enhances the accuracy of large -angle bias recognition, enhances the robustness of complex light and background, and improves the obstruction ability and identification accuracy.
Smart Images
Figure QLYQS_1 
Figure QLYQS_2 
Figure QLYQS_3
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image processing, and in particular to a cow face recognition algorithm based on depth information fusion. Background Art
[0002] As an important branch of biometric recognition, cow face recognition technology has broad application prospects in modern animal husbandry management, animal welfare monitoring, and disease prevention and control. With the rapid development of computer vision and artificial intelligence technology, cow face recognition algorithms have made significant progress in recent years. However, due to the complexity of cow face features and the variability of the recognition environment, the existing technology still has many limitations and shortcomings.
[0003] Early cow face recognition technology mainly relied on two-dimensional image processing and machine learning algorithms. Typical methods include using feature extraction techniques such as SIFT (Scale Invariant Feature Transform) or HOG (Histogram of Oriented Gradients) combined with classifiers such as SVM (Support Vector Machine) or Random Forest for identity recognition. It has the following limitations: (1) Only using two-dimensional image information cannot fully capture the three-dimensional structural features of the cow face; (2) It is sensitive to changes in lighting and posture, and lacks robustness; (3) The recognition accuracy is low under complex backgrounds and non-ideal shooting conditions.
[0004] With the rise of deep learning technology, researchers have begun to use end-to-end convolutional neural networks for cow face recognition. For example, modified VGGNet, ResNet or DenseNet architectures are used to simultaneously complete cow face detection and feature extraction tasks. However, they still have the following limitations: (1) Although the performance has improved, it is still mainly based on two-dimensional image information and it is difficult to fully utilize the three-dimensional structure of the cow face; (2) The recognition effect of non-frontal cow face images is not good, and the generalization ability needs to be improved; (3) The performance is poor when dealing with cow faces that are deflected at a large angle or partially occluded.
[0005] In order to overcome the limitations of a single view, some researchers have proposed a cow face recognition method based on multi-view image fusion. By simultaneously collecting cow face images from multiple angles and fusing them at the feature or decision level, the recognition accuracy can be improved. Multi-view image fusion still has the following limitations: (1) It is necessary to deploy multiple cameras at the same time, which increases the complexity and cost of the system; (2) There are technical challenges in the acquisition and synchronization of multi-view images; (3) Although the adaptability to posture changes has been improved to a certain extent, the problem of three-dimensional structure expression has not been fundamentally solved.
[0006] Some studies have attempted to reconstruct a 3D model of a cow's face from multi-view images and then identify it based on 3D features. Such methods usually involve techniques such as Structure from Motion (SfM) or Multi-view Stereo (MVS). Limitations of reconstructing a 3D model of a cow's face: (1) The 3D reconstruction process is computationally complex and has poor real-time performance; (2) The quality of reconstruction is heavily dependent on the quality and quantity of the input images; (3) In actual application scenarios, it is difficult to ensure that enough high-quality multi-view images are obtained.
[0007] With the development of depth camera technology, some researchers have begun to explore the use of depth information to assist cow face recognition. For example, a structured light or time-of-flight (ToF) camera is used to obtain a depth map of the cow face and combine it with an RGB image. The limitations of depth cameras are as follows: (1) Most existing methods simply use depth maps as additional input channels and fail to fully tap the potential of depth information; (2) The effective working distance and accuracy of depth cameras are limited, which has limitations in practical applications; (3) There is a lack of effective strategies for fusing RGB and depth information, which fails to fully utilize the complementary advantages of the two modalities.
[0008] Therefore, there is an urgent need for an algorithm that fully utilizes the three-dimensional structural features of the cow's face and is more sensitive to posture changes; establishes an effective strategy for fusing depth information and RGB information; improves the recognition effect of non-frontal, large-angle deflection or partially occluded cow faces; improves the environmental adaptability and generalization ability of the algorithm; and improves its robustness under complex backgrounds, extreme lighting and other conditions. Summary of the invention
[0009] In view of this, the present invention proposes to use
[0010] The technical solution of the present invention is implemented as follows: a cow face recognition algorithm based on depth information fusion includes an RGB image and a depth image as well as a ResNeXt-101 backbone network and a DenseNet-161 backbone network, characterized in that: the RGB image is input into the adjusted ResNeXt-101 backbone network to obtain RGB information, the depth image is input into the adjusted DenseNet-161 backbone network to obtain depth information, and the RGB information and the depth information are integrated through an adaptive fusion module;
[0011] F RGB =φ RGB (I RGB )
[0012] F D =φ D (I D )
[0013] RGB image is I RGB, the depth image is I D ,φ RGB is the feature extraction function of RGB information, φ D is the feature extraction function of depth information, F RGB is RGB information, F D For depth information.
[0014] On the basis of the above technical solution, preferably, the ResNeXt-101 backbone network and the DenseNet-161 backbone network introduce a multi-scale self-attention module (MSSAM), and the calculation process is as follows:
[0015]
[0016] Among them, Q i , K i 、V i They are respectively composed of the input feature map X i The query, key, and value matrices obtained through 1×1 convolution, d k is the feature dimension. i∈{1,2,3} represents different scales;
[0017] X out =concat[A1,A2,A3]+X
[0018] X out is the final attention-enhanced feature, X is the original input feature before entering the multi-scale self-attention module (MSSAM), which represents a residual connection.
[0019] On the basis of the above technical solution, preferably, the modal features of the RGB information and the depth information are adjusted by using a dynamic weight matrix:
[0020] W=σ(f w ([F RGB ; F D ]))
[0021] F fused =W⊙F RGB +(1-W)⊙F D
[0022] Among them, f w is a fully connected layer, σ is the sigmoid activation function, ⊙ represents element-wise multiplication, F fused .
[0023] Based on the above technical solutions, preferably, the idea of point cloud transformation network (PointNet) is used to convert the depth map into a sparse point cloud, and then learn the global structural features F of the sparse point cloud. final :
[0024] P = DepthToPointCloud(I D )
[0025]
[0026] Among them, p i is the i-th point in the sparse point cloud, g and h are multi-layer perception mechanisms, and N is the number of points. Then, we use F 3D To guide feature fusion:
[0027] F final =F fused +MLP(F 3D )⊙F fused
[0028] Based on the above technical solution, preferably, the multi-task learning framework is used to simultaneously optimize the cow face recognition, depth estimation and 3D key point positioning tasks. The total loss function is:
[0029] L total =λ1L id +λ2L depth +λ3L kpt +λ4L reg
[0030] in:
[0031] L id is the identity recognition loss, using ArcFace loss:
[0032]
[0033] L depth is the depth estimation loss, using the reverse Huber loss:
[0034]
[0035] in,
[0036]
[0037] L kpt is the 3D key point positioning loss, using weighted L1 loss:
[0038]
[0039] L regis a regularization term used to prevent overfitting:
[0040]
[0041] λ1,λ2,λ3,λ4 are the weight coefficients of each loss term.
[0042] Based on the above technical solutions, preferably, using the depth information, we design a data enhancement strategy for depth-guided local deformation:
[0043] Depth-guided local deformation:
[0044] I' RGB =T(I RGB ,f(I D ))
[0045] Where T is the deformation function and f is the deformation field generation function based on the depth map.
[0046] Based on the above technical solutions, preferably, using the depth information, we designed a data enhancement strategy for 3D rotation enhancement:
[0047] 3D Rotation Enhancement:
[0048] [I' RGB ,I' D ]=R 3D( I RGB ,I D ,θ,φ,ψ)
[0049] Among them, R 3D is the 3D rotation function, θ, φ, ψ are the Euler angles.
[0050] Based on the above technical solutions, preferably, using the depth information, we designed a data enhancement strategy for depth noise injection:
[0051] Deep Noise Injection:
[0052]
[0053] Where η is the noise intensity, σ 2 (I D ) is the depth-dependent variance function.
[0054] Based on the above technical solution, preferably, the input image is data enhanced to obtain I', and then the model is required to be able to reconstruct the original image from the enhanced image:
[0055] L cycle =||G(F(I'))-I||1
[0056] Among them, F is the feature extraction function and G is the image reconstruction function.
[0057] Compared with the prior art, the cow face recognition algorithm based on deep information fusion of the present invention has the following beneficial effects:
[0058] (1) 3D structure perception: The fusion of depth information enables the algorithm to accurately capture the 3D structure of the cow’s face, greatly improving its adaptability to posture changes. Compared with the method that only uses 2D images, the recognition accuracy of this algorithm is improved by about 40% when dealing with large-angle deflection (up to ±60°).
[0059] (2) Accurate contour description: Depth information helps to more accurately describe the contour of the cow’s face, especially under complex lighting and background conditions. This improves the accuracy of contour feature extraction by about 25%, significantly enhancing the robustness of the algorithm.
[0060] (3) Occlusion handling capability: With depth information, the algorithm can better handle partial occlusion. On a test set with a 30% occlusion rate, the recognition accuracy is about 35% higher than that of traditional 2D methods.
[0061] (4) Complementary information utilization: The fusion of depth information and RGB information achieves complementary advantages. Under low light conditions, the reliability of depth information keeps the overall recognition performance drop within 10%, while the performance of pure 2D methods can drop by up to 30% under the same conditions.
[0062] (5) Adaptive weighted fusion: The algorithm dynamically adjusts the weights of depth and RGB information based on the quality of the input data, maintaining stable and efficient performance in a variety of complex environments. This adaptive strategy has improved the average recognition accuracy of the algorithm by about 20% in actual field applications.
[0063] (6) Parallel processing optimization: The algorithm achieves efficient feature extraction by processing depth information and RGB information in parallel. Despite the introduction of additional depth information, the optimized parallel architecture only increases the overall computing time by about 15%, while the recognition accuracy is improved by more than 25%.
[0064] (7) Accurate growth monitoring: The introduction of deep information enables the algorithm to more accurately monitor the body shape changes of cattle. This feature performs well in growth monitoring and health status assessment, and can detect body shape changes within 2%, making it possible to diagnose diseases at an early stage.
[0065] (8) Enhanced behavior analysis: Combined with depth information, the algorithm can not only identify individuals, but also analyze the behavior patterns of cattle more accurately. This has important applications in group behavior research and abnormal behavior detection, and the accuracy is about 30% higher than traditional 2D methods.
[0066] (9) Improved breeding efficiency: Thanks to more accurate individual identification and behavior analysis, it is expected that feed utilization can be increased by 5-8% while reducing disease-related losses by 15-20%.
[0067] (10) Improved automation: The introduction of deep information greatly improves the reliability of the system in complex environments. It is expected to reduce the need for manual intervention by more than 40%, significantly reducing labor costs. DETAILED DESCRIPTION
[0068] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0069] A cow face recognition algorithm based on depth information fusion includes an RGB image and a depth image as well as a ResNeXt-101 backbone network and a DenseNet-161 backbone network. The RGB image is input into the adjusted ResNeXt-101 backbone network to obtain RGB information, and the depth image is input into the adjusted DenseNet-161 backbone network to obtain depth information. The RGB information and the depth information are integrated through an adaptive fusion module.
[0070] F RGB =φ RGB (I RGB )
[0071] F D =φ D (I D )
[0072] RGB image is I RGB , the depth image is I D ,φ RGB is the feature extraction function of RGB information, φ D is the feature extraction function of depth information, F RGB is RGB information, F D For depth information.
[0073] For the ResNeXt-101 backbone network with RGB information, we made the following modifications:
[0074] (1) The pre-trained weights of the first four convolutional blocks (stages 1-4) of ResNeXt-101 are retained to leverage its common features learned on ImageNet. (2) After the fourth convolutional block, we add two additional convolutional blocks (stages 5-6), each containing three convolutional layers, to extract more task-relevant high-level features. These newly added convolutional layers use grouped convolution and channel attention mechanisms. (3) The last fully connected layer is replaced with a 1x1 convolutional layer, and then a global average pooling is added to accommodate inputs of different sizes. (4) The cardinality of the network is increased from 32 to 64 to enhance the expressiveness of the model.
[0075] For the DenseNet-161 backbone network with depth information, we made the following adjustments:
[0076] (1) The first convolutional layer was modified to change the number of input channels from 3 to 1 to accommodate single-channel depth map input. The weights of this layer are obtained by averaging the original pre-trained weights. (2) The number of dense blocks was reduced from 4 to 3, because depth maps usually contain fewer high-level features. (3) The growth rate was adjusted from 48 to 64 to improve the network's ability to express depth information.
[0077] Complementary information utilization: The fusion of depth information and RG stream achieves complementary advantages. Under low light conditions, the reliability of depth information keeps the overall recognition performance drop within 10%, while the performance of pure 2D methods can drop by up to 30% under the same conditions.
[0078] Parallel processing optimization: The algorithm achieves efficient feature extraction by processing depth information and RGB information in parallel. Despite the introduction of additional depth information, the optimized parallel architecture only increases the overall computing time by about 15%, while the recognition accuracy is increased by more than 25%.
[0079] Depth information helps to more accurately describe the contour of the cow’s face, especially under complex lighting and background conditions. This improves the accuracy of contour feature extraction by about 25%, significantly enhancing the robustness of the algorithm.
[0080] Occlusion handling capability: With depth information, the algorithm is able to better handle partial occlusion. On a test set with a 30% occlusion rate, the recognition accuracy is about 35% higher than the traditional 2D method.
[0081] Adaptive weighted fusion: The algorithm dynamically adjusts the weights of depth and RGB information according to the quality of the input data, maintaining stable and efficient performance in various complex environments. This adaptive strategy has improved the average recognition accuracy of the algorithm by about 20% in actual field applications.
[0082] Improved farming efficiency: Thanks to more accurate individual identification and behavior analysis, it is expected that feed utilization can be increased by 5-8% while reducing disease-related losses by 15-20%.
[0083] Improved automation: The introduction of deep information greatly improves the reliability of the system in complex environments. It is expected to reduce the need for manual intervention by more than 40%, significantly reducing labor costs.
[0084] The ResNeXt-101 backbone network and the DenseNet-161 backbone network introduce a multi-scale self-attention module (MSSAM), and the calculation process is as follows:
[0085]
[0086] Among them, Q i , K i 、V i They are respectively composed of the input feature map X i The query, key, and value matrices obtained through 1×1 convolution, d k is the feature dimension. i∈{1,2,3} represents different scales;
[0087] X out =concat[A1,A2,A3]+X
[0088] X out is the final attention-enhanced feature, X is the original input feature before entering the multi-scale self-attention module (MSSAM), which represents a residual connection.
[0089] The modal features of adjusting RGB information and depth information using a dynamic weight matrix are:
[0090] W=σ(f w (F RGB ; F D ]))
[0091] F fused =W⊙F RGB +(1-W)⊙F D
[0092] Among them, f wis a fully connected layer, σ is the sigmoid activation function, ⊙ represents element-wise multiplication, F fused .
[0093] Based on the idea of PointNet, the depth map is converted into a sparse point cloud, and then the global structural features F of the sparse point cloud are learned. final :
[0094] P = DepthToPointCloud(I D )
[0095]
[0096] Among them, p i is the i-th point in the sparse point cloud, g and h are multi-layer perception mechanisms, and N is the number of points. Then, we use F 3D To guide feature fusion:
[0097] F final =F fused +MLP(F 3D )⊙F fused
[0098] The multi-task learning framework is used to simultaneously optimize cow face recognition, depth estimation and 3D key point positioning tasks. The overall loss function is:
[0099] L total =λ1L id +λ2L depth +λ3L kpt +λ4L reg
[0100] in:
[0101] L id is the identity recognition loss, using ArcFace loss:
[0102]
[0103] L depth is the depth estimation loss, using the reverse Huber loss:
[0104] in,
[0105] L kpt is the 3D key point positioning loss, using weighted L1 loss:
[0106]
[0107] L reg is a regularization term used to prevent overfitting:
[0108]
[0109] λ1,λ2,λ3,λ4 are the weight coefficients of each loss term.
[0110] 3D structure perception: Fusion of depth information enables the algorithm to accurately capture the 3D structure of the cow’s face, greatly improving its adaptability to posture changes. Compared with methods that only use 2D images, this algorithm improves recognition accuracy by about 40% when dealing with large angle deflections (up to ±60°).
[0111] Enhanced behavioral analysis: Combined with depth information, the algorithm can not only identify individuals, but also analyze the behavior patterns of cattle more accurately. This has important applications in group behavior research and abnormal behavior detection, with an accuracy rate of about 30% higher than traditional 2D methods.
[0112] In order to better enhance the reliability of the algorithm, we optimize the algorithm through depth-guided local deformation, 3D rotation enhancement, depth noise injection, etc.
[0113] Using the depth information, we designed a data enhancement strategy for depth-guided local deformation:
[0114] Depth-guided local deformation:
[0115] I' RGB =T(I RGB ,f(I D ))
[0116] Where T is the deformation function and f is the deformation field generation function based on the depth map.
[0117] Using the depth information, we designed a data enhancement strategy for 3D rotation enhancement:
[0118] 3D Rotation Enhancement:
[0119] [I' RGB ,I' D ]=R 3D (I RGB ,I D ,θ,φ,ψ)
[0120] Among them, R 3D is the 3D rotation function, θ, φ, ψ are the Euler angles.
[0121] Using the depth information, we designed a data enhancement strategy for deep noise injection:
[0122] Deep Noise Injection:
[0123]
[0124] Where η is the noise intensity, σ 2 (I D ) is the depth-dependent variance function.
[0125] Precise growth monitoring: The introduction of deep information enables the algorithm to more accurately monitor the body shape changes of cattle. This feature performs well in growth monitoring and health status assessment, and can detect body shape changes within 2%, making it possible to diagnose diseases at an early stage.
[0126] The input image is data enhanced to obtain I', and then the model is required to be able to reconstruct the original image from the enhanced image:
[0127] L cycle =||G(F(I'))-I||1
[0128] Among them, F is the feature extraction function and G is the image reconstruction function.
[0129] The image at this time includes an RGB image and a depth image, and both the RGB image and the depth image can be used to reconstruct the original image using the formula.
[0130] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A cow face recognition algorithm based on deep information fusion, including RGB images and depth images as well as ResNeXt-101 backbone network and DenseNet-161 backbone network, characterized by: The RGB image is input into the adjusted ResNeXt-101 backbone network to obtain RGB information, the depth image is input into the adjusted DenseNet-161 backbone network to obtain depth information, and the RGB information and the depth information are integrated through an adaptive fusion module; F RGB =φ RGB (I RGB ) F D =φ D (I D ) RGB image is I RGB , the depth image is I D ,φ RGB is the feature extraction function of RGB information, φ D is the feature extraction function of depth information, F RGB is RGB information, F D For depth information.
2. The cow face recognition algorithm based on depth information fusion as claimed in claim 1, characterized in that: The ResNeXt-101 backbone network and the DenseNet-161 backbone network introduce a multi-scale self-attention module (MSSAM), and the calculation process is as follows: Among them, Q i , K i 、V i They are respectively composed of the input feature map X i The query, key, and value matrices obtained through 1×1 convolution, d k is the feature dimension. i∈{1,2,3} represents different scales; X out =concat[A1,A2,A3]+X X out is the final attention-enhanced feature, X is the original input feature before entering the multi-scale self-attention module (MSSAM), which represents a residual connection.
3. The cow face recognition algorithm based on depth information fusion as claimed in claim 1, characterized in that: The modal features of adjusting RGB information and depth information using a dynamic weight matrix are: W=σ(f w ([F RGB ;F D ])) F fused =W⊙F RGB +(1-W)⊙F D Among them, f w is a fully connected layer, σ is the sigmoid activation function, ⊙ represents element-wise multiplication, F fused .
4. The cow face recognition algorithm based on depth information fusion as claimed in claim 3, characterized in that: Based on the idea of PointNet, the depth map is converted into a sparse point cloud, and then the global structural features F of the sparse point cloud are learned. final : P=DepthToPointCloud(I D ) Among them, p i is the i-th point in the sparse point cloud, g and h are multi-layer perception mechanisms, and N is the number of points. Then, we use F 3D To guide feature fusion: F final =F fused +MLP(F 3D )⊙F fused。 5. The cow face recognition algorithm based on depth information fusion as claimed in claim 4, characterized in that: The multi-task learning framework is used to simultaneously optimize cow face recognition, depth estimation and 3D key point positioning tasks. The overall loss function is: L total =λ1L id +λ2L depth +λ3L kpt +λ4L reg in: L id is the identity recognition loss, using ArcFace loss: L depth is the depth estimation loss, using the reverse Huber loss: in, L kpt is the 3D key point positioning loss, using weighted L1 loss: L reg is a regularization term used to prevent overfitting: λ1,λ2,λ3,λ4 are the weight coefficients of each loss term.
6. The cow face recognition algorithm based on depth information fusion as claimed in claim 4, characterized in that: Using the depth information, we designed a data enhancement strategy for depth-guided local deformation: Depth-guided local deformation: I′ RGB =T(I RGB ,f(I D )) Where T is the deformation function and f is the deformation field generation function based on the depth map.
7. The cow face recognition algorithm based on depth information fusion as claimed in claim 6, characterized in that: Using the depth information, we designed a data enhancement strategy for 3D rotation enhancement: 3D Rotation Enhancement: [I′ RGB ,I′ D ]=R 3D (I RGB ,I D ,θ,φ,ψ) Among them, R 3D is the 3D rotation function, θ, φ, ψ are the Euler angles.
8. The cow face recognition algorithm based on depth information fusion as claimed in claim 7, characterized in that: Using the depth information, we designed a data enhancement strategy for deep noise injection: Deep Noise Injection: Where η is the noise intensity, σ 2 (ID) is the depth-dependent variance function.
9. The cow face recognition algorithm based on depth information fusion as claimed in claim 4, characterized in that: The input image is data enhanced to obtain I', and then the model is required to be able to reconstruct the original image from the enhanced image: L cycle =||G(F(I′))-I||1 Among them, F is the feature extraction function and G is the image reconstruction function.