A multi-species pet mixed identity recognition method, device and storage medium

Abstract

The application relates to the technical field of computer vision recognition, and provides a multi-species pet mixed identity recognition method, which comprises the following steps: decoding a video stream to obtain continuous frames, detecting pet faces through a light-weight model to obtain boundary box information, confidence and a predicted category, performing multi-species multi-target tracking based on overlapping degree correlation and distributing unique IDs after screening high-confidence detection boxes, obtaining key point coordinates through a heat map regression model, correcting a posture through standard key points corresponding to the pet category to generate a standard front image, extracting an L2 normalized feature vector through a light-weight network, comparing the L2 normalized feature vector with a template library, judging an identity according to a threshold value, outputting an identity result and correlating the identity result with a tracking ID, and realizing cross-frame identity tracking between multi-species. The technical scheme of the application realizes high-robustness, low-delay pet identity recognition through posture correction and a light-weight network.

Description

Technical Field

[0001] This application relates to the field of computer vision recognition technology, and in particular to a method, device and storage medium for mixed identification of multiple species of pets. Background Technology

[0002] With the booming development and increasing diversification of the pet economy, the market demand for accurate pet identification and long-term behavior tracking continues to rise. Current mainstream solutions mostly rely on radio frequency identification (RFID) tags or subcutaneous implanted chips. While these technologies can provide basic identification, they have inherent drawbacks such as short identification distance, invasiveness (requiring physical contact or surgical implantation), and the potential to cause stress reactions in pets and poor user experience. They are unable to meet the core requirements of smart pet devices (such as automatic feeders and smart access control systems) for contactless and painless identification.

[0003] Against this backdrop, computer vision-based pet facial recognition technology has emerged as a new solution due to its advantages such as being non-contact, painless, and highly scalable. However, existing visual recognition solutions still face the following key bottlenecks in real-world, complex scenarios.

[0004] 1) Poor posture robustness is due to the fact that pets' heads often undergo large-scale deflections (head up, head down, side turn), and existing technologies lack an effective posture normalization mechanism, resulting in unstable input quality of the recognition model.

[0005] 2) Insufficient consistency in feature extraction is due to differences in shooting angles. The spatial distribution of facial images of the same pet is inconsistent, and directly inputting them into the recognition network will significantly reduce the discriminative power of features.

[0006] 3) Real-time bottleneck: To ensure accuracy, complex network structures are used, resulting in high computational overhead for the model, making it difficult to achieve real-time processing on edge devices (e.g., video stream analysis).

[0007] 4) Weak species generalization ability: existing solutions are mostly optimized for a single species (e.g., only for cats or only for dogs). Their key point definitions and pose correction standards are fixed and difficult to adapt to the differences in facial structure and key point distribution between different species (e.g., cats and dogs). If the same standard is forcibly used to process multiple species, the correction images will be distorted or information will be lost due to the mismatch of geometric features between species, which will seriously affect the accuracy and robustness of the cross-species unified identity recognition model. Summary of the Invention

[0008] This application provides a method, device, and storage medium for multi-species pet hybrid identity recognition, which achieves highly robust and low-latency pet identity recognition through posture correction and lightweight networks.

[0009] On the one hand, this application provides a method for recognizing mixed identities of pets of multiple species, the method comprising:

[0010] Acquire the input video stream and decode it into consecutive image frames;

[0011] A lightweight pet face detection model is used to process each frame of the continuous image frames, and the bounding box information, confidence score and detection category of the pet face region are output.

[0012] Based on the confidence level of the pet's face region, detection boxes of pet face regions with a confidence level higher than the first preset threshold are selected.

[0013] A multi-target tracking algorithm based on detection box overlap correlation is used to correlate the detection boxes of the pet face region in the consecutive image frames, and a unique tracking ID is assigned to the same pet target;

[0014] The detected pet face region is processed by a key point detection model based on heatmap regression to obtain the coordinate information of predefined key feature points;

[0015] Based on the output of the lightweight pet face detection model, the species category information of the pet's face is determined, and the corresponding standard key point template is selected.

[0016] Based on the species category information and the coordinate information of key feature points, a transformation matrix is ​​calculated using similarity transformation to perform pose correction on the original pet face region image, generating a standard frontal pet face image. The target key point positions of the similarity transformation correspond to the species category information.

[0017] The standard frontal pet face image is used to extract features through a lightweight pet identification network to obtain an L2-normalized feature vector.

[0018] The L2-normalized feature vector is compared with the pre-registered template feature library for similarity, and the pet identity is determined according to the threshold to obtain the pet identity recognition result.

[0019] The pet identification result is output and associated with the tracking ID to achieve cross-frame identity tracking.

[0020] On the other hand, this application provides a multi-species pet mixed identity recognition device, the device comprising:

[0021] The image acquisition module is used to acquire the input video stream and decode it into continuous image frames;

[0022] The first processing module is used to process each frame of the continuous image frames using a lightweight pet face detection model, and output the bounding box information, confidence score and detection category of the pet face region.

[0023] The filtering module is used to filter out detection boxes of pet face regions with a confidence level higher than a first preset threshold based on the confidence level of the pet face region.

[0024] The first association module is used to associate the detection boxes of the pet face region in the consecutive image frames using a multi-target tracking algorithm based on the overlap of detection boxes, and to assign a unique tracking ID to the same pet target;

[0025] The second processing module is used to process the detected pet face region through a key point detection model based on heatmap regression to obtain species category information and coordinate information of predefined key feature points;

[0026] The determination module is used to determine the species category information of the pet's face based on the output of the lightweight pet face detection model, and select the corresponding standard key point template;

[0027] The correction module is used to calculate a transformation matrix using similarity transformation based on the species category information and the coordinate information of key feature points, and to perform pose correction on the original pet face region image to generate a standard frontal pet face image. The target key point position of the similarity transformation corresponds to the species category information.

[0028] The feature extraction module is used to extract features from the standard frontal pet face image through a lightweight pet identification network to obtain an L2-normalized feature vector.

[0029] The determination module is used to compare the L2-normalized feature vector with the pre-registered template feature library for similarity, and determine the pet identity based on the threshold to obtain the pet identity recognition result;

[0030] The second association module is used to output the pet identification result and associate it with the tracking ID to achieve cross-frame identity tracking.

[0031] Thirdly, this application provides an electronic device, the device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the technical solution of the multi-species pet mixed identity recognition method described above.

[0032] Fourthly, this application provides a storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described method for recognizing mixed identities of multiple pet species.

[0033] As can be seen from the technical solution provided in this application, on the one hand, by adopting the technical path of key point detection based on heatmap regression, species-adaptive similarity transformation pose correction, and standardized image input recognition network, the key points of the face are accurately located through heatmap regression, solving the problem of key point drift caused by pose changes. By automatically selecting the corresponding standard key point template according to the detected pet species (e.g., cats, dogs), and using similarity transformation to geometrically correct the pet face image of any pose to a standard frontal view that conforms to its species characteristics, spatial deformation caused by shooting angle and head rotation can be eliminated. This ensures that the subsequent identity recognition network always receives standardized input with consistent spatial distribution, significantly improving the discriminative power of feature extraction. The proposed solution offers several advantages. First, it provides stability and cross-species generalization capabilities. Second, the lightweight detection model provides species category information while performing face detection. A multi-target tracking algorithm based on bounding box overlap correlates cross-frame targets, resolving the identity jump problem caused by high-speed motion. Combined with a lightweight identity recognition network, it achieves low-latency processing throughout the entire detection, tracking, and recognition process, meeting the real-time requirements of edge devices. Third, the pose correction stage provides structurally standardized input to the recognition network, reducing the interference of complex poses on recognition accuracy. The lightweight model design ensures deployment capability on resource-constrained devices, while the integrated processing throughout the entire process enhances the long-term reliability of smart pet devices in dynamic scenarios. In summary, the technical solution of this application achieves highly robust and low-latency pet identity recognition through pose correction and a lightweight network. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This is a flowchart of the multi-species pet mixed identity recognition method provided in the embodiments of this application;

[0036] Figure 2 This is a schematic diagram of the multi-species pet mixed identity recognition device provided in the embodiments of this application;

[0037] Figure 3 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0038] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0039] In this specification, adjectives such as "first" and "second" are used only to distinguish one element or action from another, without necessarily requiring or implying any actual such relationship or order. Where circumstances permit, reference to an element or component or step (etc.) should not be construed as being limited to only one of the elements, components, or steps, but may be one or more of the elements, components, or steps, etc.

[0040] For ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn to actual scale.

[0041] Computer vision-based pet face recognition technology has become an emerging solution due to its advantages such as being non-contact, painless, and highly scalable. However, existing visual recognition solutions still face the following key bottlenecks in real-world complex scenarios: 1) Poor pose robustness, as pets' heads often undergo significant changes (head tilting, head tilting, side turning), and existing technologies lack effective pose normalization mechanisms, leading to unstable input quality of the recognition model; 2) Insufficient consistency in feature extraction, due to differences in shooting perspectives, resulting in inconsistent spatial distribution of the same pet's face in images, and directly inputting it into the recognition network significantly reduces feature discriminative power; 3) Real-time bottleneck, namely, the use of complex network structures to ensure accuracy leads to high computational overhead, making it difficult to achieve real-time processing on edge devices (e.g., video stream analysis). For example, existing solutions typically directly process the original detection box... Feature extraction from pet face images does not eliminate the interference of pose differences on the feature space; at the same time, the process of cross-frame tracking and identity recognition is disconnected, making it difficult to meet the comprehensive requirements of smart devices for low latency and high robustness; 4) Weak species generalization ability, that is, existing solutions are mostly optimized for a single species (e.g., only for cats or only for dogs), and their key point definitions and pose correction standards are fixed, making it difficult to adapt to the differences in facial structure and key point distribution between different species (e.g., cats and dogs); if the same standard is forcibly used to process multiple species, the correction images will be distorted or information will be lost due to the mismatch of geometric features between species, which will seriously affect the accuracy and robustness of the cross-species unified identity recognition model.

[0042] To address the problems of the existing technology, this application proposes a method for mixed identification of multiple species of pets, the flowchart of which is attached. Figure 1 As shown, the main steps include S101 to S110, which are detailed below:

[0043] Step S101: Acquire the input video stream and decode it into consecutive image frames.

[0044] In this embodiment, the input video stream is a video file recorded by a surveillance camera or mobile phone camera onto a pet. Since the deep learning model processes static image data, rather than a continuous compressed video stream, it is necessary to use a standard multimedia processing library to convert the compressed video stream into a sequence of single image frames (e.g., an RGB pixel matrix) that the model can process, i.e., decode it into continuous image frames.

[0045] Step S102: Process each frame of the continuous image frames using a lightweight pet face detection model, and output the bounding box information, confidence score, and detection category of the pet face region.

[0046] In this embodiment, the lightweight pet face detection model employs depthwise separable convolution to construct the basic feature extraction layer and introduces a linear bottleneck layer and a channel attention mechanism to dynamically calibrate the feature response. The depthwise separable convolution in the lightweight pet face detection model decouples the spatial filtering and channel fusion steps and performs them sequentially, significantly reducing the number of parameters and computational complexity. Based on this, a linear bottleneck layer and a channel attention mechanism are introduced. Since depthwise separable convolution effectively avoids information loss and maintains feature richness through nonlinear transformations in low-dimensional space, while the linear bottleneck layer and channel attention mechanism dynamically calibrate the channel feature response, the lightweight pet face detection model can adaptively focus on the feature channels with the richest information when processing pet faces with different patterns and colors. Therefore, this design enables the lightweight pet face detection model to achieve high-speed feature extraction even on mobile or embedded devices with extremely limited computing resources, laying the foundation for real-time detection.

[0047] Before deployment, a lightweight pet face detection model needs to be trained. In this embodiment, the training process of the lightweight pet face detection model can be as follows: A transfer learning strategy is employed, loading pre-trained model weights on a general large dataset as initial weights; multimodal data augmentation strategies are applied, including color perturbation, geometric transformation, random occlusion, and automatic combination enhancement; wherein, color perturbation includes adjusting HSV saturation and brightness to simulate indoor lighting conditions such as backlighting and warm light; geometric transformation includes random horizontal flipping, scaling, and translation; random occlusion includes randomly erasing local textures; and automatic combination enhancement includes enabling mosaic enhancement with 100% probability and stitching four images into one. In the above training scheme for the lightweight pet face detection model, the advantage of the transfer learning strategy is that it retains to some extent the basic classification ability of the model trained on a large dataset, allowing for faster convergence and improved accuracy when training on small and medium datasets. In the multimodal data augmentation strategy, mosaic augmentation is enabled with 100% probability, and four images are stitched together into one. This increases the number of scenes where complex backgrounds and small objects coexist, improving the model's localization ability in crowded environments. Combined with the automatic combination of random erasure and RandAugment augmentation, random occlusion or distortion of local textures forces the network to focus more on the key contours and facial features of the pet's face, rather than easily variable details such as fur texture. Therefore, the multimodal data augmentation strategy maximizes the distribution boundary of pet face data without introducing unrealistic deformations, enabling the detector to robustly and accurately recognize pet faces even under complex lighting, occlusion, pose, and scale variations in the real world.

[0048] As one embodiment of this application, the lightweight pet face detection model processes each frame of a continuous image frame and outputs the bounding box information, confidence score, and detection category of the pet face region. Specifically, the lightweight pet face detection model performs forward inference through its depthwise separable convolutional backbone network to extract multi-scale features of the image; generates multiple candidate bounding boxes and their confidence scores in the output layer; then applies a non-maximum suppression algorithm to filter the candidate boxes, removing low-confidence boxes with high overlap, and outputs the finally determined, non-overlapping bounding box information, confidence score, and detection category of the pet face region.

[0049] Furthermore, to accurately detect pet faces at different scales, the lightweight pet face detection model in this application employs a multi-scale feature pyramid structure and depthwise separable convolutions to construct the basic feature extraction layer. Through a bidirectional pathway combining top-down and bottom-up approaches, and by introducing a linear bottleneck layer and channel attention mechanism, it achieves dynamic calibration of feature responses while deeply fusing high-level semantic information and low-level detail information. This structure ensures that both large-scale main cat faces and small-scale distant cat faces can obtain sufficiently rich and appropriately resolution feature maps for prediction, significantly improving the model's recall rate for targets at different scales. Specifically, the deep fusion of high-level semantic information and low-level detail information through a two-way approach combining top-down and bottom-up methods can be achieved by: upsampling the low-resolution feature map with high semantic information output from the deep layers of the lightweight pet face detection model to double its resolution; fusing the upsampled feature map with the corresponding low-semantic feature map with high detail information at the same resolution from the shallow layers of the lightweight pet face detection model; using the fused feature map as a new input, repeatedly performing upsampling and addition operations with the feature map output from the shallower layers of the network to iteratively construct a multi-scale fusion feature pyramid that combines high-level semantic information and low-level detail information; and performing pet face bounding box prediction and classification at each level of the multi-scale fusion feature pyramid.

[0050] Step S103: Based on the confidence level of the pet's face region, filter out the detection boxes of the pet's face region with a confidence level higher than the first preset threshold.

[0051] In pet detection, confidence levels are typically used to filter out low-confidence bounding boxes or to determine the reliability of bounding boxes. Since subsequent tracking algorithms usually only process bounding boxes with confidence levels above a certain threshold (i.e., high-confidence bounding boxes) to avoid false detections, in this application, bounding boxes for the pet's face region with confidence levels above a first preset threshold can be selected based on the confidence level of the pet's face region; these bounding boxes are considered high-confidence bounding boxes.

[0052] Step S104: Use a multi-target tracking algorithm based on detection box overlap to associate the detection boxes of the pet face region in consecutive image frames and assign a unique tracking ID to the same pet target.

[0053] As an embodiment of this application, step S104 can be implemented as follows: for each pet face region detection box in the current frame of a continuous image frame, calculate its intersection-union (IOU) value with all pet face region detection boxes in the previous frame; if the IOU value of a certain pet face region detection box in the current frame and a certain pet face region detection box in the previous frame exceeds a second preset threshold, then determine that the two pet face region detection boxes belong to the same pet target; assign the tracking ID corresponding to the pet face region detection box in the previous frame to the matching pet face region detection box in the current frame; for pet face region detection boxes in the current frame that fail to match any pet face region detection box in the previous frame, assign them a new unique tracking ID.

[0054] Step S105: Process the detected pet face region using a key point detection model based on heatmap regression to obtain the coordinate information of predefined key feature points.

[0055] As mentioned earlier, changes in a pet's facial posture (e.g., head tilting, side turning) can cause drastic changes in the appearance and position of key points (e.g., corners of the eyes, tip of the nose). For small-scale features, to address the two major bottlenecks of poor posture robustness and insufficient detection accuracy, traditional solutions either employ direct coordinate regression or template matching-based methods. The former involves using fully connected layers at the network end to directly output the coordinates (x, y) of key points. However, this is a highly nonlinear mapping, making the training process very unstable, prone to getting stuck in local optima, sensitive to small changes in the input image (e.g., translation), with weak generalization ability and a lack of spatial generalization, making it difficult to handle extreme postures not seen in the training set. Template matching-based methods also have significant drawbacks, including high computational cost, difficulty in real-time processing, extreme dependence on template completeness, high sensitivity to occlusion and appearance changes, and poor reliability in the dynamic real-world scenarios of pets. To overcome these shortcomings of traditional solutions, this application uses a heatmap-based regression-based key point detection model to process the detected pet facial region, obtaining species category information and the coordinate information of predefined key feature points. On the one hand, because heatmaps provide rich spatial context information, key point detection models can better understand the relative relationship between key points and surrounding features, are less sensitive to local occlusion and appearance changes, and are particularly good at locating small-scale features such as the tip of the nose and the corner of the eye. On the other hand, the loss function of heatmap regression (e.g., mean squared error) is easier to optimize than that of direct coordinate regression, the model converges faster and is more stable, and the high-precision key point coordinates provide an absolute prerequisite for subsequent high-precision pose correction.

[0056] Similar to the lightweight pet face detection model in the aforementioned embodiments, which requires training before deployment, the keypoint detection model based on heatmap regression in this application also requires training. The process includes: using model weights pre-trained on a large-scale human face keypoint dataset for transfer learning as the starting point for pet keypoint detection training; applying the same affine transformation to the input pet face training image and the corresponding ground truth coordinates of keypoints to simulate different shooting angles; applying a small Gaussian random perturbation with a mean of zero and a standard deviation of 2 pixels to the ground truth coordinates of keypoints, and simultaneously generating the perturbed heatmap ground truth; and randomly scaling and translating the detection box boundaries of the pet face in the pet face training image. In this study, transfer learning is performed using model weights pre-trained on a large-scale facial landmark dataset. The starting point for training pet landmark detection can be as follows: Load the model weights pre-trained on the large-scale facial landmark dataset as the initial weights for the pet landmark detection model; based on the definition and number of pet facial landmarks, retain and reuse the feature extraction backbone network of the pre-trained model, resetting or adjusting only the final prediction output layer to adapt to the prediction requirements of pet landmarks; in the initial training stage, freeze the weights of the feature extraction backbone network, training only the reset output layer; after the loss function converges, unfreeze all network weights and perform end-to-end fine-tuning with a low learning rate; and then apply the input pet... The same affine transformation is applied to both the face training image and its corresponding ground truth coordinates to simulate different shooting angles. Specifically, this can be achieved by: randomly generating an affine transformation matrix containing rotation, scaling, translation, and cropping parameters; using the same affine transformation matrix, performing an affine transformation operation on the input pet face training image, and performing the same matrix transformation operation on its corresponding ground truth coordinates to synchronize the perspective transformation of the image with the labeled data, thus simulating different shooting angles; applying a small Gaussian random perturbation with a mean of zero and a standard deviation of 2 pixels to the ground truth coordinates of the key points, and synchronously generating the perturbed heatmap ground truth. Specifically, this can be achieved by: applying a two-dimensional perturbation with a mean of zero and a standard deviation of 2 pixels to the ground truth coordinates of the key points. Gaussian random perturbation generates perturbed coordinates with minor annotation errors. Based on these perturbed coordinates, the corresponding Gaussian distributed heatmap ground truth is recalculated and generated to replace the original heatmap ground truth, serving as a supervision signal for the current training samples to improve the model's tolerance and robustness to annotation errors. Specifically, random scaling and translation perturbation of the detection box boundaries in the pet face training images can be performed to simulate localization deviations in actual detection processes. Based on the perturbed detection boxes, the ground truth coordinates of key points within the boxes are adjusted accordingly in terms of scale and displacement to ensure the consistency of the enhanced data.

[0057] The training scheme for the keypoint detection model based on heatmap regression not only enhances the adaptability of the keypoint detection model to changes in pose and scale and improves its tolerance to annotation errors, but also improves the stability of the keypoint detection model in complex scenarios by simulating localization deviations in actual detection.

[0058] Step S106: Based on the output of the lightweight pet face detection model, determine the species category information of the pet's face and select the corresponding standard key point template.

[0059] In this embodiment, the lightweight pet face detection model possesses multi-task learning capabilities. Besides outputting bounding boxes and confidence scores, it simultaneously outputs the detected category of the pet's face in the image (e.g., cat, dog, etc.). During training, the model uses a dataset containing species annotations, enabling it to distinguish between different species. Based on the detected species category, the system retrieves the target coordinates of key points from a pre-stored standard keypoint template library, corresponding to the species' standard frontal facial image. For example, although both cats and dogs define keypoints such as left ear, right ear, left eye, right eye, and nose tip, their target coordinates on the standard frontal image are independently defined based on their respective species' facial anatomy (e.g., eye distance, face aspect ratio, etc.). This ensures that the subsequent geometric transformation for pose correction conforms to the specific species' geometric characteristics, providing high-quality, standardized input for unified cross-species identity recognition.

[0060] Step S107: Based on the species category information and the coordinate information of key feature points, calculate the transformation matrix using similarity transformation, perform pose correction on the original pet face region image, and generate a standard frontal pet face image, wherein the target key point position of the similarity transformation corresponds to the species category information.

[0061] Because recognition networks struggle to handle inputs with varying poses, features extracted from the same pet in different poses can differ significantly in the feature space (manifesting as large intra-class differences), leading to recognition failures. Therefore, to address this challenge, existing solutions involve training an extremely robust recognition model using massive amounts of data encompassing various poses, allowing the model to learn to ignore pose changes. However, this approach suffers from high data requirements, extremely high training costs, and a highly complex model with a huge number of parameters, contradicting the core objectives of lightweight design and edge deployment. Ultimately, performance often remains a bottleneck, failing to completely solve the problem. Another existing approach uses complex 3D models for pose estimation and reconstruction. However, this approach suffers from extremely high computational complexity, making it unsuitable for real-time requirements, and also requires precise 3D prior information, rendering it highly impractical for objects like pets that lack standardized 3D models.

[0062] On the one hand, to unify pet face images of different poses and scales into a standardized space (frontal, fixed size), thereby eliminating nonlinear geometric changes caused by shooting angle and head rotation; on the other hand, to allow the network to focus on learning identity-related discriminative features (e.g., texture, markings, etc.) without having to learn complex pose invariance, thus significantly improving recognition accuracy, especially for individuals with similar appearances, this application can use similarity transformation to calculate a transformation matrix based on species category information and the coordinate information of key feature points to perform pose correction on the original pet face region image, generating a standard frontal pet face image. The target key point position of the similarity transformation corresponds to the species category information. Prior to this, standard key point templates corresponding to different species categories are pre-stored. Thus, calculating the transformation matrix using similarity transformation based on species category information and the coordinate information of key feature points can be done by selecting the corresponding standard key point template as the target position for similarity transformation based on the species category information. Since similarity transformation is fast and involves only simple matrix operations, it is very suitable for real-time processing in pipelines, perfectly meeting the low-power requirements of edge computing scenarios.

[0063] Specifically, as an embodiment of this application, step S106 can be implemented as follows: From the keypoint detection results, select the coordinates of five points—the left root of the left ear, the right root of the right ear, the center of the left eyeball, the center of the right eyeball, and the tip of the nose—as source keypoints; match the selected source keypoints with the target positions of corresponding keypoints in a predefined standard frontal pet face image; based on the matched source and target keypoints, solve for the optimal similarity transformation matrix using the least squares method. This matrix includes rotation, scaling, and translation parameters; using the solved similarity transformation matrix, perform bilinear interpolation affine transformation on the original pet face region image to generate a pose-corrected standard frontal image. In the above embodiment, the similarity transformation matrix calculation uses iterative reweighted optimization, i.e., constructing a weight matrix. The initial weight is 1, and the weighted least squares problem is solved. Calculate the initial transformation; dynamically adjust the weights based on the reprojection error, reducing the weights of points with an error greater than 3 pixels to a preset threshold (e.g., 0.2), iterating until the rate of change of the weights is less than the threshold, e.g., 5%; solve the problem using SVD decomposition after each iteration. The least squares solution, i.e.: calculate The SVD decomposition is performed, and the rotation matrix is ​​constructed by taking the vector corresponding to the minimum singular value. The translation parameter is calculated by the difference in the mean of the displacement vectors. In the above embodiment, This represents the source point set matrix, which is the matrix composed of the coordinates of the key points on the pet's face detected from the current image frame. This represents the target point set matrix, which is the matrix composed of the coordinates of predefined key points in a standard frontal pet face template. Let represent the transformation matrix to be solved. yes The transpose of .

[0064] Step S108: Extract features from a standard frontal pet face image using a lightweight pet identification network to obtain an L2-normalized feature vector.

[0065] Pet identification suffers from high model complexity and low accuracy. Existing solutions address these issues by using large, complex networks (e.g., ResNet50, VGG). While this approach may achieve high accuracy, its massive parameter count and computational demands prevent real-time inference on edge computing chips like the RK3588 (where the frame rate would be far below the required 55 FPS), failing to meet practical product requirements. Existing solutions using unnormalized feature vectors for similarity calculation suffer from inconsistent feature vector magnitudes, interfering with similarity metrics. For example, a negative sample with a longer magnitude might have similar cosine values ​​to a positive sample with a shorter magnitude, leading to misclassification and a significant drop in recognition performance. To address these shortcomings, this application utilizes a lightweight pet identification network to extract features from standard frontal pet face images, obtaining L2-normalized feature vectors. In one embodiment of this application, the lightweight pet identification network is constructed using 16 layers of inverted residual modules. It dynamically balances computational complexity and accuracy through a width multiplier, with an input resolution of 112×112 and an output of a 512-dimensional feature vector. On one hand, by employing depthwise separable convolution and inverted residual design, the model's representational capabilities are preserved as much as possible while significantly reducing model complexity and computational cost. This allows the high-precision identification model to run efficiently on resource-constrained edge devices. On the other hand, L2 normalization is applied to the extracted feature vectors, projecting them onto a unit hypersphere. This ensures the effectiveness and consistency of cosine similarity calculation. At this point, the cosine similarity between feature vectors is equivalent to their Euclidean distance, becoming a stable and reliable metric.

[0066] The lightweight pet identification network in the above embodiment is trained using a dynamically configurable multi-boundary constraint loss function, CombinedMarginLoss. Parameters m1, m2, and m3 control the combined application of boundary constraint strategies. Further, the parameters of CombinedMarginLoss are configured as follows: m1=1.0 controls the multiplication boundary of angles, m2=0.5 controls the addition boundary of angles, and m3=0.2 controls the subtraction boundary of cosines, achieving a combined application of ArcFace and CosFace strategies. The training process of the lightweight pet identification network also includes a negative sample mining and filtering mechanism based on inter-class similarity, focusing on strengthening the penalty for easily confused sample pairs. In addition, the training of the lightweight pet identification network systematically integrates various data augmentation methods, including: illumination and color perturbation, geometric deformation enhancement, occlusion and degradation simulation, and strategic combination enhancement, etc. These are detailed below.

[0067] In this embodiment, the backbone of the lightweight pet identification network employs a deep fusion design combining deep separable convolutions and efficient residual connections. Through clever channel compression and expansion strategies, the number of parameters and computational cost are significantly reduced while preserving the network's strong ability to discriminate subtle facial features of pets (e.g., fur texture, markings, eye details, etc.). Global depthwise convolutions (GDCs) are used instead of traditional fully connected layers as the classification output layer. This design significantly compresses the number of parameters at the model's end, avoiding overfitting, while enhancing the model's awareness of the global image context, effectively improving the model's robustness to recognition under small sample sizes, partial occlusion, or low-quality input. Furthermore, the lightweight pet identification network is designed and employs a dynamically configurable multi-boundary constraint loss optimization framework. The core of this framework is the multi-boundary constraint loss function CombinedMarginLoss, which supports the combined use of various boundary constraint strategies, such as a combination of ArcFace (m1=1.0, m2=0.5) and CosFace (m3=0.0). This framework can dynamically select or combine different boundary constraint strategies to actively regulate the aggregation degree of intra-class features and the separation degree of inter-class features in the feature space, thereby strengthening the feature separability of highly similar pet individuals. Simultaneously, it introduces a negative sample mining and filtering mechanism based on inter-class similarity, focusing on mining easily confused sample pairs during training and strengthening the penalty for these difficult sample pairs, thus continuously improving the model's ability to distinguish highly similar individuals. Specifically, ArcFace enhances inter-class separability by directly adding an additive boundary (m) to the angle space. For the true label category, it adds a boundary value to the angle between the feature vector and the category center, making similar samples more clustered and dissimilar samples better separated. CosFace increases the inter-class distance and improves the discriminative ability of features by subtracting a boundary value (m) from the features of the true label category in the cosine space. CombinedMarginLoss can apply multiple boundary constraint strategies simultaneously or selectively, unifying different types of boundary constraints such as ArcFace and CosFace into a single framework, controlling the combination of different types of boundaries through parameters. The training algorithm settings involve setting three parameters: m1, m2, and m3. m1 controls the multiplication boundary of the angle, m2 controls the addition boundary of the angle, and m3 controls the subtraction boundary of the cosine. When m1=1, m2>0, and m3>0, it is a combination of ArcFace and CosFace. In some embodiments, the CombinedMarginLoss parameter is set to (1.0, 0.5, 0.2).A lightweight pet identification network systematically integrates multiple data augmentation methods, constructing an augmentation process that includes illumination and color perturbation, geometric deformation enhancement, occlusion and degradation simulation, and strategic combination enhancement. This method greatly enriches and simulates real-world edge scenarios, effectively improving the robustness and generalization ability of the lightweight pet identification network in complex practical applications. The above embodiments mention introducing a negative sample mining and filtering mechanism based on inter-class similarity to focus on mining easily confused sample pairs during training. However, to address the problem of insufficient discrimination ability for high-similarity samples, dynamic threshold control can be implemented during the training of the lightweight pet identification network to mine difficult samples, i.e., based on the average inter-class similarity of the current epoch. Set the mining threshold This ensures that the training process focuses on distinguishing highly similar samples in the later stages.

[0068] Furthermore, the training of the lightweight pet identification network employs knowledge distillation, specifically, using a ResNet50 model as the teacher model and employing feature-level mean squared error loss to ensure the student model's feature distribution aligns with the teacher model. Further, the knowledge distillation process uses dynamically adjusted distillation weights to progressively enhance the learning of the teacher model's feature representations during training. Specifically, to obtain the final lightweight pet identification network, this application uses knowledge distillation. During training, a feature space-aligned distillation method is employed. By calculating the feature-level mean squared error loss, the lightweight student model is forced to maintain consistency between its embedded feature distribution and that of the more accurate and complex teacher model. Here, the teacher model uses the ResNet50 architecture, with its input and output aligned with the student model. By designing dynamically adjusted distillation weights and progressively enhancing the learning of the teacher model's feature representations during training, this scheme significantly improves the final recognition accuracy of the student model while maintaining its lightweight and low-latency characteristics.

[0069] Furthermore, to address the issue of decreased convergence efficiency in the later stages of training, dynamic feature distillation can be performed on the training of the lightweight pet identification network, namely: constructing a temperature coefficient... The soft teacher label is achieved by using feature-level mean square loss to make the student model output distribution approximate the soft probability distribution of the teacher model. The teacher model parameters are updated every 10 epochs. Furthermore, in the later stages of training the lightweight pet identification network using dynamic feature distillation, hard sample reinforcement can be activated. That is, when the training accuracy is greater than a preset threshold, such as 85%, the top preset proportion (e.g., 1 / 5) of samples with the largest feature differences between the teacher model and the student model are selected, and their distillation loss weights are increased by a preset factor (e.g., 3 times), forcing the network to focus on learning the feature representation of hard samples.

[0070] Step S109: Compare the L2-normalized feature vector with the pre-registered template feature library for similarity, and determine the pet's identity based on the threshold to obtain the pet identity recognition result.

[0071] The L2-normalized feature vector is compared with the pre-registered template feature library. By calculating the cosine similarity, if the highest similarity score exceeds the preset threshold, the face image is determined to belong to the pet of that ID, and its identity information is output. If the highest similarity score does not exceed the threshold, it can be determined to be an unknown individual, and it can be added to the database to expand the recognition range.

[0072] Step S110: Output the pet identification result and associate it with the unique tracking ID assigned to the same pet target to achieve cross-frame identity tracking.

[0073] Specifically, the pet identification results are overlaid on the original image frame along with the tracking box, key points, and other information output in step S104 for visualization output, or the structured recognition results are uploaded to the server through a data interface to complete the entire process of pet face detection, tracking, and identification.

[0074] From the above appendix Figure 1 The example of a multi-species pet hybrid identity recognition method demonstrates that, on the one hand, by employing a technical approach based on heatmap regression for keypoint detection, species-adaptive similarity transformation for pose correction, and a standardized image input recognition network, it accurately locates facial keypoints through heatmap regression, solving the keypoint drift problem caused by pose changes. By automatically selecting the corresponding standard keypoint template based on the detected pet species (e.g., cats, dogs), and using similarity transformation to geometrically correct pet facial images in any pose to a standard frontal view consistent with their species characteristics, it eliminates spatial distortions caused by shooting angle and head rotation. This ensures that the subsequent identity recognition network always receives standardized input with consistent spatial distribution, significantly improving the discriminative power of feature extraction. On the one hand, the lightweight detection model provides species category information while performing face detection. The multi-target tracking algorithm based on bounding box overlap correlates cross-frame targets, solving the identity jump problem caused by high-speed motion. Combined with a lightweight identity recognition network, it achieves low-latency processing throughout the entire detection, tracking, and recognition process, meeting the real-time requirements of edge devices. On the other hand, the pose correction stage provides structurally standardized input to the recognition network, reducing the interference of complex poses on recognition accuracy. The lightweight model design ensures deployment capability on resource-constrained devices, while the integrated processing throughout the entire process improves the long-term behavior tracking reliability of smart pet devices in dynamic scenarios. In summary, the technical solution of this application achieves highly robust and low-latency pet identity recognition through pose correction and a lightweight network.

[0075] Please see the appendix Figure 2 This application provides a multi-species pet mixed identity recognition device. The device may include an image acquisition module 201, a first processing module 202, a filtering module 203, a first association module 204, a second processing module 205, a determination module 206, a correction module 207, a feature extraction module 208, a judgment module 209, and a second association module 210, as detailed below:

[0076] Image acquisition module 201 is used to acquire the input video stream and decode it into continuous image frames;

[0077] The first processing module 202 is used to process each frame of a continuous image frame using a lightweight pet face detection model, and output the bounding box information, confidence score and detection category of the pet face region.

[0078] The filtering module 203 is used to filter out detection boxes of pet face regions with a confidence level higher than a first preset threshold based on the confidence level of the pet face region.

[0079] The first association module 204 is used to associate the detection boxes of the pet face region in consecutive image frames using a multi-target tracking algorithm based on the overlap of detection boxes, and to assign a unique tracking ID to the same pet target;

[0080] The second processing module 205 is used to process the detected pet face region through a key point detection model based on heatmap regression to obtain the coordinate information of predefined key feature points;

[0081] The determination module 206 is used to determine the species category information of the pet's face based on the output of the lightweight pet face detection model, and select the corresponding standard key point template.

[0082] The correction module 207 is used to calculate the transformation matrix using similarity transformation based on species category information and the coordinate information of key feature points, and to perform pose correction on the original pet face region image to generate a standard frontal pet face image. The target key point position of the similarity transformation corresponds to the species category information.

[0083] The feature extraction module 208 is used to extract features from a standard frontal pet face image through a lightweight pet identification network to obtain an L2-normalized feature vector.

[0084] The determination module 209 is used to compare the L2 normalized feature vector with the pre-registered template feature library for similarity, and determine the pet identity based on the threshold to obtain the pet identity recognition result.

[0085] The second association module 210 is used to output the pet identification result and associate it with the unique tracking ID assigned to the same pet target to realize cross-frame identity tracking.

[0086] From the above appendix Figure 2 As demonstrated by the example of a multi-species pet hybrid identification device, on the one hand, by employing a technical approach based on heatmap regression for keypoint detection, species-adaptive similarity transformation for pose correction, and a standardized image input recognition network, the device accurately locates facial keypoints through heatmap regression, solving the keypoint drift problem caused by pose changes. Furthermore, by automatically selecting the corresponding standard keypoint template based on the detected pet species (e.g., cats, dogs), and using similarity transformation to geometrically correct pet facial images of any pose to a standard frontal view consistent with their species characteristics, the device eliminates spatial distortions caused by shooting angle and head rotation. This ensures that the subsequent identification network always receives standardized input with a consistent spatial distribution, significantly improving the discriminative power of feature extraction. On the one hand, the lightweight detection model provides species category information while performing face detection. The multi-target tracking algorithm based on bounding box overlap correlates cross-frame targets, solving the identity jump problem caused by high-speed motion. Combined with a lightweight identity recognition network, it achieves low-latency processing throughout the entire detection, tracking, and recognition process, meeting the real-time requirements of edge devices. On the other hand, the pose correction stage provides structurally standardized input to the recognition network, reducing the interference of complex poses on recognition accuracy. The lightweight model design ensures deployment capability on resource-constrained devices, while the integrated processing throughout the entire process improves the long-term behavior tracking reliability of smart pet devices in dynamic scenarios. In summary, the technical solution of this application achieves highly robust and low-latency pet identity recognition through pose correction and a lightweight network.

[0087] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. For example... Figure 3 As shown, the electronic device 3 in this embodiment mainly includes: a processor 30, a memory 31, and a computer program 32 stored in the memory 31 and executable on the processor 30, such as a program for a multi-species pet mixed identity recognition method. When the processor 30 executes the computer program 32, it implements the steps described in the above-described multi-species pet mixed identity recognition method embodiment, for example... Figure 1 The steps S101 to S110 are shown. Alternatively, when the processor 30 executes the computer program 32, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 2 The functions of the image acquisition module 201, the first processing module 202, the filtering module 203, the first association module 204, the second processing module 205, the determination module 206, the correction module 207, the feature extraction module 208, the judgment module 209, and the second association module 210 are shown.

[0088] For example, the computer program 32 of the multi-species pet hybrid identity recognition method mainly includes: acquiring an input video stream and decoding it into continuous image frames; processing each frame of the continuous image frames using a lightweight pet face detection model to output the bounding box information, confidence level, and detection category of the pet face region; filtering out pet face region detection boxes with confidence levels higher than a first preset threshold based on the confidence level of the pet face region; associating the pet face region detection boxes in the continuous image frames using a multi-target tracking algorithm based on detection box overlap to assign a unique tracking ID to the same pet target; processing the detected pet face regions using a key point detection model based on heatmap regression to obtain the coordinate information of predefined key feature points; and using a lightweight pet face detection model... The system outputs a model to determine the species category information of the pet's face and selects the corresponding standard key point template. Based on the species category information and the coordinate information of the key feature points, a transformation matrix is ​​calculated using similarity transformation to perform pose correction on the original pet face region image, generating a standard frontal pet face image. The target key point positions of the similarity transformation correspond to the species category information. A lightweight pet identification network is used to extract features from the standard frontal pet face image, obtaining an L2-normalized feature vector. The L2-normalized feature vector is compared with a pre-registered template feature library for similarity, and the pet's identity is determined based on a threshold to obtain the pet identification result. The pet identification result is output and associated with a unique tracking ID assigned to the same pet target, achieving cross-frame identity tracking. The computer program 32 can be divided into one or more modules / units, one or more of which are stored in memory 31 and executed by processor 30 to complete this application. One or more modules / units can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program 32 in the electronic device 3.For example, computer program 32 can be divided into the functions of image acquisition module 201, first processing module 202, filtering module 203, first association module 204, second processing module 205, determination module 206, correction module 207, feature extraction module 208, judgment module 209, and second association module 210 (a module in the virtual device). The specific functions of each module are as follows: Image acquisition module 201 is used to acquire the input video stream and decode it into continuous image frames; First processing module 202 is used to process each frame of the continuous image frames using a lightweight pet face detection model, and output the bounding box information, confidence level, and detection category of the pet face region; Filtering module 203 is used to filter out pet face region detection boxes with a confidence level higher than a first preset threshold based on the confidence level of the pet face region; First association module 204 is used to associate pet face region detection boxes in continuous image frames using a multi-target tracking algorithm based on detection box overlap association, and assign a unique tracking ID to the same pet target; Second processing module 205 is used to associate the detection boxes of the pet face region in the continuous image frames using a multi-target tracking algorithm based on heatmap regression. The keypoint detection model processes the detected pet face region to obtain the coordinate information of predefined key feature points; the determination module 206 is used to determine the species category information of the pet face based on the output of the lightweight pet face detection model and select the corresponding standard keypoint template; the correction module 207 is used to calculate the transformation matrix using similarity transformation based on the species category information and the coordinate information of key feature points, and perform pose correction on the original pet face region image to generate a standard frontal pet face image, wherein the target keypoint position of the similarity transformation corresponds to the species category information; the feature extraction module 208 is used to extract features from the standard frontal pet face image through a lightweight pet identity recognition network to obtain an L2 normalized feature vector; the judgment module 209 is used to compare the L2 normalized feature vector with the pre-registered template feature library for similarity, and determine the pet identity based on a threshold to obtain the pet identity recognition result; the second association module 210 is used to output the pet identity recognition result and associate it with the unique tracking ID assigned to the same pet target to realize cross-frame identity tracking.

[0089] Electronic device 3 may include, but is not limited to, processor 30 and memory 31. Those skilled in the art will understand that... Figure 3 This is merely an example of electronic device 3 and does not constitute a limitation on electronic device 3. It may include more or fewer components than shown, or combine certain components, or different components. For example, electronic device may also include input / output devices, network access devices, buses, etc.

[0090] The processor 30 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0091] The memory 31 can be an internal storage unit of the electronic device 3, such as a hard disk or RAM. The memory 31 can also be an external storage device of the electronic device 3, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory 31 can include both internal and external storage units of the electronic device 3. The memory 31 is used to store computer programs and other programs and data required by the electronic device. The memory 31 can also be used to temporarily store data that has been output or will be output.

[0092] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed. That is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above-described device can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0093] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0094] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0095] In the embodiments provided in this application, it should be understood that the disclosed apparatus / device and method can be implemented in other ways. For example, the apparatus / device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0096] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0097] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0098] If integrated modules / units are implemented as software functional units and sold or used as independent products, they can be stored in a storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program for the multi-species pet hybrid identity recognition method can be stored in a storage medium. When the computer program is executed by a processor, it can implement the steps of the above method embodiments, namely, acquiring the input video stream and decoding it into continuous image frames; processing each frame of the continuous image frames using a lightweight pet face detection model, and outputting the bounding box information, confidence level, and detection category of the pet face region; filtering out pet face region detection boxes with confidence levels higher than a first preset threshold based on the confidence level of the pet face region; associating the pet face region detection boxes in the continuous image frames using a multi-target tracking algorithm based on detection box overlap correlation, and assigning a unique tracking ID to the same pet target; and detecting the target using a key point detection model based on heatmap regression. The system processes the captured pet face region to obtain the coordinates of predefined key feature points. Based on the output of a lightweight pet face detection model, it determines the species category of the pet face and selects the corresponding standard key point template. Using the species category information and the coordinates of the key feature points, it calculates a transformation matrix using similarity transformation to correct the pose of the original pet face region image, generating a standard frontal pet face image. The target key point positions of the similarity transformation correspond to the species category information. A lightweight pet identification network extracts features from the standard frontal pet face image to obtain L2-normalized feature vectors. The L2-normalized feature vectors are compared with a pre-registered template feature library for similarity, and the pet's identity is determined based on a threshold to obtain the pet identification result. The pet identification result is output and associated with a unique tracking ID assigned to the same pet target, achieving cross-frame identity tracking. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or some intermediate form. Storage media can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the contents of storage media can be appropriately added to or removed according to the requirements of legislation and patent practice in a jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, storage media may not include electrical carrier signals and telecommunication signals.

[0099] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. These modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application. The specific embodiments described above further illustrate the purpose, technical solutions, and beneficial effects of this application. It should be understood that the above descriptions are merely specific embodiments of this application and are not intended to limit the protection scope of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this invention.

Claims

1. A method for recognizing mixed identities of multiple pet species, characterized in that, The method includes: Pre-store standard key point templates corresponding to different species categories; Acquire the input video stream and decode it into consecutive image frames; A lightweight pet face detection model is used to process each frame of the continuous image frames, and the bounding box information, confidence score and detection category of the pet face region are output. Based on the confidence level of the pet's face region, detection boxes of pet face regions with a confidence level higher than the first preset threshold are selected. A multi-target tracking algorithm based on detection box overlap correlation is used to correlate the detection boxes of the pet face region in the consecutive image frames, and a unique tracking ID is assigned to the same pet target; The detected pet face region is processed by a key point detection model based on heatmap regression to obtain the coordinate information of predefined key feature points; Based on the output of the lightweight pet face detection model, the species category information of the pet's face is determined, and the corresponding standard key point template is selected. Based on the species category information and the coordinate information of key feature points, a transformation matrix is ​​calculated using similarity transformation to perform pose correction on the original pet face region image, generating a standard frontal pet face image. The target key point positions of the similarity transformation correspond to the species category information. The standard frontal pet face image is used to extract features through a lightweight pet identification network to obtain an L2-normalized feature vector. The L2-normalized feature vector is compared with the pre-registered template feature library for similarity, and the pet identity is determined according to the threshold to obtain the pet identity recognition result. Output the pet identification result and associate it with the tracking ID to achieve cross-frame identity tracking; The step of calculating the transformation matrix using similarity transformation based on the species category information and the coordinate information of key feature points includes: selecting the corresponding standard key point template as the target position for similarity transformation based on the species category information.

2. The method for recognizing mixed identities of multiple pet species as described in claim 1, characterized in that, The lightweight pet face detection model employs a multi-scale feature pyramid structure and depthwise separable convolution to construct the basic feature extraction layer. By combining a top-down and bottom-up bidirectional pathway and introducing a linear bottleneck layer and channel attention mechanism, it achieves dynamic calibration of feature responses while deeply integrating high-level semantic information and low-level detailed information.

3. The method for recognizing mixed identities of multiple pet species as described in claim 1, characterized in that, The keypoint detection model adopts a lightweight, high-resolution network structure based on depthwise separable convolution. It maintains the spatial resolution of the feature map through progressive upsampling and predicts the coordinates of keypoints through Gaussian heatmap regression, outputting the keypoint detection results.

4. The method for recognizing mixed identities of multiple pet species as described in claim 3, characterized in that, The training of the keypoint detection model includes: We used the model weights pre-trained on a large-scale facial landmark dataset for transfer learning as the starting point for training pet landmark detection. The same affine transformation is applied to the input pet face training image and the corresponding key point ground value coordinates to simulate different shooting angles; A small Gaussian random perturbation with a mean of zero and a standard deviation of 2 pixels is applied to the true coordinate position of the key point, and the perturbed true heat map is generated simultaneously. The detection bounding boxes of the pet's face in the training images of the pet's face are randomly scaled and translated.

5. The method for recognizing mixed identities of multiple pet species as described in claim 3, characterized in that, The step of calculating a transformation matrix using a similarity transformation algorithm based on the coordinate information of the key feature points, performing pose correction on the original pet face region image, and generating a standard frontal pet face image includes: The coordinates of five points—the left root of the left ear, the right root of the right ear, the center of the left eyeball, the center of the right eyeball, and the tip of the nose—are selected from the key point detection results as source key points. The source key points are matched with the target positions of corresponding key points in a predefined standard frontal pet face image; Based on the matching source keypoints and target keypoints, the optimal similarity transformation matrix is ​​solved by the least squares method. Using the obtained similarity transformation matrix, a bilinear interpolation affine transformation is performed on the original pet face region image to generate a standard frontal image after pose correction.

6. The method for recognizing mixed identities of multiple pet species as described in claim 1, characterized in that, The lightweight pet identification network is composed of 16 layers of inverted residual modules stacked together. It dynamically balances computational load and accuracy through width multiplier, with an input resolution of 112×112 and an output of 512-dimensional feature vector.

7. A multi-species pet mixed identification device, characterized in that, The device includes: A module for pre-storing standard keypoint templates corresponding to different species categories; The image acquisition module is used to acquire the input video stream and decode it into continuous image frames; The first processing module is used to process each frame of the continuous image frames using a lightweight pet face detection model, and output the bounding box information, confidence score and detection category of the pet face region. The filtering module is used to filter out detection boxes of pet face regions with a confidence level higher than a first preset threshold based on the confidence level of the pet face region. The first association module is used to associate the detection boxes of the pet face region in the consecutive image frames using a multi-target tracking algorithm based on the overlap of detection boxes, and to assign a unique tracking ID to the same pet target; The second processing module is used to process the detected pet face region through a key point detection model based on heatmap regression to obtain species category information and coordinate information of predefined key feature points; The determination module is used to determine the species category information of the pet's face based on the output of the lightweight pet face detection model, and select the corresponding standard key point template; The correction module is used to calculate a transformation matrix using similarity transformation based on the species category information and the coordinate information of key feature points, and to perform pose correction on the original pet face region image to generate a standard frontal pet face image. The target key point position of the similarity transformation corresponds to the species category information. The step of calculating the transformation matrix using similarity transformation based on the species category information and the coordinate information of key feature points includes: selecting the corresponding standard key point template as the target position of the similarity transformation based on the species category information. The feature extraction module is used to extract features from the standard frontal pet face image through a lightweight pet identification network to obtain an L2-normalized feature vector. The determination module is used to compare the L2-normalized feature vector with the pre-registered template feature library for similarity, and determine the pet identity based on the threshold to obtain the pet identity recognition result; The second association module is used to output the pet identification result and associate it with the tracking ID to achieve cross-frame identity tracking.

8. An electronic device, the device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 6.

9. A storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 6.

Images