Pet identification method and device, storage medium and computer device

Abstract

This application provides a pet recognition method, apparatus, storage medium, and computer device. The method involves acquiring pet images collected by an edge pet recognition device; calling a preset multi-task model to perform pet recognition on the pet images, obtaining pet recognition results; the multi-task model includes an image encoder, a pet detection branch, and a pet recognition branch. The training process of the multi-task model includes: performing unsupervised pre-training on the image encoder based on historical pet images collected by the edge pet recognition device; selecting a set of positive sample images and a set of negative sample images from the historical pet images, and updating the model parameters of the multi-task model based on the positive and negative sample image sets; acquiring training sample data with multi-task labels, and updating the parameters of the multi-task model based on the training sample data. This method can improve the accuracy of pet recognition.

Description

Technical Field

[0001] This disclosure relates to the field of pet intelligence technology, specifically to a pet identification method, device, electronic device, and storage medium. Background Technology

[0002] In recent years, with the ever-accelerating pace of life, people's life pressure has been constantly increasing. The companionship of pets can greatly alleviate people's mental stress and bring them joy, which has led to the rapid development of the pet supplies market in recent years.

[0003] Among the related technologies are pet management applications that analyze pet behavior by collecting videos or images to determine habits, health status, and other information. Based on the analysis results, appropriate education or treatment can then be provided to the pet.

[0004] Collecting videos or images of pet behavior requires a high level of pet recognition ability. However, current pet recognition methods have relatively low accuracy. Summary of the Invention

[0005] The main objective of this application is to provide a pet identification method, device, electronic device, and storage medium, aiming to improve the accuracy of pet identification.

[0006] To achieve the above objectives, a first aspect of this application proposes a pet identification method, the method comprising:

[0007] Acquire pet images captured by the edge pet device;

[0008] A preset multi-task model is invoked to perform pet recognition on the pet image, and the pet recognition result is obtained;

[0009] The multi-task model includes an image encoder, a pet detection branch, and a pet recognition branch. The training process of the multi-task model includes:

[0010] The image encoder is pre-trained unsupervised based on historical pet images collected by the edge pet device.

[0011] A set of positive sample images and a set of negative sample images are selected from the historical pet images. The set of positive sample images includes multiple different images of the same pet, and the set of negative sample images includes multiple images of different pets. The model parameters of the multi-task model are updated based on the set of positive sample images and the set of negative sample images.

[0012] Acquire training sample data with multi-task labels. The training sample data includes multiple sample pet images and pet detection labels and pet recognition labels corresponding to each sample pet image. Update the parameters of the multi-task model based on the training sample data.

[0013] To achieve the above objectives, a second aspect of this application provides a pet identification device, the device comprising:

[0014] The acquisition unit is used to acquire pet images collected by the edge pet device;

[0015] The recognition unit is used to call a preset multi-task model to perform pet recognition on the pet image and obtain the pet recognition result;

[0016] The multi-task model includes an image encoder, a pet detection branch, and a pet recognition branch. The training process of the multi-task model includes:

[0017] The image encoder is pre-trained unsupervised based on historical pet images collected by the edge pet device.

[0018] A set of positive sample images and a set of negative sample images are selected from the historical pet images. The set of positive sample images includes multiple different images of the same pet, and the set of negative sample images includes multiple images of different pets. The model parameters of the multi-task model are updated based on the set of positive sample images and the set of negative sample images.

[0019] Acquire training sample data with multi-task labels. The training sample data includes multiple sample pet images and pet detection labels and pet recognition labels corresponding to each sample pet image. Update the parameters of the multi-task model based on the training sample data.

[0020] In some embodiments, this application also provides a model training apparatus, including:

[0021] The pre-training unit is used to perform unsupervised pre-training on the image encoder based on historical pet images collected by the edge pet device;

[0022] The first update unit is used to filter out a set of positive sample images and a set of negative sample images from the historical pet images. The set of positive sample images includes multiple different images of the same pet, and the set of negative sample images includes multiple images of different pets. The model parameters of the multi-task model are updated based on the set of positive sample images and the set of negative sample images.

[0023] The second update unit is used to acquire training sample data with multi-task labels. The training sample data includes multiple sample pet images and pet detection labels and pet recognition labels corresponding to each sample pet image. The parameters of the multi-task model are updated based on the training sample data.

[0024] Optionally, in some embodiments, the first updating unit includes:

[0025] The first filtering subunit is used to filter out multiple different images of the same pet taken from different shooting angles from the historical pet images to obtain a positive sample image set.

[0026] The second filtering subunit is used to filter out images of different pets of the same breed from the historical pet images from multiple angles to obtain a negative sample image set.

[0027] The first update subunit is used to calculate the contrastive learning loss based on the positive sample image set and the negative sample image set, and update the model parameters of the multi-task model based on the contrastive learning loss.

[0028] Optionally, in some embodiments, the first update subunit includes:

[0029] The first calculation module is used to calculate the contrastive learning loss based on the positive sample image set and the negative sample image set;

[0030] The first update module is used to update the parameters of the image encoder and the pet recognition branch according to the contrastive learning loss.

[0031] Optionally, in some embodiments, the second updating unit includes:

[0032] The first acquisition subunit is used to acquire training sample data with multi-task labels. The training sample data includes multiple sample pet images and pet detection labels and pet recognition labels corresponding to each sample pet image.

[0033] The prediction subunit is used to input the sample pet image into the multi-task model to obtain the detection result output by the pet detection branch and the recognition result output by the pet recognition branch.

[0034] The first calculation subunit is used to calculate the pet detection loss based on the detection results and the corresponding pet detection tags;

[0035] The second calculation subunit is used to calculate the pet recognition loss based on the recognition result and the pet recognition tag;

[0036] The second update subunit is used to calculate the target loss based on the pet detection loss and the pet recognition loss, and to update the parameters of the pet detection branch and the pet recognition branch based on the target loss.

[0037] Optionally, in some embodiments, the second update subunit includes:

[0038] The determination module is used to determine the first weighting coefficient corresponding to the pet detection loss and the second weighting coefficient corresponding to the pet recognition loss;

[0039] The second calculation module is used to perform weighted calculation of the pet detection loss and the pet recognition loss based on the first weighting coefficient and the second weighting coefficient to obtain the target loss;

[0040] The second update module is used to update the parameters of the pet detection branch and the pet recognition branch based on the target loss.

[0041] Optionally, in some embodiments, the determining module includes:

[0042] The acquisition submodule is used to acquire the task difficulty and convergence status corresponding to the pet detection branch and the pet recognition branch;

[0043] The determination submodule is used to determine the first weight coefficient corresponding to the pet detection loss and the second weight coefficient corresponding to the pet recognition loss based on the task difficulty and the convergence state.

[0044] Optionally, in some embodiments, the prediction subunit includes:

[0045] The first prediction module is used to input the sample pet image into the image encoder to obtain the output image features;

[0046] The second prediction module is used to process the image features based on the first gating network to obtain pet detection features, and input the pet detection features into the pet detection branch to obtain the detection result;

[0047] The third prediction module is used to process the image features based on the second gating network to obtain pet recognition features, and input the pet recognition features into the pet recognition branch to obtain the recognition result.

[0048] Optionally, in some embodiments, the identification unit includes:

[0049] The training subunit is used to perform knowledge distillation based on the pet recognition branch of the preset multi-task model to obtain the student model;

[0050] A deployment subunit is used to deploy the student model in the edge pet device;

[0051] The recognition subunit is used to call the student model to perform pet recognition on the pet image and obtain the pet recognition result.

[0052] Optionally, in some embodiments, the model training apparatus provided in this application further includes:

[0053] The second acquisition subunit is used to acquire the recognition and correction data uploaded by the user, and generate fine-tuning samples based on the recognition and correction data;

[0054] The fine-tuning subunit is used to periodically fine-tune the student model based on the fine-tuning samples.

[0055] To achieve the above objectives, a third aspect of this application provides an electronic device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the pet recognition method described in the first aspect.

[0056] To achieve the above objectives, a fourth aspect of the present application provides a storage medium storing a computer program that, when executed by a processor, implements the pet identification method described in the first aspect.

[0057] To achieve the above objectives, a fifth aspect of this application provides a computer program product comprising a computer program that is read and executed by a processor of a computer device, causing the computer device to perform the pet identification method described in the first aspect.

[0058] The pet recognition method proposed in this application acquires pet images collected by an edge pet device; calls a preset multi-task model to perform pet recognition on the pet images, and obtains pet recognition results; the multi-task model includes an image encoder, a pet detection branch, and a pet recognition branch. The training process of the multi-task model includes: performing unsupervised pre-training on the image encoder based on historical pet images collected by the edge pet device; selecting a positive sample image set and a negative sample image set from the historical pet images, the positive sample image set including multiple different images of the same pet, and the negative sample image set including multiple images of different pets; updating the model parameters of the multi-task model based on the positive sample image set and the negative sample image set; acquiring training sample data with multi-task labels, the training sample data including multiple sample pet images and pet detection labels and pet recognition labels corresponding to each sample pet image; updating the parameters of the multi-task model based on the training sample data.

[0059] Therefore, the pet recognition method provided in this application trains the pet recognition model through a three-stage learning architecture. Firstly, a large number of historical pet images are used to pre-train the image encoder in the model, enabling it to learn good image feature extraction capabilities. Then, positive and negative samples are used to compare and learn the model, thereby improving its generalization ability. Furthermore, the model in this application employs a multi-task model that includes pet detection and recognition tasks, allowing the detection and recognition tasks to mutually reinforce each other, thus improving the model's pet recognition ability. Therefore, the pet recognition method provided in this application can significantly improve the accuracy of pet recognition. Attached Figure Description

[0060] The accompanying drawings are used to provide a further understanding of the technical solutions of this application and constitute a part of the specification. They are used together with the embodiments of this application to explain the technical solutions of this application and do not constitute a limitation on the technical solutions of this application.

[0061] Figure 1 A flowchart illustrating the pet identification method provided in this application;

[0062] Figure 2 A schematic diagram of the model structure of the multi-task model provided in this application;

[0063] Figure 3 A flowchart illustrating the model training method provided in this application;

[0064] Figure 4 A structural diagram of the student model provided for this application;

[0065] Figure 5 A schematic diagram of the structure of the pet identification device provided in this application;

[0066] Figure 6 A schematic diagram of the structure of the model training device provided in this application;

[0067] Figure 7 This is a schematic diagram of the hardware structure of the computer device provided in the embodiments of this application. Detailed Implementation

[0068] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0069] Before providing a further detailed description of the embodiments of this application, the nouns and terms used in the embodiments of this application are explained, and the nouns and terms used in the embodiments of this application shall be interpreted as follows:

[0070] Contrastive learning aims to teach models to distinguish between similar and dissimilar data samples. During training, the model maximizes the similarity of positive sample pairs while minimizing the similarity of negative sample pairs. A similarity function (such as cosine similarity) is typically used to measure the similarity of sample pairs, and model parameters are updated by optimizing the objective function (such as contrastive loss). Contrastive learning can utilize unlabeled data for pre-training, reducing reliance on labeled data and saving human and material costs. By comparing positive and negative samples, the model can learn the inherent structure and semantic information of the data, obtaining feature representations with strong generalization ability and improving the model's performance on downstream tasks.

[0071] Knowledge Distillation (KD): A model compression technique whose core idea is to transfer knowledge from a large, complex teacher model to a small, simple student model, so that the student model can maintain a small size while achieving performance as close as possible to that of the teacher model.

[0072] In related technologies, various edge pet devices, such as pet feeders, smart pet toilets, and pet toys, only have simple command execution functions, such as feeding pets at set times and in set quantities according to a feeding schedule, and cleaning up pet excrement after detecting that the pet has used the toilet. However, the edge pet device provided in this case, in addition to performing the above functions, can also be equipped with image / video acquisition devices and data processing devices. The edge pet device can acquire pet images or videos in real time, and then use the data processing device to detect the pet's behavior or health status based on the pet images or videos, and then provide corresponding guidance or medical advice based on the detection results.

[0073] In some multi-pet scenarios, such as multi-pet households or pet stores, after the edge pet device collects pet images, it needs to perform pet recognition first. Based on the pet recognition results, it needs to filter out accurate pet images for each pet, and then use the accurate pet images to detect behavioral habits or health status, thereby obtaining accurate detection results.

[0074] However, current pet detection methods typically employ a pet detection model to identify the presence of pets in an image and, if present, their location. Then, a pet recognition model is used to identify the individual pet. This independent or cascaded model architecture handles pet detection and individual recognition tasks separately, failing to fully utilize feature sharing between tasks. This results in insufficient pet recognition accuracy in complex scenes.

[0075] In summary, current methods for pet identification from pet images are not very accurate.

[0076] Based on this, embodiments of this application provide a pet identification method, device, electronic device, and storage medium, which aim to improve the accuracy of pet identification to a certain extent.

[0077] Reference Figure 1 In some embodiments, the pet identification method provided in this application includes, but is not limited to, steps S101 to S102.

[0078] Step S101: Obtain pet images captured by the edge pet device;

[0079] Step S102: Call the preset multi-task model to perform pet recognition on the pet image and obtain the pet recognition result.

[0080] In steps S101 to S102 of this embodiment, the edge pet device can be a pet feeder, smart pet toilet, or similar device as described above. The edge pet device can be equipped with an image acquisition device, such as a camera, to capture images of the pet in the environment. The image acquisition device can capture images in real time, when a pet is detected in the field of view, or when triggered by a specific command. This specific command can be a pet behavior analysis command, a pet health analysis command, or a pet food intake assessment command, etc. These commands can trigger image acquisition of the pet, and then perform tasks such as pet behavior analysis, health analysis, and food intake assessment based on the acquired pet images.

[0081] After acquiring pet images, edge pet devices can first perform pet recognition to identify individual pet information in each image. Then, based on this individual pet information, the captured pet images are categorized according to the different pets, resulting in a set of pet images corresponding to each individual pet. This allows for further downstream tasks such as pet behavior analysis, pet health analysis, or pet food intake assessment based on each individual pet's corresponding set of images.

[0082] In this embodiment, the specific process of pet recognition from pet images involves an edge pet device calling a preset multi-task model to perform pet recognition. This preset multi-task model can be a multi-task model deployed on the edge pet device or a multi-task model deployed on a cloud server. The multi-task model provided in this application includes an image encoder, a pet detection branch, and a pet recognition branch. Figure 2The figure shows a schematic diagram of the multi-task model provided in this application. As shown, the multi-task model includes an image encoder 210, a pet detection branch 220, and a pet recognition branch 230. The pet detection branch 220 and the pet recognition branch 230 share the underlying image encoder 210 for image feature extraction. Specifically, to improve the pet recognition capability, this application also provides a method for training the above-mentioned multi-task model, such as... Figure 3 The diagram shown is a flowchart of the model training method provided in this application, which includes, but is not limited to, steps S301 to S303.

[0083] Step S301: Perform unsupervised pre-training on the image encoder based on historical pet images collected by the edge pet device;

[0084] Step S302: Select a set of positive sample images and a set of negative sample images from historical pet images. The set of positive sample images includes multiple different images of the same pet, and the set of negative sample images includes multiple images of different pets. Update the model parameters of the multi-task model based on the set of positive sample images and the set of negative sample images.

[0085] Step S303: Obtain training sample data with multi-task labels. The training sample data includes multiple sample pet images and pet detection labels and pet recognition labels corresponding to each sample pet image. Update the parameters of the multi-task model based on the training sample data.

[0086] The model training method provided in this application employs an innovative three-stage training process. First, the image encoder used for shared feature extraction undergoes unsupervised pre-training, enabling it to learn the ability to extract rich image features from input images. Then, a contrastive learning method is used to train the model's ability to distinguish between different pets. Finally, a certain amount of labeled samples is used to train the model's ability to identify individual pets. This method does not require a large amount of labeled information, reducing the difficulty of obtaining training sample data. Furthermore, addressing the issue of high similarity among pets of the same breed in pet recognition scenarios, this method combines contrastive learning and pet recognition tasks to improve the accuracy of identifying similar pets of the same breed. In addition, the model provided in this application adopts a multi-task head structure. During end-to-end training with labeled sample data, the simultaneous updating of parameters in two task branches allows for mutual improvement between tasks, thereby enhancing the model's accuracy in pet recognition.

[0087] Specifically, unsupervised pre-training of the image encoder can be performed using historical pet images captured by an edge pet device as training samples. Edge pet devices can be deployed in the pet's living environment to capture a large number of pet images, which are then used to train the image encoder. During unsupervised training of the image encoder based on these historical pet images, self-supervised tasks such as image reconstruction and contrast prediction can be used for pre-training. In related technologies, training pet recognition models typically involves a large amount of labeled data for supervised training. However, supervised training is difficult due to the high difficulty of labeling and obtaining samples, resulting in insufficient sample size. This application uses pet images captured in real-time by an edge pet device to periodically perform unsupervised training on the model's image encoder. This enables the image encoder to learn richer feature representations, and allows its ability to capture pet features to evolve continuously as the pet grows and changes, thus significantly improving the model's ability to capture pet features.

[0088] In some embodiments, a set of positive sample images and a set of negative sample images are selected from historical pet images. The set of positive sample images includes multiple different images of the same pet, and the set of negative sample images includes multiple images of different pets. The model parameters of the multi-task model are updated based on the set of positive sample images and the set of negative sample images, including:

[0089] Multiple images of the same pet taken from different angles were selected from historical pet images to obtain a set of positive sample images;

[0090] A negative sample image set is obtained by selecting images of different pets of the same breed from historical pet images and capturing them from multiple angles.

[0091] The contrastive learning loss is calculated based on the positive and negative sample image sets, and the model parameters of the multi-task model are updated based on the contrastive learning loss.

[0092] In this embodiment, to improve the model's recognition ability in multi-pet environments, i.e., to enhance the model's ability to distinguish between different pets, a contrastive learning method is used to train the model's pet discrimination ability. In pet recognition scenarios, there is a challenging task: identifying different pets of the same breed. Therefore, in this embodiment, when training the model using contrastive learning, some challenging samples can be selected to train the model, thereby improving the model's ability to distinguish between different pets.

[0093] Specifically, multiple different images of the same pet taken from different shooting angles can be selected from historical pet images collected by edge pet devices to form a positive sample image set.

[0094] The system also selects images of different pets of the same breed from historical pet images collected by edge pet devices, taken from multiple angles, to form a negative sample image set. For example, images of different pets of the same breed taken from the face angle or images of different pets of the same breed taken from the body angle can form a negative sample image set.

[0095] After collecting the sets of positive and negative sample images, the contrastive learning loss can be calculated using the sets of positive and negative sample images, and then the contrastive learning loss can be used to train the multi-task model.

[0096] Positive sample pairs are not constrained by specific parts such as facial or body images, but deliberately include images of the same pet from multiple angles and poses, allowing the model to learn invariant features of the individual under various conditions. Similarly, negative sample pairs do not require specific constraints, in order to maximize the model's generalization ability in complex real-world scenarios.

[0097] In some embodiments, a contrastive learning loss is calculated based on a set of positive sample images and a set of negative sample images, and the model parameters of the multi-task model are updated based on the contrastive learning loss, including:

[0098] Calculate the contrastive learning loss based on the positive sample image set and the negative sample image set;

[0099] The parameters of the image encoder and the pet recognition branch are updated based on the contrastive learning loss.

[0100] In this embodiment, after obtaining the positive sample image set and the negative sample image set, the multi-task model provided in this application can be trained unsupervised based on the positive sample image set and the negative sample image set. Specifically, the contrastive learning loss corresponding to the pet recognition task can be calculated based on the positive sample image set and the negative sample image set. Specifically, positive sample pairs can be collected from the positive sample image set and negative sample pairs can be collected from the negative sample image set respectively. Then, the contrastive learning loss is calculated by using the InfoNCE loss method. The goal of this loss is to bring the positive sample pairs closer together and push the negative sample pairs further apart.

[0101] After calculating the contrastive learning loss, the model parameters of the pet recognition branch can be updated based on the contrastive learning loss, thereby pre-training the pet recognition branch and enabling it to distinguish between different pets.

[0102] The second stage of the contrastive learning training process in this application builds upon the unsupervised pre-training of the image encoder in the first stage, enabling the model to learn general representations of pets from different angles, poses, and body parts (such as the face and body). Based on the model's ability to extract accurate general representations of pets, positive and negative samples are used to further update the parameters of the image encoder and the pet recognition branch. This allows the model to further learn accurate pet differentiation capabilities, building upon its already accurate pet image feature extraction capabilities.

[0103] In some embodiments, training sample data with multi-task labels is obtained. The training sample data includes multiple sample pet images and pet detection labels and pet recognition labels corresponding to each sample pet image. The parameters of the multi-task model are updated based on the training sample data, including:

[0104] Obtain training sample data with multi-task labels. The training sample data includes multiple sample pet images and pet detection labels and pet recognition labels corresponding to each sample pet image.

[0105] Input the sample pet images into the multi-task model to obtain the detection results output by the pet detection branch and the recognition results output by the pet recognition branch;

[0106] Calculate pet testing loss based on the test results and the corresponding pet testing tags;

[0107] Calculate the pet recognition loss based on the recognition results and pet identification tags;

[0108] The target loss is calculated based on the pet detection loss and the pet recognition loss, and the parameters of the pet detection branch and the pet recognition branch are updated based on the target loss.

[0109] The image encoder in the multi-task model is pre-trained using a large number of unlabeled pet images captured by edge pet detection devices. Then, positive and negative sample pairs selected from the images captured by the edge pet detection devices are used to comparatively train the image encoder and the pet recognition branch network. Further training sample data with multi-task labels can be obtained. This training sample data can include multiple sample pet images and corresponding pet detection and recognition labels for each image. The pet detection label can be the bounding box of the pet in the image, and the pet recognition label can be the pet ID. A single sample pet image can include multiple pet detection and recognition labels.

[0110] Specifically, after obtaining the training sample data, the pet images included in each training sample data can be first input into the image encoder for image feature extraction. Then, the image features are input into the pet detection branch and the pet recognition branch respectively for pet detection and pet recognition, obtaining the output detection results and recognition results. Further, the pet detection loss can be calculated based on the detection results and the corresponding pet detection labels, and the pet recognition loss can be calculated based on the recognition results and the pet recognition labels. Further, the target loss is calculated based on the pet detection loss and the pet recognition loss, and the parameters of the pet detection branch and the pet recognition branch are further updated based on the target loss to obtain the trained multi-task model.

[0111] In related technologies, pet detection and individual recognition tasks are typically handled separately using independent or cascaded model architectures, preventing feature sharing between tasks. Furthermore, the separate model design increases parameter redundancy, leading to high computational overhead during inference and making deployment difficult on resource-constrained edge devices. In addition, isolated feature learning methods prevent the model from benefiting from multi-task collaboration, thus limiting overall model performance improvement. In contrast, the model architecture and corresponding training method provided in this application integrate pet detection and pet recognition tasks into a single model, which is then trained end-to-end. This enables feature sharing and complementarity within the model, significantly improving the accuracy of pet recognition.

[0112] The multi-task model provided in this application differs from the dual-tower multi-gate hybrid expert network structure. In this application, both the pet detection and pet recognition branches rely on image features output by a pre-trained image encoder as shared features. These shared features possess stronger generalization capabilities and are incorporated into subsequent contrastive learning and multi-task fine-tuning stages, enabling knowledge transfer and complementarity between tasks.

[0113] In some embodiments, a target loss is calculated based on the pet detection loss and the pet recognition loss, and the parameters of the pet detection branch and the pet recognition branch are updated based on the target loss, including:

[0114] Determine the first weighting coefficient corresponding to the pet detection loss and the second weighting coefficient corresponding to the pet recognition loss;

[0115] The target loss is obtained by weighting the pet detection loss and the pet recognition loss based on the first weighting coefficient and the second weighting coefficient.

[0116] The parameters of the pet detection branch and the pet recognition branch are updated based on the target loss.

[0117] In this embodiment, different weights can be set for the multi-task branches to adjust the model's performance. Specifically, a first weight coefficient corresponding to the pet detection loss and a second weight coefficient corresponding to the pet recognition loss can be determined first. Then, the pet detection loss and pet recognition loss are weighted and calculated based on the first and second weight coefficients to obtain the target loss.

[0118] The weight coefficients corresponding to the pet detection loss and pet recognition loss can be set to fixed weight coefficients, such as 0.5 and 0.5, depending on the importance of the task; or, to increase the importance of the pet recognition branch, the weight coefficients can be set to 0.4 and 0.6. In some embodiments, the weight coefficients corresponding to the pet detection loss and pet recognition loss can also be dynamic weights. That is, in the early stage of training, the pet detection loss and pet recognition loss are kept consistent, and then, as the training iterations increase, the weight of the pet detection loss is continuously decreased and the weight of the pet recognition loss is increased, thereby continuously increasing the importance of the pet recognition branch.

[0119] In some embodiments, determining a first weighting coefficient corresponding to the pet detection loss and a second weighting coefficient corresponding to the pet recognition loss includes:

[0120] Obtain the task difficulty and convergence status corresponding to the pet detection branch and the pet recognition branch;

[0121] The first weight coefficient for pet detection loss and the second weight coefficient for pet recognition loss are determined based on the task difficulty and convergence status.

[0122] This application provides an uncertainty-based multi-task loss weighting method that allows the model to automatically learn the relative weights between different tasks. This method uses the inherent uncertainty of each task as the basis for determining the weights, enabling the model to dynamically adjust the contributions of each task based on the difficulty of the pet detection and pet recognition tasks and the convergence status of the corresponding task branches. This adaptive weighting strategy avoids the subjectivity caused by manually set fixed weights, allowing the model to naturally balance the various tasks during training, thereby achieving better training results.

[0123] Specifically, during supervised training of the multi-task model based on training sample data, the task difficulty and convergence status of the pet detection and pet recognition branches can be obtained in real time. Then, based on the task difficulty and convergence status, the first weight coefficient corresponding to the pet detection loss and the second weight coefficient corresponding to the pet recognition loss are determined. Finally, the target loss is calculated based on the first and second weight coefficients, and the model parameters are updated according to the target loss.

[0124] In some embodiments, when performing supervised fine-tuning based on training sample data, a progressive training strategy can also be adopted to avoid interference between tasks, thereby ensuring the accuracy of recognition.

[0125] In some embodiments, a sample pet image is input into a multi-task model to obtain the detection result output by the pet detection branch and the recognition result output by the pet recognition branch, including:

[0126] Input the sample pet image into the image encoder to obtain the output image features;

[0127] The image features are processed based on the first gating network to obtain pet detection features. The pet detection features are then input into the pet detection branch to obtain the detection results.

[0128] The image features are processed using the second gating network to obtain pet recognition features. These pet recognition features are then input into the pet recognition branch to obtain the recognition result.

[0129] In this embodiment, the model structure can also include a gating network for each task branch. This gating network is used to adjust the adaptability between the image encoder output and each task branch. When there is no significant conflict between the pet detection task and the pet recognition task, the output of the image encoder can be directly used as the input features of the pet detection branch and the pet recognition branch, meaning the gating network can be inactive at this time. When significant conflicts are identified between multiple task branches during multi-task fine-tuning, the gating network corresponding to each task branch can be used to separate the feature learning path. In this case, the sample pet image can be input into the image encoder to obtain the output image features. Then, the image features are processed based on the first gating network corresponding to the pet detection branch to obtain pet detection features, and then the pet detection features are input into the pet detection branch to obtain the detection result; and the image features are processed based on the second gating network corresponding to the pet recognition branch to obtain pet recognition features, and then the pet recognition features are input into the pet recognition branch to obtain the recognition result.

[0130] Furthermore, based on the detection results, recognition results, and corresponding pet detection and recognition tags, the corresponding loss values ​​are calculated, and the model parameters are updated based on the loss values.

[0131] In some embodiments, a preset multi-task model is invoked to perform pet recognition on the pet image, and the pet recognition result is obtained, including:

[0132] The student model is obtained by knowledge distillation based on the pet recognition branch of the pre-defined multi-task model.

[0133] Deploying student models in edge pet devices;

[0134] The student model is used to perform pet recognition on pet images, and the pet recognition results are obtained.

[0135] As mentioned earlier, when edge pet devices use preset multi-task models to recognize pet images, they can call either locally deployed multi-task models or models deployed on cloud servers. However, pet recognition of images collected by edge pet devices is generally a sub-task performed within tasks such as pet behavior analysis and pet health analysis based on pet images. The pet images collected in this case are typically everyday images of pet owners. Uploading these images to cloud servers for pet recognition could easily lead to privacy leaks. Furthermore, uploading a large number of pet images to the cloud consumes significant bandwidth and transmission time. Therefore, it is necessary to be able to deploy models locally on edge pet devices to complete pet recognition. However, due to insufficient hardware limitations of edge pet devices, their computing resources are limited, making it impossible to meet the deployment requirements of large-scale models.

[0136] In this embodiment, a knowledge distillation method is employed. First, a complete multi-task model is trained in the cloud. Then, the capabilities of the image encoder and its corresponding pet recognition branch from the multi-task model are transferred to a student model using knowledge distillation, thus training the student model. This student model can then be deployed on an edge pet device. During model training, the pet detection task branch can promote and complement the pet recognition task branch, thereby improving the recognition capability of the pet recognition branch. During model inference, the features output by the image encoder can be directly input into the pet recognition branch to obtain the pet recognition result; that is, the pet detection branch of the model is not needed during inference. Thus, in the distillation learning phase, only the image encoder and the pet recognition branch are distilled to obtain the student model. Figure 4 The diagram shown is a schematic representation of the student model provided in this application. The student model includes a feature extraction layer 410 and a pet recognition layer 420. The feature extraction layer is used to extract image features from pet images, and the pet recognition layer is used to predict pet recognition results based on image features. The feature extraction layer 410 and the pet recognition layer 420 are obtained through distillation learning based on the image encoder 210 and the pet recognition branch 230.

[0137] After deploying the student model obtained through distillation learning on the edge pet device, when a pet image is acquired, the student model deployed on the edge pet device can be directly called to perform pet recognition on the acquired pet image, thereby obtaining accurate pet recognition results.

[0138] In this embodiment, the teacher model deployed in the cloud can be continuously trained using anonymized data uploaded by users, while the student model deployed on the edge pet device can periodically obtain updates from the cloud using an incremental update mechanism to adapt to new pet individuals and changes in characteristics. Specifically, during the distillation learning phase, the student model can learn the intermediate feature representations of the teacher model and mimic the prediction distribution of the teacher model to learn.

[0139] In some embodiments, the model training method provided in this application further includes:

[0140] Obtain the recognition and correction data uploaded by users, and generate fine-tuning samples based on the recognition and correction data;

[0141] The student model is periodically fine-tuned based on the fine-tuning samples.

[0142] In pet management applications, some tasks provide feedback to users regarding pet identification results. For example, in pet behavior analysis tasks, when abnormal pet behavior is detected, the application can provide the identified pet information (e.g., pet ID), abnormal pet information (e.g., destructive behavior), and a corresponding pet image as evidence of the abnormality. Pet owners can then use the provided pet image to determine if the pet in the image matches the pet ID. If they match, the pet identification is accurate; if not, it's inaccurate. In this case, pet owners can provide feedback through a pre-defined feedback interface, such as requesting the accurate pet ID corresponding to the pet image as correction data.

[0143] In this embodiment, the identification and correction data uploaded by users can be collected and summarized, then fine-tuning samples can be generated based on the identification and correction data, and finally the student model can be fine-tuned periodically based on the fine-tuning samples.

[0144] In existing technologies, training pet recognition models typically involves collecting training sample data. However, this method fails to differentiate the importance and quality of training samples, hindering targeted improvements to the model's capabilities. In contrast, this application utilizes the correlation between pet recognition tasks and other tasks, along with interactive features between the task and the user, to collect user feedback on the model's recognition results. From this feedback data, high-quality fine-tuning samples that can be used to specifically improve certain aspects of the model's capabilities can be extracted. These fine-tuning samples are then used to continuously refine the student model, thereby enhancing its pet recognition abilities.

[0145] Reference Figure 5 In some embodiments, this application also provides a pet identification device 500, which includes:

[0146] The acquisition unit 510 is used to acquire pet images collected by the edge pet device;

[0147] The recognition unit 520 is used to call a preset multi-task model to perform pet recognition on pet images and obtain pet recognition results.

[0148] Reference Figure 6 In some embodiments, this application also provides a model training apparatus 600, which includes:

[0149] Pre-training unit 610 is used for unsupervised pre-training of the image encoder based on historical pet images collected by edge pet devices;

[0150] The first update unit 620 is used to filter out a set of positive sample images and a set of negative sample images from historical pet images. The set of positive sample images includes multiple different images of the same pet, and the set of negative sample images includes multiple images of different pets. The model parameters of the multi-task model are updated based on the set of positive sample images and the set of negative sample images.

[0151] The second update unit 630 is used to acquire training sample data with multi-task labels. The training sample data includes multiple sample pet images and pet detection labels and pet recognition labels corresponding to each sample pet image. The parameters of the multi-task model are updated based on the training sample data.

[0152] Optionally, in some embodiments, the first updating unit includes:

[0153] The first filtering subunit is used to filter out multiple different images of the same pet taken from different shooting angles from historical pet images to obtain a positive sample image set.

[0154] The second filtering subunit is used to filter out images of different pets of the same breed from multiple angles from historical pet images to obtain a negative sample image set;

[0155] The first update subunit is used to calculate the contrastive learning loss based on the positive sample image set and the negative sample image set, and to update the model parameters of the multi-task model based on the contrastive learning loss.

[0156] Optionally, in some embodiments, the first update subunit includes:

[0157] The first calculation module is used to calculate the contrastive learning loss based on the positive sample image set and the negative sample image set;

[0158] The first update module is used to update the parameters of the image encoder and the pet recognition branch based on the contrastive learning loss.

[0159] Optionally, in some embodiments, the second updating unit includes:

[0160] The first acquisition subunit is used to acquire training sample data with multi-task labels. The training sample data includes multiple sample pet images and pet detection labels and pet recognition labels corresponding to each sample pet image.

[0161] The prediction subunit is used to input sample pet images into the multi-task model to obtain the detection results output by the pet detection branch and the recognition results output by the pet recognition branch.

[0162] The first calculation subunit is used to calculate the pet detection loss based on the detection results and the corresponding pet detection tags;

[0163] The second calculation subunit is used to calculate the pet recognition loss based on the recognition results and the pet recognition tag.

[0164] The second update subunit is used to calculate the target loss based on the pet detection loss and the pet recognition loss, and to update the parameters of the pet detection branch and the pet recognition branch based on the target loss.

[0165] Optionally, in some embodiments, the second update subunit includes:

[0166] The determination module is used to determine the first weighting coefficient corresponding to the pet detection loss and the second weighting coefficient corresponding to the pet recognition loss.

[0167] The second calculation module is used to perform weighted calculation of pet detection loss and pet recognition loss based on the first weighting coefficient and the second weighting coefficient to obtain the target loss;

[0168] The second update module is used to update the parameters of the pet detection branch and the pet recognition branch based on the target loss.

[0169] Optionally, in some embodiments, the determining module includes:

[0170] The acquisition submodule is used to obtain the task difficulty and convergence status corresponding to the pet detection branch and the pet recognition branch;

[0171] The determination submodule is used to determine the first weight coefficient corresponding to the pet detection loss and the second weight coefficient corresponding to the pet recognition loss based on the task difficulty and convergence status.

[0172] Optionally, in some embodiments, the prediction subunit includes:

[0173] The first prediction module is used to input sample pet images into the image encoder to obtain the output image features;

[0174] The second prediction module is used to process image features based on the first gating network to obtain pet detection features, and input the pet detection features into the pet detection branch to obtain the detection result;

[0175] The third prediction module is used to process image features based on the second gating network to obtain pet recognition features. The pet recognition features are then input into the pet recognition branch to obtain the recognition result.

[0176] Optionally, in some embodiments, the identification unit includes:

[0177] The training subunit is used to perform knowledge distillation on the pet recognition branch of the preset multi-task model to obtain the student model;

[0178] Deployment subunits are used to deploy student models in edge pet devices;

[0179] The recognition subunit is used to call the student model to perform pet recognition on the pet image and obtain the pet recognition result.

[0180] Optionally, in some embodiments, the model training apparatus provided in this application further includes:

[0181] The second acquisition subunit is used to acquire the recognition and correction data uploaded by the user and generate fine-tuning samples based on the recognition and correction data;

[0182] The fine-tuning subunit is used to periodically fine-tune the student model based on fine-tuning samples.

[0183] Reference Figure 7 , Figure 7 The hardware structure of a computer device according to another embodiment is illustrated. The computer device includes:

[0184] The processor 701 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0185] The memory 702 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 702 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 702 and is called and executed by the processor 701 using the pet recognition method of the embodiments of this application.

[0186] The input / output interface 703 is used to implement information input and output;

[0187] The communication interface 704 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0188] Bus 705 transmits information between various components of the device (e.g., processor 701, memory 702, input / output interface 703, and communication interface 704);

[0189] The processor 701, memory 702, input / output interface 703, and communication interface 704 are connected to each other within the device via bus 705.

[0190] This application also provides a computer program product, which includes a computer program. The processor of a computer device reads and executes the computer program, causing the computer device to perform the pet recognition method described above.

[0191] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in this disclosure and the foregoing drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “including,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatuses.

[0192] It should be understood that in this disclosure, "at least one item" means one or more, and "more than one" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0193] It should be understood that in the description of the embodiments of this application, "multiple" means two or more, "greater than", "less than", "exceeding" etc. are understood to exclude the number itself, and "above", "below", "within" etc. are understood to include the number itself.

[0194] In the several embodiments provided in this disclosure, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0195] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0196] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0197] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0198] It should also be understood that the various implementation methods provided in this application can be combined arbitrarily to achieve different technical effects.

[0199] The above is a detailed description of the embodiments of this disclosure. However, this disclosure is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this disclosure. All such equivalent modifications or substitutions are included within the scope defined by the claims of this disclosure.

Claims

1. A pet identification method, characterized in that, The method is applied to edge pet devices, and the method includes: Acquire pet images captured by the edge pet device; A student model is obtained by knowledge distillation based on the pet recognition branch of a preset multi-task model; the student model is deployed in the edge pet device; the student model is called to perform pet recognition on the pet image to obtain the pet recognition result; the student model is periodically fine-tuned based on fine-tuning samples generated by user-uploaded recognition correction data. The multi-task model includes an image encoder, a pet detection branch, and a pet recognition branch. The training process of the multi-task model includes: The image encoder is pre-trained unsupervised based on historical pet images collected by the edge pet device. Multiple images of the same pet taken from different angles are selected from the historical pet images to obtain a positive sample image set; images of different pets of the same breed taken from multiple angles are selected from the historical pet images to obtain a negative sample image set; a contrastive learning loss is calculated based on the positive sample image set and the negative sample image set, and the model parameters of the multi-task model are updated based on the contrastive learning loss; Acquire training sample data with multi-task labels, the training sample data including multiple sample pet images and pet detection labels and pet recognition labels corresponding to each sample pet image, and update the parameters of the multi-task model based on the training sample data.

2. The method according to claim 1, characterized in that, The step of calculating the contrastive learning loss based on the positive sample image set and the negative sample image set, and updating the model parameters of the multi-task model based on the contrastive learning loss, includes: The contrastive learning loss is calculated based on the set of positive sample images and the set of negative sample images. The parameters of the image encoder and the pet recognition branch are updated based on the contrastive learning loss.

3. The method according to claim 1, characterized in that, The step involves acquiring training sample data with multi-task labels, the training sample data including multiple sample pet images and pet detection and pet recognition labels corresponding to each sample pet image, and updating the parameters of the multi-task model based on the training sample data, including: Acquire training sample data with multi-task labels, the training sample data including multiple sample pet images and pet detection labels and pet recognition labels corresponding to each sample pet image; The sample pet image is input into the multi-task model to obtain the detection result output by the pet detection branch and the recognition result output by the pet recognition branch; The pet detection loss is calculated based on the detection results and the corresponding pet detection tags. Calculate the pet recognition loss based on the recognition results and the pet identification tag; The target loss is calculated based on the pet detection loss and the pet recognition loss, and the parameters of the pet detection branch and the pet recognition branch are updated based on the target loss.

4. The method according to claim 3, characterized in that, The step of calculating the target loss based on the pet detection loss and the pet recognition loss, and updating the parameters of the pet detection branch and the pet recognition branch based on the target loss, includes: Determine the first weighting coefficient corresponding to the pet detection loss and the second weighting coefficient corresponding to the pet recognition loss; The target loss is obtained by weighting the pet detection loss and the pet recognition loss based on the first weighting coefficient and the second weighting coefficient. The parameters of the pet detection branch and the pet recognition branch are updated based on the target loss.

5. The method according to claim 4, characterized in that, Determining the first weighting coefficient corresponding to the pet detection loss and the second weighting coefficient corresponding to the pet recognition loss includes: Obtain the task difficulty and convergence status corresponding to the pet detection branch and the pet recognition branch; The first weighting coefficient corresponding to the pet detection loss and the second weighting coefficient corresponding to the pet recognition loss are determined based on the task difficulty and the convergence state.

6. The method according to claim 3, characterized in that, The step of inputting the sample pet image into the multi-task model to obtain the detection result output by the pet detection branch and the recognition result output by the pet recognition branch includes: The sample pet image is input into the image encoder to obtain the output image features; The image features are processed based on the first gating network to obtain pet detection features. The pet detection features are then input into the pet detection branch to obtain the detection result. The image features are processed based on the second gating network to obtain pet recognition features. The pet recognition features are then input into the pet recognition branch to obtain the recognition result.

7. A pet identification device, characterized in that, The device is used in edge pet devices, and the device includes: The acquisition unit is used to acquire pet images collected by the edge pet device; The recognition unit is used to perform knowledge distillation on the pet recognition branch of a preset multi-task model to obtain a student model; deploy the student model in the edge pet device; call the student model to perform pet recognition on the pet image to obtain a pet recognition result; the student model is periodically fine-tuned based on fine-tuning samples generated by user-uploaded recognition correction data; The multi-task model includes an image encoder, a pet detection branch, and a pet recognition branch. The training process of the multi-task model includes: The image encoder is pre-trained unsupervised based on historical pet images collected by the edge pet device. Multiple images of the same pet taken from different angles are selected from the historical pet images to obtain a positive sample image set; images of different pets of the same breed taken from multiple angles are selected from the historical pet images to obtain a negative sample image set; a contrastive learning loss is calculated based on the positive sample image set and the negative sample image set, and the model parameters of the multi-task model are updated based on the contrastive learning loss; Acquire training sample data with multi-task labels, the training sample data including multiple sample pet images and pet detection labels and pet recognition labels corresponding to each sample pet image, and update the parameters of the multi-task model based on the training sample data.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the pet identification method according to any one of claims 1 to 6.

9. A storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the pet identification method according to any one of claims 1 to 6.

10. A computer program product comprising a computer program, characterized in that, The computer program is read and executed by the processor of the computer device, causing the computer device to perform the pet identification method according to any one of claims 1 to 6.

Images