Pet identification method, device and storage medium

Abstract

The present application is applicable to the field of image recognition technology, and provides a pet identity recognition method, device and storage medium, the method comprising: acquiring image data and sound data; wherein the image data comprises pet images, user images and environment images; the sound data comprises pet sounds; segmenting the pet images to obtain a number of pet part images; inputting the number of pet part images into multiple first models to obtain a first feature; obtaining a second feature based on the pet sounds, the user images and the environment images; and accurately determining the identity of the pet based on the first feature and the second feature.

Description

Technical Field

[0001] The present application belongs to the field of image recognition technology, and in particular, relates to a pet identity recognition method, device and storage medium. Background Art

[0002] With the development of society, more and more people keep pets, such as dogs. As a result, pet identification has become increasingly important. For example, pupil recognition or nose print recognition. However, pets are different from humans. They will not actively cooperate with photo taking, which makes it more difficult to obtain high-quality pupil images. Relying solely on pupil recognition or nose print recognition, there are problems with inaccurate pet identification or even failure to identify pets. Summary of the invention

[0003] The embodiments of the present application provide a pet identification method, device and storage medium, which can solve the problem of inaccurate pet identification or the problem of being unable to identify pets in the prior art.

[0004] In a first aspect, an embodiment of the present application provides a pet identification method, comprising:

[0005] Acquire image data and sound data; wherein the image data includes a pet image, a user image, and an environment image; and the sound data includes a pet sound;

[0006] Segmenting the pet image to obtain a plurality of pet part images;

[0007] Inputting a plurality of pet body part images into a plurality of first models to obtain a first feature;

[0008] obtaining a second feature based on the pet sound, the user image, and the environment image;

[0009] Based on the first feature and the second feature, the identity of the pet is determined.

[0010] In a second aspect, an embodiment of the present application provides a pet identification device, comprising:

[0011] An acquisition module, used to acquire image data and sound data; wherein the image data includes pet images, user images and environment images; and the sound data includes pet sounds;

[0012] A segmentation module, used for segmenting the pet image to obtain a plurality of pet part images;

[0013] A first feature acquisition module, used for inputting a plurality of pet part images into a plurality of first models to obtain a first feature;

[0014] A second feature acquisition module, configured to obtain a second feature based on the pet sound, the user image and the environment image;

[0015] A determination module is used to determine the identity of the pet based on the first feature and the second feature.

[0016] In a third aspect, an embodiment of the present application provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the pet identification method described in any one of the first aspects above is implemented.

[0017] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the pet identification method described in any one of the first aspects above is implemented.

[0018] In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product is run on a terminal device, the terminal device executes the pet identification method described in any one of the first aspects above.

[0019] Compared with the prior art, the embodiments of the present application have the following beneficial effects: obtaining image data and sound data; wherein the image data includes pet images, user images and environment images; the sound data includes pet sounds; segmenting the pet images to obtain a number of pet part images; inputting a number of pet part images into a plurality of first models to obtain a first feature; obtaining a second feature based on the pet sounds, the user images and the environment images; determining the identity of the pet based on the first feature and the second feature. Among them, multi-modal image data and sound data provide more identification information, which can reduce misjudgment and improve the accuracy of identity recognition; when certain features (such as images) are affected by the environment (such as light, occlusion), other modalities (such as sound) can provide auxiliary information to ensure the stability of recognition; the confirmation of the pet's identity can be assisted in a variety of ways (such as taking pictures, recording), and the operation is more flexible.

[0020] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0022] Figure 1 It is a flowchart of a pet identification method provided by an embodiment of the present application;

[0023] Figure 2 is a schematic diagram of the structure of a pet identification device provided in one embodiment of the present application;

[0024] Figure 3 It is a structural diagram of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0025] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0026] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0027] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0028] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0029] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0030] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0031] Figure 1 A schematic flow chart of a pet identification method provided by the present application is shown.

[0032] S101, acquiring image data and sound data.

[0033] The image data includes pet images, user images and environment images; and the sound data includes pet sounds.

[0034] S102, segmenting the pet image to obtain a plurality of pet part images.

[0035] In this application, the pet image to be segmented is input into the trained model, and the model will output the classification result of each pixel. The image is segmented into different parts according to the classification result, and the image of each part is extracted, such as: ear image, eye image, mouth image, tail image, etc.

[0036] S103, inputting a plurality of pet body part images into a plurality of first models to obtain a first feature.

[0037] The first model is a lightweight model, which refers to a model that is relatively small in terms of model size, computational complexity, and memory usage. Each first model corresponds to some features.

[0038] In an optional embodiment, the plurality of pet body part images include a nose image and an eye image, and S103 specifically includes:

[0039] Perform feature extraction on the nose image to generate a high-dimensional feature of nose print; perform feature extraction on the eye image to generate a high-dimensional feature of pupil; and concatenate the high-dimensional feature of nose print and the high-dimensional feature of pupil to serve as the first feature.

[0040] In the embodiment of the present application, the first model is used to extract features from the nose image, extract the geometric shape and texture pattern of the nose print, and output a set of high-dimensional feature vectors, namely, the high-dimensional features of the nose print. The first model is used to extract features from the eye image, extract the edge, shape, texture and other features of the pupil, and output a set of high-dimensional feature vectors, namely, the high-dimensional features of the pupil. The high-dimensional features of the nose print and the high-dimensional features of the pupil are concatenated to obtain a new joint feature vector as the first feature.

[0041] The splicing can be a simple front-to-back splicing, that is, the high-dimensional features of the nose print are placed in the front, and the high-dimensional features of the pupil are placed in the back.

[0042] S104: Obtain a second feature based on the pet sound, the user image and the environment image.

[0043] In an optional embodiment, S104 specifically includes: extracting features from the pet sound to obtain time series features of the sound; extracting features from the user image to obtain portrait features; extracting features from the environment image to obtain environment features; and fusing the time series features, the portrait features and the environment features through an attention mechanism to obtain the second feature.

[0044] Among them, feature extraction is an existing feature extraction technology and will not be described in detail here.

[0045] In an optional embodiment, the time series feature, the image feature and the environment feature are fused through an attention mechanism to obtain the second feature, including:

[0046] a, determine the confidence C of the pet's state based on the time series characteristics T ; Determine the confidence level C of the environmental state based on the environmental characteristics E ; Determine the confidence level C of the user identity and user status based on the portrait features I .

[0047] Among them, the time series feature is set as T, the environmental feature is set as E, and the portrait feature is set as I.

[0048] Among them, starting from the confidence of each state, the confidence is used to adjust the weight of each feature. After identifying the pet state, user state and environment state through various features, a confidence can be calculated for each state. This confidence can be obtained through the probability distribution of the model output, fuzzy reasoning system, etc. Specifically:

[0049] The confidence C of the pet's status derived from time series features TFor example, changes in a pet’s heart rate may correspond to multiple potential states, such as anxiety, excitement, etc. The model will give a confidence value (0-1) indicating the certainty that the system believes the pet is in a certain state.

[0050] The confidence level C of the environmental state derived from environmental characteristics E For example, environments with high temperatures and loud noises will have corresponding confidence levels, indicating that the current environment is not conducive to the comfort of the pet.

[0051] The confidence C of the user status obtained through the portrait features I Through facial expression analysis, the system can give a confidence level as to whether the user is anxious.

[0052] b, based on the confidence C of the pet's state T , the confidence level C of the environmental state E and the confidence C of the user state I , determine the pet state modulation factor M T , environmental state modulation factor M E and user state modulation factor M I . Specifically:

[0053] Pet state modulation factor M T :M T = C T + I TU * C I + I TE * C E .

[0054] Environmental state modulation factor M E :M E = C E + I ET * C T + I EU * C I .

[0055] User state modulation factor M I :M I = C I + I IU * C T + I IE * C E .

[0056] Among them, I TU ,I TE ,I ET ,I EU ,I IU ,I IEIt is the interaction coefficient between states, indicating the strength of the correlation between states. It can be a constant or not, can change dynamically according to the specific scenario and model, or can be learned through training data. The following are several ways to handle the interaction coefficient in different scenarios:

[0057] Static constant: In some simple scenarios, the interaction coefficient can be set to a constant, representing the fixed strength of the association between states. The setting of the constant value can be based on experience or prior knowledge. For example:

[0058] Fixed strong interaction: If you determine that your pet's anxiety state is often strongly correlated with environmental noise, you can set I TE Set it to a large constant, such as 0.8.

[0059] Fixed weak interactions: If the pet state is weakly correlated with the user state, I TU Set it to a small constant, such as 0.2.

[0060] Dynamic coefficients, a more flexible approach is to make these interaction coefficients change dynamically according to the current features or states. This approach can use neural networks or other machine learning models to learn these coefficients.

[0061] Adaptive coefficient calculation method:

[0062] Feature correlation calculation: By calculating the similarity or correlation between features in different states, the interaction coefficient is dynamically adjusted. For example, cosine similarity, Pearson correlation coefficient, etc. are used to measure the correlation between two features.

[0063] I TU =Cov(T,U) / σ T σ U , Cov(T,U) is the covariance of the features between the pet state and the user state, σ T and σ U are the standard deviations of pet status and user status features, respectively.

[0064] Neural network adaptive learning: A small neural network can be introduced to learn the interaction influence coefficient. The input is the state characteristics of the pet, environment and user, and the output is the interaction influence coefficient. For example:

[0065] I TU =f θ (T,U), where f θ It is a neural network, the input is the pet state T and the user state U, and the output is the adaptive interaction influence coefficient I TU .

[0066] Among them, only I is described in the above embodiment.TU The relevant calculation for I TE ,I ET ,I EU ,I IU ,I IE The relevant calculations are similar and will not be repeated here.

[0067] c. Calculate the similarity between any two of the time series features, the portrait features, and the environmental features; based on the similarity calculation result, obtain the first weight α of the time series feature. T , the second weight α of the image feature I and the third weight α of the environmental feature E .

[0068] Among them, through the time series features T, environmental features E and portrait features I obtained above, it is necessary to calculate the similarity or correlation between them, so as to determine the importance weight of each feature according to these similarities.

[0069] Calculate the similarity S between time series features and environmental features through dot product TE : Used to determine the correlation between the pet's state and the environment's state. High similarity may mean that the pet's behavior is significantly affected by the environment. For example, if a pet shows an anxious state in a noisy environment (time series features show an increased heart rate and high noise), the pet's anxious state may be related to the noise.

[0070] Among them, S is calculated by dot product TE =T·E T .

[0071] Among them, the first weight α T The calculation formula is:

[0072] .

[0073] Calculate the similarity S between image features and environmental features through dot product IE :Used to determine whether the user's state is affected by the environment. For example, the user may be irritable because the environment is overheated. For example: the user is irritable in a high temperature environment, and the expression data is associated with the ambient temperature data.

[0074] Among them, S is calculated by dot product IE =I·E T .

[0075] Among them, the second weight α I The calculation formula is:

[0076] .

[0077] Calculate the similarity S between time series features and image features through dot product TI : Used to determine the correlation between pet behavior and user emotions. The pet's state and the user's emotions may be highly correlated, especially in interactive scenarios. For example, if the user is anxious and the pet's heart rate also increases, it may mean that the pet is responding to the owner's emotions.

[0078] Among them, S is calculated by dot product TI =T·I T .

[0079] Among them, the third weight α E The calculation formula is:

[0080] .

[0081] d, based on the first weight α T The second weight α I , the third weight α E , the pet state modulation factor M T , the environmental state modulation factor M E and the user state modulation factor M I , and obtain the second feature.

[0082] Among them, the first weight α of the time series feature is obtained T , the second weight α of the image feature I and the third weight α of the environmental characteristics E Then, based on the first weight α T , the second weight α I , the third weight α E , pet state modulation factor M T , environmental state modulation factor M E and user state modulation factor M I , update the above weights. Specifically:

[0083]

[0084] Among them, after obtaining the updated weight, the second feature F is obtained by weighted summing based on the updated weight, time series feature T, environmental feature E, and image feature I combined .

[0085]

[0086] This weighted second feature F combinedIt contains time series, portrait and environment features, and adjusts the weights according to the needs of the current situation. This fusion feature will be used for subsequent tasks (such as classification or decision making).

[0087] S105: Determine the identity of the pet based on the first feature and the second feature.

[0088] In an optional embodiment, S105 specifically includes: merging the first feature and the second feature in a fusion layer, and then performing joint learning through a shared hidden layer to output the identity of the pet.

[0089] Among them, the shared hidden layer is usually a fully connected layer, or a series of deep neural network layers, which accept the fused features as input and learn the interactions and potential relationships between multi-modalities.

[0090] In the embodiment of the present application, the first feature and the second feature are spliced ​​in the fusion layer, and the information of multiple modalities such as image, sound, user and environment are integrated through the splicing operation to form the final comprehensive feature. The fused comprehensive feature is passed to the shared hidden layer of the second model (the model specification of the second model is larger than the model specification of the first model) for multimodal joint learning. The goal is to allow the features of different modalities to interact and optimize in the same latent space, and finally capture the shared information of all modalities.

[0091] After joint learning of the shared hidden layer, the final output features are used to classify the pet's identity. At this point, the model can output the specific pet's identity through a Softmax classifier. The output of Softmax is a probability distribution that represents the model's confidence in each possible pet identity. Finally, by maximizing the probability value of the Softmax output, the pet's identity is determined and the recognition result is output.

[0092] In an embodiment of the present application, a number of pet part images are first input into a plurality of first models to obtain a first feature, and at the same time, a second feature is obtained based on the pet sound, the user image and the environment image. The first feature and the second feature are input into the second model together, and multimodal joint learning is performed to determine the identity of the pet. Among them, multimodal image data and sound data provide more identification information, which can reduce misjudgment and improve the accuracy of identity recognition; when certain features (such as images) are affected by the environment (such as light, occlusion), other modalities (such as sound) can provide auxiliary information to ensure the stability of recognition; the confirmation of the pet's identity can be assisted in a variety of ways (such as taking pictures, recording), and the operation is more flexible.

[0093] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0094] Figure 2 A structural block diagram of a pet identification device provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0095] Reference Figure 2 , the pet identification device comprises:

[0096] An acquisition module, used to acquire image data and sound data; wherein the image data includes pet images, user images and environment images; and the sound data includes pet sounds;

[0097] A segmentation module, used for segmenting the pet image to obtain a plurality of pet part images;

[0098] A first feature acquisition module, used for inputting a plurality of pet part images into a plurality of first models to obtain a first feature;

[0099] A second feature acquisition module, configured to obtain a second feature based on the pet sound, the user image and the environment image;

[0100] A determination module is used to determine the identity of the pet based on the first feature and the second feature.

[0101] It should be noted that the information interaction, execution process and other contents between the above-mentioned modules are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0102] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 3 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a pet identification method is implemented.

[0103] Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0104] In one embodiment, a computer device is provided, comprising: a memory, a processor, and a pet identification program stored in the memory and executable on the processor, wherein the pet identification program, when executed by the processor, implements the steps of the pet identification method described in any of the above embodiments.

[0105] In one embodiment, a computer-readable storage medium is provided, on which a pet identification program is stored. When the pet identification program is executed by a processor, the steps of the pet identification method described in any of the above embodiments are implemented.

[0106] In one embodiment, a computer program product is provided, the computer program product comprising a pet identification program, and the pet identification program, when executed by a processor, implements the steps of the pet identification method as described in any of the above embodiments.

[0107] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0108] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0109] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A pet identification method, characterized in that: include: Acquire image data and sound data; wherein the image data includes a pet image, a user image, and an environment image; and the sound data includes a pet sound; Segmenting the pet image to obtain a plurality of pet part images; Inputting a plurality of pet body part images into a plurality of first models to obtain a first feature; obtaining a second feature based on the pet sound, the user image, and the environment image; Determining the identity of the pet based on the first feature and the second feature; Wherein, the plurality of pet part images include nose images and eye images; The step of inputting a plurality of pet body part images into a plurality of first models to obtain a first feature includes: extracting features from the nose image to generate a high-dimensional feature of a nose print; extracting features from the eye image to generate a high-dimensional feature of a pupil; and concatenating the high-dimensional feature of the nose print and the high-dimensional feature of the pupil as the first feature; The obtaining of the second feature based on the pet sound, the user image and the environment image includes: extracting features of the pet sound to obtain a time series feature T of the sound; extracting features of the user image to obtain a portrait feature I; extracting features of the environment image to obtain an environment feature E; fusing the time series feature T, the portrait feature I and the environment feature E through an attention mechanism to obtain the second feature; The second feature is obtained by fusing the time series feature T, the image feature I and the environment feature E through the attention mechanism, including: Determine the confidence C of the pet's state based on the time series feature T T ; Determine the confidence C of the environmental state based on the environmental feature E E ; Determine the confidence C of the user identity and user status based on the portrait feature I I ; Based on the confidence C of the pet's state T , the confidence level C of the environmental state E and the confidence C of the user state I , determine the pet state modulation factor M T , environmental state modulation factor M E and user state modulation factor M I ; Among them, the pet state modulation factor M T = C T + I TU * C I + I TE * C E , environmental state modulation factor M E = C E + I ET * C T + I EU * C I , user state modulation factor M I = C I + I IU * C T + I IE * C E ; Among them, I TU ,I TE ,I ET ,I EU ,I IU ,I IE is the interaction coefficient between states; Calculate the similarity between any two of the time series feature T, the image feature I and the environment feature E; based on the similarity calculation result, obtain the first weight α of the time series feature T T , the second weight α of the image feature I I and the third weight α of the environmental feature E E ; Among them, the first weight α T The calculation formula is: ; The second weight α I The calculation formula is: ; The third weight α E The calculation formula is: ; S TE =T·E T , S IE =I·E T , S TI =T·I T ; Among them, S TE Indicates the similarity between time series feature T and environmental feature E; S IE Indicates the similarity between image feature I and environment feature E calculated by dot product; S TI Indicates that the similarity between the time series feature T and the image feature I is calculated by dot product; In getting the first weight α of the time series feature T T , the second weight α of image feature I I and the third weight α of the environmental feature E E Then, based on the first weight α T , the second weight α I , the third weight α E , pet state modulation factor M T , environmental state modulation factor M E and user state modulation factor M I , update the above weights, After obtaining the updated weights, the second feature F is obtained by weighted summing based on the updated weights, time series features T, environmental features E, and image features I. combined ; 。 2. The pet identification method according to claim 1, characterized in that: The determining the identity of the pet based on the first feature and the second feature includes: The first feature and the second feature are merged in a fusion layer, and then jointly learned through a shared hidden layer to output the identity of the pet.

3. A pet identification device, characterized in that: include: An acquisition module, used to acquire image data and sound data; wherein the image data includes pet images, user images and environment images; and the sound data includes pet sounds; A segmentation module, used for segmenting the pet image to obtain a plurality of pet part images; A first feature acquisition module, used for inputting a plurality of pet part images into a plurality of first models to obtain a first feature; A second feature acquisition module, configured to obtain a second feature based on the pet sound, the user image and the environment image; A determination module, configured to determine the identity of the pet based on the first feature and the second feature; The plurality of pet body part images include nose images and eye images; the first feature acquisition module is used to: extract features from the nose image to generate a high-dimensional feature of nose print; extract features from the eye image to generate a high-dimensional feature of pupil; and concatenate the high-dimensional feature of nose print and the high-dimensional feature of pupil as the first feature; The second feature acquisition module is used to: extract features from the pet sound to obtain a time series feature T of the sound; extract features from the user image to obtain a portrait feature I; extract features from the environment image to obtain an environment feature E; and fuse the time series feature T, the portrait feature I and the environment feature E through an attention mechanism to obtain the second feature; The second feature is obtained by fusing the time series feature T, the image feature I and the environment feature E through the attention mechanism, including: Determine the confidence C of the pet's state based on the time series feature T T ; Determine the confidence C of the environmental state based on the environmental feature E E ; Determine the confidence C of the user identity and user status based on the portrait feature I I ; Based on the confidence C of the pet's state T , the confidence level C of the environmental state E and the confidence C of the user state I , determine the pet state modulation factor M T , environmental state modulation factor M E and user state modulation factor M I ; Among them, the pet state modulation factor M T = C T + I TU * C I + I TE * C E , environmental state modulation factor M E = C E + I ET * C T + I EU * C I , user state modulation factor M I = C I + I IU * C T + I IE * C E ; Among them, I TU ,I TE ,I ET ,I EU ,I IU ,I IE is the interaction coefficient between states; Calculate the similarity between any two of the time series feature T, the image feature I and the environment feature E; based on the similarity calculation result, obtain the first weight α of the time series feature T T , the second weight α of the image feature I I and the third weight α of the environmental feature E E ; Among them, the first weight α T The calculation formula is: ; The second weight α I The calculation formula is: ; The third weight α E The calculation formula is: ; S TE =T·E T , S IE =I·E T , S TI =T·I T ; Among them, S TE Indicates the similarity between time series feature T and environmental feature E; S IE Indicates the similarity between image feature I and environment feature E calculated by dot product; S TI Indicates that the similarity between the time series feature T and the image feature I is calculated by dot product; In getting the first weight α of the time series feature T T , the second weight α of image feature I I and the third weight α of the environmental feature E E Then, based on the first weight α T , the second weight α I , the third weight α E , pet state modulation factor M T , environmental state modulation factor M E and user state modulation factor M I , update the above weights, After obtaining the updated weights, the second feature F is obtained by weighted summing based on the updated weights, time series features T, environmental features E, and image features I. combined ; 。 4. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 2 is implemented.

5. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 2 is implemented.

Images