Management method based on pet nose patterns and related equipment

Abstract

The invention discloses a pet nose print-based management method and related equipment. The method is applied to an intelligent management device, the intelligent management device is located in an intelligent management system, the intelligent management system further comprises an excretion device, the excretion device is provided with a weight sensor, a smell sensor and camera equipment, and the method comprises the following steps: acquiring a first video of a pet outside the excretion device through the camera equipment; judging whether the pet has an excretion intention or not based on the first video; if the excretion intention exists, acquiring a facial image of the pet through camera equipment; determining nose print features of the pet based on the face image; performing identity authentication on the pet based on the nose pattern features, and if authentication succeeds, starting an excretion device; acquiring the excrement weight of the pet through a weight sensor, and acquiring the excrement smell of the pet through a smell sensor; generating excretion condition data of the pet based on the excretion weight and the excretion smell; and generating a management scheme corresponding to the pet based on the nose print features and the excretion condition data.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a management method based on pet nose prints and related equipment. Background Art

[0002] For pet owners who need to go out to work, they cannot manage their pets' behavior activities at home (such as excretion), and thus cannot accurately grasp their pets' data; and for various types of pet clubs, pet hospitals and other places, there are a large number of pets, and the management of pets' behavior activities requires a lot of human resources, which is rather cumbersome. Summary of the invention

[0003] The present application provides a management method and related equipment based on pet nose prints, which intelligently manages pet's behavioral activities such as excretion behavior. It is simple and efficient, and facilitates users to grasp pet's data such as excretion data in a timely and accurate manner.

[0004] In a first aspect, the present application provides a management method based on pet nose prints, the method is applied to an intelligent management device, the intelligent management device is located in an intelligent management system, the intelligent management system also includes an excretion device, the excretion device is provided with a weight sensor, an odor sensor and a camera device, the method includes:

[0005] Acquire a first video of the pet outside the excretion device by means of a camera device;

[0006] Based on the first video, determining whether the pet has an intention to defecate;

[0007] If there is an intention to defecate, obtain the pet's facial image through a camera device;

[0008] Based on the facial image, determine the pet's nose print features;

[0009] The pet's identity is authenticated based on the nose print feature. If the authentication is successful, the excretion device is turned on;

[0010] The weight of the pet's excrement is obtained through a weight sensor, and the smell of the pet's excrement is obtained through an odor sensor;

[0011] Generate pet excretion data based on excrement weight and excrement odor;

[0012] Generate a corresponding management plan for the pet based on nose print features and excretion data.

[0013] In a second aspect, the present application provides an intelligent management device, which is located in an intelligent management system, and the intelligent management system also includes an excretion device, and the excretion device is provided with a weight sensor, an odor sensor and a camera device, and the device includes: an acquisition unit and a processing unit;

[0014] An acquisition unit, used for acquiring a first video of the pet outside the excretion device through a camera device;

[0015] A processing unit, configured to determine whether the pet has an intention to defecate based on the first video;

[0016] The acquisition unit is further used to acquire a facial image of the pet through a camera device if there is an intention to defecate;

[0017] The processing unit is further used to determine the nose print features of the pet based on the facial image; perform identity authentication on the pet based on the nose print features, and if the authentication is successful, activate the excretion device;

[0018] The acquisition unit is further used to acquire the weight of the pet's excrement through the weight sensor, and acquire the odor of the pet's excrement through the odor sensor;

[0019] The processing unit is also used to generate the pet's excretion data based on the excrement weight and excrement odor; and to generate a corresponding management plan for the pet based on the nose print features and excretion data.

[0020] In a third aspect, the present application provides a management method based on pet nose prints, which is applied to an intelligent management system, the system comprising an intelligent management device and an excretion device, the excretion device being provided with a weight sensor, an odor sensor and a camera device;

[0021] The camera device obtains a first video of the pet outside the excretion device, and sends the first video to the intelligent management device;

[0022] The intelligent management device determines whether the pet has an intention to defecate based on the first video;

[0023] If there is an intention to defecate, the camera device obtains the facial image of the pet and sends the facial image to the intelligent management device;

[0024] The intelligent management device determines the nose print features of the pet based on the facial image; and authenticates the pet based on the nose print features, and if the authentication is successful, activates the excretion device;

[0025] The weight sensor obtains the weight of the pet's excrement;

[0026] The odor sensor picks up the smell of pet excrement;

[0027] The intelligent management device generates the pet's excretion data based on the excrement weight and excrement odor; based on the nose print characteristics and excretion data, it generates a corresponding management plan for the pet.

[0028] In a fourth aspect, the present application provides an intelligent management system, which includes an intelligent management device and an excretion device, wherein the excretion device is provided with a weight sensor, an odor sensor and a camera device;

[0029] A camera device, used for acquiring a first video of the pet outside the excretion device, and sending the first video to the intelligent management device;

[0030] An intelligent management device, for determining whether the pet has an intention to defecate based on the first video;

[0031] If there is an intention to defecate, the camera device is also used to obtain the facial image of the pet and send the facial image to the intelligent management device;

[0032] The intelligent management device is also used to determine the nose print characteristics of the pet based on the facial image; the pet is authenticated based on the nose print characteristics, and if the authentication is successful, the excretion device is activated;

[0033] A weight sensor for obtaining the weight of pet excrement;

[0034] Odor sensor, used to pick up the smell of pet excrement;

[0035] The intelligent management device is also used to generate the pet's excretion data based on the weight and odor of the excrement; and to generate a corresponding management plan for the pet based on the nose print features and the excretion data.

[0036] In a fifth aspect, the present application provides an electronic device, comprising: a processor and a memory, the processor being connected to the memory, the memory being used to store a computer program, and the processor being used to execute the computer program stored in the memory, so that the electronic device executes the method of the first aspect or the third aspect.

[0037] In a sixth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program enables a computer to execute the method of the first aspect.

[0038] In a seventh aspect, the present application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and is computer-operable to cause the computer to execute the method of the first aspect or the third aspect.

[0039] Implementing this application has the following beneficial effects:

[0040] The intelligent management device first obtains a first video of the pet outside the excretion device through a camera device; then the intelligent management device determines whether the pet has an intention to excrete based on the first video; then if there is an intention to excrete, the intelligent management device obtains the facial image of the pet through a camera device; then the intelligent management device determines the nose print characteristics of the pet based on the facial image; then the intelligent management device authenticates the pet based on the nose print characteristics, and if the authentication is successful, the excretion device is turned on; then the intelligent management device obtains the weight of the pet's excrement through a weight sensor, and obtains the odor of the pet's excrement through an odor sensor; then the intelligent management device generates the pet's excretion data based on the excrement weight and the excrement odor; finally, the intelligent management device generates a corresponding management plan for the pet based on the nose print characteristics and the excretion data, and intelligently manages the pet's excretion behavior, which is simple and efficient, and is convenient for users to timely and accurately grasp the pet's excretion data, and determine the management plan corresponding to the pet, thereby realizing intelligent pet management and improving convenience. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0042] Figure 1 A schematic diagram of an intelligent management system provided in an embodiment of the present application;

[0043] Figure 2 A schematic diagram of a process of a pet nose print management method provided in an embodiment of the present application;

[0044] Figure 3 A schematic diagram of a method for generating pet excretion data provided in an embodiment of the present application;

[0045] Figure 4 A schematic diagram of another pet nose print management method provided in an embodiment of the present application;

[0046] Figure 5 A schematic diagram of a scenario of pet nose print-based management provided in an embodiment of the present application;

[0047] Figure 6 A block diagram of the functional units of an intelligent management device provided in an embodiment of the present application;

[0048] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0050] The terms "first", "second", "third" and "fourth" etc. in the specification, claims and drawings of the present application are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0051] Reference to "embodiments" herein means that a particular feature, result, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0052] See also Figure 1 , Figure 1 A schematic diagram of an intelligent management system provided in an embodiment of the present application.

[0053] Figure 1 The intelligent management system shown includes an intelligent management device and an excretion device, wherein the intelligent management device can be an intelligent terminal or a server. Figure 1The intelligent management device is taken as an example for explanation; intelligent terminals such as smart phones, tablet computers, laptops, desktop computers, smart TVs, desktop computers, smart watches, smart car-mounted and other intelligent terminals are not specifically limited in this application; the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, as well as basic cloud computing services such as big data and artificial intelligence platforms, which are not specifically limited in this application; the excretion device is provided with a weight sensor, an odor sensor and a camera device, the weight sensor is used to obtain the weight of the pet's excrement, and the odor sensor is used to obtain the odor of the pet's excrement. The weight sensor and the odor sensor are both arranged in the excretion device, for example, the weight sensor can be arranged in the excretion plate or excretion trough where the pet excretes in the excretion device, and the odor sensor can be arranged on the inner top of the excretion device. Of course, the weight sensor and the odor sensor can also have other settings, which are not limited in this application; the number of excretion devices can be one or more, and each excretion device can be bound to one or more pets, Figure 1 The following mainly uses an excretion device as an example for schematic illustration.

[0054] First, the camera device on the excretion device is used to obtain a first video of the pet outside the excretion device, and send the first video to the intelligent management device; then the intelligent management device is used to determine whether the pet has an intention to excrete based on the first video; if there is an intention to excrete, the camera device is also used to obtain the pet's facial image and send the facial image to the intelligent management device; then the intelligent management device is used to determine the pet's nose print features based on the facial image, and authenticate the pet based on the nose print features, and if the authentication is successful, the excretion device is turned on; then the weight sensor on the excretion device is used to obtain the pet's excrement weight, and the odor sensor on the excretion device is used to obtain the pet's excrement odor; then the intelligent management device is also used to generate the pet's excretion data based on the excrement weight and excrement odor, and generate the pet's corresponding management plan based on the nose print features and excretion data. Further, the intelligent management device can also send the pet's corresponding management plan to the user terminal corresponding to the pet (such as the pet owner's user terminal, the terminal of the pet hospital, etc.) for the pet owner, the pet hospital staff, etc. to view.

[0055] See also Figure 2 , Figure 2A flowchart of a pet nose print management method provided in an embodiment of the present application. The method is applied to the intelligent management system in the above embodiment. The intelligent management system includes an intelligent management device and an excretion device. The excretion device is provided with a weight sensor, an odor sensor and a camera device. The method includes but is not limited to the following steps S201-S215:

[0056] S201. A camera device obtains a first video of a pet outside an excretion device.

[0057] In an embodiment of the present application, a camera device is installed on the excretion device. When the pet is outside the excretion device, such as around the excretion device or at a door, the camera device can capture a first video corresponding to the pet, wherein the duration, format, etc. of the first video are not limited in this application.

[0058] S202: The camera device sends a first video to the intelligent management device.

[0059] S203. The intelligent management device determines whether the pet has the intention to defecate based on the first video. If so, execute step S204.

[0060] In an embodiment of the present application, for example, based on the first video, the length of time the pet stays at the excretion device can be determined; then based on the length of time, it is determined whether the pet has the intention to excrete. For example, if the length of time is greater than a first threshold, it is determined that the pet has the intention to excrete; if the length of time is less than the first threshold, it is determined that the pet does not have the intention to excrete.

[0061] Or in an optional embodiment, after determining the length of time the pet stays at the excretion device based on the first video, the first video may be subjected to image extraction to obtain multiple second images corresponding to the pet, wherein each second image includes the face of the pet; and the first video may be subjected to audio frame extraction to obtain multiple second audio frames corresponding to the pet. Then, based on multiple preset third images, the multiple second images are screened to obtain a fourth image, wherein the multiple third images represent images when the pet has an intention to excrete, each third image includes a facial image when the pet has an intention to excrete, and the fourth image is an image obtained by screening to represent the specific intention of the pet to excrete. For example, the similarity between each second image and each third image is calculated to obtain multiple similarities corresponding to each second image, and then the second image with a similarity greater than the second threshold is determined as the fourth image, that is, the number of fourth images is one or more at this time. Then, based on the length of stay, the fourth image, and multiple second audio frames, the probability of the pet's intention to excrete is predicted, specifically:

[0062] Based on the length of stay and the fourth image, predict the first probability that the pet has the intention to defecate. For example, obtain the first mapping relationship between the preset length of stay and the probability of the intention to defecate, and obtain the second mapping relationship between the image including the pet's face when the pet has the intention to defecate and the probability of the intention to defecate; then based on the length of stay and the first mapping relationship, determine the first sub-probability corresponding to the length of stay; and based on the number of the fourth images and the second mapping relationship, determine the second sub-probability corresponding to the fourth image; then based on the first sub-probability and the second sub-probability, determine the first probability, such as summing the first sub-probability and the second sub-probability, summing and averaging or weighted averaging, etc., to obtain the first probability, which is not limited in this application.

[0063] Alternatively, the first probability can also be obtained by formula (1):

[0064]

[0065] Wherein, P1 represents the first probability, T represents the duration of stay, N represents the number of fourth images, W1 is the first weight corresponding to the duration of stay, and W2 is the second weight corresponding to the fourth image. It should be noted that if the number of fourth images is greater than the preset number threshold, the unit of the duration of stay may be seconds, and if the number of fourth images is less than the preset number threshold, the unit of the duration of stay may be minutes. It can be seen that the longer the pet stays at the excretion device and the more fourth images there are, the greater the probability that the pet has the intention to excrete, and vice versa.

[0066] Then, based on the multiple second audio frames, a second probability that the pet has an intention to defecate is predicted. Exemplarily, multiple preset third audio frames are obtained; then, based on the multiple third audio frames, multiple second audio frames are screened to obtain a fourth audio frame, wherein the fourth audio frame includes any one of the audio frames corresponding to the multiple sounds emitted when the pet has an intention to defecate; then a third mapping relationship between the number of audio frames corresponding to when the pet has an intention to defecate and the probability of the intention to defecate is obtained; then, based on the number of the fourth audio frames and the third mapping relationship, the above-mentioned second probability is determined.

[0067] Or optionally, the second probability can also be determined based on a pre-trained defecation intention recognition model based on pet audio, wherein the training method of the defecation intention recognition model based on pet audio can include: obtaining the pet audio to be trained corresponding to the pet having the defecation intention; performing audio frame extraction on the pet audio to be trained to obtain multiple first training audio frames; then performing feature extraction on each first training audio frame to obtain the first training audio feature corresponding to each first training audio frame; then performing attention fusion on the multiple first training audio features corresponding to the multiple first training audio frames to obtain a first training audio feature matrix; then performing probability prediction based on the first training audio feature matrix to obtain a predicted probability; then determining the first training loss based on the predicted probability and the label corresponding to the pet audio to be trained; then training the pre-trained model based on the first training loss until a trained defecation intention recognition model based on pet audio is obtained. Therefore, multiple second audio frames can be input into the trained pet audio-based defecation intention recognition model to output the second probability, that is, feature extraction is performed on each second audio frame to obtain the first audio feature corresponding to each second audio frame, and then attention fusion is performed based on the multiple first audio features corresponding to the multiple second audio frames to obtain the first audio feature matrix, and then probability prediction is performed based on the first audio frame feature matrix to obtain the above-mentioned second probability.

[0068] Furthermore, after determining the first probability and the second probability, the first probability and the second probability can be summed, averaged, weighted averaged, or any other processing can be performed to obtain the probability of defecation intention; finally, based on the probability of defecation intention, it is determined whether the pet has the intention to defecate. For example, if the probability of defecation intention is greater than a preset probability threshold, it is determined that the pet has the intention to defecate; if the probability of defecation intention is less than the preset probability threshold, it is determined that the pet does not have the intention to defecate.

[0069] S204: The intelligent management device sends a facial image acquisition instruction to the camera device.

[0070] S205: The camera device obtains the facial image of the pet.

[0071] When it is determined that there is an intention to defecate, the intelligent management device can send a facial image acquisition instruction to the camera device, and then the camera device acquires the facial image of the pet in response to the facial image acquisition instruction.

[0072] S206: The camera device sends the facial image to the intelligent management device.

[0073] S207: The intelligent management device determines the nose print features of the pet based on the facial image.

[0074] In an embodiment of the present application, the facial image of the pet is segmented to obtain the nose print image of the pet; then, feature extraction is performed on the nose print image of the pet to obtain the nose print features of the pet.

[0075] S208. The intelligent management device performs identity authentication on the pet based on the nose print feature. If the authentication is successful, execute step S209.

[0076] In an embodiment of the present application, a nose print image bound to an excretion device can be obtained; then, a feature extraction is performed on the nose print image to obtain a first nose print feature; then, based on the nose print feature of the pet and the first nose print feature, the nose print similarity is determined, wherein the present application does not limit the calculation method of the nose print similarity; if the nose print similarity is greater than or equal to the third threshold value, it means that the pet identity authentication is successful, and then step S208 is executed to open the excretion device, such as opening the door of the excretion device. It should be explained that if there are multiple excretion devices, each excretion device can be bound to one or more nose print images, and each nose print image corresponds to a pet. Therefore, when the pet wants to defecate, the nose print verification is added to help the pet develop a standardized and orderly excretion habit, and the pet can be excreted at the excretion device belonging to itself, which is also convenient for users such as pet owners, pet doctors, etc. to manage and monitor the pet in an orderly manner. Especially for many pets, the corresponding relationship between the pet and the excretion device can be realized, so that the pet can develop a habit, and it can also avoid random excretion between pets, causing cross infection and other problems.

[0077] S209, the intelligent management device turns on the excretion device.

[0078] In an embodiment of the present application, if the pet identity authentication is successful, the intelligent management device may send an opening instruction to the excretion device, and accordingly, the excretion device responds to the opening instruction and opens.

[0079] S210: The weight sensor obtains the weight of the pet's excrement.

[0080] S211. The weight sensor sends the weight of the excrement to the intelligent management device.

[0081] S212: The odor sensor obtains the odor of the pet's excrement.

[0082] After the excretion device is turned on, if the pet enters the excretion device to excrete, the weight of the pet's excrement can be obtained through the weight sensor on the excretion device, and the smell of the pet's excrement can be obtained through the odor sensor.

[0083] S213. The odor sensor sends the excrement odor to the intelligent management device.

[0084] It should be noted that the present application does not limit the order in which step S210 and step S212 are executed.

[0085] S214. The intelligent management device generates pet excretion data based on the weight and smell of the excrement.

[0086] For example, see Figure 3 , Figure 3 A schematic diagram of a method for generating pet excretion data provided in an embodiment of the present application. The method is applied to the above-mentioned intelligent management device, and the method includes but is not limited to steps S301-S304:

[0087] S301. Generate first sub-excretion situation data based on excrement weight and excrement odor.

[0088] In an embodiment of the present application, the concentration of the main odor components (such as ammonia and hydrogen sulfide) can be determined based on the odor of the excrement, and then the first sub-excretion situation data can be generated based on the excrement weight and the concentration of the main odor components, that is, the first sub-excretion situation data includes the excrement weight and the concentration of the main odor components.

[0089] S302, obtaining a video of the pet defecating in the excretion device, and obtaining an image of the pet's excrement.

[0090] In an embodiment of the present application, the camera device can not only obtain the first video outside the excretion device, but also obtain the excretion video of the pet in the excretion device, and obtain the image of the pet's excrement. For example, the camera device can be a 360-degree camera, a rotatable camera, etc., or optionally, the camera device can also include two cameras, one camera is used to obtain the above-mentioned first video, and the other camera is used to obtain the excretion video of the pet in the excretion device, and obtain the image of the pet's excrement. The present application does not limit the camera device.

[0091] S303: Generate second sub-excretion situation data based on the excretion image and the excretion video.

[0092] Exemplarily, step S303 may include but is not limited to steps A1-A3:

[0093] A1: Based on the excretion video, determine the target emotional information that represents the pet during the excretion process.

[0094] For example, an excretion video is input into a pre-trained emotion recognition model to obtain target emotion information of the pet during the excretion process. The training method of the emotion recognition model includes: obtaining an excretion video to be trained; then extracting audio frames from the excretion video to be trained in time sequence to obtain multiple second training audio frames; then performing feature fusion of the multiple second training audio frames in time sequence dimension, such as graph attention mechanism, to obtain a second training audio feature matrix; then performing emotion classification based on the second training audio feature matrix to obtain the classification probability of each emotion category, and then determining the emotion category with a classification probability greater than a preset classification probability threshold as the emotion category corresponding to the excretion video to be trained; then determining a second training loss based on the emotion category corresponding to the excretion video to be trained and the label corresponding to the excretion video to be trained, and then training the pre-trained model based on the second training loss until a trained emotion recognition model is obtained. Therefore, the defecation video can be input into a trained emotion recognition model to output target emotion information representing the pet's defecation process, and the target emotion information can include one or more emotion categories. The principle of inputting the defecation video into a trained emotion recognition model to output target emotion information representing the pet's defecation process is similar to the above-mentioned principle of obtaining the emotion category corresponding to the defecation video to be trained based on the defecation video to be trained, and will not be repeated here.

[0095] Alternatively, step A1 may include but is not limited to steps A11-A14:

[0096] A11: Segment the excretion video to obtain a second video and a third video.

[0097] Among them, the second video is a video of the pet defecating, and the third video is a video of the pet after defecating.

[0098] A12: Emotion classification is performed based on the second video to obtain a first predicted probability of each emotion category of the pet during defecation.

[0099] For example, the second video can be classified based on the pre-trained emotion recognition model to obtain the first predicted probability of the pet in each emotion category during defecation. Alternatively, when determining the first predicted probability of the pet in each emotion category during defecation, the second video can also be subjected to audio frame extraction to obtain multiple first audio frames, and the second video can be subjected to image extraction to obtain multiple first images corresponding to the multiple first audio frames, wherein the multiple first audio frames correspond to the multiple first images one-to-one, for example, the first audio frames and the first images at the same time can correspond; then, feature extraction is performed on each audio frame to obtain a first feature matrix, and feature extraction is performed on each image to obtain a second feature matrix; then, the feature representation of each audio frame in the first feature matrix and the feature representation of the image corresponding to each audio frame in the second feature matrix are fused to obtain a third feature matrix, for example, the feature representation of each audio frame in the first feature matrix and the feature representation of the image corresponding to each audio frame in the second feature matrix can be fused by any method such as splicing, addition, etc. to obtain the third feature matrix.

[0100] Or optionally, when determining the third feature matrix, the first feature matrix may be subjected to attention processing to obtain a first attention matrix, and the second feature matrix may be subjected to attention processing to obtain a second attention matrix; and then the feature representation of each audio frame in the first attention matrix and the feature representation of the image corresponding to each audio frame in the second attention matrix may be fused, such as by splicing, addition, etc., to obtain the third feature matrix.

[0101] Alternatively, when determining the third feature matrix, after performing attention processing on the first feature matrix to obtain the first attention matrix and performing attention processing on the second feature matrix to obtain the second attention matrix, the feature representation of each audio frame in the first attention matrix and the feature representation of the image corresponding to each audio frame in the second attention matrix are fused, such as by splicing, adding, etc., to obtain a first fused feature matrix; and the first attention matrix is ​​subjected to graph convolution processing to obtain a first aggregate feature matrix, and the second attention matrix is ​​subjected to graph convolution processing to obtain a second aggregate feature matrix; then the first aggregate feature matrix and the second aggregate feature matrix are fused, such as by splicing, adding, etc., the feature representation of each audio frame in the first aggregate feature matrix and the feature representation of the image corresponding to each audio frame in the second aggregate feature matrix are fused, such as by splicing, adding, etc., to obtain a second fused feature matrix; then the first fused feature matrix and the second fused feature matrix are fused to obtain the above-mentioned third feature matrix, such as by splicing, adding, etc., the feature representation of each audio frame in the first fused feature matrix and the feature representation of the image corresponding to each audio frame in the second fused feature matrix. It should be noted that the present application does not limit the method of obtaining the third feature matrix.

[0102] Then, a first adjacency matrix corresponding to the multiple first audio frames is obtained, and a second adjacency matrix corresponding to the multiple first images is obtained. For example, the first adjacency matrix and the second adjacency matrix can be undirected, and the audio frame nodes or image nodes that are adjacent in time sequence are connected according to the timing information corresponding to the audio frames or images, so as to obtain the corresponding adjacency matrix.

[0103] Finally, based on the third feature matrix, the first adjacency matrix, and the second adjacency matrix, the first predicted probability of the pet in each emotion category during excretion is determined. Exemplarily, based on the third feature matrix, the target feature weights corresponding to each audio frame and each image are determined, such as using an activation function such as a sigmoid function to activate the third feature matrix, and obtaining the third feature weights corresponding to each audio frame or each image, which are used to characterize the importance of different time nodes; then, based on the target feature weights corresponding to each audio frame and each image, each feature representation in the third feature matrix is ​​weighted to obtain a fourth feature matrix.

[0104] Then, for the i-th first feature processing, the i-th first feature processing is performed based on the first adjacency matrix, the fifth feature matrix obtained by the i-1-th first feature processing, and the feature representation of the neighboring audio frames of each audio frame in the fifth feature matrix corresponding to the i-1-th first feature processing, until multiple first feature processings are performed to obtain a sixth feature matrix, wherein the i-th first feature processing is any one of the multiple first feature processings, and when i=1, the fifth feature matrix is ​​the first feature matrix; in addition, the neighboring audio frames of each audio frame are the neighboring nodes of each audio frame, and the neighboring audio frames of each audio frame can be determined based on the first adjacency matrix. Among them, the i-th first feature processing can be specifically:

[0105] The feature representations of the neighboring audio frames of each audio frame in the fifth feature matrix corresponding to the i-1th first feature processing are feature aggregated to obtain the first aggregated features corresponding to each audio frame in the i-th first feature processing. For example, for the z-th audio frame in the fifth feature matrix corresponding to the i-1th first feature processing, the feature representations of the neighboring audio frames of the z-th audio frame, the degree matrix corresponding to the first adjacency matrix, and the unit matrix corresponding to the first adjacency matrix are multiplied to obtain the aggregated features corresponding to the z-th audio frame. Similarly, the aggregated features corresponding to each audio frame in the i-th first feature processing can be obtained. Then, the first adjacency matrix, the unit matrix corresponding to the first adjacency matrix, and the fifth feature matrix obtained by the i-1th first feature processing are multiplied to obtain the second aggregated features corresponding to each audio frame in the i-th first feature processing. Then, the first aggregated features corresponding to each audio frame in the i-th first feature processing and the second aggregated features corresponding to each audio frame in the i-th first feature processing are added to obtain the fifth feature matrix corresponding to the i-th first feature processing, until the first feature processing is performed multiple times to obtain the sixth feature matrix.

[0106] At the same time, for the k-th second feature processing, the k-th second feature processing is performed based on the second adjacency matrix, the seventh feature matrix obtained by the k-1-th second feature processing, and the feature representation of the neighbor image of each image in the seventh feature matrix corresponding to the k-1-th second feature processing, until multiple second feature processings are performed to obtain an eighth feature matrix, wherein the k-th second feature processing is any one of the multiple second feature processings, and when i=1, the seventh feature matrix is ​​the second feature matrix, wherein the principle of obtaining the eighth feature matrix is ​​similar to that of obtaining the sixth feature matrix mentioned above, and will not be repeated here.

[0107] Then, classification is performed based on the fourth feature matrix, the sixth feature matrix, and the eighth feature matrix to obtain the first predicted probability of the pet in each emotion category during excretion. For example, the fourth feature matrix, the sixth feature matrix, and the eighth feature matrix are first fused according to the dimensions of the audio frame or the image, such as splicing, adding, etc., to obtain the third fused feature matrix; and then emotion classification is performed based on the third fused feature matrix to obtain the first predicted probability of the pet in each emotion category during excretion.

[0108] A13: Emotion classification is performed based on the third video to obtain a second predicted probability of the pet in each emotion category after defecation.

[0109] Similarly, the principle of classifying the emotions of the third video to obtain the second predicted probability of the pet in each emotion category after defecation is similar to the principle of classifying the emotions of the second video to obtain the first predicted probability of the pet in each emotion category during defecation, and will not be repeated here.

[0110] A14: Determine target emotion information based on the first prediction probability and the second prediction probability of the pet in each emotion category.

[0111] Exemplarily, the first predicted probability and the second predicted probability of the pet under each emotion category can be summed, averaged, weighted averaged, etc. according to the emotion category to obtain the third predicted probability of the pet under each emotion category, and then the emotion category with the third predicted probability greater than or equal to the fourth threshold is determined as the above-mentioned target emotion information, that is, the target emotion information can include one or more emotion categories at this time. Or optionally, the emotion category with the first predicted probability greater than or equal to the fourth threshold and the emotion category with the second predicted probability greater than or equal to the fourth threshold can also be determined as the target emotion information, that is, at this time, the target emotion information includes the emotion category of the pet during excretion and the emotion category after excretion, and the emotion category during excretion and the emotion category after excretion may also have the same emotion. Or optionally, the emotion category with the first predicted probability and / or the second predicted probability greater than or equal to the fourth threshold can also be determined as the target emotion information. It should be noted that the present application does not limit the method for determining the target emotion information.

[0112] A2: Determine the pet's excrement status based on the excrement image.

[0113] Exemplarily, the shape and color of the pet's excrement can be determined based on the excrement image, and then the pet's excrement status can be generated based on the shape and color of the excrement, that is, the excrement status includes the shape and color of the excrement. The shape and color of the excrement can be described below: (1) Normal shape: soft but not thin, complete shape; color: usually brown, uniform color; (2) Loose stool shape: strong fluidity, incomplete shape, usually liquid; color: yellow, brown or green, uneven color; (3) Hard stool shape: hard, usually spherical or dry block; color: brown, dark; (4) Bloody stool shape: bright red or dark red, usually accompanied by other abnormal conditions; color: obvious blood stains, related to food, gastrointestinal bleeding and other factors; (5) Mucous stool shape: viscous, with obvious mucus secretion; color: usually yellow or transparent; etc., which are not elaborated one by one in this application.

[0114] A3: Generate the second sub-excretion situation data based on the emotional information and excretion status.

[0115] In the embodiment of the present application, the second sub-excretion situation data can be directly generated based on the emotional information and the excretion status, that is, the second sub-excretion situation data includes the emotional information and the excretion status.

[0116] S304: Generate the pet's excretion situation data based on the first sub-excretion situation data and the second sub-excretion situation data.

[0117] Furthermore, the first sub-excretion situation data and the second sub-excretion situation data may be combined and sorted to obtain the pet's excretion situation data.

[0118] S215. The intelligent management device generates a management plan corresponding to the pet based on the nose print features and excretion data.

[0119] Exemplarily, based on the nose print feature, a plurality of first historical excretion data corresponding to the nose print feature can be obtained; then, based on the excretion data, a plurality of historical excretion data are screened for similarity to obtain a plurality of second historical excretion data; then, a historical diet plan of the pet corresponding to each second historical excretion data is obtained; then, based on the pet's feedback on each historical diet plan, the diet management plan of the pet under the excretion data is determined, such as determining the historical diet plan with the largest feedback as the diet management plan of the pet under the excretion data. Alternatively, a plurality of historical diet plans with a feedback degree greater than or equal to a preset threshold are obtained, and then the plurality of historical diet plans are split and randomly recombined to obtain a plurality of new diet plans; then, the quality score of each new diet plan is calculated, such as calculating the diet similarity between each new diet plan and the above-mentioned historical diet plan with the largest feedback, and then obtaining a fourth mapping relationship between the diet similarity and the quality score, and based on the diet similarity and the fourth mapping relationship corresponding to each new diet plan, the quality score corresponding to each new diet plan is determined; then, based on the quality score of each new diet plan, the management plan corresponding to the pet is determined, such as determining the diet plan with the highest quality score as the management plan corresponding to the pet. It should be noted that the present application does not limit the method of generating a management plan corresponding to a pet.

[0120] It should be noted that when the intelligent management device detects that the pet has defecated and left the excretion device based on the data obtained by the camera equipment, the intelligent management device can send a closing command to the excretion device, and then the excretion device responds to the closing command and closes, such as locking the automatic door on the excretion device.

[0121] It can be seen that in the embodiment of the present application, the intelligent management device first obtains a first video of the pet outside the excretion device through a camera device; then the intelligent management device determines whether the pet has the intention to excrete based on the first video; then if there is the intention to excrete, the intelligent management device obtains the facial image of the pet through a camera device; then the intelligent management device determines the nose print characteristics of the pet based on the facial image; then the intelligent management device authenticates the identity of the pet based on the nose print characteristics, and if the authentication is successful, the excretion device is turned on; then the intelligent management device obtains the weight of the pet's excrement through a weight sensor, and obtains the odor of the pet's excrement through an odor sensor; then the intelligent management device generates the pet's excretion data based on the excrement weight and the excrement odor; finally, the intelligent management device generates a corresponding management plan for the pet based on the nose print characteristics and the excretion data, and intelligently manages the pet's excretion behavior, which is simple and efficient, and is convenient for users to timely and accurately grasp the pet's excretion data, and determine the management plan corresponding to the pet, so as to realize intelligent pet management and improve convenience.

[0122] See also Figure 4 , Figure 4 A flowchart of another pet nose print management method provided in an embodiment of the present application. The method is applied to the above-mentioned intelligent management device, that is, the execution subject is the intelligent management device, the intelligent management device is located in the intelligent management system, and the intelligent management system also includes an excretion device, which is provided with a weight sensor, an odor sensor and a camera device. The method includes but is not limited to the following steps S401-S408:

[0123] S401. Obtain a first video of the pet outside the excretion device through a camera device.

[0124] S402: Based on the first video, determine whether the pet has the intention to defecate.

[0125] S403: If there is an intention to defecate, obtain the pet's facial image through a camera device.

[0126] S404: Determine the nose print features of the pet based on the facial image.

[0127] S405: Authenticate the pet based on the nose print feature. If the authentication is successful, activate the excretion device.

[0128] S406: Obtain the weight of the pet's excrement through a weight sensor, and obtain the odor of the pet's excrement through an odor sensor.

[0129] S407: Generate pet excretion data based on excrement weight and excrement odor.

[0130] S408: Generate a management plan corresponding to the pet based on the nose print features and excretion data.

[0131] It should be noted that the specific implementation principles of steps S401-S408 can refer to the above Figure 2 The principle explanation of the corresponding embodiment in the pet nose print-based management method provided in the embodiment will not be repeated here.

[0132] See also Figure 5 , Figure 5 A schematic diagram of a pet nose print management scenario provided in an embodiment of the present application. The scenario includes an intelligent management device, an excretion device, and a user device; first, the excretion device is closed, i.e. Figure 5 The door of the excretion device shown is closed. When the pet is outside the excretion device, the first video of the pet outside the excretion device is obtained by the camera device on the excretion device; then the camera device sends the first video to the intelligent management device; then the intelligent management device determines whether the pet has the intention to excrete based on the first video; if there is the intention to excrete, the intelligent management device can send a facial image acquisition instruction to the camera device; then the camera device obtains the facial image of the pet in response to the facial image acquisition instruction; then the camera device sends the facial image of the pet to the intelligent management device; then the intelligent management device determines the nose print features of the pet based on the facial image; then the intelligent management device authenticates the pet based on the nose print features; if the authentication is successful, the intelligent management device sends an opening instruction to the excretion device, and then the excretion device responds to the opening instruction to open the door of the excretion device. Open; then the pet can enter the excretion device to defecate; when it is detected that the pet has finished defecation and left the excretion device, the excretion device is closed, that is, the door of the excretion device is closed, and the weight of the pet's excrement is obtained through the weight sensor, and the smell of the pet's excrement is obtained through the odor sensor; then the weight sensor sends the weight of the excrement to the intelligent management device, and the odor sensor sends the odor of the excrement to the intelligent management device; then the intelligent management device generates the pet's excretion data based on the excrement weight and the excrement odor; then the intelligent management device generates a corresponding management plan for the pet based on the nose print features and the excretion data; further, the intelligent management device can also send the management plan corresponding to the pet to the user device (that is, the device of the user related to the pet) for the user to view and understand the pet's excretion data and other data.

[0133] See also Figure 6 , Figure 6 This is a functional unit composition block diagram of an intelligent management device provided in an embodiment of the present application. The intelligent management device includes: an acquisition unit 601 and a processing unit 602;

[0134] An acquisition unit 601 is used to acquire a first video of the pet outside the excretion device through a camera device;

[0135] The processing unit 602 is used to determine whether the pet has an intention to defecate based on the first video;

[0136] The acquisition unit 601 is further used to acquire a facial image of the pet through a camera device if there is an intention to defecate;

[0137] The processing unit 602 is further used to determine the nose print features of the pet based on the facial image; perform identity authentication on the pet based on the nose print features, and if the authentication is successful, activate the excretion device;

[0138] The acquisition unit 601 is further used to acquire the weight of the pet's excrement through the weight sensor, and acquire the odor of the pet's excrement through the odor sensor;

[0139] The processing unit 602 is also used to generate the pet's excretion data based on the excrement weight and excrement odor; and to generate a corresponding management plan for the pet based on the nose print features and the excretion data.

[0140] In one embodiment of the present application, in terms of generating the pet's excretion situation data based on the excretion weight and the excretion odor, the processing unit 602 is also used to generate the first sub-excretion situation data based on the excretion weight and the excretion odor; the acquisition unit 601 is also used to acquire the excretion video of the pet in the excretion device, and acquire the pet's excretion image; the processing unit 602 is also used to generate the second sub-excretion situation data based on the excretion image and the excretion video; the pet's excretion situation data is generated based on the first sub-excretion situation data and the second sub-excretion situation data.

[0141] In one embodiment of the present application, in terms of generating the second sub-excretion situation data based on the excretion image and the excretion video, the processing unit 602 is specifically configured to:

[0142] Based on the defecation video, determine the target emotional information that represents the pet during the defecation process;

[0143] Determine the excrement status of the pet based on the excrement image;

[0144] Based on the emotional information and the excretion status, the second sub-excretion situation data is generated.

[0145] In one embodiment of the present application, in determining target emotional information representing the pet during the defecation process based on the defecation video, the processing unit 602 is specifically configured to:

[0146] Segment the excretion video to obtain a second video and a third video, wherein the second video is a video of the pet excreting, and the third video is a video of the pet after excreting;

[0147] Performing emotion classification based on the second video to obtain a first prediction probability of the pet in each emotion category during defecation;

[0148] Performing emotion classification based on the third video to obtain a second predicted probability of the pet in each emotion category after defecation;

[0149] The target emotion information is determined based on the first prediction probability and the second prediction probability of the pet in each emotion category.

[0150] In one embodiment of the present application, in terms of performing emotion classification based on the second video to obtain a first predicted probability of the pet in each emotion category during defecation, the processing unit 602 is specifically configured to:

[0151] Extracting audio frames from the second video to obtain a plurality of first audio frames, and extracting images from the second video to obtain a plurality of first images corresponding to the plurality of first audio frames;

[0152] Performing feature extraction on each audio frame to obtain a first feature matrix, and performing feature extraction on each image to obtain a second feature matrix;

[0153] The feature representation of each audio frame in the first feature matrix and the feature representation of the image corresponding to each audio frame in the second feature matrix are merged to obtain a third feature matrix;

[0154] Acquire a first adjacency matrix corresponding to the plurality of first audio frames, and acquire a second adjacency matrix corresponding to the plurality of first images;

[0155] Based on the third feature matrix, the first adjacency matrix, and the second adjacency matrix, a first predicted probability of the pet in each emotion category during defecation is determined.

[0156] In one embodiment of the present application, in determining the first predicted probability of the pet in each emotion category during defecation based on the third feature matrix, the first adjacency matrix, and the second adjacency matrix, the processing unit 602 is specifically configured to:

[0157] Determining target feature weights corresponding to each audio frame and each image based on the third feature matrix;

[0158] Based on the target feature weights corresponding to each audio frame and each image, weight each feature representation in the third feature matrix to obtain a fourth feature matrix;

[0159] For the i-th first feature processing, the i-th first feature processing is performed based on the first adjacency matrix, the fifth feature matrix obtained by the i-1-th first feature processing, and the feature representation of the neighboring audio frames of each audio frame in the fifth feature matrix corresponding to the i-1-th first feature processing, until multiple first feature processings are performed to obtain a sixth feature matrix, wherein the i-th first feature processing is any one of the multiple first feature processings, and when i=1, the fifth feature matrix is ​​the first feature matrix;

[0160] For the k-th second feature processing, the k-th second feature processing is performed based on the second adjacency matrix, the seventh feature matrix obtained by the k-1-th second feature processing, and the feature representation of the neighbor image of each image in the seventh feature matrix corresponding to the k-1-th second feature processing, until multiple second feature processings are performed to obtain an eighth feature matrix, wherein the k-th second feature processing is any one of the multiple second feature processings, and when i=1, the seventh feature matrix is ​​the second feature matrix;

[0161] Classification is performed based on the fourth characteristic matrix, the sixth characteristic matrix, and the eighth characteristic matrix to obtain the first predicted probability of the pet in each emotion category during defecation.

[0162] In one embodiment of the present application, in determining whether the pet has an intention to defecate based on the first video, the processing unit 602 is specifically configured to:

[0163] Based on the first video, determining the length of time the pet stays at the excretion device;

[0164] Performing image extraction on the first video to obtain a plurality of second images corresponding to the pet;

[0165] Extracting audio frames from the first video to obtain a plurality of second audio frames corresponding to the pet;

[0166] Based on a plurality of preset third images, the plurality of second images are screened to obtain a fourth image, wherein the plurality of third images represent images when the pet has an intention to defecate;

[0167] Predicting the probability of the pet's defecation intention based on the stay duration, the fourth image, and the plurality of second audio frames;

[0168] Based on the probability of defecation intention, determine whether the pet has defecation intention.

[0169] In a specific implementation, the acquisition unit 601 and the processing unit 602 described in the embodiment of the present invention may execute other implementations described in the embodiment of the pet nose print-based management method provided in the embodiment of the present invention, which will not be described in detail here.

[0170] See also Figure 7 , Figure 7This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 7 As shown, the electronic device 700 includes a transceiver 701, a processor 702 and a memory 703. They are connected via a bus 704. The memory 703 is used to store computer programs and data, and can transmit the data stored in the memory 703 to the processor 702.

[0171] The processor 702 is used to read the computer program in the memory 703 and perform the following operations:

[0172] Control the transceiver 701 to obtain a first video of the pet outside the excretion device through a camera device;

[0173] Based on the first video, determining whether the pet has an intention to defecate;

[0174] If there is an intention to defecate, the transceiver 701 is controlled to obtain a facial image of the pet through a camera device;

[0175] Based on the facial image, determine the pet's nose print features;

[0176] The pet's identity is authenticated based on the nose print feature. If the authentication is successful, the excretion device is turned on;

[0177] Controlling the transceiver 701 to obtain the weight of the pet's excrement through the weight sensor, and to obtain the smell of the pet's excrement through the smell sensor;

[0178] Generate pet excretion data based on excrement weight and excrement odor;

[0179] Generate a corresponding management plan for the pet based on nose print features and excretion data.

[0180] In one embodiment of the present application, in terms of generating pet excretion data based on excrement weight and excrement odor, the processor 702 is specifically configured to perform the following steps:

[0181] Generate first sub-excretion situation data based on the excrement weight and excrement odor;

[0182] Obtaining a video of a pet defecating in an excretion device, and obtaining an image of the pet's excrement;

[0183] Based on the excrement image and the excretion video, generating second sub-excretion situation data;

[0184] The excretion situation data of the pet is generated based on the first sub-excretion situation data and the second sub-excretion situation data.

[0185] In one embodiment of the present application, in terms of generating the second sub-excretion situation data based on the excretion image and the excretion video, the processor 702 is specifically configured to perform the following steps:

[0186] Based on the defecation video, determine the target emotional information that represents the pet during the defecation process;

[0187] Determine the excrement status of the pet based on the excrement image;

[0188] Based on the emotional information and the excretion status, the second sub-excretion situation data is generated.

[0189] In one embodiment of the present application, in determining target emotional information representing the pet during the defecation process based on the defecation video, the processor 702 is specifically configured to perform the following steps:

[0190] Segment the excretion video to obtain a second video and a third video, wherein the second video is a video of the pet excreting, and the third video is a video of the pet after excreting;

[0191] Performing emotion classification based on the second video to obtain a first prediction probability of the pet in each emotion category during defecation;

[0192] Performing emotion classification based on the third video to obtain a second predicted probability of the pet in each emotion category after defecation;

[0193] The target emotion information is determined based on the first prediction probability and the second prediction probability of the pet in each emotion category.

[0194] In one embodiment of the present application, in terms of performing emotion classification based on the second video to obtain a first predicted probability of the pet in each emotion category during defecation, the processor 702 is specifically configured to perform the following steps:

[0195] Extracting audio frames from the second video to obtain a plurality of first audio frames, and extracting images from the second video to obtain a plurality of first images corresponding to the plurality of first audio frames;

[0196] Performing feature extraction on each audio frame to obtain a first feature matrix, and performing feature extraction on each image to obtain a second feature matrix;

[0197] The feature representation of each audio frame in the first feature matrix and the feature representation of the image corresponding to each audio frame in the second feature matrix are merged to obtain a third feature matrix;

[0198] Acquire a first adjacency matrix corresponding to the plurality of first audio frames, and acquire a second adjacency matrix corresponding to the plurality of first images;

[0199] Based on the third feature matrix, the first adjacency matrix, and the second adjacency matrix, a first predicted probability of the pet in each emotion category during defecation is determined.

[0200] In one embodiment of the present application, in determining the first predicted probability of the pet in each emotion category during defecation based on the third feature matrix, the first adjacency matrix, and the second adjacency matrix, the processor 702 is specifically configured to perform the following steps:

[0201] Determining target feature weights corresponding to each audio frame and each image based on the third feature matrix;

[0202] Based on the target feature weights corresponding to each audio frame and each image, weight each feature representation in the third feature matrix to obtain a fourth feature matrix;

[0203] For the i-th first feature processing, the i-th first feature processing is performed based on the first adjacency matrix, the fifth feature matrix obtained by the i-1-th first feature processing, and the feature representation of the neighboring audio frames of each audio frame in the fifth feature matrix corresponding to the i-1-th first feature processing, until multiple first feature processings are performed to obtain a sixth feature matrix, wherein the i-th first feature processing is any one of the multiple first feature processings, and when i=1, the fifth feature matrix is ​​the first feature matrix;

[0204] For the k-th second feature processing, the k-th second feature processing is performed based on the second adjacency matrix, the seventh feature matrix obtained by the k-1-th second feature processing, and the feature representation of the neighbor image of each image in the seventh feature matrix corresponding to the k-1-th second feature processing, until multiple second feature processings are performed to obtain an eighth feature matrix, wherein the k-th second feature processing is any one of the multiple second feature processings, and when i=1, the seventh feature matrix is ​​the second feature matrix;

[0205] Classification is performed based on the fourth characteristic matrix, the sixth characteristic matrix, and the eighth characteristic matrix to obtain the first predicted probability of the pet in each emotion category during defecation.

[0206] In one embodiment of the present application, in determining whether the pet has an intention to defecate based on the first video, the processor 702 is specifically configured to perform the following steps:

[0207] Based on the first video, determining the length of time the pet stays at the excretion device;

[0208] Performing image extraction on the first video to obtain a plurality of second images corresponding to the pet;

[0209] Extracting audio frames from the first video to obtain a plurality of second audio frames corresponding to the pet;

[0210] Based on a plurality of preset third images, the plurality of second images are screened to obtain a fourth image, wherein the plurality of third images represent images when the pet has an intention to defecate;

[0211] Predicting the probability of the pet's defecation intention based on the stay duration, the fourth image, and the plurality of second audio frames;

[0212] Based on the probability of defecation intention, determine whether the pet has defecation intention.

[0213] Specifically, the transceiver 701 may be Figure 6 The acquisition unit 601 of the intelligent management device of the embodiment, the processor 702 may be Figure 6 The processing unit 602 of the intelligent management device of the embodiment.

[0214] In a specific implementation, the transceiver 701 and the processor 702 described in the embodiment of the present invention may also execute other implementations described in the embodiment of the pet nose print-based management method provided in the embodiment of the present invention, which will not be described in detail here.

[0215] It should be understood that the electronic device in this application can be an intelligent management device, which can be a server or a terminal device. The server can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, and basic cloud computing services such as big data and artificial intelligence platforms. This application does not make specific restrictions. The above electronic devices are only examples, not exhaustive, and include but are not limited to the above electronic devices.

[0216] It should be understood that the embodiments of the present application also provide a computer-readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement part or all of the steps of any one of the pet nose print-based management methods recorded in the above method embodiments.

[0217] An embodiment of the present application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute part or all of the steps of any one of the pet nose print-based management methods recorded in the above method embodiments.

[0218] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present application.

[0219] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0220] In the several embodiments provided in the present application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0221] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0222] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of a software program module.

[0223] If the integrated unit is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of each embodiment method of the present application. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0224] A person skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable memory, and the memory can include: a flash drive, a read-only memory (English: Read-Only Memory, abbreviated as: ROM), a random access memory (English: Random Access Memory, abbreviated as: RAM), a magnetic disk or an optical disk, etc.

[0225] The embodiments of the present application are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A management method based on pet nose prints, characterized in that: Applied to an intelligent management device, the intelligent management device is located in an intelligent management system, the intelligent management system also includes an excretion device, the excretion device is provided with a weight sensor, an odor sensor and a camera device, the method includes: Acquire a first video of the pet outside the excretion device by using the camera device; Based on the first video, determining whether the pet has an intention to defecate; If there is an intention to defecate, obtaining a facial image of the pet through the camera device; Based on the facial image, determining nose print features of the pet; Performing identity authentication on the pet based on the nose print feature, and if the authentication is successful, opening the excretion device; The weight of the pet's excrement is obtained by the weight sensor, and the odor of the pet's excrement is obtained by the odor sensor; Generate excretion data of the pet based on the excrement weight and the excrement smell; Based on the nose print features and the excretion data, a management plan corresponding to the pet is generated.

2. The method according to claim 1, characterized in that: The step of generating the pet's excretion data based on the excrement weight and the excrement odor includes: generating first sub-excretion condition data based on the excrement weight and the excrement odor; Acquire a video of the pet defecating in the excretion device, and acquire an image of the pet's excrement; generating second sub-excretion situation data based on the excretion image and the excretion video; The excretion situation data of the pet is generated based on the first sub-excretion situation data and the second sub-excretion situation data.

3. The method according to claim 2, characterized in that The step of generating second sub-excretion situation data based on the excretion image and the excretion video includes: Based on the excretion video, determining target emotional information representing the pet during the excretion process; Determining the excrement status of the pet based on the excrement image; The second sub-excretion situation data is generated based on the emotion information and the excretion state.

4. The method according to claim 3, characterized in that Determining target emotional information representing the pet during the excretion process based on the excretion video includes: Segmenting the excretion video to obtain a second video and a third video, wherein the second video is a video of the pet excreting, and the third video is a video of the pet after excreting; Performing emotion classification based on the second video to obtain a first predicted probability of the pet in each emotion category during defecation; Performing emotion classification based on the third video to obtain a second predicted probability of the pet in each emotion category after defecation; The target emotion information is determined based on the first predicted probability and the second predicted probability of the pet in each emotion category.

5. The method according to claim 4, characterized in that The performing emotion classification based on the second video to obtain a first predicted probability of the pet in each emotion category during defecation includes: Extracting audio frames from the second video to obtain a plurality of first audio frames, and extracting images from the second video to obtain a plurality of first images corresponding to the plurality of first audio frames; Performing feature extraction on each audio frame to obtain a first feature matrix, and performing feature extraction on each image to obtain a second feature matrix; fusing a feature representation of each audio frame in the first feature matrix and a feature representation of an image corresponding to each audio frame in the second feature matrix to obtain a third feature matrix; Acquire a first adjacency matrix corresponding to the plurality of first audio frames, and acquire a second adjacency matrix corresponding to the plurality of first images; Based on the third feature matrix, the first adjacency matrix, and the second adjacency matrix, a first predicted probability of the pet in each emotion category during defecation is determined.

6. The method according to claim 5, characterized in that The determining, based on the third feature matrix, the first adjacency matrix, and the second adjacency matrix, a first predicted probability of the pet in each emotion category during defecation includes: Based on the third feature matrix, determining a target feature weight corresponding to each audio frame and each image; Based on the target feature weights corresponding to each audio frame and each image, weighting each feature representation in the third feature matrix to obtain a fourth feature matrix; For the i-th first feature processing, the i-th first feature processing is performed based on the first adjacency matrix, the fifth feature matrix obtained by the i-1-th first feature processing, and the feature representation of the neighboring audio frames of each audio frame in the fifth feature matrix corresponding to the i-1-th first feature processing, until multiple first feature processings are performed to obtain a sixth feature matrix, wherein the i-th first feature processing is any one of the multiple first feature processings, and when i=1, the fifth feature matrix is ​​the first feature matrix; For the k-th second feature processing, the k-th second feature processing is performed based on the second adjacency matrix, the seventh feature matrix obtained by the k-1-th second feature processing, and the feature representation of the neighbor image of each image in the seventh feature matrix corresponding to the k-1-th second feature processing, until multiple second feature processings are performed to obtain an eighth feature matrix, wherein the k-th second feature processing is any one of the multiple second feature processings, and when i=1, the seventh feature matrix is ​​the second feature matrix; Classification is performed based on the fourth characteristic matrix, the sixth characteristic matrix, and the eighth characteristic matrix to obtain a first predicted probability of the pet in each emotion category during defecation.

7. The method according to any one of claims 1 to 6, characterized in that: The determining, based on the first video, whether the pet has an intention to defecate includes: Based on the first video, determining the length of time the pet stays at the excretion device; Performing image extraction on the first video to obtain a plurality of second images corresponding to the pet; Extracting audio frames from the first video to obtain a plurality of second audio frames corresponding to the pet; Based on a plurality of preset third images, the plurality of second images are screened to obtain a fourth image, wherein the plurality of third images represent images when the pet has an intention to defecate; Based on the stay duration, the fourth image, and the plurality of second audio frames, predicting the probability of the pet's intention to defecate; Based on the defecation intention probability, it is determined whether the pet has the defecation intention.

8. An intelligent management device, characterized in that: The device is located in the intelligent management system, and the intelligent management system also includes an excretion device, and the excretion device is provided with a weight sensor, an odor sensor and a camera device. The device includes: an acquisition unit and a processing unit; The acquisition unit is used to acquire a first video of the pet outside the excretion device through the camera device; The processing unit is used to determine whether the pet has an intention to defecate based on the first video; The acquisition unit is further configured to acquire a facial image of the pet through the camera device if there is an intention to defecate; The processing unit is further configured to determine the nose print features of the pet based on the facial image; perform identity authentication on the pet based on the nose print features, and activate the excretion device if the authentication is successful; The acquisition unit is further used to acquire the weight of the pet's excrement through the weight sensor, and acquire the odor of the pet's excrement through the odor sensor; The processing unit is further used to generate the pet's excretion data based on the excrement weight and the excrement odor; and to generate a corresponding management plan for the pet based on the nose print features and the excretion data.

9. An electronic device, characterized in that: include: A processor and a memory, wherein the processor is connected to the memory, the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the electronic device executes the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method according to any one of claims 1 to 7.

Images