Emotion determination system and emotion determination method

Abstract

To provide an emotion determination system that determines an emotion of an animal with a simple method.SOLUTION: An emotion determination system includes: an acquisition part for acquiring an image of an animal; a behavior information generation part for generating behavior information relating to behaviors of an animal from an image of the animal acquired by the acquisition part; and an emotion determination part for determining an emotion of the animal on the basis of behavior information generated by the behavior information generation part.SELECTED DRAWING: Figure 1

Description

【Technical Field】 【0001】 The present invention relates to an emotion determination system and an emotion determination method, and more particularly to an emotion determination system and an emotion determination method that provide a determination result regarding the emotion of an animal from an image of the animal. 【Background Art】 【0002】 Pets such as dogs, cats, and rabbits are invaluable to humans, and various pet services are widespread and developing. 【0003】 Animals kept as pets are individuals with their own personalities and living beings with emotions. If the emotions of animals can be grasped, the communication between pets and their owners will become smoother and deeper, preventing misunderstandings and mismatches between pets and their owners, and ultimately reducing unfortunate cases such as abandonment of breeding. 【0004】 However, since animals cannot speak, they cannot explain their own emotions. 【0005】 Patent Document 1 discloses an animal photographing device that photographs the state of an animal during excretion and a health determination system that determines at least one of the health state and the emotion of the animal based on the photographed image of the animal, but does not disclose emotion determination based on the behavior of the animal. 【0006】 Patent Document 2 discloses an information processing device including a state determination unit that determines the state of the animal using an image in which the animal is photographed, an emotion determination unit that determines the emotion of the animal corresponding to the state of the animal determined by the state determination unit based on information indicating the relationship between the state of the animal and the emotion, and a notification processing unit that notifies a terminal of a statement indicating the emotion of the animal determined by the emotion determination unit, but does not disclose emotion determination based on the behavior of the animal. 【Prior Art Documents】 【Patent Documents】 【0007】 [Patent Document 1] Japanese Patent Publication No. 2020-5558 [Patent Document 2] Japanese Patent Publication No. 2020-170916 [Overview of the project] [Problems that the invention aims to solve] 【0008】 Therefore, the present invention aims to provide an emotion determination system and emotion determination method that can determine the emotions of animals in a simple manner. [Means for solving the problem] 【0009】 As a result of diligent research to solve the above problems, the inventors have discovered that by analyzing images of animals and obtaining behavioral information about the animals' actions in those images, it is possible to determine or estimate the emotions of those animals, thus completing the present invention. 【0010】 In other words, the present invention is as follows [1] to

[13] . [1] An emotion determination system comprising: an acquisition unit for acquiring images of animals; an action information generation unit for generating action information relating to the animal's behavior from the images of animals acquired by the acquisition unit; and an emotion determination unit for determining the animal's emotions based on the action information generated by the action information generation unit. [2] The emotion determination system of [1], wherein the behavior information generation unit extracts features from an image of an animal and generates behavior information by determining the animal's behavior based on the features. [3] An emotion determination system according to [1], wherein the behavior information generation unit includes a trained model that has learned the relationship between an image of an animal and the behavior of the animal, and the trained model determines the behavior and generates behavior information. [4] An emotion determination system of any of the [1] to [3], wherein the behavior information generation unit generates behavior information for each of the multiple types of behaviors included in the image. [5] An emotion determination system of any of [1] to [4] in which the behavior information generation unit weights the behavior information according to the length of the duration of the behavior when generating the behavior information. [6] Furthermore, an emotion determination system of any of the following [1] to [5] is provided, which includes a database that stores the correspondence between behavioral information and emotions, and the emotion determination unit determines the emotions of the animal by referring to the database. [7] An emotion determination system of any of [1] to [6], comprising a breed determination unit that determines the breed of the animal from the image of the animal acquired by the acquisition unit. [8] An emotion determination system of any of the following [1] to [7], wherein the database is a database that stores the correspondence between behavioral information and emotions for each species or variety. [9] The database comprises multiple emotion determination systems [1] to [7], each of which is configured according to a classification of animal breeds based on the average weight of adult animals of that breed.

[10] A method for generating a trained model that determines the behavior of an animal from an image of that animal, characterized in that an image of an animal and a label relating to the behavior of that animal are input into a computer as training data, and the artificial intelligence is trained.

[11] An emotion determination method comprising the steps of: preparing an image of an animal; generating behavioral information from the image of the animal using a computer; and determining the emotion of the animal based on the behavioral information using a computer.

[12] The emotion determination method of

[11] , wherein the behavior information generation step is a step of generating behavior information by extracting features from an image of an animal and determining the animal's behavior based on the features.

[13] The emotion determination method of

[12] , wherein the behavior information generation step is a step of determining behavior using a trained model that has learned the relationship between an image of an animal and the behavior of the animal, and generating behavior information. 【Advantages of the Invention】 【0011】 According to the present invention, it becomes possible to provide an emotion determination system and an emotion determination method for determining the emotions of animals in a simple manner. 【Brief Description of the Drawings】 【0012】 [Figure 1] It is a diagram showing an example of an image used as teacher data in an embodiment. [Figure 2] It is a diagram showing an example of an image used as teacher data in an embodiment. [Figure 3] It is a diagram showing an example of an image used as teacher data in an embodiment. [Figure 4] It is a schematic configuration diagram showing an embodiment of the emotion determination system of the present invention. [Figure 5] It is a flowchart diagram showing an example of the flow of emotion determination by the emotion determination system of the present invention. [Figure 6] It is a schematic configuration diagram showing an embodiment of the emotion determination system of the present invention (a case where breed determination is performed before emotion determination). [Figure 7] It is a flowchart diagram showing an example of the flow of emotion determination by the emotion determination system of the present invention (a case where breed determination is performed before emotion determination). 【Modes for Carrying Out the Invention】 【0013】 <Emotion Determination System> An emotion determination system according to an embodiment of the present invention includes an acquisition unit that acquires an image of an animal, an action information generation unit that generates action information related to the action of the animal from the image of the animal acquired by the acquisition unit, and an emotion determination unit that determines the emotion of the animal based on the action information generated by the action information generation unit. Note that these general or specific aspects may be implemented by a system, an apparatus, a method, an integrated circuit, a computer program, or a recording medium, or may be implemented by any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium. 【0014】 [Acquisition unit] The acquisition unit acquires an image of an animal. Unless otherwise specified, the "image" includes both images and videos. The acquisition unit may be configured to receive the transmission and input of images from an external terminal or a computer, or may be a camera including a lens and an imaging element. Examples of animals include mammals such as dogs, cats, and rabbits, with dogs and cats being preferred. The method of receiving the image may be any method such as scanning, inputting image data, transmitting, or capturing an image on the spot. The format of the image is not particularly limited and may be a still image or a moving image. However, since multiple still images can be acquired from a moving image, the emotion can be determined more accurately. The part of the animal shown in the image is not particularly limited, but the image of the animal is preferably an image that makes it easy to determine what action the animal is taking, for example, an image showing a part where emotion is likely to be expressed such as the face (especially the eyes and ears), the tail, or an image showing the whole body. The image may be in black and white, grayscale, or color. Images whose shape is edited with image editing software, images showing multiple animals, images in which the eyes and ears are shown too small to be distinguished, or unclear images are not preferred. The image may be normalized or have a unified resolution. 【0015】 [Action information generation unit] The behavioral information generation unit generates behavioral information about the animal's behavior from the animal's image acquired by the acquisition unit. For example, the behavioral information generation unit analyzes the image data to determine the pet's behavior classification. Preferably, the behavioral information generation unit generates behavioral information by extracting features from the animal's image and determining the animal's behavior based on these features. It is also preferable that the behavioral information generation unit includes a trained model that has learned the relationship between the animal's image and the animal's behavior, and generates behavioral information by determining the behavior using the trained model. 【0016】 Behavioral information refers to information about behavior, such as tags, and includes information about the classification of behavior. In the case of dogs, a classification of behavior may include one or more behaviors selected from a group consisting of, for example, tail wagging, tail drooping, tail curling, tail tucking between legs, jumping, circling, barking, growling, snorting, licking nose, yawning, scratching, shivering, averting eyes, sniffing, lifting one front paw, nudging a person's foot with the nose, licking, biting, running, lying down, lying on one's back and showing the belly, sitting, ears erect, ears flattened back, mouth open, digging, and panting. In the case of cats, behavioral information can include one or more actions selected from the following group: pupils dilating, eyes becoming sharp, eyes narrowing, ears pointing forward, ears pointing backward, fangs bared and mouth wide open, lips pursed, whiskers relaxing, whiskers pointing upward, head lowered below shoulders, head rubbing, head fur standing on end, lying down showing belly, body rubbing against owner, body rubbing against wall, forequarters lowered and hindquarters raised, claws extended and bracing, limbs tensed and back raised, tail raised, tail fur standing on end, tail curled, tail slammed down, and tail hanging down. These are merely examples, and other actions may also be included as behavioral information. The system does not need to be structured so that all of these examples can be classified as behavioral information; only actions strongly linked to emotions may be included. Furthermore, behavioral information is associated with tags or numbers assigned to each action. Even symbols or other forms of information are acceptable. 【0017】 If the behavior information generation unit includes a pre-trained model, the pre-trained model can be generated, for example, by supervised learning or unsupervised learning. In the case of supervised learning, training data can include, for example, images of animals and data or labels related to those animals' behaviors. Images of animals can also be used as training data by attaching tags or labels corresponding to the animals' behaviors. 【0018】 The format of the animal images used as training data is not particularly limited. They can be still images or videos. The parts of the animals shown in the images are not particularly limited, but images that make it easy to identify the animal's behavior are preferred, such as images showing the face or tail, or images showing the whole body. Images can be black and white, grayscale, or color. Images whose shape has been edited with image editing software, images showing multiple animals, images where the eyes or ears are too small to be distinguished, or images that are blurry are not preferred. Images that have been normalized or have a uniform resolution are preferred. It is even more preferable that the resolution of the images used as training data and the images received by the acquisition unit are the same. 【0019】 The behavior of animals captured in images used as training data can be identified from the video itself. Other methods for identifying behavior include, for example, surveys with pet owners or information provided during pet insurance applications. 【0020】 As a pre-trained model, artificial intelligence (AI) is preferred. Artificial intelligence (AI) is software or a system that imitates the intelligent tasks performed by the human brain. Specifically, it refers to computer programs that understand natural language used by humans, perform logical reasoning, and learn from experience. The AI ​​can be either general-purpose or specialized, and specific examples include logistic regression, decision trees, k-means algorithms, and multilayer architectures. Any known neural network, such as a perceptron, recurrent neural network, deep neural network, or convolutional neural network, may be used, and publicly available software can be used. 【0021】 To generate a trained model, artificial intelligence is trained. While either machine learning or deep learning can be used for training, deep learning is preferred. Deep learning is an evolution of machine learning and is characterized by its ability to automatically identify features. 【0022】 There are no particular restrictions on the training method used to generate a pre-trained model; publicly available software and libraries can be used. Transfer learning is also acceptable as the training method. For example, NVIDIA's DIGITS (the Deep Learning GPUTraining System) can be used. Alternatively, for example, a pre-trained model can be generated using transfer learning with artificial intelligence (neural networks) such as ResNet, MobileNet, or EfficientNet, and a machine learning library (Deep Learning library) such as PyTorch. Furthermore, For example, you may use a publicly available support vector machine, such as one published in "Introduction to Support Vector Machines" (Kyoritsu Shuppan). 【0023】 The behavior information generation unit preferably generates behavior information for each behavior when the image contains multiple types of behavior. For example, if a dog wags its tail for 30 seconds and then raises one front paw for 30 seconds in a one-minute video, the behavior information generated will be "wags tail" and "raises one front paw". If the dog wags its tail and raises one front paw simultaneously in the video, it may generate a single behavior information such as "wags tail and raises front paw", or it may generate behavior information separately for each of the actions "wags tail" and "raises one front paw". Good. Preferably, behavioral information is generated for each action. Furthermore, when the behavioral information generation unit generates behavioral information, it is preferable to weight each piece of behavioral information according to the duration of the action, and if weighting is performed, it is preferable to generate information about the weighting along with each piece of behavioral information. For example, if in a 1 minute 40 second video the dog wags its tail for 70 seconds and then raises one front paw for 30 seconds, the behavioral information generated would be "wags tail" and "raises one front paw," with the weight for the "wags tail" action being 0.7 and the weight for the "raises one front paw" action being 0.3. If the dog keeps its tail down for the entire duration of the 1 minute video, the behavioral information would be "lowers tail," with a weight of 1.0. Furthermore, when weighting is performed, it is preferable to multiply the emotional points assigned to each piece of behavioral information by the weight to determine the final emotion, and if multiple pieces of behavioral information are generated, it is preferable to sum the emotional points for each piece of behavioral information to determine the emotion. 【0024】 [Emotion Judgment Department] The emotion determination unit determines the emotion of the animal based on the behavioral information generated by the behavioral information generation unit. For example, specific behavioral information may be pre-associated with specific emotions and stored in memory. When the emotion determination unit receives specific behavioral information, it selects and determines the corresponding emotion. Determining an emotion may also involve predicting or inferring the emotion. Alternatively, it may involve determining which tendency each emotional item has, or assigning a score to each emotion. The determination can also be a graded evaluation based on the strength of the tendency for each emotional item. 【0025】 Furthermore, it is preferable that the emotion determination system of the present invention includes a database that stores the correspondence between behavioral information and emotions, and that the emotion determination unit determines emotions from behavioral information by referring to the database. Preferably, the database is a database that stores the correspondence between behavioral information and emotions for each species or breed. For example, a behavioral and emotion database for dogs, a behavioral and emotion database for cats, a behavioral and emotion database for rabbits, and so on. Examples of breed-specific databases include a behavioral and emotion database for Chihuahuas and a behavioral and emotion database for Siberian Huskies. It is also preferable that the database includes multiple databases according to classifications of animal breeds based on the average weight of adult dogs of that breed. For example, a configuration that includes databases for large dogs, medium-sized dogs, small dogs, and toddler dogs. As for classification, for example, in the case of dogs, breeds with an average adult weight of less than 4 kg can be classified as toddler dogs, breeds with an average adult weight of less than 10 kg can be classified as small dogs, breeds with an average adult weight of less than 25 kg can be classified as medium dogs, and breeds with an average adult weight of 25 kg or more can be classified as large dogs, based on average weight. In addition, breeds may be classified by other factors such as height, body length, and genetic relationships. It should be noted that a database is not essential in this invention; for example, if the emotion determination unit includes dictionary data relating behavioral information to emotions, a separate database is not required. 【0026】 Furthermore, the emotion determination unit may be configured to include a trained model that has learned the relationship between the type of animal behavior and emotion, and to determine emotion from behavioral information using this trained model. The trained model can be generated, for example, by supervised learning or unsupervised learning. When using a trained model, a database storing the relationship between behavioral information and emotion may not be necessary. Examples of training data in the case of supervised learning include data and labels related to the animal's behavior and its emotions. Images of animals may also be used as training data by attaching tags and labels corresponding to the animal's behavior and emotions. Other configurations of the trained model are the same as those described for the behavioral information generation unit above. 【0027】 [Type determination department] The emotion determination system of the present invention preferably further includes a breed determination unit. The breed determination unit uses a trained model for breed determination to determine the movement of an animal from an image of an animal input to the acquisition unit. This is a means of determining the variety of an object. If the variety determination unit is not included, the system can be configured to allow the user to input variety information from a separate terminal or other device. 【0028】 In humans, there isn't much difference in the position or shape of key facial features (such as eyes, nose, and mouth), so there's no reason to include a breed identification unit. However, because pets have been selectively bred by humans, their behavioral and emotional tendencies, as well as the correspondence between behavior and emotion, can differ from breed to breed. Therefore, including a breed identification unit makes it possible to determine emotions more accurately. 【0029】 The details of the images related to the variety determination unit are the same as those for the behavioral information generation unit described above. 【0030】 The trained model for breed determination is a trained model that has learned the relationship between an animal image and the animal's breed. Preferably, it includes a trained model that is trained using an animal image and its breed as training data, and takes an animal image as input and outputs the determination of the animal's breed. The animal images used as training data are the same as those used in the emotion determination unit described above. Furthermore, the trained model for breed determination is the same as the trained model in the emotion determination unit described above, except that it has learned the relationship between an animal image and the animal's breed. Different algorithms, software, libraries, and training methods may be used for the trained models in the behavior information generation unit and emotion determination unit described above, or the same algorithms, software, libraries, and training methods may be used. 【0031】 A breed is a unit of biological group below the species level. For example, in the case of dogs, dog breeds are also called dog breeds, and specific examples include Toy Poodle, Chihuahua, Miniature Dachshund, Shiba Inu, Pomeranian, Yorkshire Terrier, Miniature Schnauzer, Shih Tzu, French Bulldog, Papillon, Maltese, Labrador, Dalmatian, and Chow Chow. In the case of cats, cat breeds are also called cat breeds, and examples include Scottish Fold, American Shorthair, Norwegian Forest Cat, Russian Blue, British Shorthair, Ragdoll, Maine Coon, and Persian. In the case of rabbits, examples include Netherland Dwarf, Holland Lop, Lop-eared, Mini Rex, Dwarf Lop, and American Fuzzy Lop. 【0032】 The emotion determination system of the present invention may also include multiple trained models as an emotion determination unit, and may determine the emotion of an animal from an image of the animal input to the acquisition unit using the trained models corresponding to the breed determination result of the breed determination unit and the breed information input simultaneously when the image is received by the acquisition unit. When the emotion determination system of the present invention includes a breed determination unit, it is preferable to determine the breed of the animal from the image of the animal input to the acquisition unit, and then perform emotion determination in the emotion determination unit using the corresponding trained model based on the determination result and behavioral information. 【0033】 Preferably, the trained model corresponding to a breed is one that has been trained using images of animals of one or more specific breeds and labels related to the emotions of those animals as training data. For example, a trained model that has been trained using images of animals of a specific breed and labels related to the emotions of those animals. In this case, the images of animals of a specific breed may be only images of one breed, for example, only images of toy poodles, or it may be images of multiple breeds, for example, only images of toy poodles, pomeranians, and miniature dachshunds. In the case of multiple breeds, the breeds can be classified as follows: a trained model for large breeds trained using images of large breeds, a trained model for medium breeds trained using images of medium breeds, a trained model for small breeds trained using images of small breeds, and a trained model for very small breeds trained using images of very small breeds. The system may be configured to include pre-trained models corresponding to each classification. For example, in the case of dogs, breeds can be classified by average weight: breeds with an average adult weight of less than 4 kg are called extra-small dogs, breeds with an average adult weight of 4 to less than 10 kg are called small dogs, breeds with an average adult weight of 10 to less than 25 kg are called medium dogs, and breeds with an average adult weight of 25 kg or more are called large dogs. Breeds can also be classified by other factors such as height, body length, and genetic relationships. 【0034】 [output] The emotion determination unit determines the animal's emotions based on its behavioral information. The format of the output of the determination results is not particularly limited. For example, it could display the type of emotion as text or icons on the screen of the user's computer or mobile device, such as "Happy," "Relaxed," "Affectionate," "Having Fun," "In a Good Mood," or "Warning." Alternatively, it could provide multiple emotion categories and display the tendency for each category as a score or percentage. Specifically, "Happy" might be 0.45 points, and "Affectionate" might be 0.35 points (the higher the score, the stronger the tendency for that emotion). The emotion determination system of the present invention may also include an output unit that receives the determination result from the emotion determination unit and outputs the determination result, and a transmission unit that transmits the determination result to a personal computer or mobile terminal used by the user via a communication line. 【0035】 Hereinafter, one embodiment of the emotion determination system of the present invention will be described with reference to Figure 4. 【0036】 In Figure 4, terminal 40 is a terminal used by a user. The user is, for example, a pet owner who wants to know the emotions of their pet. Examples of terminal 40 include personal computers, smartphones, tablet devices, and other portable information terminals. Terminal 40 is composed of a processing unit such as a CPU, a storage unit such as a hard disk, ROM or RAM, a display unit such as an LCD panel, an input unit such as a mouse, keyboard, or touch panel, and a communication unit such as a network adapter. 【0037】 The user accesses the server from terminal 40 and inputs and transmits images (photographs) of animals, as well as information such as the animal's species, breed, sex, and weight. Alternatively, when using this emotion determination system, the user may take photos or videos of the target animal using the camera of their smartphone on the spot and input and transmit them. For example, the user takes a video of the target animal according to the instructions displayed on the screen of terminal 40, and if a suitable video is taken, transmits it to the server via the internet. In this case, the server may also be equipped with a separate photo-taking assistance means consisting of an image determination program, which determines whether the video is suitable for emotion determination, such as whether the video captures the animal's behavior, and transmits the determination result to the user through an interface or terminal. Furthermore, the user can receive the results of the emotion assessment on the server by having terminal 40 access the server. 【0038】 In this embodiment, the server is configured as a computer, but it may be any device as long as it has the functions according to the present invention. The server may also be a server located in the cloud. 【0039】 The memory unit 10 is composed of, for example, ROM, RAM, or a hard disk. The memory unit 10 stores information processing programs for operating each part of the server, and also stores the behavior information generation unit 11 and the emotion determination unit 12. 【0040】 The behavior information generation unit 11 analyzes images of animals obtained through user input, transmission, etc., determines the behavior of the animals in the images, and generates animal behavior information. The emotion determination unit 12 determines the animal The system determines the corresponding emotion of an animal based on its behavioral information. If a database 13 exists that stores the correspondence between animal behavioral information and emotions, the emotion determination unit selects and determines an emotion from the behavioral information by referring to the database 13. The behavioral information generation unit 11 and the emotion determination unit 12 in this embodiment may be configured to include, for example, a deep neural network or a convolutional neural network. 【0041】 The processing unit 20 generates behavioral information using the information processing program of the behavioral information generation unit 11 stored in the memory unit. The processing unit 20 also performs emotion determination using the information processing program of the emotion determination unit stored in the memory unit. 【0042】 The interface unit (communication unit) 30 includes an acquisition unit 31 and an output unit 32, and receives images of animals and other information such as the species and breed of the animals from the user's terminal, and outputs and transmits the emotion judgment results to the user's terminal. 【0043】 With the emotion determination system of this embodiment, users can easily obtain the result of determining their pet's emotions by uploading photos or videos of their pet to the server. 【0044】 In this embodiment, the behavioral information generation unit, emotion determination unit, and acquisition unit are described as being stored in a server and connected to the user's terminal via a connection means such as the Internet or LAN. However, the present invention is not limited to this, and may also be configured such that the behavioral information generation unit, emotion determination unit, database, and interface unit are stored in a single server or device, or each is stored in a separate server, or a separate terminal used by the user is not required. 【0045】 Figure 5 shows a flowchart of emotion determination based on one embodiment of the emotion determination system of the present invention. The user inputs basic information such as the species, breed, age, and sex of the target animal into the acquisition unit and uploads an image of the animal (Step S1). The server's processing unit uses the behavior information generation unit to determine the behavior of the animal from the uploaded image (Step S2). The emotion determination unit determines the emotion of the animal from the information generated by the behavior information generation unit (Step S3), and the output unit outputs the derived determination result, such as by displaying it on the screen, and presents it to the user (Step S4). 【0046】 <Embodiment equipped with a variety determination unit> Hereinafter, an embodiment of the emotion determination system of the present invention, which includes a variety determination unit, will be described with reference to Figure 6. 【0047】 The emotion determination system shown in Figure 6 is the same as the embodiment described above, except that the memory unit 10 stores a behavior information generation unit 11, an emotion determination unit 12, as well as a database for small dogs 14, a database for medium-sized dogs 15, a database for large dogs 16, and a breed determination unit 17. 【0048】 In this emotion determination system according to this embodiment, when an image of an animal is input to the acquisition unit 31, the breed determination unit 17 determines the breed of the animal. The breed determination may be a determination of a specific breed name, or it may be a determination of classification such as whether it is a large breed, a medium breed, or a small breed. 【0049】 Based on the determination result by the breed determination unit 17, a database to be used for emotion determination is selected. The system may also include a program or software to select the most suitable database from multiple databases depending on the breed and breed classification. For example, if the image of an animal input to the acquisition unit is determined to be an image of a large breed, the large dog database 16 is selected, and emotion determination from behavioral information is performed. 【0050】 In this embodiment, the configuration is based on the premise of using one emotion determination system for each species, such as for dogs and cats. However, an emotion determination system that supports multiple species of animals is also possible. In this case, it is preferable to include a species determination unit that determines the species of the animal from an image of the animal, and it is preferable to have multiple databases for each species and breed. 【0051】 Figure 7 shows a flowchart of emotion determination based on an embodiment in which the emotion determination system of the present invention is equipped with a breed determination unit. The user inputs basic information such as the type of animal to be determined to the acquisition unit and uploads an image of the animal (step S21). The server's processing unit uses the behavior information generation unit to determine the behavior of the animal from the uploaded image and generates behavior information (step S22). Next, the breed determination unit determines the breed from the image of the animal (step S23). Then, a database corresponding to the breed is selected (step S24), and the emotion of the animal is determined from the behavior information by referring to the selected database (step S25). The output unit outputs the derived determination result by displaying it on the screen or by other means and presenting it to the user (step S26). Note that breed determination by the breed determination unit may be performed before the generation of the animal's behavior information, or it may be performed simultaneously with the generation of the animal's behavior information. 【0052】 The animal images, trained models, behavioral information generation, and emotion determination are the same as in the emotion determination system of the present invention described above. [Examples] 【0053】 [Generating a pre-trained model for the behavioral information generation unit] As training data, we prepared 761 images of cats with their tails raised (breakdown: images from behind, from the front, from the side, and sitting) and images of cats with their tails down (breakdown: images from behind, from the front, from the side, and sitting). The number of images for each category is shown in Table 1 below. These images were used as training data to train the neural network ResNet. 【0054】 [Table 1] 【0055】 Next, the trained model was evaluated. As shown in Table 2 below, test images (images separate from the training data) were prepared, including images with tails raised and images with tails not raised, and the tests were conducted. The number of correct answers for the tests is also shown in Table 2. 【0056】 [Table 2] 【0057】 As described above, we were able to generate a pre-trained model capable of accurately determining cat behavior from cat images. 【0058】 [Creating a database on the correspondence between behavior and emotion] Based on the following literature and other sources, we created a database of cat behavioral information and its corresponding emotions, as shown in Table 3 below. Note that in Table 3, "distress" refers to emotional distress, not physical health, and is synonymous with "depression." • The Complete Guide to Living with Cats (https: / / test.anicom-sompo.co.jp / nekonoshiori / ) ·Facial expressions of pain in cats: the development and validation of a Feline Grimace Scale(Sci Rep. 2019 Dec 13;9(1):19128. doi:10.1038 / s41598-019-55693-8.) ·The quality of being sociable: The influence of humanattentional state, population, and human familiarity on domestic catsociability(ehav Processes . 2019 Jan;158:11-17. doi: 10.1016 / j.beproc.2018.10.026. Epub 2018 Nov 2.) ·Attachment bonds between domestic cats and humans(CORRESPONDENCE| VOLUME 29, ISSUE 18, PR864-R865, SEPTEMBER 23, 2019) ·Early weaning increases aggression and stereotypicbehaviour in cats(Sci. Rep. volume 7, Article number:10412 (2017)) ·The ‘Feline Five’: An exploration of personality in pet cats (Felis catus)(PLOS ONE August 23, 2017) ·Social interaction, food, scent or toys? A formalassessment of domestic pet and shelter cat (Felis silvestris catus) preferences(Behav Processes . 2017 Aug;141(Pt 3):322-328. doi: 10.1016 / j.beproc.2017.03.016.) ·Social referencing and cat-human communication(Animal Cognition volume 18, pages639-648 (2015)) ·Even healthy cats act sick when their routine is disrupted(https: / / www.sciencedaily.com / releases / 2011 / 01 / 110103110357.htm) 【0059】 [Table 3] 【0060】 In each of the following examples, videos of cats were filmed and their behavior was observed. The resolution was 12 million pixels, and the frame rate was 5 FPS (0.2 seconds / frame). 【0061】 [Example 1] The cat was an American Shorthair, and the video was recorded for 13.4 seconds. When the video was input into a computer program for determining animal behavior (a trained model that learned the relationship between cat images and behaviors using cat images and information about the types of behaviors as training data), the behavior information "tail raised" was generated, as shown in Table 4 below. Since the duration was equal to the recording time, the weight was set to 1. 【0062】 [Table 4] 【0063】 Based on the behavioral information described above, the database listed in Table 3 was consulted, and the cat's emotions in this video were judged to be "happy" at 0.5 points and "affectionate" at 0.5 points. 【0064】 [Example 2] The cat was a mixed breed, and the video was recorded for 9.8 seconds. When the video was input into a computer program for analyzing animal behavior, behavioral information such as "raising its tail" and "rubbing its body against its owner" was generated, as shown in Table 5 below. The duration of "raising its tail" was 6.4 seconds, and the duration of "rubbing its body against its owner" was 3.4 seconds. Based on the ratio of these durations, weights of 0.65 and 0.35 were assigned to these behaviors, respectively. 【0065】 [Table 5] 【0066】 Based on the behavioral information described above, the database listed in Table 3 was referenced. The emotions of the cat in this video were determined as follows: "raising its tail" was assigned 0.5 points for "happy" and 0.5 points for "affectionate," and "rubbing its body against its owner" was assigned 1 point for "affectionate." After multiplying each point by a weight, the total result was determined that the emotions of the cat in this video were "happy" at 0.327 points and "affectionate" at 0.673 points. 【0067】 [Example 3] The cat was a Scottish Fold, and the video was recorded for 10 seconds. When the video was input into a computer program for determining animal behavior, behavioral information such as "tail raised" and "other" was generated, as shown in Table 6 below. The duration of "tail raised" was 7.0 seconds and "other" was 3.0 seconds, so weights of 0.7 and 0.3 were assigned based on the ratio of duration. 【0068】 [Table 6] 【0069】 Based on the behavioral information described above, the database listed in Table 3 was referenced. The emotions of the cat in this video were determined as follows: "Tail raised" was assigned 0.5 points for "Happy" and 0.5 points for "Affectionate." No particular emotion was assigned to "Other." After multiplying each point by a weight and summing them up, the cat's emotions in this video were determined to be "Happy" at 0.35 points and "Affectionate" at 0.35 points. 【0070】 [Example 4] The cat was a mixed breed, and the recording time was 8.4 seconds. When the video was input into a computer program for determining animal behavior, behavioral information was generated as shown in Table 7 below: "ears turned backward," "fangs bared and mouth opened wide," "hair on head standing on end," and "tail hair standing on end." All of these behaviors were observed simultaneously, and the duration of each behavior was the same as the recording time. Therefore, each behavior was weighted at 1 / 4, resulting in a weight of 0.25 for all four behaviors. 【0071】 [Table 7] 【0072】 Based on the behavioral information described above, the database listed in Table 3 was referenced, and the emotions of the cat in this video were determined as follows: "ears turned back" was assigned 0.5 points for "alert" and 0.5 points for "angry"; "baring fangs and opening mouth wide" was assigned 1 point for "angry"; "hair on head standing on end" was assigned 1 point for "angry"; and "hair on tail standing on end" was assigned 1 point for "angry". After multiplying each point by a weight, the total result was determined that the emotions of the cat in this video were "alert" at 0.125 points and "angry" at 0.875 points. 【0073】 [Example 5] The cat was a Japanese cat (Waneko), and the video was recorded for 16 seconds. When the video was input into a computer program for determining animal behavior, behavioral information was generated, as shown in Table 8 below, including "narrowing eyes," "pursing lips," and "lying down and showing the belly." Of these behaviors, "narrowing eyes" and "pursing lips" were observed to some extent simultaneously. The durations were 7.2 seconds for "narrowing eyes," 4.2 seconds for "pursing lips," and 8.2 seconds for "lying down and showing the belly," so the weights were set to 0.37, 0.21, and 0.42, respectively. 【0074】 [Table 8] 【0075】 Based on the behavioral information described above, the database listed in Table 3 was referenced, and the emotions of the cat in this video were determined as follows: "narrowing eyes" was assigned 0.5 points for "relaxed" and 0.5 points for "unwell"; "pursing lips" was assigned 0.5 points for "happy" and 0.5 points for "good mood"; and "lying down showing its belly" was assigned 1 point for "relaxed". After multiplying each point by a weight, the total result was determined that the emotions of the cat in this video were "happy" at 0.107 points, "relaxed" at 0.602 points, "good mood" at 0.107 points, and "unwell" at 0.184 points.

Claims

[Claim 1] The acquisition unit acquires animal videos, The system includes a determination unit that determines the animal's behavior from the video and determines the animal's emotions based on that behavior. The determination unit is an emotion determination system that determines emotions by weighting based on the duration of the animal's behavior. [Claim 2] The emotion determination system according to claim 1, wherein the aforementioned video consists of multiple still images and does not include sound. [Claim 3] The emotion determination system according to claim 1, further comprising a breed determination unit that determines the breed of an animal from the video of the animal acquired by the acquisition unit. [Claim 4] Steps to prepare animal videos, The computer has the step of determining the animal's behavior from the video and determining the animal's emotions based on that behavior. An emotion determination method, wherein the determination step is a step of determining an emotion by weighting it based on the duration of the animal's behavior. [Claim 5] On the computer, A step of determining the behavior of an animal from a video of the animal, and determining the emotion of the animal based on the behavior, wherein the determination of the emotion is performed by weighting based on the duration of the animal's behavior. A program to execute.

Images