Information processing systems, information processing methods, and programs

Abstract

To provide an information processing device, information processing method, and program that can estimate the emotions of an animal and transmit information based on the estimated emotions of the animal. [Solution] The information processing device comprises an estimation unit that estimates the animal's emotions based on the animal's biological information, a processing unit that causes an imaging unit to capture an image of what the animal is looking at based on the animal's emotions estimated by the estimation unit, and a creation unit that creates transmission information including text information based on the image of what the animal is looking at captured by the imaging unit.

Description

Technical Field

[0001] Embodiments of the present invention relate to an information processing system, an information processing method, and a program.

Background Art

[0002] A technique for estimating the emotions of dogs based on canine heart rate variability (HRV) data is known (see, for example, Patent Document 1 and Non-Patent Document 1).

Prior Art Documents

Patent Documents

[0003] [[ID=2,2]]

Patent Document 1

Non-Patent Documents

[0004] [[ID=3,1]]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In the above-described technique, it is possible to estimate the degree of interest and stress of an animal, estimate the emotions of the animal based on the estimated degree of stress and interest, and represent the emotions of the animal by the lighting color of an LED light or a face image indicating the emotions of the animal. However, in the above-described technique, information cannot be transmitted based on the estimated emotions. Furthermore, in the aforementioned technology, for example, an animal emotion estimation device that measures heart rate variability is attached to the dog's body surface. However, the dog's fur and movements can interfere with the measurement of heart rate variability, potentially affecting the accuracy of the measurement. For this reason, it is difficult to collect heart rate variability data from a dog during a walk and estimate its emotions.

[0006] The object of the present invention is to provide an information processing system, information processing method, and program that can estimate the emotions of an animal and transmit information based on the estimated emotions of the animal. [Means for solving the problem]

[0007] (1) One embodiment of the present invention is an information processing system comprising: an estimation unit that estimates the emotions of an animal based on the animal's biological information; a processing unit that causes an imaging unit to image the area in front of the animal's gaze based on the emotions of the animal estimated by the estimation unit; and a creation unit that creates transmitted information including textual information based on the image of the area in front of the animal's gaze captured by the imaging unit.

[0008] (2) One embodiment of the present invention further comprises an image generation unit that generates image information taken from a direction different from the direction of the animal's gaze based on image information taken from the direction of the animal's gaze by the imaging unit, and the creation unit creates transmission information including text information based on the image information taken from a direction different from the direction of the animal's gaze generated by the image generation unit.

[0009] (3) One embodiment of the present invention further comprises a content generation unit that generates content based on other image information of the line of sight generated by the image generation unit, and the creation unit creates transmission information including text information based on the content generated by the content generation unit.

[0010] (4) One embodiment of the above-described information processing system further comprises a content generation unit that generates content based on image information of the animal's line of sight captured by the imaging unit, and the creation unit creates transmission information including text information based on the content generated by the content generation unit.

[0011] (5) One embodiment of the above-described information processing system further comprises a biological information measuring unit attached to an animal and measuring the animal's ecological information, wherein the estimation unit estimates the animal's emotions based on the biological information of the animal measured by the biological information measuring unit.

[0012] (6) In one embodiment of the present invention, in the information processing system described above, the biological information of the animal includes at least one of the animal's heart rate, heart rate variability, respiratory rate, pulse wave, electroencephalogram, and skin temperature.

[0013] (7) One embodiment of the present invention is the information processing system described above, in which the emotional information of the animal includes either negative emotions or positive emotions.

[0014] (8) One embodiment of the present invention is the information processing system described above, wherein the imaging unit captures an image of the direction in which the animal is looking if the emotion of the animal estimated by the estimation unit is either a positive emotion or a negative emotion, or both.

[0015] (9) In one embodiment of the present invention, in the information processing system described above, the creation unit creates posting information to be posted to a social networking service.

[0016] (10) One embodiment of the present invention is an information processing method performed by a computer, which estimates the emotions of an animal based on the animal's biological information, takes an image of what the animal is looking at based on the estimated emotions of the animal, and creates transmission information including text information based on the image of what the animal is looking at that has been captured.

[0017] (11) One embodiment of the present invention is a program that causes a computer to estimate the emotion of an animal based on the biological information of the animal, cause an imaging unit to image the direction of the line of sight of the animal based on the estimated emotion of the animal, and create transmission information including text information based on the image of the direction of the line of sight of the imaged animal.

Advantages of the Invention

[0018] According to an embodiment of the present invention, it is possible to provide an information processing system, an information processing method, and a program that can estimate the emotion of an animal and transmit information based on the estimated emotion of the animal.

Brief Description of the Drawings

[0019] [Figure 1] It is a diagram showing a configuration example of an information processing system 1 according to an embodiment of the present invention. [Figure 2A] It is a flowchart showing an example of the operation of an information processing apparatus 100 according to the present embodiment. [Figure 2B] It is a flowchart showing an example of the operation of an information processing apparatus 100 according to the present embodiment. [Figure 3] It is a diagram showing a configuration example of an information processing system 1 according to a modified example of the embodiment. [Figure 4A] It is a flowchart showing an example of the operation of an information processing apparatus 400 according to a modified example of the embodiment. [Figure 4B] It is a flowchart showing an example of the operation of an information processing apparatus 400 according to a modified example of the embodiment. [Figure 5A] It is a schematic diagram showing an example of a wearable device 210 according to the present embodiment. [Figure 5B] It is a schematic diagram showing an example of a wearable device 210 according to the present embodiment. [Figure 6] It is a schematic diagram showing an example of a method for attaching a wearable device 210 according to the present embodiment. [Figure 7] It is a diagram showing an example of canine heart rate variability data. [Figure 8]This figure shows an example of processing performed by the information processing device 400 according to this embodiment. [Figure 9A] This is a schematic diagram showing an example of how to mount the imaging unit 300 according to this embodiment. [Figure 9B] This figure shows an example of an image captured by the imaging unit 300 according to this embodiment. [Figure 10A] This figure shows an example of an image captured by the imaging unit 300 according to this embodiment. [Figure 10B] This figure shows an example of the operation of the information processing device 400 according to this embodiment. [Figure 11] This figure shows an example of the operation of the information processing device 400 according to this embodiment. [Modes for carrying out the invention]

[0020] Next, the information processing system, information processing method, and program of this embodiment will be described with reference to the drawings. The embodiments described below are merely examples, and the embodiments to which the present invention is applied are not limited to the embodiments described below. In all the figures used to illustrate the embodiments, components with the same function are given the same reference numerals, and repeated explanations are omitted. Furthermore, in this application, "based on XX" means "based on at least XX," and includes cases where it is based on another element in addition to XX. Also, "based on XX" is not limited to cases where XX is used directly, but also includes cases where it is based on something that has been calculated or processed from XX. "XX" is any element (for example, any information).

[0021] (Embodiment) (Information Processing System) Figure 1 shows an example configuration of an information processing system 1 according to an embodiment of the present invention. The information processing system 1 includes an information processing device 100, a biological information measurement unit 200, and an imaging unit 300. The information processing device 100 and the biological information measurement unit 200 are connected by wire or wireless. The information processing device 100 and the imaging unit 300 are connected by wire or wireless.

[0022] The information processing device 100 receives biological information of an animal from the biological information measurement unit 200. Based on the input biological information of the animal, the information processing device 100 estimates the animal's emotions. Based on the estimated emotions of the animal, the information processing device 100 instructs the imaging unit 300 to capture an image of what the animal is looking at. Based on the image of what the animal is looking at captured by the imaging unit 300, the information processing device 100 creates transmission information including text information. The information processing device 100 transmits the created transmission information. For example, the information processing device 100 creates a post to be posted to a social networking service as transmission information. The information processing device 100 posts the created post to the social networking service.

[0023] The information processing device 100 is implemented by a device such as a personal computer, server, smartphone, tablet computer, or industrial computer. The information processing device 100 includes, for example, an input unit 101, an estimation unit 102, a processing unit 103, a creation unit 106, an output unit 107, and a storage unit 110.

[0024] The input unit 101 receives information. For example, the input unit 101 may receive information from an external device. As another example, the input unit 101 may have an operation unit such as a keyboard and a mouse. In this case, the input unit 101 receives information corresponding to the operation performed by the user on the operation unit.

[0025] The input unit 101 receives biological information of an animal from the biological information measurement unit 200. The animal may be an animal classified as a mammal. The animal may be livestock. The animal may or may not include humans. An example of the biological information measurement unit 200 is a heart rate sensor, pulse wave sensor, non-contact skin temperature sensor, blood pressure sensor, and respiration sensor. Heart rate sensors detect the R wave of the heartbeat and analyze the periodicity and frequency of the heart rate and R-r interval. Pulse wave sensors measure pulse rate and pulse wave amplitude from changes in blood flow in arterioles. In practice, pulse rate is the same as heart rate and is often used as a substitute for heart rate. Pulse wave amplitude increases or decreases depending on the relaxation or tension of the muscles wrapped around the arterioles. Similar to heart rate, it is possible to examine tension and relaxation states.

[0026] Non-contact skin temperature sensors detect skin temperature. Skin temperature is maintained by the heat of blood flow in the arterioles. When the arterioles tense and constrict, skin temperature decreases, and when relaxed, it increases. This can measure mental tension and relaxation. Blood pressure sensors measure blood pressure. Blood pressure is the pressure of the blood pumped from the heart. Blood pressure is determined by two factors: cardiac output per beat and peripheral vascular resistance. Blood pressure is an indicator used to estimate tension and relaxation. Respiratory sensors measure respiration (rate). Respiration can be consciously controlled, but it is also controlled autonomously. Respiration becomes shallow and rapid when one is unwell.

[0027] The estimation unit 102 acquires biological information of the animal from the input unit 101. Based on the acquired biological information, the estimation unit 102 estimates the animal's emotions. For example, the estimation unit 102 estimates whether the animal has positive or negative emotions. Positive emotions include feelings of joy and excitement, while negative emotions include feelings of anxiety, tension, stress, and fatigue. Specifically, a first range of biometric information is determined in advance, which is the range of biometric information when the animal is experiencing positive emotions, and a second range is determined, which is the range of biometric information when the animal is experiencing negative emotions. The estimation unit 102 determines whether the acquired biometric information falls into the first range or the second range. If the acquired biometric information falls into the first range, the estimation unit 102 estimates that the animal is experiencing positive emotions, and if it falls into the second range, it estimates that the animal is experiencing negative emotions.

[0028] The processing unit 103 obtains the estimated result of the animal's emotion from the estimation unit 102. Based on the obtained estimated result of the animal's emotion, if the estimated result of the animal's emotion is positive, the processing unit 103 outputs an imaging command to the imaging unit 300 to cause it to take an image. If the estimated result of the animal's emotion is negative, the processing unit 103 may terminate the process or output an imaging command to the imaging unit 300 to cause it to take an image.

[0029] The imaging unit 300 is attached to the animal so that it can image the direction in which the animal is looking. The imaging unit 300 is a camera that images real space and generates an image using various components such as an image sensor such as a CMOS (Complementary Metal Oxide Semiconductor) or a CCD (Charge Coupled Device), and a lens for controlling the imaging of the subject onto the image sensor. The imaging unit 300 may capture still images or motion pictures. The imaging unit 300 may capture three-dimensional images.

[0030] The imaging unit 300 captures an image in the direction of the animal's gaze and creates image information based on the imaging command output by the information processing device 100. For example, the imaging unit 300 may capture an image once or multiple times. If it captures an image multiple times, it may capture an image multiple times at predetermined time intervals. The imaging unit 300 outputs one or more image information of the direction of the animal's gaze that it has created to the processing unit 103. The processing unit 103 acquires one or more image information of the direction of the animal's gaze that was output by the imaging unit 300.

[0031] The creation unit 106 acquires one or more image information of the animal's line of sight from the processing unit 103. The creation unit 106 creates text information based on the acquired one or more image information of the animal's line of sight. The creation unit 106 creates transmission information that includes one or more image information of the animal's line of sight and the created text information. The creation unit 106 may also create transmission information that includes the created text information but does not include one or more image information of the animal's line of sight. For example, the creation unit 106 acquires text information corresponding to images similar to each of the one or more images of the animal's line of sight acquired from the processing unit 103 from a table that associates multiple images with text information corresponding to each of those multiple images. Alternatively, for example, the creation unit 106 may be configured to include Generative Artificial Intelligence (AI) and generate various text information from each of the one or more images of the animal's line of sight acquired from the processing unit 103. The output unit 107 obtains the transmission information from the creation unit 106. The output unit 107 outputs the obtained transmission information. For example, the output unit 107 may post the transmission information to a social networking service.

[0032] The storage unit 110 is implemented using HDD (Hard Disk Drive), flash memory, RAM (Random Access Memory), ROM (Read Only Memory), etc., and stores information. For example, the storage unit 110 stores a table that associates multiple images with text information corresponding to each of those multiple images.

[0033] The input unit 101, estimation unit 102, processing unit 103, creation unit 106, and output unit 107 are realized, for example, by a hardware processor such as a CPU (Central Processing Unit) executing a computer program (software) stored in the memory unit 110.

[0034] Furthermore, some or all of these functional components may be implemented by hardware (including circuitry) such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), and GPU (Graphics Processing Unit), or by collaboration between software and hardware.

[0035] Computer programs may be stored in advance on storage devices such as HDDs (Hard Disk Drives) or flash memory, or they may be stored on removable storage media such as DVDs (Digital Versatile Discs) or CD-ROMs, and installed when the storage media is inserted into a drive device.

[0036] (Operation of information processing device) Figures 2A and 2B are flowcharts illustrating an example of the operation of the information processing device 100 according to this embodiment. Referring to Figure 2A, the operation when an animal's biological information is input to the information processing device 100 and the device determines that the input animal's biological information indicates a positive emotion will be described. (Step S1a-1) The biological information of the animal is input to the input unit 101 from the biological information measurement unit 200. (Step S2a-1) The estimation unit 102 acquires biological information of the animal from the input unit 101 and estimates the animal's emotions based on the acquired biological information.

[0037] (Step S3a-1) The processing unit 103 obtains the estimated result of the animal's emotion from the estimation unit 102 and determines whether the estimated result of the animal's emotion is positive or negative based on the obtained result. If the estimated result of the animal's emotion is negative, the processing unit 103 terminates the process. (Step S4a-1) If the processing unit 103 determines that the estimated emotion of the animal is positive, it outputs an imaging command to the imaging unit 300 to cause it to take an image. Based on the imaging command output by the information processing device 100, the imaging unit 300 takes one or more images in the direction the animal is looking and creates one or more image information. The imaging unit 300 outputs the created one or more image information in the direction the animal is looking to the processing unit 103. The processing unit 103 acquires the one or more image information output by the imaging unit 300.

[0038] (Step S5a-1) The creation unit 106 acquires one or more image information from the processing unit 103 and creates text information from one or more images in the direction of the animal's gaze based on the acquired image information. The creation unit 106 creates transmission information that includes one or more image information in the direction of the animal's gaze and the created text information. (Step S6a-1) The output unit 107 obtains the transmission information from the creation unit 106 and outputs the obtained transmission information. For example, the output unit 107 outputs the transmission information to a social networking service. The same process may be repeated thereafter.

[0039] Next, referring to Figure 2B, we will explain the operation when the information processing device 100 receives biological information of an animal and outputs information in both cases, such as when it determines that the animal is experiencing a positive emotion or a negative emotion based on the input biological information of the animal. Steps S1b-1 to S6b-1 can be explained by applying steps S1a-1 to S6a-1 in Figure 2A. (Step S7b-1) If the processing unit 103 determines in step S3b-1 that the estimated emotion of the animal is not a positive emotion, it determines, based on the estimated emotion of the animal, whether or not the estimated emotion is negative. If the processing unit 103 determines that the estimated emotion of the animal is not negative, it terminates the process.

[0040] (Step S8b-1) If the processing unit 103 determines that the estimated emotion of the animal is negative, it outputs an imaging command to the imaging unit 300 to cause it to take an image. Based on the imaging command output by the information processing device 100, the imaging unit 300 takes one or more images in the direction the animal is looking and creates one or more image information. The imaging unit 300 outputs the created one or more image information in the direction the animal is looking to the processing unit 103. The processing unit 103 acquires the one or more image information output by the imaging unit 300.

[0041] (Step S9b-1) The creation unit 106 acquires one or more image information from the processing unit 103 and creates text information from one or more images in the direction of the animal's gaze based on the acquired image information. The creation unit 106 creates transmission information that includes one or more image information in the direction of the animal's gaze and the created text information. (Step S10b-1) The output unit 107 obtains the transmission information from the creation unit 106 and outputs the obtained transmission information. For example, the output unit 107 outputs the transmission information to a social networking service. The same process may be repeated thereafter.

[0042] In the embodiments described above, the information processing device 100 was described in which the imaging unit 300 takes an image of what the animal is looking at when the animal has positive emotions, and when the animal has both positive and negative emotions. However, the invention is not limited to these examples. For example, when the animal has negative emotions, the processing unit 103 may have the imaging unit 300 take one or more images of what the animal is looking at. The creation unit 106 of the information processing device 100 may create text information based on one or more image pieces of what the animal is looking at, as captured by the imaging unit 300. The creation unit 106 may create and output transmission information that includes one or more image pieces of what the animal is looking at and the created text information. For example, it may be output to a social networking service, or it may be transmitted to the terminal device of the animal's owner. This allows the animal's owner to know what the animal is feeling negative emotions towards.

[0043] Furthermore, the information processing device 100 may combine the information it generates when an animal has positive emotions and the information it generates when an animal has negative emotions, and output them to, for example, a social networking service. This allows the information to be transmitted about what the animal has positive and negative emotions towards. As an example of an animal, an example of text information included in the information transmitted by a dog would be: "It was a beautiful day today, and I had a great walk! There was a poodle in the park, and it was fun playing with it! After playing, I also saw a turtle, but it looked scary, and it was really frightening!"

[0044] In the embodiments described above, the information processing device 100 was described in the case where it estimates whether an animal has positive or negative emotions, but it is not limited to this example. For example, it may be estimated that the animal has neither positive nor negative emotions, or it may be estimated to have emotions other than positive or negative emotions.

[0045] In the embodiments described above, the estimation unit 102 may estimate whether an animal has positive or negative emotions based on one type of biological information. Alternatively, the estimation unit 102 may estimate whether an animal has positive or negative emotions based on multiple types of biological information. When the estimation unit 102 estimates whether an animal has positive or negative emotions based on multiple types of biological information, it may estimate whether an animal has positive or negative emotions based on any combination of the multiple types of biological information. Any combination of the multiple types of biological information may be associated with the type of biological information included in each of the combinations of the multiple types of biological information, and with information indicating whether the animal has positive or negative emotions.

[0046] When estimating an animal's emotions based on any combination of multiple types of biometric information, the estimation unit 102 may have a trained model built by machine learning that establishes the relationship between any biometric information from the multiple types of biometric information and information indicating the emotions of an animal possessing that biometric information. In this case, the estimation unit 102 may input the biometric information obtained from the input unit 101 into the trained model and obtain the information indicating the animal's emotions output by the trained model to estimate the animal's emotions.

[0047] (modified version) (Information processing device) Figure 3 shows an example configuration of an information processing device 400 as a modified embodiment. In addition to the information processing device 400, Figure 3 also shows a biological information measurement unit 200 and an imaging unit 300. The information processing device 400 and the biological information measurement unit 200 are connected by wire or wireless. The information processing device 400 and the imaging unit 300 are connected by wire or wireless.

[0048] The information processing device 400 receives biological information of an animal from the biological information measurement unit 200. Based on the input biological information of the animal, the information processing device 400 estimates the animal's emotions. Based on the estimated emotions of the animal, the information processing device 400 instructs the imaging unit 300 to capture an image of what the animal is looking at. Based on the image of what the animal is looking at captured by the imaging unit 300, the information processing device 400 generates image information captured from a direction different from the direction of the animal's gaze. Based on the generated image information captured from a direction different from the direction of the animal's gaze, the information processing device 400 creates text information and creates transmission information including the created text information. The information processing device 400 outputs the created transmission information. For example, the information processing device 400 creates a post to be posted to a social networking service as transmission information. The information processing device 400 posts the created post to the social networking service.

[0049] The information processing device 400 is implemented by a device such as a personal computer, server, smartphone, tablet computer, or industrial computer. The information processing device 400 includes, for example, an input unit 401, an estimation unit 402, a processing unit 403, an image generation unit 404, a content generation unit 405, a creation unit 406, an output unit 407, and a storage unit 410.

[0050] The input unit 401, estimation unit 402, processing unit 403, output unit 407, and storage unit 410 can be described using the same method as the input unit 101, estimation unit 102, processing unit 103, output unit 107, and storage unit 110 described above, so their description is omitted. The image generation unit 404 acquires one or more image information from the processing unit 403 that is in the direction of the animal's gaze. Based on the acquired one or more image information in the direction of the animal's gaze, the image generation unit 404 generates one or more image information taken from one or more images in the direction of the animal's gaze, from a direction different from the direction of the animal's gaze. For example, the image generation unit 404 is configured to include an image generation AI and can generate various images. Based on the acquired one or more image information in the direction of the animal's gaze, the image generation unit 404 generates one or more image information taken from a direction different from the direction of the animal's gaze. Specifically, the image generation unit 404 may generate an image that looks like the animal took a selfie by generating image information taken from a direction opposite to the direction of the animal's gaze, or it may generate an image that looks like a third party took a picture of the animal.

[0051] The content generation unit 405 acquires one or more image information from the image generation unit 404. Based on the acquired one or more image information, the content generation unit 405 creates content, such as text information. For example, the content generation unit 405 is configured to include a generation AI and generates various text information based on the image information. The creation unit 406 acquires one or more image information generated by the image generation unit 404 and acquires text information from the content generation unit 405. The creation unit 406 creates outgoing information that includes one or more image information and text information. For example, the creation unit 406 may create posting information to be posted to a social networking service, which includes one or more image information and text information, as outgoing information.

[0052] The image generation unit 404, the content generation unit 405, and the creation unit 406 are realized, for example, by a hardware processor such as a CPU executing a computer program (software) stored in the storage unit 410. Furthermore, some or all of these functional components may be implemented by hardware (including circuitry) such as LSIs, ASICs, FPGAs, and GPUs, or by the collaboration of software and hardware.

[0053] Computer programs may be stored in advance on a storage device such as an HDD or flash memory, or they may be stored on a removable storage medium such as a DVD or CD-ROM and installed when the storage medium is inserted into a drive device.

[0054] (Operation of information processing device) Figures 4A and 4B are flowcharts illustrating an example of the operation of the information processing device 400 according to a modified embodiment. Referring to Figure 4A, the operation when an animal's biological information is input to the information processing device 400 and it is determined that the input animal's biological information indicates a positive emotion will be output. Steps S1b-2 to S4b-2 and S8b-2 can be explained by applying steps S1a-1 to S4a-1 and S6a-1 in Figure 2A.

[0055] (Step S5b-2) The image generation unit 404 acquires one or more image information from the processing unit 403 that is in the direction of the animal's gaze, and generates one or more image information captured from a direction different from the direction of the animal's gaze based on the acquired one or more image information in the direction of the animal's gaze. (Step S6b-2) The content generation unit 405 acquires one or more image information captured from the image generation unit 404 from a direction different from the direction of the animal's gaze, and creates content, such as text information, based on the acquired one or more image information captured from a direction different from the direction of the animal's gaze. For example, the content generation unit 405 creates content to post to a social networking service based on one or more image information captured from a direction different from the direction of the animal's gaze.

[0056] (Step S7b-2) The creation unit 406 acquires one or more image information captured from the image generation unit 404 from a direction different from the direction of the animal's gaze, and acquires text information from the content generation unit 405. Based on the acquired one or more image information and text information, the creation unit 406 creates transmission information. For example, the creation unit 406 creates transmission information that includes one or more image information and text information to be posted to a social networking service.

[0057] Next, referring to Figure 4B, we will explain the operation when the information processing device 400 receives biological information of an animal and outputs information in both cases, such as when it determines that the animal is experiencing a positive emotion or a negative emotion based on the input biological information of the animal. Steps S1b-2 to S8b-2 can be explained by applying steps S1a-1 to S4a-1 and S6a-1 in Figure 2A and steps S5b-1 to S7b-2 in Figure 4A. (Step S9b-2) If the processing unit 403 determines in step S3b-2 that the estimated emotion of the animal is not a positive emotion, it determines, based on the estimated emotion of the animal, whether or not the estimated emotion is negative. If the processing unit 103 determines that the estimated emotion of the animal is not a negative emotion, it terminates the process.

[0058] (Step S10b-2) If the processing unit 403 determines that the estimated emotion of the animal is negative, it outputs an imaging command to the imaging unit 300 to cause it to take an image. Based on the imaging command output by the information processing device 400, the imaging unit 300 takes one or more images in the direction the animal is looking and creates one or more image information. The imaging unit 300 outputs the created one or more image information in the direction the animal is looking to the processing unit 403. The processing unit 403 acquires the one or more image information output by the imaging unit 300.

[0059] (Step S11b-2) The image generation unit 404 acquires one or more image information from the processing unit 403 that is in the direction of the animal's gaze, and generates one or more image information captured from a direction different from the direction of the animal's gaze based on the acquired one or more image information in the direction of the animal's gaze. (Step S12b-2) The content generation unit 405 acquires one or more image information captured from the image generation unit 404 from a direction different from the direction of the animal's gaze, and creates content, such as text information, based on the acquired one or more image information captured from a direction different from the direction of the animal's gaze. For example, the content generation unit 405 generates content to post to a social networking service based on one or more image information captured from a direction different from the direction of the animal's gaze.

[0060] (Step S13b-2) The creation unit 406 acquires one or more image information captured from the image generation unit 404 from a direction different from the direction of the animal's gaze, and acquires text information from the content generation unit 405. Based on the acquired one or more image information and text information, the creation unit 406 creates transmission information. For example, the creation unit 406 creates transmission information that includes one or more image information and text information to be posted to a social networking service. (Step S14b-1) The output unit 407 acquires the transmission information from the creation unit 406 and outputs the acquired transmission information. For example, the output unit 407 outputs the transmission information to a social networking service.

[0061] In the embodiment described above, the image generation unit 404 generates one or more image pieces of information taken from a direction different from the direction of the animal's gaze, based on one or more image pieces of information taken from the direction of the animal's gaze that was acquired. Here, a direction different from the direction of the animal's gaze includes, for example, a view from above or any other direction. In the embodiments described above, the creation unit 406 was described as creating outgoing information based on one or more image information and text information, but the invention is not limited to this example. For example, the creation unit may be configured to create outgoing information based on one or more image information or text information.

[0062] (Examples of application) An example of the application of this embodiment will be described. Using a dog as an example animal, a heart rate sensor is used as the biological information measurement unit 200, and the information processing device 400 acquires heart rate variability (HRV) data of the dog will be described. For example, on a typical sunny day, a dog is fitted with a wearable device that has a heart rate sensor attached. While the dog is wearing the wearable device, it transitions from resting to playing. The heart rate sensor dynamically captures the dog's shift from a calm state to an excited state, and the information processing device 400 estimates the dog's emotion based on the HRV detected by the heart rate sensor. If it estimates that the dog has a positive emotion, it causes the imaging unit 300 to capture an image in the direction the dog is looking.

[0063] The information processing device 400 processes the captured image to generate image information taken from a direction different from the direction of the animal's gaze. For example, the information processing device 400 generates an image depicting a dog leaping with joy while chasing a ball in a park. Based on the generated image information taken from a direction different from the direction of the animal's gaze, the information processing device 400 creates text information that contextualizes the sunny weather and time of day. The information processing device 400 generates transmission information that includes the image information taken from a direction different from the direction of the animal's gaze and the created text information. The information processing device 400 outputs the generated transmission information. For example, the information processing device 400 creates post information as transmission information that includes the image information taken from a direction different from the direction of the animal's gaze and the created text information, and posts the created post information to a social network. This allows the post information to be shared on social media.

[0064] Figures 5A and 5B are schematic diagrams showing an example of a wearable device 210 according to this embodiment. The wearable device 210 comprises a heart rate sensor 212 and a mounting member 214 for attaching the heart rate sensor 212. The heart rate sensor 212 is created by 3D (three-dimensional) printing using, for example, a lightweight and durable material. An example of the mounting member 214 includes a grip and a rubber band.

[0065] Figure 6 is a schematic diagram showing an example of how to attach the wearable device 210 according to this embodiment. In addition to the wearable device 210, Figure 6 shows a transceiver 215 that sends and receives data with the heart rate sensor 212, and a harness H for attaching the transceiver 215 to the neck of the dog DO. As shown in Figure 6, the wearable device 210 is attached to the auricle of the dog DO.

[0066] Traditionally, heart rate sensors 212 are often attached to the groin or abdomen of dogs. The groin and abdomen are sensitive areas for dogs, and their fur makes it difficult to attach the heart rate sensor. By attaching the heart rate sensor 212 as a wearable device 210 to the ear of the dog, the difficulty in attachment due to the dog's fur can be overcome, making it easier to attach than attaching it to the groin, and also making it less likely to fall off after being attached.

[0067] Figure 7 shows an example of heart rate variability data from a dog. Figure 7 shows heart rate variability data measured with a heart rate sensor attached to the groin of a dog (DO), and heart rate variability data measured with a heart rate sensor attached to the ear of a dog (DO). Figure 7 shows that the difference between the maximum and minimum values ​​is smaller in the heart rate variability data measured with the heart rate sensor attached to the ear of a dog (DO) than in the heart rate variability data measured with the heart rate sensor attached to the groin. This is presumably because the blood vessels in the groin are larger than those in the ear.

[0068] Figure 8 shows an example of the processing of the information processing device 400 according to this embodiment. The estimation unit 402 acquires heart rate variability data of dog DO input to the input unit 401. Based on the acquired heart rate variability data of dog DO, the estimation unit 402 derives the mean RRI interval. Let N be the number of RRI data points, and Ri be the value of the i-th RRI. In this case, the estimation unit 402 derives the mean RRI interval of RR intervals RR1, RR intervals RR2, ..., RR intervals RRn.

[0069] The estimation unit 402 derives the RMSSD (Root Mean Square of Successive Differences) based on the derived average RR interval. The RMSSD is calculated by squaring the RR interval in milliseconds (ms) and taking the square root of the average value. In other words, the RMSSD is defined as the root mean square of the difference between two consecutive RRI values. The RMSSD is expressed by equation (1).

number

[0070] The estimation unit 402 derives the SDNN (Standard Deviation of the NN intervals) based on the derived mean RR interval. The SDNN is the standard deviation of RRI. The SDNN is expressed by equation (2).

number

[0071] When dogs experience positive emotions, the SDNN value is low, and when dogs experience negative emotions, the RMSSD value is low (see Non-Patent Literature 1). The estimation unit 402 estimates whether the dog DO has positive or negative emotions based on the derived SDNN and RMSSD. Specifically, a first threshold for the SDNN and a second threshold for the RMSSD are predetermined to determine whether the dog has positive emotions. The estimation unit 402 estimates that the dog DO has positive emotions if the derived SDNN is less than or equal to the first threshold, and estimates that the dog DO has positive emotions if the derived RMSSD is greater than or equal to the second threshold. This makes it possible to estimate whether the dog DO has positive or negative emotions based on the dog DO's heart rate variability data.

[0072] The processing unit 403 obtains the estimated emotion of the animal from the estimation unit 402, and based on the obtained estimated emotion of the animal, outputs an imaging command to the imaging unit 300 if the estimated emotion of the animal is positive. The processing unit 403 terminates processing if the estimated emotion of the animal is not positive. The imaging unit 300 captures images in the direction of dog DO's gaze based on the imaging command output by the information processing device 400, and creates image information. This allows for imaging in response to changes in dog DO's emotions.

[0073] Figure 9A is a schematic diagram showing an example of how to attach the imaging unit 300 according to this embodiment. As shown in Figure 9A, the imaging unit 300 is attached to the neck of dog DO by a harness H. By attaching the imaging unit 300 to the neck of dog DO, the imaging direction of the imaging unit 300 can be aligned with the direction of dog DO's line of sight. Figure 9B shows an example of an image captured by the imaging unit 300 according to this embodiment. Figure 9B shows that the imaging unit 300 is capturing an image of what the dog DO is looking at. The imaging unit 300 outputs one or more image information of what the dog DO is looking at to the processing unit 403. The processing unit 403 acquires one or more image information output by the imaging unit 300.

[0074] The image generation unit 404 acquires one or more image information from the processing unit 403. Based on the acquired one or more image information, the creation unit 406 generates one or more image information from one or more images in the direction of dog DO's gaze, taken from a direction different from the direction of dog DO's gaze. Figure 10A is a diagram showing an example of an image captured by the imaging unit 300 according to this embodiment. Figure 10A shows four images captured by the imaging unit 300 from the perspective of dog DO. The number of images captured by the imaging unit 300 from the perspective of dog DO is not limited to four; it could be 1-3, or five or more. By capturing multiple images from the perspective of dog DO by the imaging unit 300, a narrative can be formed from dog DO's point of view.

[0075] Figure 10B is a diagram illustrating an example of the operation of the information processing device 400 according to this embodiment. Figure 10B shows an example of four images captured from a direction different from the direction of the dog DO's gaze, generated by the image generation unit 404. For example, the image generation unit 404 may generate an image that appears to be a selfie taken by the dog DO, or an image that appears to be a photograph taken by a third party. Figure 10B shows an example of an image that appears to be a selfie taken by the dog DO.

[0076] The content generation unit 405 acquires one or more image information captured from the image generation unit 404 from a direction different from the direction of the animal's gaze. Based on the acquired one or more image information, the content generation unit 405 creates content, such as text information. Figure 11 shows an example of the operation of the information processing device 400 according to this embodiment. Figure 11 shows an example of document information created by the content generation unit 405. According to Figure 11, it can be seen that text has been created to appeal to the image of a dog.

[0077] The creation unit 406 acquires one or more image information captured from the image generation unit 404 from a direction different from the direction of the animal's gaze, and acquires text information from the content generation unit 405. The creation unit 406 creates transmission information that includes one or more image information captured from a direction different from the direction of the animal's gaze and the text information. For example, the creation unit 406 may create transmission information that includes one or more image information and text information to be posted to a social networking service.

[0078] The output unit 107 acquires transmission information from the creation unit 106. The output unit 107 outputs the acquired transmission information. For example, if the output unit 107 acquires posting information as transmission information, it posts the posting information to the dog DO's own account on a social networking service. The dog DO's own account is created in advance. This allows information to be transmitted based on the estimated emotions of the dog DO, enabling real-time data sharing and integration with social media for interaction.

[0079] The application example described above involved the information processing device 400 acquiring heart rate variability (HRV) data of dogs, but the method is not limited to this example. For example, it can also be appropriately applied when the information processing device 100 acquires heart rate variability (HRV) data of dogs. In the application example, an accelerometer or other sensors may be attached to the dog DO, and the information processing device 400 may acquire detection data detected by the accelerometer or other sensors. By combining this with the detection data, the information processing device 400 can add functions not only for emotion estimation but also for tracking the location of the dog DO during walks and predicting its health status. By utilizing the function of tracking the location of the dog DO during walks, it is also possible to develop an algorithm that identifies places where the dog DO frequently marks its territory and recommends parks and walking routes that the dog DO likes.

[0080] An example of the application of the information processing system 1 in this embodiment will be described. (1) Home pet care Information processing system 1 is used as a device that allows dog owners to understand their pet's emotions and health status in real time. By using information processing system 1 on a daily basis, owners can monitor their dog's stress level and health status, enabling them to provide appropriate care and health management. Furthermore, by combining accelerometers and other sensors, it's possible to add features such as tracking the dog's location during walks and predicting its health status. Utilizing the location tracking function, it's possible to identify places where dogs frequently mark their territory and develop algorithms to recommend parks and walking routes that dogs prefer.

[0081] (2) Animal behavior research Research institutions and universities will utilize Information Processing System 1 for animal behavior and emotion research. In particular, it can be used as a tool to measure stress levels and emotional changes in dogs in real time and to investigate the relationship between stress levels, emotional changes, and behavioral patterns. The research results will be used to collect data that contributes to improving animal welfare.

[0082] (3) Pet-related social networking services By generating content based on pets' emotions, a new form of service can be developed on social networking services to share pets' lives. This allows pet owners to share their pets' activities and emotions in real time on social networking services, providing a platform for forming pet communities and exchanging information. Furthermore, by using image generation AI to create images from a third-person perspective of what the pet is looking at, making it appear as if the dog took a selfie, the entertainment value can be further enhanced.

[0083] (4) Marketing and sales of pet products With the expansion of the pet wearable device market, this can be positioned as a new product line. It allows for reaching a wide range of users through sales in pet shops and online stores.

[0084] (5) Health management of livestock By monitoring livestock stress levels and providing appropriate care and management, it is possible to improve animal welfare and productivity. This technology is particularly promising for applications in the dairy and livestock industries. Combining emotion estimation with location tracking and health prediction capabilities will allow for more effective livestock health management.

[0085] (6) Recommendations and surveys on toys tailored to dogs' preferences Because it can estimate a dog's emotions, it's possible to select toys that match the dog's hobbies and preferences based on those emotions. This can reduce stress and increase satisfaction for the dog. Furthermore, by conducting surveys with dogs, it's possible to understand their toy preferences and emotional states, allowing for more personalized care.

[0086] According to the information processing system of this embodiment, the system includes: an estimation unit that estimates the animal's emotions based on the animal's biological information; a processing unit that causes an imaging unit to capture an image of what the animal is looking at based on the animal's emotions estimated by the estimation unit; and a creation unit that creates transmission information including text information based on the image of what the animal is looking at captured by the imaging unit. By configuring the system in this way, the information processing system can, based on the estimated emotions of the animal, have the imaging unit capture an image of what the animal is looking at, and then create transmission information, including textual information, based on the captured image of what the animal is looking at, thereby enabling the transmission of information.

[0087] The information processing system further includes an image generation unit that generates image information from a direction different from the direction of the animal's gaze, based on image information from the direction of the animal's gaze captured by the imaging unit. The creation unit creates transmission information, including text information, based on the image information from a direction different from the direction of the animal's gaze generated by the image generation unit. By configuring the system in this way, the information processing system can generate image information taken from a different direction than the direction of the animal's gaze, based on the image information of the animal's line of sight. This allows the system to create communication information, including text, based on images that appear to be selfies taken by the animal or images taken by a third party.

[0088] The information processing system further includes a content generation unit that generates content based on image information of other points of view generated by an image generation unit. The creation unit creates transmission information, including text information, based on the content generated by the content generation unit. By configuring it in this way, the information processing system can create transmitted information, including textual information, based on the generated images of other points of view, and thus transmit information.

[0089] The information processing system further includes a content generation unit that generates content based on image information of the animal's line of sight captured by the imaging unit. The creation unit creates transmission information, including text information, based on the content generated by the content generation unit. By configuring it in this way, the information processing system can create transmitted information, including text information, based on the image information of the animal's line of sight, and thus transmit information.

[0090] The information processing system further includes a bio-information measuring unit that is attached to an animal and measures the animal's ecological information. The estimation unit estimates the animal's emotions based on the animal's bio-information measured by the bio-information measuring unit. By configuring it in this way, the information processing system can estimate the animal's emotions based on the animal's biological information measured by a biological information measuring unit that is attached to the animal and measures the animal's ecological information.

[0091] In an information processing system, the biological information of an animal includes at least one of the following: heart rate, heart rate variability, respiratory rate, pulse wave, electroencephalogram, and skin temperature. By configuring it in this way, the information processing system can estimate the animal's emotions based on at least one of the animal's heart rate, heart rate variability, respiratory rate, pulse wave, electroencephalogram, and skin temperature.

[0092] In information processing systems, animal emotional information includes either negative or positive emotions. By configuring it in this way, the information processing system can estimate whether an animal is experiencing negative or positive emotions based on its biological information.

[0093] In the information processing system, the imaging unit captures an image of the animal's gaze if the animal's emotion, as estimated by the estimation unit, is either positive, negative, or both. By configuring the system in this way, the information processing system can, based on the estimated emotions of the animal, have the imaging unit capture an image of what the animal is looking at, and then, based on the image of what the animal is looking at in either a positive emotion, a negative emotion, or both, create posting information to be posted to a social networking service, thereby enabling the dissemination of information.

[0094] In an information processing system, the creation unit creates posting information to be submitted to a social networking service. By configuring it in this way, the information processing system can share posted information in real time on social networking services.

[0095] Although embodiments of the present invention have been described above, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, modifications, and combinations can be made without departing from the spirit of the invention. For example, an embodiment may be combined with a modification 2 of the embodiment. These embodiments and their modifications are included in the scope and spirit of the invention, as well as in the claims of the invention and its equivalents.

[0096] Furthermore, the aforementioned information processing devices 100 and 400 have computers inside. The processes of each of the aforementioned devices are stored in program form on a computer-readable recording medium, and the above processes are performed when the computer reads and executes this program.

[0097] Here, computer-readable recording media refer to magnetic disks, magneto-optical disks, CD-ROMs, DVD-ROMs, semiconductor memory, etc. Alternatively, this computer program may be distributed to a computer via a communication line, and the computer that receives the program may execute it. Furthermore, the above program may be intended to implement some of the functions described above. In addition, it may be a so-called differential file (differential program) that can implement the above functions in combination with a program already recorded on the computer device. [Explanation of symbols]

[0098] 100, 400…Information processing unit, 101, 401…Input unit, 102, 402…Estimation unit, 103, 403…Processing unit, 106, 406…Creation unit, 107, 407…Output unit, 110, 410…Storage unit, 200…Biological information measurement unit, 300…Imaging unit, 404…Image generation unit, 405…Content generation unit

Claims

1. An estimation unit that estimates the emotions of the animal based on the animal's biological information, Based on the emotions of the animal estimated by the estimation unit, the processing unit causes the imaging unit to capture an image of what the animal is looking at, A creation unit creates transmission information including textual information based on the image of the animal's line of sight captured by the imaging unit, An information processing system equipped with the following features.

2. Image generation unit generates image information taken from a direction different from the direction of the animal's gaze, based on the image information of the animal's line of sight captured by the imaging unit. Furthermore, The information processing system according to claim 1, wherein the creation unit creates transmission information including text information based on the image information generated by the image generation unit.

3. A content generation unit generates content based on the image information generated by the image generation unit. Furthermore, The information processing system according to claim 2, wherein the creation unit creates transmission information including text information based on the content generated by the content generation unit.

4. A content generation unit generates content based on the image information of the animal's line of sight captured by the imaging unit. Furthermore, The information processing system according to claim 1, wherein the creation unit creates transmission information including text information based on the content generated by the content generation unit.

5. A biological information measuring unit attached to an animal to measure the animal's ecological information. Furthermore, The information processing system according to claim 1, wherein the estimation unit estimates the emotions of the animal based on the biological information of the animal measured by the biological information measurement unit.

6. The information processing system according to any one of claims 1 to 5, wherein the biological information of the animal includes at least one of the animal's heart rate, heart rate variability, respiratory rate, pulse wave, electroencephalogram, and skin temperature.

7. The information processing system according to claim 6, wherein the emotional information of the animal includes either negative emotions or positive emotions.

8. The information processing system according to claim 7, wherein the imaging unit captures an image of the direction the animal is looking if the emotion of the animal estimated by the estimation unit is either a positive emotion or a negative emotion, or both.

9. The information processing system according to any one of claims 1 to 5, wherein the creation unit creates posting information to be posted to a social networking service.

10. A method of information processing performed by a computer, Based on the animal's biological information, the emotions of the said animal are estimated. Based on the estimated emotions of the animal, the direction in which the animal is looking is captured in an image. Based on the image of the animal's line of sight, which has been captured, information including text information is generated. Information processing methods.

11. On the computer, Based on the animal's biological information, the emotions of the said animal are estimated. Based on the estimated emotions of the animal, the imaging unit is instructed to capture an image of what the animal is looking at. Based on the image of the animal's line of sight, the system generates communication information including textual information. program.

Images