Information processing device, information processing method, and program

Abstract

This information processing device comprises an image acquisition means, a state estimation means, an information retrieval means and an output means. The image acquisition means acquires an image of a subject organism captured by a terminal device. The state estimation means estimates a state of the subject organism from the image by using a machine learning model that has learned a relationship between an image of a subject organism and a state of a subject organism. The information retrieval means retrieves action information for taking action on the state of the subject organism. The outputs means outputs the action information to the terminal device.

Description

Information processing device, information processing method, and recording medium

[0001] The present disclosure relates to a technology for predicting illness or injury and searching for a treatment method.

[0002] When a pet animal (hereinafter referred to as "pet") becomes sick or injured, the pet owner needs to take appropriate measures. For example, Patent Document 1 describes a health examination device that estimates the illness of a pet from a video taken of the pet.

[0003] Japanese Patent Application Laid-Open No. 2014-223063

[0004] However, even with the method disclosed in Patent Document 1, it is difficult for pet owners to provide appropriate treatment for their pets.

[0005] One object of the present disclosure is to provide an information processing device capable of providing information necessary for a user to take appropriate measures for an illness or injury.

[0006] In order to solve the above problem, in one aspect of the present disclosure, an information processing device includes: an image acquisition means for acquiring an image of a target organism photographed by a terminal device; a state estimation means for estimating a state of the target organism from the image using a machine learning model that has learned the relationship between the image of the target organism and the state of the target organism; an information search means for searching for response information for dealing with the state of the target organism; and an output means for outputting the response information to the terminal device.

[0007] In another aspect of the present disclosure, an information processing method includes acquiring an image of a target organism photographed with a terminal device, estimating the state of the target organism from the image using a machine learning model that has learned the relationship between the image of the target organism and the state of the target organism, searching for response information to address the state of the target organism, and outputting the response information to the terminal device.

[0008] In yet another aspect of the present disclosure, a recording medium records a program that causes a computer to execute the following processes: acquire an image of a target organism photographed with a terminal device; estimate the state of the target organism from the image using a machine learning model that has learned the relationship between the image of the target organism and the state of the target organism; search for response information to deal with the state of the target organism; and output the response information to the terminal device.

[0009] According to the present disclosure, users can take appropriate measures against illness or injury.

[0010] 1 shows the overall configuration of a communication system to which an information processing device according to a first embodiment is applied; FIG. 2 is a block diagram showing the configurations of a server and a user terminal; FIG. 3 is a block diagram showing the configuration of a pet terminal; FIG. 4 is a block diagram showing the functional configuration of a server; FIG. 5 is a block diagram showing the functional configuration of a learning device; FIG. 6 shows an example of handling information; FIG. 7 is a flowchart of display processing by an information processing device; FIG. 8 is a block diagram showing the functional configuration of a first modified example of the first embodiment; FIG. 9 is a block diagram showing the functional configuration of an information processing device according to a second embodiment; FIG. 10 is a flowchart of processing by an information processing device according to the second embodiment.

[0011] First Embodiment [Overall Configuration] Fig. 1 shows the overall configuration of a communication system to which an information processing device according to the present disclosure is applied. The communication system 1 includes a server 200 and a user terminal 300 used by a pet owner. A pet P is located within a predetermined range of the pet owner, and a pet terminal 20 is attached to the pet P. The server 200 and the owner's user terminal 300 are capable of wireless communication. The user terminal 300 and the pet terminal 20 are also capable of wireless communication. Note that providing the pet terminal 20 is not essential.

[0012] As a basic operation, when pet P shows symptoms of illness or is injured, the owner makes a predetermined statement such as "hospital." When the user terminal 300 recognizes the predetermined statement from the owner, it activates the camera and captures a video of pet P. Then, the user terminal 300 transmits the captured video (hereinafter also referred to as "captured video") to the server 200. The server 200 estimates the condition of pet P from the video captured by the user terminal 300 using image analysis by AI (Artificial Intelligence) or the like. Then, the server 200 searches for treatment information based on the estimated condition of pet P and transmits the search results to the user terminal 300. This allows the owner to know how to treat pet P's illness or injury.

[0013] The condition of pet P refers to the state of illness or injury of pet P. For example, illnesses include coughing and convulsions. Injuries include the size of the wound and the degree of bleeding. The server 200 estimates the condition of pet P from the behavior and appearance of pet P contained in the video captured by the user terminal 300. The treatment information is information related to how to treat illness or injury. The treatment information includes methods for treating illness or injury and hospital information. The server 200 searches for treatment information such as appropriate treatment methods and hospital information depending on the condition of pet P.

[0014] 2A is a block diagram showing the configuration of the server 200. The server 200 mainly includes a communication unit 211, a processor 212, a memory 213, a recording medium 214, and a database (DB) 215.

[0015] The communication unit 211 transmits and receives data to and from external devices. Specifically, the communication unit 211 transmits and receives information to and from the user terminal 300 of the owner.

[0016] The processor 212 is a computer such as a CPU (Central Processing Unit), and executes a prepared program to control the entire server 200. The processor 212 may be a GPU (Graphics Processing Unit), an FPGA (Field-Programmable Gate Array), a DSP (Demand-Side Platform), an ASIC (Application Specific Integrated Circuit), or the like.

[0017] The memory 213 is configured by a ROM (Read Only Memory), a RAM (Random Access Memory), etc. The memory 213 is also used as a working memory while the processor 212 is executing various processes. The memory 213 also temporarily stores a series of videos captured by the user terminal 300 under the control of the processor 212. The videos are stored in the memory 213 in association with, for example, the owner's identification information, timestamp information, etc.

[0018] The recording medium 214 is a non-volatile, non-transitory recording medium such as a disk-shaped recording medium or a semiconductor memory, and is configured to be detachable from the server 200. The recording medium 214 records various programs executed by the processor 212.

[0019] Response information for each state of the pet is stored in database (DB) 215. DB 215 may include an external storage device such as a hard disk connected to or built into server 200, or may include a storage medium such as a removable flash memory. Note that instead of providing DB 215 in server 200, DB 215 may be provided in an external server, and response information may be obtained from the server via communication.

[0020] The server 200 may also include an input unit such as a keyboard and a mouse for an administrator to give instructions and input, and a display unit such as a liquid crystal display.

[0021] 2B is a block diagram showing the internal configuration of a user terminal 300 used by a pet owner. The user terminal 300 is a terminal device such as smart glasses or a smartphone. The user terminal 300 includes a communication unit 311, a processor 312, a memory 313, a display unit 314, a camera 315, a microphone 316, and a GPS receiver 317.

[0022] The communication unit 311 transmits and receives data to and from external devices. Specifically, the communication unit 311 transmits and receives information to and from the server 200.

[0023] The processor 312 is a computer such as a CPU, and executes a program prepared in advance to control the entire user terminal 300. The processor 312 may be a GPU, FPGA, DSP, ASIC, etc. The processor 312 executes the program prepared in advance to transmit video captured by the camera 315 to the server 200.

[0024] The memory 313 is composed of a ROM, a RAM, etc. The memory 313 stores various programs executed by the processor 312. The memory 313 is also used as a working memory while the processor 312 is executing various processes. Video captured by the camera 315 is stored in the memory 313 and then transmitted to the server 200. The display unit 314 is, for example, a liquid crystal display device, and displays the video captured by the camera 315, handling information received by the user terminal 300, etc.

[0025] The camera 315 is a camera that captures the user's field of view. The camera 315 captures the user's field of view, including subjects such as pets, and transmits the captured images to the server 200. This allows the server 200 to obtain video footage of subjects such as pets. The microphone 316 collects the user's voice and surrounding sounds and transmits the collected audio to the server 200. The server 200 can estimate the state of the pet P based on, for example, the sounds of the pet P running around or the cries of the pet P. The GPS receiver 317 generates location information of latitude and longitude that indicates the current location of the user terminal 300.

[0026] 3 is a block diagram showing the configuration of the pet terminal 20 attached to the pet P. The pet terminal 20 is attached to the pet P in place of the collar of the pet P, for example, or is attached to the collar worn by the pet. The pet terminal 20 includes a communication unit 21, a processor 22, a memory 23, a pet camera 24, an acceleration sensor 25, an air pressure sensor 26, a biosensor 27, and a microphone 28.

[0027] The communication unit 21 communicates with external devices. Specifically, the communication unit 21 wirelessly communicates with the user terminal 300 using, for example, Wi-Fi (registered trademark) or Bluetooth (registered trademark).

[0028] The processor 22 is a computer such as a CPU, and executes a program prepared in advance to control the entire pet terminal 20. The processor 22 executes the program prepared in advance to periodically transmit information acquired by each sensor to the user terminal 300.

[0029] The memory 23 is composed of a ROM, a RAM, etc. The memory 23 stores various programs executed by the processor 22. The memory 23 is also used as a working memory while the processor 22 is executing various processes. Furthermore, the memory 23 temporarily stores information detected by each sensor.

[0030] Pet camera 24 is a camera for capturing images from the pet's point of view. Pet camera 24 may be configured to detect the orientation of pet P's neck to determine the capture direction, or may be attached near pet P's head, or may be a camera that captures a wide-angle image in front of pet P. Pet camera 24 captures an area including pet P's line of sight and transmits the captured image to user terminal 300. This allows user terminal 300 to obtain an image from the pet's point of view.

[0031] The acceleration sensor 25 is a three-axis acceleration sensor that measures the movement of the pet P in three axes and transmits the measured values ​​to the user terminal 300. Based on the output of the acceleration sensor 25, the user terminal 300 can estimate the amount of activity of the pet P. The barometric pressure sensor 26 measures the barometric pressure at the location of the pet P and transmits the measured values ​​to the user terminal 300. Based on the output of the barometric pressure sensor 26, the user terminal 300 can detect the vertical movement of the pet P, such as jumping. Although not shown in FIG. 3 , a gyro sensor may also be used. Alternatively, a six-axis sensor that combines a three-axis acceleration sensor and a three-axis gyro sensor (a three-axis angular velocity sensor) may also be used. Note that the sensor is not limited to the above, as long as it is a sensor that can measure the amount of activity of an animal.

[0032] The biosensor 27 is a sensor that measures biometric information of the pet P, such as the body temperature, heart rate, and respiratory rate of the pet P, and transmits the information to the user terminal 300. The microphone 28 collects sounds around the pet P and transmits the sounds to the user terminal 300.

[0033] 4 is a block diagram showing the functional configuration of the server 200. Functionally, the server 200 includes an image acquisition unit 411, a state estimation unit 412, an information search unit 413, and an information transmission unit 414.

[0034] The server 200 receives a captured video from the user terminal 300. The captured video is input to the image acquisition unit 411. The image acquisition unit 411 outputs the captured video to the state estimation unit 412.

[0035] The state estimation unit 412 performs image analysis based on the captured video acquired from the image acquisition unit 411 and estimates the state of the pet P. The state estimation unit 412 recognizes the behavior and appearance of the pet P included in the captured video using a previously prepared image recognition model or the like and estimates the state of the pet P. This image recognition model is a machine learning model that has been trained in advance to estimate the state of an animal, and will hereinafter also be referred to as a "state estimation model." The state estimation unit 412 estimates the state of the pet P and outputs the estimated state of the pet P to the information search unit 413.

[0036] The information search unit 413 searches for treatment information from the DB 215 based on the condition of the pet P acquired from the condition estimation unit 412. The DB 215 includes a treatment DB 215a and a hospital DB 215b. The treatment DB 215a stores treatment methods for each condition of the pet. The information search unit 413 searches for treatment methods corresponding to the condition of the pet P from the treatment DB 215a. The hospital DB 215b stores hospital information for each condition of the pet. The information search unit 413 searches for hospital information corresponding to the condition of the pet P from the treatment DB 215b. Note that although the information search unit 413 searches for treatment information from the database 215, the information search unit 413 may instead search for treatment information from the Internet. The information search unit 413 outputs the condition of the pet P acquired from the condition estimation unit 412 and the treatment information to the information transmission unit 414.

[0037] The information transmitting unit 414 outputs the treatment information acquired from the information searching unit 413 to the user terminal 300. This eliminates the need for the owner of pet P to investigate the illness or injury of pet P, and allows the owner to concentrate on treating pet P. The information transmitting unit 414 may also transmit the condition of pet P to the hospital in advance. This allows hospital personnel to know in advance the symptoms of pet P, who is expected to visit the hospital soon.

[0038] In addition to the state of the pet P, the state estimation unit 412 may estimate attribute information of the pet P, which will be described later, and output the information search unit 413. The attribute information includes the type of animal; for example, the state estimation unit 412 estimates the type of animal of the pet P, such as whether the pet P is a dog or a cat. The information search unit 413 searches for treatment information from the database 215 or the Internet based on the state of the pet P and the attribute information of the pet P acquired from the state estimation unit 412. This enables the information search unit 413 to search for treatment methods for different types of animals, such as treatment methods for dogs and treatment methods for cats, and information on hospitals for different types of animals, such as specialized hospitals for dogs and specialized hospitals for cats, allowing the owner of the pet P to find treatment information that is more suitable for the pet P.

[0039] [Learning of State Estimation Model] Next, an example of learning of the state estimation model used by the state estimation unit 412 will be described. The state estimation model is generated by so-called supervised learning. Fig. 5 is a block diagram showing a method for learning the state estimation model, and includes learning data 511 and a learning device 512.

[0040] The training data 511 is pre-labeled still image data or video data (hereinafter also referred to as "training data"). The training data is created by extracting areas of an animal from still image data or video data of an animal and assigning labels related to the animal's behavior, appearance, etc. to the extracted areas. The animal's behavior and appearance include symptoms that appear when the animal is sick or injured (hereinafter also referred to as "animal condition"). The learning device 512 generates a state estimation model that learns the relationship between the image of the animal and the animal's condition based on the training data 511. The state estimation unit 412 uses the generated state estimation model to estimate the condition of the pet P included in the input video. For example, the state estimation unit 412 outputs an estimation result such as the pet P is coughing or bleeding.

[0041] Furthermore, by using still image data or video data to which labels related to the type of animal, in addition to labels related to the behavior and appearance of the animal, as training data for machine learning, the learning device 512 can generate a state estimation model that has learned the relationship between the image of the animal, the state of the animal, and the type of animal. As a result, the state estimation unit 412 can estimate the type of pet P and the state of the pet P from the input captured video using the generated state estimation model. For example, the state estimation unit 412 can output an estimation result such as "the dog is coughing," "the dog is bleeding," or "the cat is coughing."

[0042] [Example of Response Information] Next, a description will be given of the response information searched by the information search unit 413. Fig. 6 shows an example of treatment methods and hospital information as the response information.

[0043] FIG. 6A shows an example of a treatment method, and includes a treatment method 611 and an explanatory video 612. The treatment method 611 is text information or image information describing a method for treating an illness or injury. Examples of treatment methods for an illness or injury include a method for fixing a broken bone or a method for providing artificial respiration. The information search unit 413 searches for such treatment methods according to the condition of the pet P. The explanatory video 612 is a video about a method for treating an illness or injury, and includes a video demonstrating the treatment. The explanatory video 612 is used when it is difficult to perform treatment using only the information in the treatment method 611, for example, artificial respiration for a dog.

[0044] FIG. 6B shows an example of hospital information, including basic information 613 and map information 614. The basic information 613 is basic information about the hospital, including the name of the hospital, the animals treated, the medical department, the hospital's address, and the hospital's telephone number. The information search unit 413 searches for hospital information based on the type of pet P and the condition of the pet P. For example, if the pet P is a dog and has an injury, the information search unit 413 searches for hospitals that include dogs as a medical department and surgery as a medical department. The information search unit 413 also searches for hospitals within a predetermined range of the user terminal 300 based on the location information of the user terminal 300 and the hospital's address. The map information 614 is route information from the current location to the hospital. The map information 614 includes information such as a map showing the route from the current location to the hospital and the required time to get from the current location to the hospital.

[0045] [Display Example] Next, a display example by the user terminal 300 will be described. Fig. 7 shows a display example of response information transmitted by display processing by an information processing device. In the display example of Fig. 7, the treatment method of Fig. 6(A) and hospital information of Fig. 6(B) are displayed as response information within the user terminal 300. By using smart glasses as the user terminal 300, the owner of pet P can treat pet P with both hands while viewing the response information. Note that instead of smart glasses, other eyeglass-type wearable devices such as AR (Augmented Reality) glasses, MR (Mixed Reality) glasses, or VR (Virtual Reality) glasses may be used.

[0046] [Display Processing] Next, the display processing for performing the above-described display will be described. Fig. 8 is a flowchart of the display processing by the information processing device. This processing is realized by the processor 212 shown in Fig. 2 executing a program prepared in advance and operating as each element shown in Fig. 4.

[0047] First, when pet P becomes ill or injured, the owner of pet P makes a predetermined statement, such as "hospital." The user terminal 300 recognizes the owner's predetermined statement and starts recording a video. The user terminal 300 then transmits the recorded video to the server 200. The image acquisition unit 411 of the server 200 acquires the recorded video from the user terminal 300 (step S11). The image acquisition unit 411 outputs the recorded video to the state estimation unit 412. Next, the state estimation unit 412 estimates the state of pet P based on the recorded video acquired from the image acquisition unit 411 using a state estimation model that has been trained in advance (step S12). The state of pet P refers to the state of illness or injury of pet P. The state estimation unit 412 outputs the estimation result to the information search unit 413.

[0048] Next, the information search unit 413 searches for countermeasure information from the database 215 based on the state of the pet P acquired from the state estimation unit 412 (step S13). Countermeasure information is stored in the database 215 for each state of the pet. The countermeasure information also includes treatment methods for illnesses and injuries, hospital information, and the like. The information search unit 413 may also search for countermeasure information from the Internet. The information search unit 413 outputs the searched countermeasure information to the information transmission unit 414. Next, the information transmission unit 414 transmits the countermeasure information acquired from the information search unit 413 to the user terminal 300 (step S14). Then, the processing ends.

[0049] [Modifications] Next, modifications of the first embodiment will be described. The following modifications can be applied to the first embodiment in appropriate combinations. (Modification 1) In the first embodiment described above, the server 200 estimates the state of the pet P based on the video captured by the user terminal 300. Instead, the server 200 may estimate the state of the pet P using biological information of the pet P in addition to the captured video. FIG. 9 shows the functional configuration of the server 200a of Modification 1. As shown in the figure, in Modification 1, a sensor information acquisition unit 415 is provided in the server 200a. The sensor information acquisition unit 415 acquires biological information of the pet P detected by the pet terminal 20 from the user terminal 300. The biological information of the pet P includes body temperature, heart rate, respiratory rate, etc. The sensor information acquisition unit 415 outputs the acquired biological information of the pet P to the state estimation unit 412.

[0050] The state estimation model used by the state estimation unit 412 is a trained model that has been trained in advance to estimate the state of the pet P based on the captured video and biological information. The state estimation unit 412 estimates the behavior and appearance of the pet P based on the captured video. The state estimation unit 412 also estimates whether the pet P's body temperature, heart rate, respiratory rate, etc. are normal or abnormal based on the pet P's biological information. For example, if the pet P's body temperature is equal to or higher than a predetermined threshold TH1 or if the pet P's body temperature is equal to or lower than a predetermined threshold TH2, the state estimation unit 412 estimates the pet P's body temperature as "abnormal." On the other hand, if the pet P's body temperature is lower than the predetermined threshold TH1 and higher than the predetermined threshold TH2, the state estimation unit 412 estimates the pet P's body temperature as "normal." The state estimation unit 412 then integrates the results estimated from the captured video and the pet P's biological information and outputs the integrated results to the information search unit 413 as the state of the pet P. The condition estimation unit 412 outputs to the information search unit 413 the condition of pet P, such as, for example, "Pet P is coughing, body temperature is normal, heart rate is normal, and respiratory rate is normal" or "Pet P is coughing, body temperature is abnormal (high), heart rate is abnormal (high), and respiratory rate is abnormal (high)."

[0051] The information search unit 413 searches for handling information from the DB 215 or the Internet based on the state of the pet P acquired from the state estimation unit 412. The DB 215 stores handling information in advance for each state of the pet P. The information search unit 413 outputs the searched handling information to the information transmission unit 414. Then, the information transmission unit 414 transmits the handling information acquired from the information search unit 413 to the user terminal 300.

[0052] As described above, by using the captured video as well as the biological information of the pet P, the server 200 can narrow the search range from among the many pieces of response information contained in the DB 215, and can narrow down the search, thereby making it possible to acquire response information that is more suitable for the pet P. Note that it is not essential to estimate the condition of the pet using the biological information from the pet terminal 20 as described above.

[0053] (Variation 2) In the first embodiment described above, the server 200 estimates the condition of the pet P based on a video of the pet P and transmits response information to the user terminal 300. However, the target living thing (also referred to as the "target living thing") is not limited to a pet. For example, the server 200 may estimate the condition of a human based on a video of a human and transmit response information to the user terminal 300. This enables the user to obtain appropriate response information for a person who cannot verbally describe the symptoms of an illness or injury, such as an infant.

[0054] (Variation 3) In the first embodiment described above, the server 200 estimates the state of the pet P. However, the server 200 may estimate attribute information of the target creature in addition to the state of the target creature such as the pet P. The attribute information includes the type of creature (e.g., human, dog, cat), age (e.g., infant, toddler, elderly), and gender. The state estimation unit 412 estimates the state of the target creature and the attribute information of the target creature from the captured video using a pre-prepared image recognition model or the like. The state estimation unit 412 outputs the state of the target creature and the attribute information of the target creature to the information search unit 413. Then, the information search unit 413 searches the database 215 or the Internet for response information based on the state of the target creature and the attribute information of the target creature acquired from the state estimation unit 412. Note that the database 215 stores response information in advance for each state of the target creature and each attribute information of the target creature. For example, if the condition of the target organism is a burn and the attribute information of the target organism is an infant, the information search unit 413 searches for a method for treating burns in infants, information on pediatric hospitals, etc. This allows the user to find information on how to deal with the target organism that is more suitable for the target organism.

[0055] (Variation 4) In the first embodiment, when the owner of pet P makes a predetermined statement such as "hospital," the server 200 searches for response information corresponding to the pet P's condition at that time and transmits the information to the owner's user terminal 300. In addition to the above, the server 200 may use previously recorded videos of pet P to detect behaviors that could lead to pet P's illness or injury (hereinafter also referred to as "risky behaviors") and transmit a warning message to the owner's user terminal 300. Previously recorded videos of pet P include videos of pet P's daily life and videos recorded when pet P became ill or injured. The server 200 detects pet P's risky behaviors from previously recorded videos of pet P using a prepared image recognition model or the like. Examples of pet P's risky behaviors include pet P engaging in behaviors that may lead to accidental ingestion or pet P exhibiting warning behaviors of epilepsy. If the number of times pet P engages in risky behaviors exceeds a predetermined number, the server 200 transmits a warning message to the owner's user terminal 300. This allows the owner to prevent illness or injury to the pet P in advance.

[0056] (Variation 5) In the first embodiment described above, information acquired by the user terminal 300 is basically transmitted as is to the server 200, and the server 200 estimates the state of the pet P based on the received information and transmits handling information to the user terminal 300. Alternatively, the user terminal 300 may estimate the state of the pet P and search for handling information. This can reduce the communication load from the user terminal 300 to the server 200. In these cases, the user terminal 300 is an example of an information processing device.

[0057] Furthermore, when the user terminal 300 searches for the response information and the response information includes hospital information, the user terminal 300 may ask the user by voice whether or not to contact a hospital. If the user replies that they will contact a hospital, the user terminal 300 may start a calling application and call the hospital. This allows the user to confirm in advance with the hospital whether they can accept an emergency patient.

[0058] 10 is a block diagram showing the functional configuration of an information processing device 50 according to a second embodiment. The information processing device 50 according to the second embodiment includes an image acquisition unit 51, a state estimation unit 52, an information search unit 53, and an output unit 54.

[0059] 11 is a flowchart of processing by the information processing device 50. The image acquisition means 51 acquires an image of the target organism photographed by a terminal device (step S51). The state estimation means 52 estimates the state of the target organism from the image using a machine learning model that has learned the relationship between the image of the target organism and the state of the target organism (step S52). The information search means 53 searches for response information for addressing the state of the target organism (step S53). The output means 54 outputs the response information to the terminal device (step S54).

[0060] According to the information processing device 50 of the second embodiment, the user can take appropriate measures for illness or injury of the target organism.

[0061] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes.

[0062] (Supplementary Note 1) An information processing device comprising: an image acquisition means for acquiring an image of a target organism photographed with a terminal device; a state estimation means for estimating the state of the target organism from the image using a machine learning model that has learned the relationship between the image of the target organism and the state of the target organism; an information search means for searching for response information to deal with the state of the target organism; and an output means for outputting the response information to the terminal device.

[0063] (Supplementary Note 2) The information processing device according to Supplementary Note 1, further comprising: a sensor information acquisition means for acquiring biometric information of the target organism; and the state estimation means for estimating the state of the target organism based on the image and the biometric information.

[0064] (Supplementary Note 3) The information processing device according to Supplementary Note 1 or 2, wherein the countermeasure information includes a method of treating an illness or injury, and the output means transmits the countermeasure information including the method of treatment.

[0065] (Supplementary Note 4) The information processing device according to any one of Supplementary Notes 1 to 3, wherein the response information includes hospital information, and the output unit transmits the response information including the hospital information.

[0066] (Supplementary Note 5) The information processing device according to any one of Supplementary Notes 1 to 4, wherein the state estimation means estimates attribute information of the target organism, and the information search means searches for handling information based on the state and attribute information of the target organism.

[0067] (Appendix 6) An information processing device as described in Appendix 5, wherein the target organism is a pet, the attribute information is the type of the pet, and the information search means searches for hospital information regarding hospitals that treat the pet based on the type of the pet.

[0068] (Supplementary Note 7) The information processing device according to Supplementary Note 1, wherein the output means outputs the estimation result of the state estimation means to a terminal of a medical worker.

[0069] (Supplementary Note 8) The information processing device according to Supplementary Note 1, wherein the state estimation means estimates behavior that may lead to illness or injury of the target organism from the image, and the output means outputs an estimation result of the state estimation means to the terminal device.

[0070] (Supplementary Note 9) An information processing method that acquires an image of a target organism taken with a terminal device, estimates the state of the target organism from the image using a machine learning model that has learned the relationship between the image of the target organism and the state of the target organism, searches for response information to deal with the state of the target organism, and outputs the response information to the terminal device.

[0071] (Supplementary Note 10) A recording medium having recorded thereon a program that causes a computer to execute the following processes: acquire an image of a target organism photographed with a terminal device; estimate the state of the target organism from the image using a machine learning model that has learned the relationship between the image of the target organism and the state of the target organism; search for response information to deal with the state of the target organism; and output the response information to the terminal device.

[0072] Although the present disclosure has been described above with reference to the embodiments and examples, the present disclosure is not limited to the above-described embodiments and examples. Various modifications that can be understood by a person skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure.

[0073] 20 Pet terminal 200 Server 215 Database (DB) 300 User terminal 411 Image acquisition unit 412 State estimation unit 413 Information search unit 414 Information transmission unit 511 Learning data 512 Learning device

Claims

1. An image acquisition means for acquiring an image of a target organism captured by a terminal device; A state estimation means for estimating a state of the target organism from the image using a machine learning model that has learned the relationship between the image of the target organism and the state of the target organism; An information search means for searching for treatment information for treating the condition of the target organism; an output means for outputting the handling information to the terminal device; An information processing device comprising:

2. a sensor information acquiring means for acquiring biological information of the target organism, The information processing apparatus according to claim 1 , wherein the state estimation means estimates the state of the target organism based on the image and the biological information.

3. The treatment information includes a method for treating an illness or injury, The information processing apparatus according to claim 1 , wherein the output unit transmits handling information including the treatment method.

4. The response information includes hospital information, The information processing apparatus according to claim 1 , wherein the output means transmits treatment information including the hospital information.

5. The state estimation means estimates attribute information of the target organism, The information processing apparatus according to claim 1 , wherein the information search means searches for treatment information based on the state and attribute information of the target organism.

6. The subject is a companion animal, the attribute information is the type of the pet animal; 6. The information processing apparatus according to claim 5, wherein said information retrieval means retrieves hospital information relating to hospitals that treat said pets based on the type of said pet.

7. The information processing apparatus according to claim 1 , wherein the output means outputs the estimation result of the state estimation means to a terminal of a medical worker.

8. The state estimation means estimates a behavior that may lead to illness or injury of the target organism from the image, The information processing apparatus according to claim 1 , wherein the output means outputs an estimation result of the state estimation means to the terminal device.

9. An information processing method implemented by a computer, comprising: Acquire an image of the target organism using a terminal device; Estimating the state of the target organism from the image using a machine learning model that has learned the relationship between the image of the target organism and the state of the target organism; Searching for treatment information for treating the condition of the target organism; The information processing method includes outputting the handling information to the terminal device.

10. Acquire an image of the target organism using a terminal device; Estimating the state of the target organism from the image using a machine learning model that has learned the relationship between the image of the target organism and the state of the target organism; Searching for treatment information for treating the condition of the target organism; A program that causes a computer to execute a process of outputting the handling information to the terminal device.