System

Abstract

A system is provided.SOLUTION: A system comprising: means for acquiring motion, voice, and body temperature data of a pet in real time; means for analyzing the acquired data and estimating an emotion of the pet; means for notifying a user of the estimated emotion; means for monitoring a health condition of the pet and notifying an animal medical institution when an abnormality is detected; and means for generating and playing favorite music of the pet.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] For many families who welcome pets into their homes, managing their pets' health and understanding their emotions are important issues. However, it is difficult to accurately grasp a pet's emotions based solely on its movements and vocalizations, and there are limited means for early detection of health abnormalities. Furthermore, there are no fully established methods for providing individually tailored music to relax pets. The present invention aims to solve these problems by providing a system that facilitates communication between pets and their owners and efficiently manages their pets' health. [Means for solving the problem]

[0005] The present invention solves the above-mentioned problems with a system that includes the following means: means for acquiring pet movement, voice, and body temperature data in real time; means for analyzing the acquired data and inferring the pet's emotions; means for notifying the user of the inferred emotions; means for monitoring the pet's health condition and notifying a veterinary medical facility if an abnormality is detected; and means for generating and playing music preferred by the pet. This system enables accurate understanding of the pet's emotions and health condition and appropriate management. The means for acquiring pet movement, voice, and body temperature data include cameras and sensors, which collect detailed data. Furthermore, health monitoring is performed using an AI model for analysis, making it possible to detect abnormalities early.

[0006] "Movement" is information that indicates the pet's physical movements and actions.

[0007] "Audio" refers to information that refers to the cries and sounds made by pets.

[0008] "Body temperature data" is measurement information for obtaining a pet's body temperature in real time.

[0009] "Real-time acquisition means" refers to devices or technologies that instantly collect and record pet movement, voice, and temperature data.

[0010] "Analysis" is the process of deciphering information and deriving meaning and patterns from acquired data.

[0011] "Emotions" refer to a pet's internal state, such as happiness, sadness, excitement, or relaxation.

[0012] "Means of inference" refers to technologies and algorithms that predict pet emotions based on the analysis results.

[0013] "Means for notifying the user" refers to a device or mechanism for communicating the analysis results and inferred emotions to the owner.

[0014] "Health status" refers to information about the pet's physical condition and health.

[0015] "Monitoring" refers to the act of continuously checking a pet's health and detecting any changes or abnormalities.

[0016] "Abnormal" refers to a deviation from normal health.

[0017] "Means for notifying veterinary medical institutions" refers to devices or mechanisms that notify veterinary hospitals or other medical institutions by sending signals or messages when abnormalities in a pet's health are detected.

[0018] "Preferred music" refers to the type and characteristics of music that a pet finds relaxing or comfortable.

[0019] "Generating means" refers to the technology and algorithms used to create and deliver music of your choice.

[0020] "Means for playing" refers to a device or mechanism for playing the generated music.

[0021] A "camera" is a device that captures your pet's movements on video or in still images.

[0022] A "sensor" is a device that detects and acquires pet voice and body temperature data.

[0023] An "AI model" is a mathematical model or algorithm that uses artificial intelligence to analyze data and make inferences and predictions. [Brief explanation of the drawings]

[0024] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3]FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0025] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0026] First, the terms used in the following description will be explained.

[0027] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0028] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0029] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0030] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0031] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0032] [First embodiment]

[0033] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0034] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0035] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0036] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0037] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0038] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0039] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0040] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0041] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0042] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0043] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0044] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0045] The present invention is a system that acquires pet movement, voice, and body temperature data in real time, analyzes this information to infer the pet's emotions, and notifies the user. It also has the function of monitoring the pet's health condition, notifying a veterinary medical institution if an abnormality is detected, and generating and playing music tailored to the pet's preferences. The following describes in detail an embodiment of this system.

[0046] System Configuration

[0047] The system includes the following main components:

[0048] 1. A device that collects pet movement, voice, and body temperature data

[0049] Camera: Records your pet's movements as video and captures movement data in real time.

[0050] Sensor: Detects your pet's cries and body temperature and collects data.

[0051] 2. Server that analyzes the data

[0052] AI model: A mathematical model that analyzes the acquired data and predicts the pet's emotions and health condition.

[0053] 3. Means of notifying users

[0054] Smartphone app: An application that notifies users of analysis results and alerts when abnormalities are detected.

[0055] 4. Means of notifying the veterinary medical facility

[0056] Automatic notification system: A system that contacts the veterinary clinic when an abnormality is detected in the pet's health.

[0057] 5. Means of generating and playing music

[0058] AI model: Generates music tailored to your pet's preferences.

[0059] Built-in speaker: Ability to play generated music.

[0060] Program processing

[0061] Data collection and transmission

[0062] Device: A camera installed by the user records the pet's movements in real time, and sensors capture sounds and body temperature data. This data is periodically sent to a server via the Internet.

[0063] Emotion and health analysis

[0064] Server: The received data is input into an AI model, which analyzes the pet's movements and voice patterns to infer its emotions. It also monitors its health from its body temperature and movement patterns, and any abnormalities detected are recorded in a database.

[0065] Notification of results

[0066] Server: The analysis results are notified to the user via a smartphone app. For example, if it is determined that the pet is happy, that information is displayed on the user's smartphone. If any abnormalities are detected in the pet's health, a notification is automatically sent to the veterinary clinic, and an alert is also sent to the user.

[0067] Relaxation music generation and playback

[0068] Device: The user registers their pet's favorite music through a smartphone app, and that information is sent to the server.

[0069] Server: Uses an AI model to generate optimal relaxation music for pets and sends the music data to the device.

[0070] Terminal: The received music data is played through the camera's built-in speaker, providing a relaxing environment for your pet.

[0071] Specific examples

[0072] For example, we will explain how a family dog ​​might use this system over the course of a day.

[0073] 1. Data Collection:

[0074] Device: The camera records the dog's movements and facial expressions, and sensors detect its barks and body temperature. The device captures the dog's wagging tail and lively barks.

[0075] 2. Data Analysis:

[0076] Server: The AI ​​model analyzes the received data and infers that the dog is excited. It also determines that the dog is in good health based on the body temperature data.

[0077] 3. Result notification:

[0078] Server: The analysis results are sent to the user's smartphone app. A message saying "The dog is excited" is displayed and the user confirms it.

[0079] 4. Health Monitoring and Notification:

[0080] Server: Regularly checks the pet's health and, if the body temperature is abnormally high, determines that the pet may have a fever and notifies the veterinary clinic. An alert is also sent to the user.

[0081] 5. Generate and play relaxation music:

[0082] Device: At night, the camera's built-in speaker plays soothing piano music to help your dog relax.

[0083] In this way, by using the system of the present invention, it is possible to accurately grasp the emotions and health status of pets, enabling appropriate management and care.

[0084] The processing flow will be explained below.

[0085] Step 1:

[0086] Device: The camera records your pet's movements in real time, and sensors capture sounds and body temperature data, which are then recorded as detailed information about your pet's movements and sounds.

[0087] Step 2:

[0088] Terminal: Collected data is temporarily stored through batch processing and sent to the server at regular intervals (e.g., every 5 minutes). The data is encrypted during transmission to ensure secure transmission.

[0089] Step 3:

[0090] Server: Decompresses and decrypts the data received from the terminal and temporarily stores it in a database. It then checks the data for consistency and eliminates missing or abnormal data.

[0091] Step 4:

[0092] Server: Inputs data into the AI ​​model and infers the pet's emotions. For example, if a large tail wag and a high-pitched bark are detected at the same time, it will determine that the pet is happy.

[0093] Step 5:

[0094] Server: Stores the estimated emotions in a database and prepares to notify the user in real time via a smartphone app.

[0095] Step 6:

[0096] Device: The user's smartphone app receives the notification sent from the server and displays "Your pet is happy." The user can check this information and communicate with their pet.

[0097] Step 7:

[0098] Server: Regularly monitors the pet's health and detects abnormalities in body temperature or behavior. For example, if the body temperature is higher than normal, it determines that the pet may have a fever.

[0099] Step 8:

[0100] Server: If an abnormality in health is detected, it automatically sends a notification to the veterinary clinic, which includes detailed health data about the pet.

[0101] Step 9:

[0102] Server: At the same time, an alert is sent to the user via the smartphone app, allowing the user to quickly check if there is anything abnormal with their pet's health.

[0103] Step 10:

[0104] Device: The user registers their pet's favorite music through a smartphone app. For example, they can enter information such as "I like classical music."

[0105] Step 11:

[0106] Server: Based on the user's music preferences, the AI ​​model generates relaxing music that is customized to suit each pet's individual preferences.

[0107] Step 12:

[0108] Server: The generated music data is sent to the device, which then plays it to the pet through the camera's built-in speaker.

[0109] Step 13:

[0110] Device: Plays the received music data to provide a relaxing environment for your pet. For example, playing calming piano music at night can help your pet relax.

[0111] Example 1

[0112] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0113] Conventional technology has limited methods for accurately understanding a pet's emotions and health condition and communicating this to the user. Furthermore, existing systems lack the functionality to generate and play relaxation music tailored to the pet's preferences. This makes it difficult to manage a pet's emotions and health condition, and for users to understand their pet's condition in real time. Therefore, a system is needed that can analyze a pet's emotions and health condition in real time and quickly notify a veterinary medical facility if there are any abnormalities.

[0114] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0115] In this invention, the server includes means for acquiring data on the pet's movements, voice, and body temperature in real time, means for analyzing the acquired data and inferring the pet's emotions, means for notifying the user of the inferred emotions, means for monitoring the pet's health condition and notifying a veterinary medical institution if an abnormality is detected, means for the user to register the type of music preferred by the pet, means for the generative AI model to generate music based on the registered music preferences, and means for playing the generated music. This makes it possible to grasp the pet's emotions and health condition in real time and provide relaxing music.

[0116] "Movement data" refers to data relating to the pet's physical movements and behavior.

[0117] "Audio data" refers to data related to the cries and sounds made by pets.

[0118] "Body temperature data" refers to data relating to the pet's body temperature.

[0119] "Real-time" refers to data acquisition and analysis occurring almost immediately.

[0120] "Means of acquisition" refers to methods of collecting data using devices such as cameras and sensors.

[0121] "Means of analysis" refers to the way in which acquired data is processed, analyzed, and converted into meaningful information.

[0122] "Means for inferring emotions" refers to a method for determining and inferring a pet's current emotions from the analyzed data.

[0123] "Means of notification" refers to the method of notifying users and related organizations of analysis results and abnormalities.

[0124] "Health monitoring measures" are methods for continuously observing a pet's temperature and behavior to assess its health.

[0125] "Means for detecting abnormalities" are methods for detecting abnormalities in health status when they occur.

[0126] "Means of notifying a veterinary medical institution" refers to a method of notifying a specialized institution such as a veterinary hospital of a detected abnormality.

[0127] The "means for generating preferred music" is a method for creating relaxation music that matches a pet's preferences using a generative AI model.

[0128] "Means for playing" refers to a method for playing the generated music through a device such as a speaker.

[0129] A "generative AI model" is a model that uses artificial intelligence to analyze data and generate music.

[0130] A "prompt sentence" is an input sentence that conveys specific instructions or requests to a generative AI model.

[0131] The present invention is a system that collects pet movement, voice, and body temperature data in real time, analyzes this data to infer the pet's emotions, and notifies the user. It also has the function of monitoring the pet's health and notifying a veterinary clinic if an abnormality is detected. It also has the function of generating and playing music tailored to the pet's preferences.

[0132] Program processing

[0133] Data collection

[0134] Device: First, the user prepares a camera and a sensor. Specifically, the camera (e.g., Logitech C920) records the pet's movements in real time, and the sensor (e.g., DHT22) captures the pet's voice and body temperature. These data are collected by a device placed near the pet.

[0135] Sending data

[0136] Device: The collected data is sent to a server over the internet, packaged in a standard data format such as JSON, and periodically sent to the server via a Wi-Fi connection.

[0137] Emotion and health analysis

[0138] Server: The received data is input into an AI model (e.g., a neural network model using TensorFlow) running on the server. The server analyzes the video data using computer vision technology (e.g., YOLO or OpenPose) to infer the pet's emotions from its movement patterns. The audio data is also analyzed using a voice recognition system (e.g., Google Speech-to-Text API). The body temperature data is compared with pre-set reference values ​​to assess the pet's health.

[0139] Notification of results

[0140] Server: The analysis results are sent to the user's smartphone app. Notifications are sent using a notification service such as Firebase Cloud Messaging. For example, if the analysis result indicates that "the dog is excited," the user's smartphone will be notified.

[0141] Health Monitoring and Notification

[0142] Server: Monitors the pet's health 24 hours a day, and if the pet's body temperature is abnormally high (e.g., above 39°C), an alert is automatically sent to a veterinary clinic. At the same time, the user is notified of the health alert via their smartphone app.

[0143] Relaxation music generation and playback

[0144] User: The user registers their pet's favorite music through a smartphone app. For example, they can specify "classical music."

[0145] Server: Based on the registered information, the server generates music using a generative AI model (e.g., OpenAI's GPT-3). An example prompt is "Generate a calming piano piece that will help my dog ​​relax at night."

[0146] Device: The generated music data is sent to the device and played through the camera's built-in speaker, providing a relaxing environment for your pet.

[0147] Specific examples

[0148] For example, we explain how a family dog ​​uses this system from morning until night.

[0149] Data collection: Cameras capture the dog's movements, and sensors capture barks and body temperature data.

[0150] Data transmission: Collected data is sent to the server every 5 seconds.

[0151] Data analysis: The data received by the server is input into the AI ​​model, which analyzes it as "the dog is excited."

[0152] Result notification: The analysis result "The dog is excited" is notified to the user's smartphone.

[0153] Health monitoring: If the body temperature exceeds 39°C, it is considered a "fever" and an animal medical facility is automatically notified.

[0154] Relaxation Music: At night, gentle piano music generated by the server is played through the camera's built-in speaker to help your dog relax.

[0155] In this way, the system of the present invention can grasp the emotions and health condition of a pet in real time and quickly take appropriate measures.

[0156] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0157] Step 1: Collect data

[0158] Device: A user-installed camera (e.g., Logitech C920) and sensor (e.g., DHT22) collect pet behavior, sound, and temperature data in real time. The camera captures video at 60 fps and acquires behavior data. The sensor periodically reads and stores temperature and sound data.

[0159] Input: Pet movement, voice, and body temperature data

[0160] Output: A package of captured motion, voice, and temperature data

[0161] Specific behavior:

[0162] The camera records the dog's walking and eating, while sensors measure its barking and temperature.

[0163] Step 2: Sending data

[0164] Device: The collected data is converted into JSON format and sent to the server via Wi-Fi connection. The data is set to be sent every 5 seconds.

[0165] Input: Data collected in step 1

[0166] Output: Movement, voice, and temperature data sent to the server

[0167] Specific behavior:

[0168] The device automatically packages the data collected by the cameras and sensors and sends it to a server via the internet.

[0169] Step 3: Emotion and health analysis

[0170] Server: The received data is input into an AI model for analysis. For example, movement data is analyzed using a computer vision model, voice data is analyzed using a voice recognition model, and body temperature data is compared with pre-defined criteria.

[0171] Input: Data sent in step 2

[0172] Output: Analysis results (e.g. dog is excited, body temperature is normal)

[0173] Specific behavior:

[0174] The server analyzes the video data and determines that the dog is wagging its tail. It analyzes the audio data and determines that the dog is barking. It also checks the body temperature and records it as 36.5°C.

[0175] Step 4: Notification of results

[0176] Server: Notifies the user of the analysis results via their smartphone app. Uses services such as Firebase Cloud Messaging to communicate information to the user in real time.

[0177] Input: Analysis results obtained in step 3

[0178] Output: Notification sent to the user

[0179] Specific behavior:

[0180] The server formats the message "The dog is excited" in JSON format and notifies the user using the Firebase Cloud Messaging API.

[0181] Step 5: Health monitoring and notification

[0182] Server: Continuously monitors body temperature and behavior data, and if an abnormality is detected, contacts a veterinary medical facility. If the standard value is exceeded, an alert is automatically issued.

[0183] Input: Body temperature data and movement data obtained in step 3

[0184] Output: Abnormal alert notification

[0185] Specific behavior:

[0186] The server monitors body temperature data 24 hours a day, and if the temperature exceeds 39°C, it sends a "fever" alert to the veterinary clinic and notifies the user at the same time.

[0187] Step 6: Generate and play relaxation music

[0188] User: Register your pet's favorite music through the smartphone app. You can specify genres such as "classical music."

[0189] Server: Generates music using a generative AI model based on the registered information. Enter a prompt such as "Generate classical music" into the AI ​​model.

[0190] Device: The generated music data is played through the camera's built-in speaker.

[0191] Input: Music type (user input), music generation prompt

[0192] Output: Generated music data, music to be played

[0193] Specific behavior:

[0194] The user selects "classical music" and sends it to the server. The server then inputs the prompt "Please generate a gentle piano piece that will help my dog ​​relax at night" into the AI ​​model and sends the generated music to the device. The device then plays the music, providing a relaxing effect to the dog.

[0195] (Application example 1)

[0196] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0197] The present invention aims to accurately grasp a pet's emotions and health status by collecting data on its movements, voice, and body temperature in real time and analyzing this information. Furthermore, there is a need for early detection of abnormalities in a pet's health and providing a means of communication for prompt response, thereby improving pet health management and pet owner peace of mind. Another important issue is providing a means for generating and playing relaxation music to relieve stress in pets.

[0198] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0199] In this invention, the server includes a means for analyzing the pet's monitoring data and notifying the smartphone app of the results, a means for automatically contacting a veterinary clinic if the pet's health is abnormal, and a means for playing relaxation music if the pet is suspected to be stressed. This allows the pet's emotions and health condition to be accurately grasped in real time, enabling early detection of health abnormalities and prompt response. It also provides an environment that relieves pet stress, improving the owner's sense of security.

[0200] "Pet's movement, voice, and body temperature data" is a general term for behavioral information indicating the pet's movements, sound data reflecting vocal expressions, and temperature data for measuring body temperature.

[0201] "Real-time acquisition means" means the technological means for obtaining current information immediately without delay.

[0202] A "means for analyzing and inferring pet emotions" is a system or algorithm designed to analyze the collected data and infer the pet's emotional state.

[0203] "Means for notifying the user" refers to communication means or interfaces for immediately conveying analysis results and important information to the user.

[0204] The "means of notifying veterinary medical institutions when an abnormality is detected" is a system that automatically contacts pre-registered veterinary medical institutions when an abnormality is detected in a pet's health condition.

[0205] "Means for generating and playing music that suits pets' preferences" refers to devices or programs that use AI or algorithms to create music that suits pets' preferences and play it back.

[0206] "Means for analyzing pet monitoring data and notifying the results to a smartphone app" refers to methods or technologies for analyzing acquired monitoring data and notifying the user of the analysis results via a smartphone app.

[0207] "Means for automatically contacting a veterinary medical institution when an abnormality in the health of a pet" refers to a system or mechanism for automatically contacting a designated veterinary medical institution when an abnormality in the health of a pet is discovered.

[0208] The "means for playing relaxation music when it is determined that a pet is feeling stressed" is a mechanism for playing music to relax a pet when it is determined that the pet is in a stressful state.

[0209] To implement the present invention, a smart delivery robot specialized for food delivery services is used. The specific configuration and operation of the system are described below.

[0210] System Configuration

[0211] 1. Device Configuration

[0212] Camera: Used to record your pet's movements in real time.

[0213] Microphone: Used to capture pet sounds.

[0214] Temperature sensor: Used to measure your pet's body temperature.

[0215] Built-in speaker: Used to play the generated relaxation music.

[0216] 2. Server Configuration

[0217] AI model: Used to analyze the acquired data and run a generative AI model to infer the pet's emotions and health status, specifically using machine learning frameworks such as TensorFlow and PyTorch.

[0218] 3. User Interface

[0219] Smartphone app: Used to notify users of analysis results and alerts when anomalies are detected. Notification APIs such as Firebase Cloud Messaging are used.

[0220] How it works

[0221] Data collection

[0222] The device records your pet's movements in real time with a camera, picks up its cries with a microphone, and measures its body temperature with a temperature sensor, all of which are immediately sent to a server.

[0223] Data analysis

[0224] The server inputs the received movement data, voice data, and body temperature data into a generative AI model to analyze the pet's emotions and health condition. For example, by inputting the movement data and voice data into the AI ​​model, it can infer that the pet is "excited," and from the body temperature data, it can determine that the pet is "in good health."

[0225] Result notification

[0226] The server notifies the user of the analysis results via a smartphone app, which displays the results as messages such as "your pet is excited" or "your pet is in good health."

[0227] Health monitoring and abnormality notification

[0228] The server monitors the health of your pet and automatically sends a notification to a veterinary clinic if an abnormality is detected. For example, if your pet's temperature is abnormally high, it will immediately determine that there is a possibility of a fever and contact the veterinary clinic. An alert will also be sent to the user.

[0229] Relaxation music generation and playback

[0230] The device uses a generative AI model to generate relaxing music and plays it through the built-in speaker to provide a relaxing environment for pets, especially at night, if the device detects that your pet is feeling stressed.

[0231] Specific examples

[0232] For example, consider the case where a food delivery robot arrives at a user's home and encounters a pet. At this time, a camera records the pet's movements in real time, a microphone picks up its meows, and a temperature sensor measures its body temperature. The data is immediately sent to a server and analyzed by a generative AI model. The resulting analysis results, such as "the pet is excited" or "its health is good," are notified to the user's smartphone. If the pet appears stressed, relaxing music is played.

[0233] Prompt Sentence Examples

[0234] Example prompt sentence:

[0235] Take your pet's motion and voice data as input and infer its emotions. Then, analyze its temperature data to determine its health status.

[0236] As a result, the system of the present invention can accurately grasp the emotions and health status of pets in real time, enabling early detection of health abnormalities and prompt response, and providing an environment that reduces stress for pets.

[0237] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0238] Step 1:

[0239] Data collection:

[0240] The device records the pet's movements in real time with a camera, picks up its cries with a microphone, and measures its body temperature with a temperature sensor. These data are collected as movement video data, audio data, and body temperature data.

[0241] Input: Pet movement, sound, temperature

[0242] Output: Motion video data, audio data, body temperature data

[0243] Step 2:

[0244] Data transmission:

[0245] The device sends the collected data to a server via the internet, and each data is protected by a secure communication protocol to prevent data loss or tampering.

[0246] Input: motion video data, audio data, body temperature data

[0247] Output: Each data sent to the server

[0248] Step 3:

[0249] Data Analysis:

[0250] The server inputs the received motion video data, audio data, and body temperature data into a generative AI model to analyze the pet's emotions and health condition. The AI ​​model infers the pet's emotions from its motion and audio patterns and evaluates its health condition based on the body temperature data.

[0251] Input: motion video data, audio data, body temperature data

[0252] Output: Emotion prediction result, health status evaluation result

[0253] Step 4:

[0254] Notification of results:

[0255] The server then notifies the user of the analysis results via a smartphone app. The notification content, which is related to the pet's emotional state and health status, is displayed in a user-friendly format.

[0256] Input: Emotion prediction results, health status assessment results

[0257] Output: A notification message to the user (e.g. "Your pet is excited" or "It is in good health")

[0258] Step 5:

[0259] Automatic notification of health abnormalities:

[0260] The server operates a system that automatically contacts animal medical institutions if an abnormality is detected in the pet's health condition. An alert is also sent to the user, enabling a prompt response.

[0261] Input: Health status assessment result (if abnormality detected)

[0262] Output: Notification message to veterinary clinic, alert to user

[0263] Step 6:

[0264] Relaxation music generation and playback:

[0265] The device uses a generative AI model to generate relaxing music for your pet, which is then played through the built-in speaker, providing a relaxing environment for your pet.

[0266] Input: Emotion prediction result (if stress is detected)

[0267] Output: Generated relaxation music data, music playback

[0268] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0269] This invention is a system that acquires pet behavior, voice, and body temperature data in real time, analyzes this information, predicts the pet's emotions, and notifies the user. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it takes into account the emotional states of both the pet and the user, enabling more appropriate communication and health management.

[0270] System Configuration

[0271] The system includes the following main components:

[0272] 1. A device that collects pet movement, voice, and body temperature data

[0273] Camera: Records your pet's movements as video and captures movement data in real time.

[0274] Sensor: Detects your pet's cries and body temperature and collects data.

[0275] 2. Server that analyzes the data

[0276] AI model: A mathematical model that analyzes the acquired data and predicts the pet's emotions and health condition.

[0277] 3. Means of notifying users

[0278] Smartphone app: An application that notifies users of analysis results and alerts when abnormalities are detected.

[0279] 4. Means of notifying the veterinary medical facility

[0280] Automatic notification system: A system that contacts the veterinary clinic when an abnormality is detected in the pet's health.

[0281] 5. Means of generating and playing music

[0282] AI model: Generates music tailored to your pet's preferences.

[0283] Built-in speaker: Ability to play generated music.

[0284] 6. Emotion engine that recognizes user emotions

[0285] Camera and microphone: Analyzes the user's facial expressions, tone of voice, and gestures to infer emotions.

[0286] Program processing

[0287] Data collection and transmission

[0288] Device: A camera installed by the user records the pet's movements in real time, and sensors capture its sounds and body temperature data, which are periodically sent to a server via the Internet.

[0289] Emotion and health analysis

[0290] Server: The received data is input into an AI model, which analyzes the pet's movements and voice patterns to infer its emotions. It also monitors its health from its body temperature and movement patterns, and any abnormalities detected are recorded in a database.

[0291] Server: Additionally, the server uses the user's camera and microphone to analyze the user's facial expressions, tone of voice, and gestures to infer emotions.

[0292] Notification of results

[0293] Server: Notifies the user of the analysis results via a smartphone app. For example, if it determines that "your pet is happy," that information is displayed on the user's smartphone. The notification content is also adjusted based on the user's emotions. For example, if the user is feeling stressed, advice such as "Try ways to help your pet relax" is provided.

[0294] Health Monitoring and Notification

[0295] Server: Regularly monitors the pet's health and detects abnormalities in body temperature or behavior. For example, if the body temperature is higher than normal, it determines that the pet may have a fever.

[0296] Server: If an abnormality in health is detected, it automatically sends a notification to the veterinary clinic, which includes detailed health data about the pet.

[0297] Server: At the same time, an alert is sent to the user via the smartphone app, allowing the user to quickly check if there is anything abnormal with their pet's health.

[0298] Relaxation music generation and playback

[0299] Device: The user registers their pet's favorite music through a smartphone app. For example, they can enter information such as "I like classical music."

[0300] Server: Based on the user's music preferences, the AI ​​model generates relaxing music that is customized to suit each pet's individual preferences.

[0301] Device: Plays the received music data to provide a relaxing environment for your pet. For example, playing calming piano music at night can help your pet relax.

[0302] Specific examples

[0303] For example, we will explain how a dog and its owner in a household might use this system over the course of a day.

[0304] 1. Data Collection:

[0305] Device: The camera records the dog's movements and facial expressions, and sensors detect its barks and body temperature. The device captures the dog's wagging tail and lively barks.

[0306] 2. User sentiment analysis:

[0307] Server: Using the user's camera and microphone, the server analyzes the user's facial expressions and tone of voice to determine whether the user is experiencing stress.

[0308] 3. Data Analysis:

[0309] Server: The AI ​​model analyzes the received data and infers that the dog is excited. It also determines that the dog is in good health based on the body temperature data.

[0310] 4. Result notification:

[0311] Server: The analysis results are sent to the user's smartphone app. A message is displayed stating that the dog is excited, and, taking into consideration that the user is feeling stressed, advice is provided to encourage relaxation.

[0312] 5. Health Monitoring and Notification:

[0313] Server: Regularly checks the pet's health and, if the body temperature is abnormally high, determines that the pet may have a fever and notifies the veterinary clinic. An alert is also sent to the user.

[0314] 6. Generate and play relaxation music:

[0315] Device: At night, the camera's built-in speaker plays soothing piano music to help your dog relax.

[0316] In this way, by using the system of the present invention, the emotions and health conditions of pets and users can be accurately understood, enabling appropriate management and care.

[0317] The processing flow will be explained below.

[0318] Step 1:

[0319] Device: The camera records your pet's movements in real time, while the sensors capture vocalization and temperature data. The camera captures video of your pet's movements and facial expressions, while the sensors record your pet's body temperature and vocalization frequency.

[0320] Step 2:

[0321] The terminal periodically processes the acquired data in batches and transmits them over the internet in encrypted form to a server. This transmission process occurs at regular intervals (e.g., every 5 minutes).

[0322] Step 3:

[0323] Server: The server decompresses and decrypts the data received from the device and temporarily stores it in a database. Here, it performs a data integrity check and eliminates any invalid or missing data.

[0324] Step 4:

[0325] Server: Inputs data into the AI ​​model, analyzes the pet's movements and vocal patterns, and infers its emotions. For example, if a vigorous tail wagging and a high-pitched bark are detected at the same time, it will determine that the pet is happy.

[0326] Step 5:

[0327] Server: Stores the estimated emotions in a database and prepares to notify the user. Notifications are set up to be sent in real time via a smartphone app.

[0328] Step 6:

[0329] Device: The user's smartphone app receives the notification sent from the server and displays the message "Your pet is happy." The user can check this information and communicate with their pet.

[0330] Step 7:

[0331] On the device: The device uses the user's camera and microphone to record the user's facial expressions, tone of voice, and gestures, which then stores the user's emotional data on the device.

[0332] Step 8:

[0333] Terminal: The user's emotional data is periodically sent to the server in encrypted format, similar to the data sent to pets.

[0334] Step 9:

[0335] Server: The received user emotion data is input into the AI ​​model to infer the user's emotions. For example, if the user's facial expression is grim and their voice tone is low, it is determined that the user is feeling stressed.

[0336] Step 10:

[0337] Server: Adjusts the content of notifications sent to pets based on the user's emotional data. For example, if the pet is excited and the user is feeling stressed, the server will provide advice such as "Try to help your pet relax."

[0338] Step 11:

[0339] Server: Regularly monitors the pet's health and detects abnormalities. For example, if the body temperature is higher than normal, it determines that the pet may have a fever.

[0340] Step 12:

[0341] Server: If an abnormality in health status is detected, a notification is automatically sent to a veterinary clinic. The notification contains detailed health data of the pet to aid in diagnosis.

[0342] Step 13:

[0343] Server: At the same time, an alert is sent to the user via the smartphone app, allowing them to quickly check if there is anything abnormal with their pet's health.

[0344] Step 14:

[0345] Device: The user registers their pet's favorite music through a smartphone app. For example, they can enter a preference such as "I like classical music."

[0346] Step 15:

[0347] Server: Based on the user's music preferences, the AI ​​model generates relaxing music that is customized to suit each pet's individual preferences.

[0348] Step 16:

[0349] Server: The generated music data is sent to the device, which then plays it through the device's built-in speaker.

[0350] Step 17:

[0351] Device: Plays the received music data to provide a relaxing environment for your pet. For example, playing calming piano music at night can help your pet relax.

[0352] This series of steps allows accurate monitoring of the emotions and health of pets and users, and allows for appropriate management and countermeasures.

[0353] Example 2

[0354] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0355] Modern pet ownership requires predicting a pet's emotions and health status and notifying the user at the appropriate time. However, current technology makes it difficult to accurately predict a pet's emotions, and responses when abnormalities occur in the pet's health are often delayed. Furthermore, there are no notification systems that take the user's emotions into account. This makes it difficult to provide appropriate care and communication.

[0356] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0357] In this invention, the server includes means for acquiring pet movement, voice, and body temperature data in real time, means for analyzing the acquired data to infer the pet's emotions and health condition, means for notifying the user of the inferred emotions and health condition, means for acquiring the user's facial expressions, tone of voice, and gestures to infer emotions, means for notifying the veterinary clinic and the user if an abnormality in the pet's health condition is detected, and means for generating and playing music preferred by the pet. This makes it possible to accurately grasp the pet's emotions and health condition and quickly notify the user and the veterinary clinic. It also enables appropriate notifications that take the user's emotions into account and the generation and playback of music tailored to the pet's preferences, thereby achieving better communication between the pet and the user and appropriate health management.

[0358] "Movement data" is information that is recorded and collected in real time about your pet's body movements and behavior.

[0359] "Audio data" refers to acoustic information collected using sensors that record the cries and sounds made by pets.

[0360] "Body temperature data" refers to temperature information collected by measuring a pet's body temperature with a sensor.

[0361] "Acquisition means" refers to a device or equipment for acquiring pet movement data, voice data, and body temperature data in real time.

[0362] "Analysis methods" refers to the processes and algorithms used to use AI models to infer a pet's emotions and health status based on the acquired data.

[0363] "Notification means" refers to a device or application for communicating analysis results to users or animal medical facilities.

[0364] "Facial expression" refers to data that refers to facial movements and expressions captured using a camera to assess the user's emotions.

[0365] "Voice tone" is voice information used to infer emotions by analyzing the pitch and tone of the user's voice.

[0366] "Gesture" is movement information that is used to analyze the user's hand movements and body movements and to infer emotions.

[0367] "AI model" refers to a mathematical and computational model that uses artificial intelligence to analyze data and infer emotions and health states.

[0368] "Veterinary medical institution" refers to a medical facility or veterinary hospital that should be contacted if any abnormalities are discovered in the health of a pet.

[0369] "Music Generation Method" refers to the algorithm or process used to create music based on the pet's preferences.

[0370] "Playback means" refers to a device or system for actually playing back the generated music.

[0371] MODE FOR CARRYING OUT THE INVENTION

[0372] The present invention is a system that acquires pet movement, voice, and body temperature data in real time, analyzes this information to estimate the pet's emotions and health condition, and notifies the user. It also includes an emotion engine for recognizing the user's emotions, enabling appropriate communication and health management that takes into account the emotional states of both the pet and the user. Specific embodiments are described below.

[0373] System Configuration

[0374] The system includes the following main components:

[0375] 1. A device that collects pet movement, voice, and body temperature data

[0376] Camera: Records your pet's movements as video and captures movement data in real time.

[0377] Example: Logitech C920

[0378] Sensor: Detects your pet's cries and body temperature and collects data.

[0379] Example: DHT11

[0380] 2. Server that analyzes the data

[0381] AI model: A mathematical model that analyzes the acquired data and predicts the pet's emotions and health condition.

[0382] Examples: TensorFlow, Keras

[0383] Database: Stores analysis results and collected data.

[0384] Example: MySQL

[0385] 3. Means of notifying users

[0386] Smartphone app: An application that notifies users of analysis results and alerts when abnormalities are detected.

[0387] Example: React Native, Flutter

[0388] 4. Means of notifying the veterinary medical facility

[0389] Automatic notification system: A system that contacts the veterinary clinic when an abnormality is detected in the pet's health.

[0390] Examples: Twilio, Firebase Cloud Messaging

[0391] 5. Means of generating and playing music

[0392] AI model: Generates music tailored to your pet's preferences.

[0393] Example: Magenta

[0394] Built-in speaker: Ability to play generated music.

[0395] Example: JBL

[0396] 6. Emotion engine that recognizes user emotions

[0397] Camera and microphone: Analyzes the user's facial expressions, tone of voice, and gestures to infer emotions.

[0398] Examples: OpenFace, Praat

[0399] Program processing

[0400] Data collection

[0401] Device: A camera installed by the user records the pet's movements in real time, and sensors capture its sounds and body temperature data, which are periodically sent to a server via the Internet.

[0402] Data analysis

[0403] Server: The received data is input into an AI model, which analyzes the pet's movements and voice patterns to infer its emotions. It also monitors its health from its body temperature and movement patterns, and any abnormalities detected are recorded in a database.

[0404] Emotion analysis

[0405] Server: Uses the user's camera and microphone to analyze the user's facial expressions, tone of voice, and gestures to infer emotions.

[0406] Result notification

[0407] Server: Notifies the user of the analysis results via a smartphone app. For example, if it determines that "your pet is happy," that information is displayed on the user's smartphone. The notification content is also adjusted based on the user's emotions. For example, if the user is feeling stressed, advice such as "Try ways to help your pet relax" is provided.

[0408] Health Monitoring and Notification

[0409] Server: Regularly monitors the pet's health and detects abnormalities in body temperature or behavior. For example, if the body temperature is higher than normal, it determines that the pet may have a fever.

[0410] Server: If an abnormality in health is detected, it automatically sends a notification to the veterinary clinic, which includes detailed health data about the pet.

[0411] Server: At the same time, an alert is sent to the user via the smartphone app, allowing the user to quickly check if there is anything abnormal with their pet's health.

[0412] Relaxation music generation and playback

[0413] Device: The user registers their pet's favorite music through a smartphone app. For example, they can enter information such as "I like classical music."

[0414] Server: Based on the user's music preferences, the AI ​​model generates relaxing music that is customized to suit each pet's individual preferences.

[0415] Device: Plays the received music data to provide a relaxing environment for your pet. For example, playing calming piano music at night can help your pet relax.

[0416] Specific examples

[0417] Data collection

[0418] Device: The camera records the dog's movements and facial expressions, and sensors detect its barks and body temperature. The device captures the dog's wagging tail and lively barks.

[0419] Emotion analysis

[0420] Server: Using the user's camera and microphone, the server analyzes the user's facial expressions and tone of voice to determine whether the user is experiencing stress.

[0421] Data analysis

[0422] Server: The AI ​​model analyzes the received data and infers that the dog is excited. It also determines that the dog is in good health based on the body temperature data.

[0423] Result notification

[0424] Server: The analysis results are sent to the user's smartphone app. A message is displayed stating that the dog is excited, and, taking into consideration that the user is feeling stressed, advice is provided to encourage relaxation.

[0425] Health Monitoring and Notification

[0426] Server: Regularly checks the pet's health and, if the body temperature is abnormally high, determines that the pet may have a fever and notifies the veterinary clinic. An alert is also sent to the user.

[0427] Relaxation music generation and playback

[0428] Device: At night, the camera's built-in speaker plays soothing piano music to help your dog relax.

[0429] Prompt Sentence Examples

[0430] Based on the following data, please generate a program for a system that will infer a pet's emotions and notify the user. Please also clearly describe the system's overview and specific processing steps.

[0431] data:

[0432] Pet movement video data

[0433] Pet sounds and body temperature data

[0434] The user's facial expressions, tone of voice, and gestures

[0435] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0436] Step 1: Collect data

[0437] Device: Captures pet movement, voice, and temperature data in real time through cameras and sensors.

[0438] Input: Pet movement video, bark, body temperature

[0439] Output: Collected motion video data, audio data, and body temperature data

[0440] Specific operation: The camera records the pet's movements, the microphone records the dog's barks, and the temperature sensor measures the dog's temperature. For example, the camera records the dog jumping, the microphone picks up the sound of the dog barking, and the temperature sensor measures the dog's temperature at 37.8°C.

[0441] Step 2: Sending data

[0442] Terminal: Sends collected data to a server via the Internet.

[0443] Input: Collected motion video data, audio data, and body temperature data

[0444] Output: Motion video data, audio data, and body temperature data sent to the server

[0445] Specific operation: Data collected by the camera and sensors is uploaded to a cloud server via Wi-Fi. For example, motion video files, audio files, and body temperature data are sent to the cloud server.

[0446] Step 3: Data analysis

[0447] Server: Inputs the received data into an AI model, analyzes the pet's movements and voice patterns to infer emotions, and monitors its health from its body temperature and movement patterns.

[0448] Input: Motion video data, audio data, and body temperature data received by the server

[0449] Output: Pet emotion prediction results, health status analysis results

[0450] Specific operation: The AI ​​model processes the received video data and, for example, analyzes it as "the dog is wagging its tail" and determines that "the dog is happy." It also determines that "the dog is in good health" based on body temperature data.

[0451] Step 4: Sentiment Analysis

[0452] Server: Uses the user's camera and microphone to analyze the user's facial expressions, tone of voice, and gestures to infer emotions.

[0453] Input: Facial expression data, tone of voice data, and gesture data captured from the user's camera and microphone

[0454] Output: User emotion estimation result

[0455] Specific operation: The system processes data from when the user speaks using facial expressions and voice, and infers that the user is relaxed. For example, it analyzes facial expressions as "smiling" and voice tone as "calm."

[0456] Step 5: Notification of results

[0457] Server: Notifies the user of the analysis results via a smartphone app.

[0458] Input: Pet emotion prediction results, health status analysis results, user emotion prediction results

[0459] Output: Message sent to the user's smartphone app

[0460] Specific operation: Based on the analysis results, a push notification is sent to the user's smartphone saying "your dog is happy." If the user feels stressed, the system provides advice such as "try ways to help your pet relax."

[0461] Step 6: Health monitoring and notification

[0462] Server: If an abnormality is detected in the pet's health condition, the server automatically notifies the veterinary clinic and the user.

[0463] Input: Pet health analysis results

[0464] Output: Alert notifications sent to veterinary clinics and users

[0465] Specific operation: If the temperature data is abnormally high, it is determined that there is a possibility of a fever. An email notification is sent to the veterinary clinic, and a push notification is sent to the user alerting them that their pet has a high temperature.

[0466] Step 7: Generate and play relaxation music

[0467] Device: Based on the user's selection, the AI ​​model generates relaxation music and plays it through the device's speakers.

[0468] Input: User's music preference data

[0469] Output: Generated relaxation music, music to be played

[0470] How it works: The user tells the app that they like classical music. The AI ​​model generates calming piano music and plays it through the device's built-in speaker at night, providing a relaxing environment for pets.

[0471] (Application example 2)

[0472] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0473] Conventional pet monitoring and health management systems provide real-time analysis of pet behavior, voice, and body temperature data, but they are unable to link the emotional state of the pet with the user based on this information, or provide appropriate feedback to the user based on the pet's estimated emotions. This has resulted in a lack of improvement in the quality of pet health management and emotional care. Furthermore, they lack the ability to immediately notify medical institutions when abnormalities are detected in the pet's health.

[0474] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0475] In this invention, the server includes a means for acquiring data on the pet's movements, voice, and body temperature in real time, a means for analyzing the acquired data to infer the pet's emotions, and a means for simultaneously analyzing the emotional states of the user and the pet and providing appropriate feedback, thereby enabling more appropriate communication and health management that takes into account the emotional states of the pet and the user.

[0476] "Behavioral data" refers to information about a pet's movements and behavior, and is primarily data obtained visually using cameras and sensors.

[0477] "Audio data" refers to information about pets' cries and vocalizations, and is data obtained using audio sensors and microphones.

[0478] "Body temperature data" refers to information about a pet's body temperature, and is data obtained using a temperature sensor.

[0479] "Real-time capture" means that data is collected and processed at the present time and in the moment, making it available almost immediately.

[0480] "Analysis" refers to the process of analyzing acquired data using mathematical models and algorithms to extract meaning and trends.

[0481] "Emotion inference" refers to estimating the emotional state of a pet from the analyzed data, including various emotional states such as joy, sadness, excitement, and anxiety.

[0482] "Notifying" refers to the act of informing the user of the analysis results or anomaly detection information, which is usually done via a smartphone or other notification means.

[0483] "Health monitoring" refers to the process of continually checking your pet's health information to ensure there are no irregularities or abnormalities.

[0484] "Notifying a medical institution" refers to the act of communicating information to an appropriate medical provider when an abnormality in a pet's health is detected.

[0485] "Generating preferred music" refers to the process of automatically creating music that will relax your pet using an AI model.

[0486] "Playing" refers to the act of making the generated music audible through speakers or other audio devices.

[0487] "Simultaneous analysis of the emotional states of the user and the pet" refers to the process of collecting the emotional states of both the pet and the user in real time and simultaneously analyzing them based on that data.

[0488] "Providing appropriate feedback" refers to the act of suggesting and notifying users of optimal actions and information based on the analysis results.

[0489] This invention relates to a system that acquires pet behavior, voice, and body temperature data in real time, analyzes this information to infer the pet's emotions, and notifies the user. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it takes into account the emotional states of both the pet and the user, enabling more appropriate communication and health management.

[0490] System Configuration

[0491] The system mainly consists of the following components:

[0492] 1. Device that collects movement, voice, and body temperature data

[0493] Camera: Records your pet's movements as video and captures movement data in real time.

[0494] Sensor: Detects your pet's cries and body temperature and collects data.

[0495] 2. Server that analyzes the data

[0496] AI model: A mathematical model that analyzes the acquired data and predicts the pet's emotions and health condition.

[0497] 3. Means of notifying users

[0498] Smartphone app: An application that notifies users of analysis results and alerts when abnormalities are detected.

[0499] 4. Means of notifying medical institutions

[0500] Automatic notification system: A system that contacts the veterinary clinic when an abnormality is detected in the pet's health.

[0501] 5. Means of generating and playing music

[0502] AI model: Generates music tailored to your pet's preferences.

[0503] Built-in speaker: Ability to play generated music.

[0504] 6. Emotion engine that recognizes user emotions

[0505] Camera and microphone: Analyzes the user's facial expressions, tone of voice, and gestures to infer emotions.

[0506] Program processing

[0507] Data collection and transmission

[0508] The device uses a camera installed by the user to record pet movements in real time, and sensors to capture sounds and body temperature data, which are then periodically sent to a server via the internet.

[0509] Emotion and health analysis

[0510] The server inputs the received data into an AI model and analyzes the pet's movements and voice patterns to infer emotions. It also monitors the pet's health status based on its body temperature and movement patterns, and any abnormalities detected are recorded in a database. Furthermore, the server uses the user's camera and microphone to analyze the user's facial expressions, tone of voice, and gestures to infer emotions.

[0511] Notification of results

[0512] The server notifies the user of the analysis results via a smartphone app. For example, if it determines that "your pet is happy," that information is displayed on the user's smartphone. The notification content is also adjusted based on the user's emotions. For example, if the user is feeling stressed, the server provides advice such as "Try ways to help your pet relax."

[0513] Health Monitoring and Notification

[0514] The server regularly monitors the pet's health and detects any abnormalities in body temperature or behavior. For example, if the body temperature is higher than normal, it determines that the pet may have a fever. If an abnormality in health is detected, a notification is automatically sent to a veterinary clinic. This notification includes detailed health data about the pet. At the same time, an alert is sent to the user via a smartphone app, allowing the user to quickly check if there is anything abnormal with their pet's health.

[0515] Relaxation music generation and playback

[0516] The user registers their pet's preferred music through a smartphone app on the device. For example, they may enter information such as "I like classical music." The server uses an AI model to generate relaxing music based on the music preference information obtained from the user. The generated music is customized to suit each pet's individual preferences. The received music data is played back to provide a relaxing environment for the pet. For example, playing calming piano music at night can promote relaxation for the pet.

[0517] Specific examples

[0518] As a concrete example, let's consider the case of a dog left at a pet hotel. A camera records the dog's movements and facial expressions, and sensors detect its barks and body temperature data. The dog's tail wagging and energetic barks are captured. The server receives this data and uses an AI model to infer that the dog is excited, and based on the body temperature data, determines that the dog is in good health. These analysis results are notified to the user's smartphone app, which displays a message saying that the dog is excited. If the user's emotions indicate stress, the system also provides advice to encourage relaxation.

[0519] Prompt Sentence Examples

[0520] For example, give the generative AI model the following prompt:

[0521] I'd like to create an application for a pet hotel that uses a system to capture and analyze pet behavior, voice, and temperature data in real time, and then predict and notify the pet's emotions and health status. I'm considering a system that notifies the user of detailed analysis results and sends alerts to hotel staff and pet owners in the event of an abnormality. Please provide an example implementation using Python and TensorFlow.

[0522] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0523] Step 1:

[0524] Data collection

[0525] The device collects movement data, audio data, and body temperature data in real time. Specifically, the camera records the pet's movements, the microphone records its cries, and the temperature sensor measures its body temperature. The input is the camera footage, audio files, and body temperature data, and the output is the raw data files.

[0526] Step 2:

[0527] Sending data

[0528] The data collected by the terminal is sent to the server via the Internet. The data is sent directly without any processing. The input is the raw data, and the output is the success status of data transmission to the server.

[0529] Step 3:

[0530] Preparing for analysis

[0531] The server preprocesses the received raw data. Specifically, it extracts the camera footage frame by frame, performs digital signal processing on the audio data, and digitizes the body temperature data. The input is raw data, and the output is preprocessed data.

[0532] Step 4:

[0533] emotion guessing

[0534] The server inputs the preprocessed data into an AI model to infer the pet's emotions. The AI ​​model uses an algorithm to identify emotions from movement patterns, vocal tones, and changes in body temperature. The input is the preprocessed data, and the output is an inferred emotional state. Specifically, it assigns labels such as "happy," "anxious," and "excited."

[0535] Step 5:

[0536] Health monitoring

[0537] The server uses an AI model to analyze the body temperature data and monitor the pet's health, particularly detecting abnormalities when the temperature is outside the normal range. The input is the body temperature data, and the output is the health status assessment result. Specific outputs include labels such as "healthy," "fever," and "low body temperature."

[0538] Step 6:

[0539] Notification of analysis results

[0540] The server notifies the user's smartphone app of the emotion prediction results and health status evaluation results. The input is the analysis results, and the output is a notification message displayed on the user's smartphone app. Specific notification content includes "Your pet is happy" or "Your pet has a fever."

[0541] Step 7:

[0542] Notification to medical institutions

[0543] If the server detects an abnormality, it automatically notifies the veterinary clinic. The input is the health status assessment result, and the output is a notification message sent to the veterinary clinic. Specific notification content might be something like, "Your pet has a fever. Immediate medical examination is required."

[0544] Step 8:

[0545] Music generation and playback

[0546] The device generates music that the pet likes and starts playing it. The server generates the music using an AI model and sends the data to the device. The input is the pet's emotional state and preferred music information, and the output is the generated music data. Specific music examples include "classical music" and "gentle piano pieces."

[0547] Step 9:

[0548] User sentiment analysis

[0549] The server analyzes the user's facial expressions, tone of voice, and gestures using a camera and microphone to infer the user's emotions. The input is the user's video and audio data, and the output is an inferred result of the user's emotional state. Specific inferred results include "stress," "relaxation," and "joy."

[0550] Step 10:

[0551] Providing feedback

[0552] Based on the analysis results, the server provides appropriate feedback to the user. The input is the estimated emotional state of the pet and the user, and the output is a feedback message to the user. Specific feedback content might be, "Spend more time relaxing with your pet."

[0553] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0554] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0555] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0556] [Second embodiment]

[0557] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0558] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0559] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0560] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0561] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0562] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0563] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0564] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0565] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0566] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0567] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0568] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0569] The present invention is a system that acquires pet movement, voice, and body temperature data in real time, analyzes this information to infer the pet's emotions, and notifies the user. It also has the function of monitoring the pet's health condition, notifying a veterinary medical institution if an abnormality is detected, and generating and playing music tailored to the pet's preferences. The following describes in detail an embodiment of this system.

[0570] System Configuration

[0571] The system includes the following main components:

[0572] 1. A device that collects pet movement, voice, and body temperature data

[0573] Camera: Records your pet's movements as video and captures movement data in real time.

[0574] Sensor: Detects your pet's cries and body temperature and collects data.

[0575] 2. Server that analyzes the data

[0576] AI model: A mathematical model that analyzes the acquired data and predicts the pet's emotions and health condition.

[0577] 3. Means of notifying users

[0578] Smartphone app: An application that notifies users of analysis results and alerts when abnormalities are detected.

[0579] 4. Means of notifying the veterinary medical facility

[0580] Automatic notification system: A system that contacts the veterinary clinic when an abnormality is detected in the pet's health.

[0581] 5. Means of generating and playing music

[0582] AI model: Generates music tailored to your pet's preferences.

[0583] Built-in speaker: Ability to play generated music.

[0584] Program processing

[0585] Data collection and transmission

[0586] Device: A camera installed by the user records the pet's movements in real time, and sensors capture sounds and body temperature data. This data is periodically sent to a server via the Internet.

[0587] Emotion and health analysis

[0588] Server: The received data is input into an AI model, which analyzes the pet's movements and voice patterns to infer its emotions. It also monitors its health from its body temperature and movement patterns, and any abnormalities detected are recorded in a database.

[0589] Notification of results

[0590] Server: The analysis results are notified to the user via a smartphone app. For example, if it is determined that the pet is happy, that information is displayed on the user's smartphone. If any abnormalities are detected in the pet's health, a notification is automatically sent to the veterinary clinic, and an alert is also sent to the user.

[0591] Relaxation music generation and playback

[0592] Device: The user registers their pet's favorite music through a smartphone app, and that information is sent to the server.

[0593] Server: Uses an AI model to generate optimal relaxation music for pets and sends the music data to the device.

[0594] Terminal: The received music data is played through the camera's built-in speaker, providing a relaxing environment for your pet.

[0595] Specific examples

[0596] For example, we will explain how a family dog ​​might use this system over the course of a day.

[0597] 1. Data Collection:

[0598] Device: The camera records the dog's movements and facial expressions, and sensors detect its barks and body temperature. The device captures the dog's wagging tail and lively barks.

[0599] 2. Data Analysis:

[0600] Server: The AI ​​model analyzes the received data and infers that the dog is excited. It also determines that the dog is in good health based on the body temperature data.

[0601] 3. Result notification:

[0602] Server: The analysis results are sent to the user's smartphone app. A message saying "The dog is excited" is displayed and the user confirms it.

[0603] 4. Health Monitoring and Notification:

[0604] Server: Regularly checks the pet's health and, if the body temperature is abnormally high, determines that the pet may have a fever and notifies the veterinary clinic. An alert is also sent to the user.

[0605] 5. Generate and play relaxation music:

[0606] Device: At night, the camera's built-in speaker plays soothing piano music to help your dog relax.

[0607] In this way, by using the system of the present invention, it is possible to accurately grasp the emotions and health status of pets, enabling appropriate management and care.

[0608] The processing flow will be explained below.

[0609] Step 1:

[0610] Device: The camera records your pet's movements in real time, and sensors capture sounds and body temperature data, which are then recorded as detailed information about your pet's movements and sounds.

[0611] Step 2:

[0612] Terminal: Collected data is temporarily stored through batch processing and sent to the server at regular intervals (e.g., every 5 minutes). The data is encrypted during transmission to ensure secure transmission.

[0613] Step 3:

[0614] Server: Decompresses and decrypts the data received from the terminal and temporarily stores it in a database. It then checks the data for consistency and eliminates missing or abnormal data.

[0615] Step 4:

[0616] Server: Inputs data into the AI ​​model and infers the pet's emotions. For example, if a large tail wag and a high-pitched bark are detected at the same time, it will determine that the pet is happy.

[0617] Step 5:

[0618] Server: Stores the estimated emotions in a database and prepares to notify the user in real time via a smartphone app.

[0619] Step 6:

[0620] Device: The user's smartphone app receives the notification sent from the server and displays "Your pet is happy." The user can check this information and communicate with their pet.

[0621] Step 7:

[0622] Server: Regularly monitors the pet's health and detects abnormalities in body temperature or behavior. For example, if the body temperature is higher than normal, it determines that the pet may have a fever.

[0623] Step 8:

[0624] Server: If an abnormality in health is detected, it automatically sends a notification to the veterinary clinic, which includes detailed health data about the pet.

[0625] Step 9:

[0626] Server: At the same time, an alert is sent to the user via the smartphone app, allowing the user to quickly check if there is anything abnormal with their pet's health.

[0627] Step 10:

[0628] Device: The user registers their pet's favorite music through a smartphone app. For example, they can enter information such as "I like classical music."

[0629] Step 11:

[0630] Server: Based on the user's music preferences, the AI ​​model generates relaxing music that is customized to suit each pet's individual preferences.

[0631] Step 12:

[0632] Server: The generated music data is sent to the device, which then plays it to the pet through the camera's built-in speaker.

[0633] Step 13:

[0634] Device: Plays the received music data to provide a relaxing environment for your pet. For example, playing calming piano music at night can help your pet relax.

[0635] Example 1

[0636] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0637] Conventional technology has limited methods for accurately understanding a pet's emotions and health condition and communicating this to the user. Furthermore, existing systems lack the functionality to generate and play relaxation music tailored to the pet's preferences. This makes it difficult to manage a pet's emotions and health condition, and for users to understand their pet's condition in real time. Therefore, a system is needed that can analyze a pet's emotions and health condition in real time and quickly notify a veterinary medical facility if there are any abnormalities.

[0638] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0639] In this invention, the server includes means for acquiring data on the pet's movements, voice, and body temperature in real time, means for analyzing the acquired data and inferring the pet's emotions, means for notifying the user of the inferred emotions, means for monitoring the pet's health condition and notifying a veterinary medical institution if an abnormality is detected, means for the user to register the type of music preferred by the pet, means for the generative AI model to generate music based on the registered music preferences, and means for playing the generated music. This makes it possible to grasp the pet's emotions and health condition in real time and provide relaxing music.

[0640] "Movement data" refers to data relating to the pet's physical movements and behavior.

[0641] "Audio data" refers to data related to the cries and sounds made by pets.

[0642] "Body temperature data" refers to data relating to the pet's body temperature.

[0643] "Real-time" refers to data acquisition and analysis occurring almost immediately.

[0644] "Means of acquisition" refers to methods of collecting data using devices such as cameras and sensors.

[0645] "Means of analysis" refers to the way in which acquired data is processed, analyzed, and converted into meaningful information.

[0646] "Means for inferring emotions" refers to a method for determining and inferring a pet's current emotions from the analyzed data.

[0647] "Means of notification" refers to the method of notifying users and related organizations of analysis results and abnormalities.

[0648] "Health monitoring measures" are methods for continuously observing a pet's temperature and behavior to assess its health.

[0649] "Means for detecting abnormalities" are methods for detecting abnormalities in health status when they occur.

[0650] "Means of notifying a veterinary medical institution" refers to a method of notifying a specialized institution such as a veterinary hospital of a detected abnormality.

[0651] The "means for generating preferred music" is a method for creating relaxation music that matches a pet's preferences using a generative AI model.

[0652] "Means for playing" refers to a method for playing the generated music through a device such as a speaker.

[0653] A "generative AI model" is a model that uses artificial intelligence to analyze data and generate music.

[0654] A "prompt sentence" is an input sentence that conveys specific instructions or requests to a generative AI model.

[0655] The present invention is a system that collects pet movement, voice, and body temperature data in real time, analyzes this data to infer the pet's emotions, and notifies the user. It also has the function of monitoring the pet's health and notifying a veterinary clinic if an abnormality is detected. It also has the function of generating and playing music tailored to the pet's preferences.

[0656] Program processing

[0657] Data collection

[0658] Device: First, the user prepares a camera and a sensor. Specifically, the camera (e.g., Logitech C920) records the pet's movements in real time, and the sensor (e.g., DHT22) captures the pet's voice and body temperature. These data are collected by a device placed near the pet.

[0659] Sending data

[0660] Device: The collected data is sent to a server over the internet, packaged in a standard data format such as JSON, and periodically sent to the server via a Wi-Fi connection.

[0661] Emotion and health analysis

[0662] Server: The received data is input into an AI model (e.g., a neural network model using TensorFlow) running on the server. The server analyzes the video data using computer vision technology (e.g., YOLO or OpenPose) to infer the pet's emotions from its movement patterns. The audio data is also analyzed using a voice recognition system (e.g., Google Speech-to-Text API). The body temperature data is compared with pre-set reference values ​​to assess the pet's health.

[0663] Notification of results

[0664] Server: The analysis results are sent to the user's smartphone app. Notifications are sent using a notification service such as Firebase Cloud Messaging. For example, if the analysis result indicates that "the dog is excited," the user's smartphone will be notified.

[0665] Health Monitoring and Notification

[0666] Server: Monitors the pet's health 24 hours a day, and if the pet's body temperature is abnormally high (e.g., above 39°C), an alert is automatically sent to a veterinary clinic. At the same time, the user is notified of the health alert via their smartphone app.

[0667] Relaxation music generation and playback

[0668] User: The user registers their pet's favorite music through a smartphone app. For example, they can specify "classical music."

[0669] Server: Based on the registered information, the server generates music using a generative AI model (e.g., OpenAI's GPT-3). An example prompt is "Generate a calming piano piece that will help my dog ​​relax at night."

[0670] Device: The generated music data is sent to the device and played through the camera's built-in speaker, providing a relaxing environment for your pet.

[0671] Specific examples

[0672] For example, we explain how a family dog ​​uses this system from morning until night.

[0673] Data collection: Cameras capture the dog's movements, and sensors capture barks and body temperature data.

[0674] Data transmission: Collected data is sent to the server every 5 seconds.

[0675] Data analysis: The data received by the server is input into the AI ​​model, which analyzes it as "the dog is excited."

[0676] Result notification: The analysis result "The dog is excited" is notified to the user's smartphone.

[0677] Health monitoring: If the body temperature exceeds 39°C, it is considered a "fever" and an animal medical facility is automatically notified.

[0678] Relaxation Music: At night, gentle piano music generated by the server is played through the camera's built-in speaker to help your dog relax.

[0679] In this way, the system of the present invention can grasp the emotions and health condition of a pet in real time and quickly take appropriate measures.

[0680] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0681] Step 1: Collect data

[0682] Device: A user-installed camera (e.g., Logitech C920) and sensor (e.g., DHT22) collect pet behavior, sound, and temperature data in real time. The camera captures video at 60 fps and acquires behavior data. The sensor periodically reads and stores temperature and sound data.

[0683] Input: Pet movement, voice, and body temperature data

[0684] Output: A package of captured motion, voice, and temperature data

[0685] Specific behavior:

[0686] The camera records the dog's walking and eating, while sensors measure its barking and temperature.

[0687] Step 2: Sending data

[0688] Device: The collected data is converted into JSON format and sent to the server via Wi-Fi connection. The data is set to be sent every 5 seconds.

[0689] Input: Data collected in step 1

[0690] Output: Movement, voice, and temperature data sent to the server

[0691] Specific behavior:

[0692] The device automatically packages the data collected by the cameras and sensors and sends it to a server via the internet.

[0693] Step 3: Emotion and health analysis

[0694] Server: The received data is input into an AI model for analysis. For example, movement data is analyzed using a computer vision model, voice data is analyzed using a voice recognition model, and body temperature data is compared with pre-defined criteria.

[0695] Input: Data sent in step 2

[0696] Output: Analysis results (e.g. dog is excited, body temperature is normal)

[0697] Specific behavior:

[0698] The server analyzes the video data and determines that the dog is wagging its tail. It analyzes the audio data and determines that the dog is barking. It also checks the body temperature and records it as 36.5°C.

[0699] Step 4: Notification of results

[0700] Server: Notifies the user of the analysis results via their smartphone app. Uses services such as Firebase Cloud Messaging to communicate information to the user in real time.

[0701] Input: Analysis results obtained in step 3

[0702] Output: Notification sent to the user

[0703] Specific behavior:

[0704] The server formats the message "The dog is excited" in JSON format and notifies the user using the Firebase Cloud Messaging API.

[0705] Step 5: Health monitoring and notification

[0706] Server: Continuously monitors body temperature and behavior data, and if an abnormality is detected, contacts a veterinary medical facility. If the standard value is exceeded, an alert is automatically issued.

[0707] Input: Body temperature data and movement data obtained in step 3

[0708] Output: Abnormal alert notification

[0709] Specific behavior:

[0710] The server monitors body temperature data 24 hours a day, and if the temperature exceeds 39°C, it sends a "fever" alert to the veterinary clinic and notifies the user at the same time.

[0711] Step 6: Generate and play relaxation music

[0712] User: Register your pet's favorite music through the smartphone app. You can specify genres such as "classical music."

[0713] Server: Generates music using a generative AI model based on the registered information. Enter a prompt such as "Generate classical music" into the AI ​​model.

[0714] Device: The generated music data is played through the camera's built-in speaker.

[0715] Input: Music type (user input), music generation prompt

[0716] Output: Generated music data, music to be played

[0717] Specific behavior:

[0718] The user selects "classical music" and sends it to the server. The server then inputs the prompt "Please generate a gentle piano piece that will help my dog ​​relax at night" into the AI ​​model and sends the generated music to the device. The device then plays the music, providing a relaxing effect to the dog.

[0719] (Application example 1)

[0720] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0721] The present invention aims to accurately grasp a pet's emotions and health status by collecting data on its movements, voice, and body temperature in real time and analyzing this information. Furthermore, there is a need for early detection of abnormalities in a pet's health and providing a means of communication for prompt response, thereby improving pet health management and pet owner peace of mind. Another important issue is providing a means for generating and playing relaxation music to relieve stress in pets.

[0722] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0723] In this invention, the server includes a means for analyzing the pet's monitoring data and notifying the smartphone app of the results, a means for automatically contacting a veterinary clinic if the pet's health is abnormal, and a means for playing relaxation music if the pet is suspected to be stressed. This allows the pet's emotions and health condition to be accurately grasped in real time, enabling early detection of health abnormalities and prompt response. It also provides an environment that relieves pet stress, improving the owner's sense of security.

[0724] "Pet's movement, voice, and body temperature data" is a general term for behavioral information indicating the pet's movements, sound data reflecting vocal expressions, and temperature data for measuring body temperature.

[0725] "Real-time acquisition means" means the technological means for obtaining current information immediately without delay.

[0726] A "means for analyzing and inferring pet emotions" is a system or algorithm designed to analyze the collected data and infer the pet's emotional state.

[0727] "Means for notifying the user" refers to communication means or interfaces for immediately conveying analysis results and important information to the user.

[0728] The "means of notifying veterinary medical institutions when an abnormality is detected" is a system that automatically contacts pre-registered veterinary medical institutions when an abnormality is detected in a pet's health condition.

[0729] "Means for generating and playing music that suits pets' preferences" refers to devices or programs that use AI or algorithms to create music that suits pets' preferences and play it back.

[0730] "Means for analyzing pet monitoring data and notifying the results to a smartphone app" refers to methods or technologies for analyzing acquired monitoring data and notifying the user of the analysis results via a smartphone app.

[0731] "Means for automatically contacting a veterinary medical institution when an abnormality in the health of a pet" refers to a system or mechanism for automatically contacting a designated veterinary medical institution when an abnormality in the health of a pet is discovered.

[0732] The "means for playing relaxation music when it is determined that a pet is feeling stressed" is a mechanism for playing music to relax a pet when it is determined that the pet is in a stressful state.

[0733] To implement the present invention, a smart delivery robot specialized for food delivery services is used. The specific configuration and operation of the system are described below.

[0734] System Configuration

[0735] 1. Device Configuration

[0736] Camera: Used to record your pet's movements in real time.

[0737] Microphone: Used to capture pet sounds.

[0738] Temperature sensor: Used to measure your pet's body temperature.

[0739] Built-in speaker: Used to play the generated relaxation music.

[0740] 2. Server Configuration

[0741] AI model: Used to analyze the acquired data and run a generative AI model to infer the pet's emotions and health status, specifically using machine learning frameworks such as TensorFlow and PyTorch.

[0742] 3. User Interface

[0743] Smartphone app: Used to notify users of analysis results and alerts when anomalies are detected. Notification APIs such as Firebase Cloud Messaging are used.

[0744] How it works

[0745] Data collection

[0746] The device records your pet's movements in real time with a camera, picks up its cries with a microphone, and measures its body temperature with a temperature sensor, all of which are immediately sent to a server.

[0747] Data analysis

[0748] The server inputs the received movement data, voice data, and body temperature data into a generative AI model to analyze the pet's emotions and health condition. For example, by inputting the movement data and voice data into the AI ​​model, it can infer that the pet is "excited," and from the body temperature data, it can determine that the pet is "in good health."

[0749] Result notification

[0750] The server notifies the user of the analysis results via a smartphone app, which displays the results as messages such as "your pet is excited" or "your pet is in good health."

[0751] Health monitoring and abnormality notification

[0752] The server monitors the health of your pet and automatically sends a notification to a veterinary clinic if an abnormality is detected. For example, if your pet's temperature is abnormally high, it will immediately determine that there is a possibility of a fever and contact the veterinary clinic. An alert will also be sent to the user.

[0753] Relaxation music generation and playback

[0754] The device uses a generative AI model to generate relaxing music and plays it through the built-in speaker to provide a relaxing environment for pets, especially at night, if the device detects that your pet is feeling stressed.

[0755] Specific examples

[0756] For example, consider the case where a food delivery robot arrives at a user's home and encounters a pet. At this time, a camera records the pet's movements in real time, a microphone picks up its meows, and a temperature sensor measures its body temperature. The data is immediately sent to a server and analyzed by a generative AI model. The resulting analysis results, such as "the pet is excited" or "its health is good," are notified to the user's smartphone. If the pet appears stressed, relaxing music is played.

[0757] Prompt Sentence Examples

[0758] Example prompt sentence:

[0759] Take your pet's motion and voice data as input and infer its emotions. Then, analyze its temperature data to determine its health status.

[0760] As a result, the system of the present invention can accurately grasp the emotions and health status of pets in real time, enabling early detection of health abnormalities and prompt response, and providing an environment that reduces stress for pets.

[0761] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0762] Step 1:

[0763] Data collection:

[0764] The device records the pet's movements in real time with a camera, picks up its cries with a microphone, and measures its body temperature with a temperature sensor. These data are collected as movement video data, audio data, and body temperature data.

[0765] Input: Pet movement, sound, temperature

[0766] Output: Motion video data, audio data, body temperature data

[0767] Step 2:

[0768] Data transmission:

[0769] The device sends the collected data to a server via the internet, and each data is protected by a secure communication protocol to prevent data loss or tampering.

[0770] Input: motion video data, audio data, body temperature data

[0771] Output: Each data sent to the server

[0772] Step 3:

[0773] Data Analysis:

[0774] The server inputs the received motion video data, audio data, and body temperature data into a generative AI model to analyze the pet's emotions and health condition. The AI ​​model infers the pet's emotions from its motion and audio patterns and evaluates its health condition based on the body temperature data.

[0775] Input: motion video data, audio data, body temperature data

[0776] Output: Emotion prediction result, health status evaluation result

[0777] Step 4:

[0778] Notification of results:

[0779] The server then notifies the user of the analysis results via a smartphone app. The notification content, which is related to the pet's emotional state and health status, is displayed in a user-friendly format.

[0780] Input: Emotion prediction results, health status assessment results

[0781] Output: A notification message to the user (e.g. "Your pet is excited" or "It is in good health")

[0782] Step 5:

[0783] Automatic notification of health abnormalities:

[0784] The server operates a system that automatically contacts animal medical institutions if an abnormality is detected in the pet's health condition. An alert is also sent to the user, enabling a prompt response.

[0785] Input: Health status assessment result (if abnormality detected)

[0786] Output: Notification message to veterinary clinic, alert to user

[0787] Step 6:

[0788] Relaxation music generation and playback:

[0789] The device uses a generative AI model to generate relaxing music for your pet, which is then played through the built-in speaker, providing a relaxing environment for your pet.

[0790] Input: Emotion prediction result (if stress is detected)

[0791] Output: Generated relaxation music data, music playback

[0792] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0793] This invention is a system that acquires pet behavior, voice, and body temperature data in real time, analyzes this information, predicts the pet's emotions, and notifies the user. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it takes into account the emotional states of both the pet and the user, enabling more appropriate communication and health management.

[0794] System Configuration

[0795] The system includes the following main components:

[0796] 1. A device that collects pet movement, voice, and body temperature data

[0797] Camera: Records your pet's movements as video and captures movement data in real time.

[0798] Sensor: Detects your pet's cries and body temperature and collects data.

[0799] 2. Server that analyzes the data

[0800] AI model: A mathematical model that analyzes the acquired data and predicts the pet's emotions and health condition.

[0801] 3. Means of notifying users

[0802] Smartphone app: An application that notifies users of analysis results and alerts when abnormalities are detected.

[0803] 4. Means of notifying the veterinary medical facility

[0804] Automatic notification system: A system that contacts the veterinary clinic when an abnormality is detected in the pet's health.

[0805] 5. Means of generating and playing music

[0806] AI model: Generates music tailored to your pet's preferences.

[0807] Built-in speaker: Ability to play generated music.

[0808] 6. Emotion engine that recognizes user emotions

[0809] Camera and microphone: Analyzes the user's facial expressions, tone of voice, and gestures to infer emotions.

[0810] Program processing

[0811] Data collection and transmission

[0812] Device: A camera installed by the user records the pet's movements in real time, and sensors capture its sounds and body temperature data, which are periodically sent to a server via the Internet.

[0813] Emotion and health analysis

[0814] Server: The received data is input into an AI model, which analyzes the pet's movements and voice patterns to infer its emotions. It also monitors its health from its body temperature and movement patterns, and any abnormalities detected are recorded in a database.

[0815] Server: Additionally, the server uses the user's camera and microphone to analyze the user's facial expressions, tone of voice, and gestures to infer emotions.

[0816] Notification of results

[0817] Server: Notifies the user of the analysis results via a smartphone app. For example, if it determines that "your pet is happy," that information is displayed on the user's smartphone. The notification content is also adjusted based on the user's emotions. For example, if the user is feeling stressed, advice such as "Try ways to help your pet relax" is provided.

[0818] Health Monitoring and Notification

[0819] Server: Regularly monitors the pet's health and detects abnormalities in body temperature or behavior. For example, if the body temperature is higher than normal, it determines that the pet may have a fever.

[0820] Server: If an abnormality in health is detected, it automatically sends a notification to the veterinary clinic, which includes detailed health data about the pet.

[0821] Server: At the same time, an alert is sent to the user via the smartphone app, allowing the user to quickly check if there is anything abnormal with their pet's health.

[0822] Relaxation music generation and playback

[0823] Device: The user registers their pet's favorite music through a smartphone app. For example, they can enter information such as "I like classical music."

[0824] Server: Based on the user's music preferences, the AI ​​model generates relaxing music that is customized to suit each pet's individual preferences.

[0825] Device: Plays the received music data to provide a relaxing environment for your pet. For example, playing calming piano music at night can help your pet relax.

[0826] Specific examples

[0827] For example, we will explain how a dog and its owner in a household might use this system over the course of a day.

[0828] 1. Data Collection:

[0829] Device: The camera records the dog's movements and facial expressions, and sensors detect its barks and body temperature. The device captures the dog's wagging tail and lively barks.

[0830] 2. User sentiment analysis:

[0831] Server: Using the user's camera and microphone, the server analyzes the user's facial expressions and tone of voice to determine whether the user is experiencing stress.

[0832] 3. Data Analysis:

[0833] Server: The AI ​​model analyzes the received data and infers that the dog is excited. It also determines that the dog is in good health based on the body temperature data.

[0834] 4. Result notification:

[0835] Server: The analysis results are sent to the user's smartphone app. A message is displayed stating that the dog is excited, and, taking into consideration that the user is feeling stressed, advice is provided to encourage relaxation.

[0836] 5. Health Monitoring and Notification:

[0837] Server: Regularly checks the pet's health and, if the body temperature is abnormally high, determines that the pet may have a fever and notifies the veterinary clinic. An alert is also sent to the user.

[0838] 6. Generate and play relaxation music:

[0839] Device: At night, the camera's built-in speaker plays soothing piano music to help your dog relax.

[0840] In this way, by using the system of the present invention, the emotions and health conditions of pets and users can be accurately understood, enabling appropriate management and care.

[0841] The processing flow will be explained below.

[0842] Step 1:

[0843] Device: The camera records your pet's movements in real time, while the sensors capture vocalization and temperature data. The camera captures video of your pet's movements and facial expressions, while the sensors record your pet's body temperature and vocalization frequency.

[0844] Step 2:

[0845] The terminal periodically processes the acquired data in batches and transmits them over the internet in encrypted form to a server. This transmission process occurs at regular intervals (e.g., every 5 minutes).

[0846] Step 3:

[0847] Server: The server decompresses and decrypts the data received from the device and temporarily stores it in a database. Here, it performs a data integrity check and eliminates any invalid or missing data.

[0848] Step 4:

[0849] Server: Inputs data into the AI ​​model, analyzes the pet's movements and vocal patterns, and infers its emotions. For example, if a vigorous tail wagging and a high-pitched bark are detected at the same time, it will determine that the pet is happy.

[0850] Step 5:

[0851] Server: Stores the estimated emotions in a database and prepares to notify the user. Notifications are set up to be sent in real time via a smartphone app.

[0852] Step 6:

[0853] Device: The user's smartphone app receives the notification sent from the server and displays the message "Your pet is happy." The user can check this information and communicate with their pet.

[0854] Step 7:

[0855] On the device: The device uses the user's camera and microphone to record the user's facial expressions, tone of voice, and gestures, which then stores the user's emotional data on the device.

[0856] Step 8:

[0857] Terminal: The user's emotional data is periodically sent to the server in encrypted format, similar to the data sent to pets.

[0858] Step 9:

[0859] Server: The received user emotion data is input into the AI ​​model to infer the user's emotions. For example, if the user's facial expression is grim and their voice tone is low, it is determined that the user is feeling stressed.

[0860] Step 10:

[0861] Server: Adjusts the content of notifications sent to pets based on the user's emotional data. For example, if the pet is excited and the user is feeling stressed, the server will provide advice such as "Try to help your pet relax."

[0862] Step 11:

[0863] Server: Regularly monitors the pet's health and detects abnormalities. For example, if the body temperature is higher than normal, it determines that the pet may have a fever.

[0864] Step 12:

[0865] Server: If an abnormality in health status is detected, a notification is automatically sent to a veterinary clinic. The notification contains detailed health data of the pet to aid in diagnosis.

[0866] Step 13:

[0867] Server: At the same time, an alert is sent to the user via the smartphone app, allowing them to quickly check if there is anything abnormal with their pet's health.

[0868] Step 14:

[0869] Device: The user registers their pet's favorite music through a smartphone app. For example, they can enter a preference such as "I like classical music."

[0870] Step 15:

[0871] Server: Based on the user's music preferences, the AI ​​model generates relaxing music that is customized to suit each pet's individual preferences.

[0872] Step 16:

[0873] Server: The generated music data is sent to the device, which then plays it through the device's built-in speaker.

[0874] Step 17:

[0875] Device: Plays the received music data to provide a relaxing environment for your pet. For example, playing calming piano music at night can help your pet relax.

[0876] This series of steps allows accurate monitoring of the emotions and health of pets and users, and allows for appropriate management and countermeasures.

[0877] Example 2

[0878] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0879] Modern pet ownership requires predicting a pet's emotions and health status and notifying the user at the appropriate time. However, current technology makes it difficult to accurately predict a pet's emotions, and responses when abnormalities occur in the pet's health are often delayed. Furthermore, there are no notification systems that take the user's emotions into account. This makes it difficult to provide appropriate care and communication.

[0880] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0881] In this invention, the server includes means for acquiring pet movement, voice, and body temperature data in real time, means for analyzing the acquired data to infer the pet's emotions and health condition, means for notifying the user of the inferred emotions and health condition, means for acquiring the user's facial expressions, tone of voice, and gestures to infer emotions, means for notifying the veterinary clinic and the user if an abnormality in the pet's health condition is detected, and means for generating and playing music preferred by the pet. This makes it possible to accurately grasp the pet's emotions and health condition and quickly notify the user and the veterinary clinic. It also enables appropriate notifications that take the user's emotions into account and the generation and playback of music tailored to the pet's preferences, thereby achieving better communication between the pet and the user and appropriate health management.

[0882] "Movement data" is information that is recorded and collected in real time about your pet's body movements and behavior.

[0883] "Audio data" refers to acoustic information collected using sensors that record the cries and sounds made by pets.

[0884] "Body temperature data" refers to temperature information collected by measuring a pet's body temperature with a sensor.

[0885] "Acquisition means" refers to a device or equipment for acquiring pet movement data, voice data, and body temperature data in real time.

[0886] "Analysis methods" refers to the processes and algorithms used to use AI models to infer a pet's emotions and health status based on the acquired data.

[0887] "Notification means" refers to a device or application for communicating analysis results to users or animal medical facilities.

[0888] "Facial expression" refers to data that refers to facial movements and expressions captured using a camera to assess the user's emotions.

[0889] "Voice tone" is voice information used to infer emotions by analyzing the pitch and tone of the user's voice.

[0890] "Gesture" is movement information that is used to analyze the user's hand movements and body movements and to infer emotions.

[0891] "AI model" refers to a mathematical and computational model that uses artificial intelligence to analyze data and infer emotions and health states.

[0892] "Veterinary medical institution" refers to a medical facility or veterinary hospital that should be contacted if any abnormalities are discovered in the health of a pet.

[0893] "Music Generation Method" refers to the algorithm or process used to create music based on the pet's preferences.

[0894] "Playback means" refers to a device or system for actually playing back the generated music.

[0895] MODE FOR CARRYING OUT THE INVENTION

[0896] The present invention is a system that acquires pet movement, voice, and body temperature data in real time, analyzes this information to estimate the pet's emotions and health condition, and notifies the user. It also includes an emotion engine for recognizing the user's emotions, enabling appropriate communication and health management that takes into account the emotional states of both the pet and the user. Specific embodiments are described below.

[0897] System Configuration

[0898] The system includes the following main components:

[0899] 1. A device that collects pet movement, voice, and body temperature data

[0900] Camera: Records your pet's movements as video and captures movement data in real time.

[0901] Example: Logitech C920

[0902] Sensor: Detects your pet's cries and body temperature and collects data.

[0903] Example: DHT11

[0904] 2. Server that analyzes the data

[0905] AI model: A mathematical model that analyzes the acquired data and predicts the pet's emotions and health condition.

[0906] Examples: TensorFlow, Keras

[0907] Database: Stores analysis results and collected data.

[0908] Example: MySQL

[0909] 3. Means of notifying users

[0910] Smartphone app: An application that notifies users of analysis results and alerts when abnormalities are detected.

[0911] Example: React Native, Flutter

[0912] 4. Means of notifying the veterinary medical facility

[0913] Automatic notification system: A system that contacts the veterinary clinic when an abnormality is detected in the pet's health.

[0914] Examples: Twilio, Firebase Cloud Messaging

[0915] 5. Means of generating and playing music

[0916] AI model: Generates music tailored to your pet's preferences.

[0917] Example: Magenta

[0918] Built-in speaker: Ability to play generated music.

[0919] Example: JBL

[0920] 6. Emotion engine that recognizes user emotions

[0921] Camera and microphone: Analyzes the user's facial expressions, tone of voice, and gestures to infer emotions.

[0922] Examples: OpenFace, Praat

[0923] Program processing

[0924] Data collection

[0925] Device: A camera installed by the user records the pet's movements in real time, and sensors capture its sounds and body temperature data, which are periodically sent to a server via the Internet.

[0926] Data analysis

[0927] Server: The received data is input into an AI model, which analyzes the pet's movements and voice patterns to infer its emotions. It also monitors its health from its body temperature and movement patterns, and any abnormalities detected are recorded in a database.

[0928] Emotion analysis

[0929] Server: Uses the user's camera and microphone to analyze the user's facial expressions, tone of voice, and gestures to infer emotions.

[0930] Result notification

[0931] Server: Notifies the user of the analysis results via a smartphone app. For example, if it determines that "your pet is happy," that information is displayed on the user's smartphone. The notification content is also adjusted based on the user's emotions. For example, if the user is feeling stressed, advice such as "Try ways to help your pet relax" is provided.

[0932] Health Monitoring and Notification

[0933] Server: Regularly monitors the pet's health and detects abnormalities in body temperature or behavior. For example, if the body temperature is higher than normal, it determines that the pet may have a fever.

[0934] Server: If an abnormality in health is detected, it automatically sends a notification to the veterinary clinic, which includes detailed health data about the pet.

[0935] Server: At the same time, an alert is sent to the user via the smartphone app, allowing the user to quickly check if there is anything abnormal with their pet's health.

[0936] Relaxation music generation and playback

[0937] Device: The user registers their pet's favorite music through a smartphone app. For example, they can enter information such as "I like classical music."

[0938] Server: Based on the user's music preferences, the AI ​​model generates relaxing music that is customized to suit each pet's individual preferences.

[0939] Device: Plays the received music data to provide a relaxing environment for your pet. For example, playing calming piano music at night can help your pet relax.

[0940] Specific examples

[0941] Data collection

[0942] Device: The camera records the dog's movements and facial expressions, and sensors detect its barks and body temperature. The device captures the dog's wagging tail and lively barks.

[0943] Emotion analysis

[0944] Server: Using the user's camera and microphone, the server analyzes the user's facial expressions and tone of voice to determine whether the user is experiencing stress.

[0945] Data analysis

[0946] Server: The AI ​​model analyzes the received data and infers that the dog is excited. It also determines that the dog is in good health based on the body temperature data.

[0947] Result notification

[0948] Server: The analysis results are sent to the user's smartphone app. A message is displayed stating that the dog is excited, and, taking into consideration that the user is feeling stressed, advice is provided to encourage relaxation.

[0949] Health Monitoring and Notification

[0950] Server: Regularly checks the pet's health and, if the body temperature is abnormally high, determines that the pet may have a fever and notifies the veterinary clinic. An alert is also sent to the user.

[0951] Relaxation music generation and playback

[0952] Device: At night, the camera's built-in speaker plays soothing piano music to help your dog relax.

[0953] Prompt Sentence Examples

[0954] Based on the following data, please generate a program for a system that will infer a pet's emotions and notify the user. Please also clearly describe the system's overview and specific processing steps.

[0955] data:

[0956] Pet movement video data

[0957] Pet sounds and body temperature data

[0958] The user's facial expressions, tone of voice, and gestures

[0959] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0960] Step 1: Collect data

[0961] Device: Captures pet movement, voice, and temperature data in real time through cameras and sensors.

[0962] Input: Pet movement video, bark, body temperature

[0963] Output: Collected motion video data, audio data, and body temperature data

[0964] Specific operation: The camera records the pet's movements, the microphone records the dog's barks, and the temperature sensor measures the dog's temperature. For example, the camera records the dog jumping, the microphone picks up the sound of the dog barking, and the temperature sensor measures the dog's temperature at 37.8°C.

[0965] Step 2: Sending data

[0966] Terminal: Sends collected data to a server via the Internet.

[0967] Input: Collected motion video data, audio data, and body temperature data

[0968] Output: Motion video data, audio data, and body temperature data sent to the server

[0969] Specific operation: Data collected by the camera and sensors is uploaded to a cloud server via Wi-Fi. For example, motion video files, audio files, and body temperature data are sent to the cloud server.

[0970] Step 3: Data analysis

[0971] Server: Inputs the received data into an AI model, analyzes the pet's movements and voice patterns to infer emotions, and monitors its health from its body temperature and movement patterns.

[0972] Input: Motion video data, audio data, and body temperature data received by the server

[0973] Output: Pet emotion prediction results, health status analysis results

[0974] Specific operation: The AI ​​model processes the received video data and, for example, analyzes it as "the dog is wagging its tail" and determines that "the dog is happy." It also determines that "the dog is in good health" based on body temperature data.

[0975] Step 4: Sentiment Analysis

[0976] Server: Uses the user's camera and microphone to analyze the user's facial expressions, tone of voice, and gestures to infer emotions.

[0977] Input: Facial expression data, tone of voice data, and gesture data captured from the user's camera and microphone

[0978] Output: User emotion estimation result

[0979] Specific operation: The system processes data from when the user speaks using facial expressions and voice, and infers that the user is relaxed. For example, it analyzes facial expressions as "smiling" and voice tone as "calm."

[0980] Step 5: Notification of results

[0981] Server: Notifies the user of the analysis results via a smartphone app.

[0982] Input: Pet emotion prediction results, health status analysis results, user emotion prediction results

[0983] Output: Message sent to the user's smartphone app

[0984] Specific operation: Based on the analysis results, a push notification is sent to the user's smartphone saying "your dog is happy." If the user feels stressed, the system provides advice such as "try ways to help your pet relax."

[0985] Step 6: Health monitoring and notification

[0986] Server: If an abnormality is detected in the pet's health condition, the server automatically notifies the veterinary clinic and the user.

[0987] Input: Pet health analysis results

[0988] Output: Alert notifications sent to veterinary clinics and users

[0989] Specific operation: If the temperature data is abnormally high, it is determined that there is a possibility of a fever. An email notification is sent to the veterinary clinic, and a push notification is sent to the user alerting them that their pet has a high temperature.

[0990] Step 7: Generate and play relaxation music

[0991] Device: Based on the user's selection, the AI ​​model generates relaxation music and plays it through the device's speakers.

[0992] Input: User's music preference data

[0993] Output: Generated relaxation music, music to be played

[0994] How it works: The user tells the app that they like classical music. The AI ​​model generates calming piano music and plays it through the device's built-in speaker at night, providing a relaxing environment for pets.

[0995] (Application example 2)

[0996] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0997] Conventional pet monitoring and health management systems provide real-time analysis of pet behavior, voice, and body temperature data, but they are unable to link the emotional state of the pet with the user based on this information, or provide appropriate feedback to the user based on the pet's estimated emotions. This has resulted in a lack of improvement in the quality of pet health management and emotional care. Furthermore, they lack the ability to immediately notify medical institutions when abnormalities are detected in the pet's health.

[0998] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0999] In this invention, the server includes a means for acquiring data on the pet's movements, voice, and body temperature in real time, a means for analyzing the acquired data to infer the pet's emotions, and a means for simultaneously analyzing the emotional states of the user and the pet and providing appropriate feedback, thereby enabling more appropriate communication and health management that takes into account the emotional states of the pet and the user.

[1000] "Behavioral data" refers to information about a pet's movements and behavior, and is primarily data obtained visually using cameras and sensors.

[1001] "Audio data" refers to information about pets' cries and vocalizations, and is data obtained using audio sensors and microphones.

[1002] "Body temperature data" refers to information about a pet's body temperature, and is data obtained using a temperature sensor.

[1003] "Real-time capture" means that data is collected and processed at the present time and in the moment, making it available almost immediately.

[1004] "Analysis" refers to the process of analyzing acquired data using mathematical models and algorithms to extract meaning and trends.

[1005] "Emotion inference" refers to estimating the emotional state of a pet from the analyzed data, including various emotional states such as joy, sadness, excitement, and anxiety.

[1006] "Notifying" refers to the act of informing the user of the analysis results or anomaly detection information, which is usually done via a smartphone or other notification means.

[1007] "Health monitoring" refers to the process of continually checking your pet's health information to ensure there are no irregularities or abnormalities.

[1008] "Notifying a medical institution" refers to the act of communicating information to an appropriate medical provider when an abnormality in a pet's health is detected.

[1009] "Generating preferred music" refers to the process of automatically creating music that will relax your pet using an AI model.

[1010] "Playing" refers to the act of making the generated music audible through speakers or other audio devices.

[1011] "Simultaneous analysis of the emotional states of the user and the pet" refers to the process of collecting the emotional states of both the pet and the user in real time and simultaneously analyzing them based on that data.

[1012] "Providing appropriate feedback" refers to the act of suggesting and notifying users of optimal actions and information based on the analysis results.

[1013] This invention relates to a system that acquires pet behavior, voice, and body temperature data in real time, analyzes this information to infer the pet's emotions, and notifies the user. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it takes into account the emotional states of both the pet and the user, enabling more appropriate communication and health management.

[1014] System Configuration

[1015] The system mainly consists of the following components:

[1016] 1. Device that collects movement, voice, and body temperature data

[1017] Camera: Records your pet's movements as video and captures movement data in real time.

[1018] Sensor: Detects your pet's cries and body temperature and collects data.

[1019] 2. Server that analyzes the data

[1020] AI model: A mathematical model that analyzes the acquired data and predicts the pet's emotions and health condition.

[1021] 3. Means of notifying users

[1022] Smartphone app: An application that notifies users of analysis results and alerts when abnormalities are detected.

[1023] 4. Means of notifying medical institutions

[1024] Automatic notification system: A system that contacts the veterinary clinic when an abnormality is detected in the pet's health.

[1025] 5. Means of generating and playing music

[1026] AI model: Generates music tailored to your pet's preferences.

[1027] Built-in speaker: Ability to play generated music.

[1028] 6. Emotion engine that recognizes user emotions

[1029] Camera and microphone: Analyzes the user's facial expressions, tone of voice, and gestures to infer emotions.

[1030] Program processing

[1031] Data collection and transmission

[1032] The device uses a camera installed by the user to record pet movements in real time, and sensors to capture sounds and body temperature data, which are then periodically sent to a server via the internet.

[1033] Emotion and health analysis

[1034] The server inputs the received data into an AI model and analyzes the pet's movements and voice patterns to infer emotions. It also monitors the pet's health status based on its body temperature and movement patterns, and any abnormalities detected are recorded in a database. Furthermore, the server uses the user's camera and microphone to analyze the user's facial expressions, tone of voice, and gestures to infer emotions.

[1035] Notification of results

[1036] The server notifies the user of the analysis results via a smartphone app. For example, if it determines that "your pet is happy," that information is displayed on the user's smartphone. The notification content is also adjusted based on the user's emotions. For example, if the user is feeling stressed, the server provides advice such as "Try ways to help your pet relax."

[1037] Health Monitoring and Notification

[1038] The server regularly monitors the pet's health and detects any abnormalities in body temperature or behavior. For example, if the body temperature is higher than normal, it determines that the pet may have a fever. If an abnormality in health is detected, a notification is automatically sent to a veterinary clinic. This notification includes detailed health data about the pet. At the same time, an alert is sent to the user via a smartphone app, allowing the user to quickly check if there is anything abnormal with their pet's health.

[1039] Relaxation music generation and playback

[1040] The user registers their pet's preferred music through a smartphone app on the device. For example, they may enter information such as "I like classical music." The server uses an AI model to generate relaxing music based on the music preference information obtained from the user. The generated music is customized to suit each pet's individual preferences. The received music data is played back to provide a relaxing environment for the pet. For example, playing calming piano music at night can promote relaxation for the pet.

[1041] Specific examples

[1042] As a concrete example, let's consider the case of a dog left at a pet hotel. A camera records the dog's movements and facial expressions, and sensors detect its barks and body temperature data. The dog's tail wagging and energetic barks are captured. The server receives this data and uses an AI model to infer that the dog is excited, and based on the body temperature data, determines that the dog is in good health. These analysis results are notified to the user's smartphone app, which displays a message saying that the dog is excited. If the user's emotions indicate stress, the system also provides advice to encourage relaxation.

[1043] Prompt Sentence Examples

[1044] For example, give the generative AI model the following prompt:

[1045] I'd like to create an application for a pet hotel that uses a system to capture and analyze pet behavior, voice, and temperature data in real time, and then predict and notify the pet's emotions and health status. I'm considering a system that notifies the user of detailed analysis results and sends alerts to hotel staff and pet owners in the event of an abnormality. Please provide an example implementation using Python and TensorFlow.

[1046] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1047] Step 1:

[1048] Data collection

[1049] The device collects movement data, audio data, and body temperature data in real time. Specifically, the camera records the pet's movements, the microphone records its cries, and the temperature sensor measures its body temperature. The input is the camera footage, audio files, and body temperature data, and the output is the raw data files.

[1050] Step 2:

[1051] Sending data

[1052] The data collected by the terminal is sent to the server via the Internet. The data is sent directly without any processing. The input is the raw data, and the output is the success status of data transmission to the server.

[1053] Step 3:

[1054] Preparing for analysis

[1055] The server preprocesses the received raw data. Specifically, it extracts the camera footage frame by frame, performs digital signal processing on the audio data, and digitizes the body temperature data. The input is raw data, and the output is preprocessed data.

[1056] Step 4:

[1057] emotion guessing

[1058] The server inputs the preprocessed data into an AI model to infer the pet's emotions. The AI ​​model uses an algorithm to identify emotions from movement patterns, vocal tones, and changes in body temperature. The input is the preprocessed data, and the output is an inferred emotional state. Specifically, it assigns labels such as "happy," "anxious," and "excited."

[1059] Step 5:

[1060] Health monitoring

[1061] The server uses an AI model to analyze the body temperature data and monitor the pet's health, particularly detecting abnormalities when the temperature is outside the normal range. The input is the body temperature data, and the output is the health status assessment result. Specific outputs include labels such as "healthy," "fever," and "low body temperature."

[1062] Step 6:

[1063] Notification of analysis results

[1064] The server notifies the user's smartphone app of the emotion prediction results and health status evaluation results. The input is the analysis results, and the output is a notification message displayed on the user's smartphone app. Specific notification content includes "Your pet is happy" or "Your pet has a fever."

[1065] Step 7:

[1066] Notification to medical institutions

[1067] If the server detects an abnormality, it automatically notifies the veterinary clinic. The input is the health status assessment result, and the output is a notification message sent to the veterinary clinic. Specific notification content might be something like, "Your pet has a fever. Immediate medical examination is required."

[1068] Step 8:

[1069] Music generation and playback

[1070] The device generates music that the pet likes and starts playing it. The server generates the music using an AI model and sends the data to the device. The input is the pet's emotional state and preferred music information, and the output is the generated music data. Specific music examples include "classical music" and "gentle piano pieces."

[1071] Step 9:

[1072] User sentiment analysis

[1073] The server analyzes the user's facial expressions, tone of voice, and gestures using a camera and microphone to infer the user's emotions. The input is the user's video and audio data, and the output is an inferred result of the user's emotional state. Specific inferred results include "stress," "relaxation," and "joy."

[1074] Step 10:

[1075] Providing feedback

[1076] Based on the analysis results, the server provides appropriate feedback to the user. The input is the estimated emotional state of the pet and the user, and the output is a feedback message to the user. Specific feedback content might be, "Spend more time relaxing with your pet."

[1077] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1078] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1079] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1080] [Third embodiment]

[1081] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1082] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1083] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1084] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1085] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1086] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1087] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1088] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1089] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1090] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1091] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1092] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1093] The present invention is a system that acquires pet movement, voice, and body temperature data in real time, analyzes this information to infer the pet's emotions, and notifies the user. It also has the function of monitoring the pet's health condition, notifying a veterinary medical institution if an abnormality is detected, and generating and playing music tailored to the pet's preferences. The following describes in detail an embodiment of this system.

[1094] System Configuration

[1095] The system includes the following main components:

[1096] 1. A device that collects pet movement, voice, and body temperature data

[1097] Camera: Records your pet's movements as video and captures movement data in real time.

[1098] Sensor: Detects your pet's cries and body temperature and collects data.

[1099] 2. Server that analyzes the data

[1100] AI model: A mathematical model that analyzes the acquired data and predicts the pet's emotions and health condition.

[1101] 3. Means of notifying users

[1102] Smartphone app: An application that notifies users of analysis results and alerts when abnormalities are detected.

[1103] 4. Means of notifying the veterinary medical facility

[1104] Automatic notification system: A system that contacts the veterinary clinic when an abnormality is detected in the pet's health.

[1105] 5. Means of generating and playing music

[1106] AI model: Generates music tailored to your pet's preferences.

[1107] Built-in speaker: Ability to play generated music.

[1108] Program processing

[1109] Data collection and transmission

[1110] Device: A camera installed by the user records the pet's movements in real time, and sensors capture sounds and body temperature data. This data is periodically sent to a server via the Internet.

[1111] Emotion and health analysis

[1112] Server: The received data is input into an AI model, which analyzes the pet's movements and voice patterns to infer its emotions. It also monitors its health from its body temperature and movement patterns, and any abnormalities detected are recorded in a database.

[1113] Notification of results

[1114] Server: The analysis results are notified to the user via a smartphone app. For example, if it is determined that the pet is happy, that information is displayed on the user's smartphone. If any abnormalities are detected in the pet's health, a notification is automatically sent to the veterinary clinic, and an alert is also sent to the user.

[1115] Relaxation music generation and playback

[1116] Device: The user registers their pet's favorite music through a smartphone app, and that information is sent to the server.

[1117] Server: Uses an AI model to generate optimal relaxation music for pets and sends the music data to the device.

[1118] Terminal: The received music data is played through the camera's built-in speaker, providing a relaxing environment for your pet.

[1119] Specific examples

[1120] For example, we will explain how a family dog ​​might use this system over the course of a day.

[1121] 1. Data Collection:

[1122] Device: The camera records the dog's movements and facial expressions, and sensors detect its barks and body temperature. The device captures the dog's wagging tail and lively barks.

[1123] 2. Data Analysis:

[1124] Server: The AI ​​model analyzes the received data and infers that the dog is excited. It also determines that the dog is in good health based on the body temperature data.

[1125] 3. Result notification:

[1126] Server: The analysis results are sent to the user's smartphone app. A message saying "The dog is excited" is displayed and the user confirms it.

[1127] 4. Health Monitoring and Notification:

[1128] Server: Regularly checks the pet's health and, if the body temperature is abnormally high, determines that the pet may have a fever and notifies the veterinary clinic. An alert is also sent to the user.

[1129] 5. Generate and play relaxation music:

[1130] Device: At night, the camera's built-in speaker plays soothing piano music to help your dog relax.

[1131] In this way, by using the system of the present invention, it is possible to accurately grasp the emotions and health status of pets, enabling appropriate management and care.

[1132] The processing flow will be explained below.

[1133] Step 1:

[1134] Device: The camera records your pet's movements in real time, and sensors capture sounds and body temperature data, which are then recorded as detailed information about your pet's movements and sounds.

[1135] Step 2:

[1136] Terminal: Collected data is temporarily stored through batch processing and sent to the server at regular intervals (e.g., every 5 minutes). The data is encrypted during transmission to ensure secure transmission.

[1137] Step 3:

[1138] Server: Decompresses and decrypts the data received from the terminal and temporarily stores it in a database. It then checks the data for consistency and eliminates missing or abnormal data.

[1139] Step 4:

[1140] Server: Inputs data into the AI ​​model and infers the pet's emotions. For example, if a large tail wag and a high-pitched bark are detected at the same time, it will determine that the pet is happy.

[1141] Step 5:

[1142] Server: Stores the estimated emotions in a database and prepares to notify the user in real time via a smartphone app.

[1143] Step 6:

[1144] Device: The user's smartphone app receives the notification sent from the server and displays "Your pet is happy." The user can check this information and communicate with their pet.

[1145] Step 7:

[1146] Server: Regularly monitors the pet's health and detects abnormalities in body temperature or behavior. For example, if the body temperature is higher than normal, it determines that the pet may have a fever.

[1147] Step 8:

[1148] Server: If an abnormality in health is detected, it automatically sends a notification to the veterinary clinic, which includes detailed health data about the pet.

[1149] Step 9:

[1150] Server: At the same time, an alert is sent to the user via the smartphone app, allowing the user to quickly check if there is anything abnormal with their pet's health.

[1151] Step 10:

[1152] Device: The user registers their pet's favorite music through a smartphone app. For example, they can enter information such as "I like classical music."

[1153] Step 11:

[1154] Server: Based on the user's music preferences, the AI ​​model generates relaxing music that is customized to suit each pet's individual preferences.

[1155] Step 12:

[1156] Server: The generated music data is sent to the device, which then plays it to the pet through the camera's built-in speaker.

[1157] Step 13:

[1158] Device: Plays the received music data to provide a relaxing environment for your pet. For example, playing calming piano music at night can help your pet relax.

[1159] Example 1

[1160] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1161] Conventional technology has limited methods for accurately understanding a pet's emotions and health condition and communicating this to the user. Furthermore, existing systems lack the functionality to generate and play relaxation music tailored to the pet's preferences. This makes it difficult to manage a pet's emotions and health condition, and for users to understand their pet's condition in real time. Therefore, a system is needed that can analyze a pet's emotions and health condition in real time and quickly notify a veterinary medical facility if there are any abnormalities.

[1162] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1163] In this invention, the server includes means for acquiring data on the pet's movements, voice, and body temperature in real time, means for analyzing the acquired data and inferring the pet's emotions, means for notifying the user of the inferred emotions, means for monitoring the pet's health condition and notifying a veterinary medical institution if an abnormality is detected, means for the user to register the type of music preferred by the pet, means for the generative AI model to generate music based on the registered music preferences, and means for playing the generated music. This makes it possible to grasp the pet's emotions and health condition in real time and provide relaxing music.

[1164] "Movement data" refers to data relating to the pet's physical movements and behavior.

[1165] "Audio data" refers to data related to the cries and sounds made by pets.

[1166] "Body temperature data" refers to data relating to the pet's body temperature.

[1167] "Real-time" refers to data acquisition and analysis occurring almost immediately.

[1168] "Means of acquisition" refers to methods of collecting data using devices such as cameras and sensors.

[1169] "Means of analysis" refers to the way in which acquired data is processed, analyzed, and converted into meaningful information.

[1170] "Means for inferring emotions" refers to a method for determining and inferring a pet's current emotions from the analyzed data.

[1171] "Means of notification" refers to the method of notifying users and related organizations of analysis results and abnormalities.

[1172] "Health monitoring measures" are methods for continuously observing a pet's temperature and behavior to assess its health.

[1173] "Means for detecting abnormalities" are methods for detecting abnormalities in health status when they occur.

[1174] "Means of notifying a veterinary medical institution" refers to a method of notifying a specialized institution such as a veterinary hospital of a detected abnormality.

[1175] The "means for generating preferred music" is a method for creating relaxation music that matches a pet's preferences using a generative AI model.

[1176] "Means for playing" refers to a method for playing the generated music through a device such as a speaker.

[1177] A "generative AI model" is a model that uses artificial intelligence to analyze data and generate music.

[1178] A "prompt sentence" is an input sentence that conveys specific instructions or requests to a generative AI model.

[1179] The present invention is a system that collects pet movement, voice, and body temperature data in real time, analyzes this data to infer the pet's emotions, and notifies the user. It also has the function of monitoring the pet's health and notifying a veterinary clinic if an abnormality is detected. It also has the function of generating and playing music tailored to the pet's preferences.

[1180] Program processing

[1181] Data collection

[1182] Device: First, the user prepares a camera and a sensor. Specifically, the camera (e.g., Logitech C920) records the pet's movements in real time, and the sensor (e.g., DHT22) captures the pet's voice and body temperature. These data are collected by a device placed near the pet.

[1183] Sending data

[1184] Device: The collected data is sent to a server over the internet, packaged in a standard data format such as JSON, and periodically sent to the server via a Wi-Fi connection.

[1185] Emotion and health analysis

[1186] Server: The received data is input into an AI model (e.g., a neural network model using TensorFlow) running on the server. The server analyzes the video data using computer vision technology (e.g., YOLO or OpenPose) to infer the pet's emotions from its movement patterns. The audio data is also analyzed using a voice recognition system (e.g., Google Speech-to-Text API). The body temperature data is compared with pre-set reference values ​​to assess the pet's health.

[1187] Notification of results

[1188] Server: The analysis results are sent to the user's smartphone app. Notifications are sent using a notification service such as Firebase Cloud Messaging. For example, if the analysis result indicates that "the dog is excited," the user's smartphone will be notified.

[1189] Health Monitoring and Notification

[1190] Server: Monitors the pet's health 24 hours a day, and if the pet's body temperature is abnormally high (e.g., above 39°C), an alert is automatically sent to a veterinary clinic. At the same time, the user is notified of the health alert via their smartphone app.

[1191] Relaxation music generation and playback

[1192] User: The user registers their pet's favorite music through a smartphone app. For example, they can specify "classical music."

[1193] Server: Based on the registered information, the server generates music using a generative AI model (e.g., OpenAI's GPT-3). An example prompt is "Generate a calming piano piece that will help my dog ​​relax at night."

[1194] Device: The generated music data is sent to the device and played through the camera's built-in speaker, providing a relaxing environment for your pet.

[1195] Specific examples

[1196] For example, we explain how a family dog ​​uses this system from morning until night.

[1197] Data collection: Cameras capture the dog's movements, and sensors capture barks and body temperature data.

[1198] Data transmission: Collected data is sent to the server every 5 seconds.

[1199] Data analysis: The data received by the server is input into the AI ​​model, which analyzes it as "the dog is excited."

[1200] Result notification: The analysis result "The dog is excited" is notified to the user's smartphone.

[1201] Health monitoring: If the body temperature exceeds 39°C, it is considered a "fever" and an animal medical facility is automatically notified.

[1202] Relaxation Music: At night, gentle piano music generated by the server is played through the camera's built-in speaker to help your dog relax.

[1203] In this way, the system of the present invention can grasp the emotions and health condition of a pet in real time and quickly take appropriate measures.

[1204] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1205] Step 1: Collect data

[1206] Device: A user-installed camera (e.g., Logitech C920) and sensor (e.g., DHT22) collect pet behavior, sound, and temperature data in real time. The camera captures video at 60 fps and acquires behavior data. The sensor periodically reads and stores temperature and sound data.

[1207] Input: Pet movement, voice, and body temperature data

[1208] Output: A package of captured motion, voice, and temperature data

[1209] Specific behavior:

[1210] The camera records the dog's walking and eating, while sensors measure its barking and temperature.

[1211] Step 2: Sending data

[1212] Device: The collected data is converted into JSON format and sent to the server via Wi-Fi connection. The data is set to be sent every 5 seconds.

[1213] Input: Data collected in step 1

[1214] Output: Movement, voice, and temperature data sent to the server

[1215] Specific behavior:

[1216] The device automatically packages the data collected by the cameras and sensors and sends it to a server via the internet.

[1217] Step 3: Emotion and health analysis

[1218] Server: The received data is input into an AI model for analysis. For example, movement data is analyzed using a computer vision model, voice data is analyzed using a voice recognition model, and body temperature data is compared with pre-defined criteria.

[1219] Input: Data sent in step 2

[1220] Output: Analysis results (e.g. dog is excited, body temperature is normal)

[1221] Specific behavior:

[1222] The server analyzes the video data and determines that the dog is wagging its tail. It analyzes the audio data and determines that the dog is barking. It also checks the body temperature and records it as 36.5°C.

[1223] Step 4: Notification of results

[1224] Server: Notifies the user of the analysis results via their smartphone app. Uses services such as Firebase Cloud Messaging to communicate information to the user in real time.

[1225] Input: Analysis results obtained in step 3

[1226] Output: Notification sent to the user

[1227] Specific behavior:

[1228] The server formats the message "The dog is excited" in JSON format and notifies the user using the Firebase Cloud Messaging API.

[1229] Step 5: Health monitoring and notification

[1230] Server: Continuously monitors body temperature and behavior data, and if an abnormality is detected, contacts a veterinary medical facility. If the standard value is exceeded, an alert is automatically issued.

[1231] Input: Body temperature data and movement data obtained in step 3

[1232] Output: Abnormal alert notification

[1233] Specific behavior:

[1234] The server monitors body temperature data 24 hours a day, and if the temperature exceeds 39°C, it sends a "fever" alert to the veterinary clinic and notifies the user at the same time.

[1235] Step 6: Generate and play relaxation music

[1236] User: Register your pet's favorite music through the smartphone app. You can specify genres such as "classical music."

[1237] Server: Generates music using a generative AI model based on the registered information. Enter a prompt such as "Generate classical music" into the AI ​​model.

[1238] Device: The generated music data is played through the camera's built-in speaker.

[1239] Input: Music type (user input), music generation prompt

[1240] Output: Generated music data, music to be played

[1241] Specific behavior:

[1242] The user selects "classical music" and sends it to the server. The server then inputs the prompt "Please generate a gentle piano piece that will help my dog ​​relax at night" into the AI ​​model and sends the generated music to the device. The device then plays the music, providing a relaxing effect to the dog.

[1243] (Application example 1)

[1244] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1245] The present invention aims to accurately grasp a pet's emotions and health status by collecting data on its movements, voice, and body temperature in real time and analyzing this information. Furthermore, there is a need for early detection of abnormalities in a pet's health and providing a means of communication for prompt response, thereby improving pet health management and pet owner peace of mind. Another important issue is providing a means for generating and playing relaxation music to relieve stress in pets.

[1246] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1247] In this invention, the server includes a means for analyzing the pet's monitoring data and notifying the smartphone app of the results, a means for automatically contacting a veterinary clinic if the pet's health is abnormal, and a means for playing relaxation music if the pet is suspected to be stressed. This allows the pet's emotions and health condition to be accurately grasped in real time, enabling early detection of health abnormalities and prompt response. It also provides an environment that relieves pet stress, improving the owner's sense of security.

[1248] "Pet's movement, voice, and body temperature data" is a general term for behavioral information indicating the pet's movements, sound data reflecting vocal expressions, and temperature data for measuring body temperature.

[1249] "Real-time acquisition means" means the technological means for obtaining current information immediately without delay.

[1250] A "means for analyzing and inferring pet emotions" is a system or algorithm designed to analyze the collected data and infer the pet's emotional state.

[1251] "Means for notifying the user" refers to communication means or interfaces for immediately conveying analysis results and important information to the user.

[1252] The "means of notifying veterinary medical institutions when an abnormality is detected" is a system that automatically contacts pre-registered veterinary medical institutions when an abnormality is detected in a pet's health condition.

[1253] "Means for generating and playing music that suits pets' preferences" refers to devices or programs that use AI or algorithms to create music that suits pets' preferences and play it back.

[1254] "Means for analyzing pet monitoring data and notifying the results to a smartphone app" refers to methods or technologies for analyzing acquired monitoring data and notifying the user of the analysis results via a smartphone app.

[1255] "Means for automatically contacting a veterinary medical institution when an abnormality in the health of a pet" refers to a system or mechanism for automatically contacting a designated veterinary medical institution when an abnormality in the health of a pet is discovered.

[1256] The "means for playing relaxation music when it is determined that a pet is feeling stressed" is a mechanism for playing music to relax a pet when it is determined that the pet is in a stressful state.

[1257] To implement the present invention, a smart delivery robot specialized for food delivery services is used. The specific configuration and operation of the system are described below.

[1258] System Configuration

[1259] 1. Device Configuration

[1260] Camera: Used to record your pet's movements in real time.

[1261] Microphone: Used to capture pet sounds.

[1262] Temperature sensor: Used to measure your pet's body temperature.

[1263] Built-in speaker: Used to play the generated relaxation music.

[1264] 2. Server Configuration

[1265] AI model: Used to analyze the acquired data and run a generative AI model to infer the pet's emotions and health status, specifically using machine learning frameworks such as TensorFlow and PyTorch.

[1266] 3. User Interface

[1267] Smartphone app: Used to notify users of analysis results and alerts when anomalies are detected. Notification APIs such as Firebase Cloud Messaging are used.

[1268] How it works

[1269] Data collection

[1270] The device records your pet's movements in real time with a camera, picks up its cries with a microphone, and measures its body temperature with a temperature sensor, all of which are immediately sent to a server.

[1271] Data analysis

[1272] The server inputs the received movement data, voice data, and body temperature data into a generative AI model to analyze the pet's emotions and health condition. For example, by inputting the movement data and voice data into the AI ​​model, it can infer that the pet is "excited," and from the body temperature data, it can determine that the pet is "in good health."

[1273] Result notification

[1274] The server notifies the user of the analysis results via a smartphone app, which displays the results as messages such as "your pet is excited" or "your pet is in good health."

[1275] Health monitoring and abnormality notification

[1276] The server monitors the health of your pet and automatically sends a notification to a veterinary clinic if an abnormality is detected. For example, if your pet's temperature is abnormally high, it will immediately determine that there is a possibility of a fever and contact the veterinary clinic. An alert will also be sent to the user.

[1277] Relaxation music generation and playback

[1278] The device uses a generative AI model to generate relaxing music and plays it through the built-in speaker to provide a relaxing environment for pets, especially at night, if the device detects that your pet is feeling stressed.

[1279] Specific examples

[1280] For example, consider the case where a food delivery robot arrives at a user's home and encounters a pet. At this time, a camera records the pet's movements in real time, a microphone picks up its meows, and a temperature sensor measures its body temperature. The data is immediately sent to a server and analyzed by a generative AI model. The resulting analysis results, such as "the pet is excited" or "its health is good," are notified to the user's smartphone. If the pet appears stressed, relaxing music is played.

[1281] Prompt Sentence Examples

[1282] Example prompt sentence:

[1283] Take your pet's motion and voice data as input and infer its emotions. Then, analyze its temperature data to determine its health status.

[1284] As a result, the system of the present invention can accurately grasp the emotions and health status of pets in real time, enabling early detection of health abnormalities and prompt response, and providing an environment that reduces stress for pets.

[1285] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1286] Step 1:

[1287] Data collection:

[1288] The device records the pet's movements in real time with a camera, picks up its cries with a microphone, and measures its body temperature with a temperature sensor. These data are collected as movement video data, audio data, and body temperature data.

[1289] Input: Pet movement, sound, temperature

[1290] Output: Motion video data, audio data, body temperature data

[1291] Step 2:

[1292] Data transmission:

[1293] The device sends the collected data to a server via the internet, and each data is protected by a secure communication protocol to prevent data loss or tampering.

[1294] Input: motion video data, audio data, body temperature data

[1295] Output: Each data sent to the server

[1296] Step 3:

[1297] Data Analysis:

[1298] The server inputs the received motion video data, audio data, and body temperature data into a generative AI model to analyze the pet's emotions and health condition. The AI ​​model infers the pet's emotions from its motion and audio patterns and evaluates its health condition based on the body temperature data.

[1299] Input: motion video data, audio data, body temperature data

[1300] Output: Emotion prediction result, health status evaluation result

[1301] Step 4:

[1302] Notification of results:

[1303] The server then notifies the user of the analysis results via a smartphone app. The notification content, which is related to the pet's emotional state and health status, is displayed in a user-friendly format.

[1304] Input: Emotion prediction results, health status assessment results

[1305] Output: A notification message to the user (e.g. "Your pet is excited" or "It is in good health")

[1306] Step 5:

[1307] Automatic notification of health abnormalities:

[1308] The server operates a system that automatically contacts animal medical institutions if an abnormality is detected in the pet's health condition. An alert is also sent to the user, enabling a prompt response.

[1309] Input: Health status assessment result (if abnormality detected)

[1310] Output: Notification message to veterinary clinic, alert to user

[1311] Step 6:

[1312] Relaxation music generation and playback:

[1313] The device uses a generative AI model to generate relaxing music for your pet, which is then played through the built-in speaker, providing a relaxing environment for your pet.

[1314] Input: Emotion prediction result (if stress is detected)

[1315] Output: Generated relaxation music data, music playback

[1316] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1317] This invention is a system that acquires pet behavior, voice, and body temperature data in real time, analyzes this information, predicts the pet's emotions, and notifies the user. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it takes into account the emotional states of both the pet and the user, enabling more appropriate communication and health management.

[1318] System Configuration

[1319] The system includes the following main components:

[1320] 1. A device that collects pet movement, voice, and body temperature data

[1321] Camera: Records your pet's movements as video and captures movement data in real time.

[1322] Sensor: Detects your pet's cries and body temperature and collects data.

[1323] 2. Server that analyzes the data

[1324] AI model: A mathematical model that analyzes the acquired data and predicts the pet's emotions and health condition.

[1325] 3. Means of notifying users

[1326] Smartphone app: An application that notifies users of analysis results and alerts when abnormalities are detected.

[1327] 4. Means of notifying the veterinary medical facility

[1328] Automatic notification system: A system that contacts the veterinary clinic when an abnormality is detected in the pet's health.

[1329] 5. Means of generating and playing music

[1330] AI model: Generates music tailored to your pet's preferences.

[1331] Built-in speaker: Ability to play generated music.

[1332] 6. Emotion engine that recognizes user emotions

[1333] Camera and microphone: Analyzes the user's facial expressions, tone of voice, and gestures to infer emotions.

[1334] Program processing

[1335] Data collection and transmission

[1336] Device: A camera installed by the user records the pet's movements in real time, and sensors capture its sounds and body temperature data, which are periodically sent to a server via the Internet.

[1337] Emotion and health analysis

[1338] Server: The received data is input into an AI model, which analyzes the pet's movements and voice patterns to infer its emotions. It also monitors its health from its body temperature and movement patterns, and any abnormalities detected are recorded in a database.

[1339] Server: Additionally, the server uses the user's camera and microphone to analyze the user's facial expressions, tone of voice, and gestures to infer emotions.

[1340] Notification of results

[1341] Server: Notifies the user of the analysis results via a smartphone app. For example, if it determines that "your pet is happy," that information is displayed on the user's smartphone. The notification content is also adjusted based on the user's emotions. For example, if the user is feeling stressed, advice such as "Try ways to help your pet relax" is provided.

[1342] Health Monitoring and Notification

[1343] Server: Regularly monitors the pet's health and detects abnormalities in body temperature or behavior. For example, if the body temperature is higher than normal, it determines that the pet may have a fever.

[1344] Server: If an abnormality in health is detected, it automatically sends a notification to the veterinary clinic, which includes detailed health data about the pet.

[1345] Server: At the same time, an alert is sent to the user via the smartphone app, allowing the user to quickly check if there is anything abnormal with their pet's health.

[1346] Relaxation music generation and playback

[1347] Device: The user registers their pet's favorite music through a smartphone app. For example, they can enter information such as "I like classical music."

[1348] Server: Based on the user's music preferences, the AI ​​model generates relaxing music that is customized to suit each pet's individual preferences.

[1349] Device: Plays the received music data to provide a relaxing environment for your pet. For example, playing calming piano music at night can help your pet relax.

[1350] Specific examples

[1351] For example, we will explain how a dog and its owner in a household might use this system over the course of a day.

[1352] 1. Data Collection:

[1353] Device: The camera records the dog's movements and facial expressions, and sensors detect its barks and body temperature. The device captures the dog's wagging tail and lively barks.

[1354] 2. User sentiment analysis:

[1355] Server: Using the user's camera and microphone, the server analyzes the user's facial expressions and tone of voice to determine whether the user is experiencing stress.

[1356] 3. Data Analysis:

[1357] Server: The AI ​​model analyzes the received data and infers that the dog is excited. It also determines that the dog is in good health based on the body temperature data.

[1358] 4. Result notification:

[1359] Server: The analysis results are sent to the user's smartphone app. A message is displayed stating that the dog is excited, and, taking into consideration that the user is feeling stressed, advice is provided to encourage relaxation.

[1360] 5. Health Monitoring and Notification:

[1361] Server: Regularly checks the pet's health and, if the body temperature is abnormally high, determines that the pet may have a fever and notifies the veterinary clinic. An alert is also sent to the user.

[1362] 6. Generate and play relaxation music:

[1363] Device: At night, the camera's built-in speaker plays soothing piano music to help your dog relax.

[1364] In this way, by using the system of the present invention, the emotions and health conditions of pets and users can be accurately understood, enabling appropriate management and care.

[1365] The processing flow will be explained below.

[1366] Step 1:

[1367] Device: The camera records your pet's movements in real time, while the sensors capture vocalization and temperature data. The camera captures video of your pet's movements and facial expressions, while the sensors record your pet's body temperature and vocalization frequency.

[1368] Step 2:

[1369] The terminal periodically processes the acquired data in batches and transmits them over the internet in encrypted form to a server. This transmission process occurs at regular intervals (e.g., every 5 minutes).

[1370] Step 3:

[1371] Server: The server decompresses and decrypts the data received from the device and temporarily stores it in a database. Here, it performs a data integrity check and eliminates any invalid or missing data.

[1372] Step 4:

[1373] Server: Inputs data into the AI ​​model, analyzes the pet's movements and vocal patterns, and infers its emotions. For example, if a vigorous tail wagging and a high-pitched bark are detected at the same time, it will determine that the pet is happy.

[1374] Step 5:

[1375] Server: Stores the estimated emotions in a database and prepares to notify the user. Notifications are set up to be sent in real time via a smartphone app.

[1376] Step 6:

[1377] Device: The user's smartphone app receives the notification sent from the server and displays the message "Your pet is happy." The user can check this information and communicate with their pet.

[1378] Step 7:

[1379] On the device: The device uses the user's camera and microphone to record the user's facial expressions, tone of voice, and gestures, which then stores the user's emotional data on the device.

[1380] Step 8:

[1381] Terminal: The user's emotional data is periodically sent to the server in encrypted format, similar to the data sent to pets.

[1382] Step 9:

[1383] Server: The received user emotion data is input into the AI ​​model to infer the user's emotions. For example, if the user's facial expression is grim and their voice tone is low, it is determined that the user is feeling stressed.

[1384] Step 10:

[1385] Server: Adjusts the content of notifications sent to pets based on the user's emotional data. For example, if the pet is excited and the user is feeling stressed, the server will provide advice such as "Try to help your pet relax."

[1386] Step 11:

[1387] Server: Regularly monitors the pet's health and detects abnormalities. For example, if the body temperature is higher than normal, it determines that the pet may have a fever.

[1388] Step 12:

[1389] Server: If an abnormality in health status is detected, a notification is automatically sent to a veterinary clinic. The notification contains detailed health data of the pet to aid in diagnosis.

[1390] Step 13:

[1391] Server: At the same time, an alert is sent to the user via the smartphone app, allowing them to quickly check if there is anything abnormal with their pet's health.

[1392] Step 14:

[1393] Device: The user registers their pet's favorite music through a smartphone app. For example, they can enter a preference such as "I like classical music."

[1394] Step 15:

[1395] Server: Based on the user's music preferences, the AI ​​model generates relaxing music that is customized to suit each pet's individual preferences.

[1396] Step 16:

[1397] Server: The generated music data is sent to the device, which then plays it through the device's built-in speaker.

[1398] Step 17:

[1399] Device: Plays the received music data to provide a relaxing environment for your pet. For example, playing calming piano music at night can help your pet relax.

[1400] This series of steps allows accurate monitoring of the emotions and health of pets and users, and allows for appropriate management and countermeasures.

[1401] Example 2

[1402] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1403] Modern pet ownership requires predicting a pet's emotions and health status and notifying the user at the appropriate time. However, current technology makes it difficult to accurately predict a pet's emotions, and responses when abnormalities occur in the pet's health are often delayed. Furthermore, there are no notification systems that take the user's emotions into account. This makes it difficult to provide appropriate care and communication.

[1404] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1405] In this invention, the server includes means for acquiring pet movement, voice, and body temperature data in real time, means for analyzing the acquired data to infer the pet's emotions and health condition, means for notifying the user of the inferred emotions and health condition, means for acquiring the user's facial expressions, tone of voice, and gestures to infer emotions, means for notifying the veterinary clinic and the user if an abnormality in the pet's health condition is detected, and means for generating and playing music preferred by the pet. This makes it possible to accurately grasp the pet's emotions and health condition and quickly notify the user and the veterinary clinic. It also enables appropriate notifications that take the user's emotions into account and the generation and playback of music tailored to the pet's preferences, thereby achieving better communication between the pet and the user and appropriate health management.

[1406] "Movement data" is information that is recorded and collected in real time about your pet's body movements and behavior.

[1407] "Audio data" refers to acoustic information collected using sensors that record the cries and sounds made by pets.

[1408] "Body temperature data" refers to temperature information collected by measuring a pet's body temperature with a sensor.

[1409] "Acquisition means" refers to a device or equipment for acquiring pet movement data, voice data, and body temperature data in real time.

[1410] "Analysis methods" refers to the processes and algorithms used to use AI models to infer a pet's emotions and health status based on the acquired data.

[1411] "Notification means" refers to a device or application for communicating analysis results to users or animal medical facilities.

[1412] "Facial expression" refers to data that refers to facial movements and expressions captured using a camera to assess the user's emotions.

[1413] "Voice tone" is voice information used to infer emotions by analyzing the pitch and tone of the user's voice.

[1414] "Gesture" is movement information that is used to analyze the user's hand movements and body movements and to infer emotions.

[1415] "AI model" refers to a mathematical and computational model that uses artificial intelligence to analyze data and infer emotions and health states.

[1416] "Veterinary medical institution" refers to a medical facility or veterinary hospital that should be contacted if any abnormalities are discovered in the health of a pet.

[1417] "Music Generation Method" refers to the algorithm or process used to create music based on the pet's preferences.

[1418] "Playback means" refers to a device or system for actually playing back the generated music.

[1419] MODE FOR CARRYING OUT THE INVENTION

[1420] The present invention is a system that acquires pet movement, voice, and body temperature data in real time, analyzes this information to estimate the pet's emotions and health condition, and notifies the user. It also includes an emotion engine for recognizing the user's emotions, enabling appropriate communication and health management that takes into account the emotional states of both the pet and the user. Specific embodiments are described below.

[1421] System Configuration

[1422] The system includes the following main components:

[1423] 1. A device that collects pet movement, voice, and body temperature data

[1424] Camera: Records your pet's movements as video and captures movement data in real time.

[1425] Example: Logitech C920

[1426] Sensor: Detects your pet's cries and body temperature and collects data.

[1427] Example: DHT11

[1428] 2. Server that analyzes the data

[1429] AI model: A mathematical model that analyzes the acquired data and predicts the pet's emotions and health condition.

[1430] Examples: TensorFlow, Keras

[1431] Database: Stores analysis results and collected data.

[1432] Example: MySQL

[1433] 3. Means of notifying users

[1434] Smartphone app: An application that notifies users of analysis results and alerts when abnormalities are detected.

[1435] Example: React Native, Flutter

[1436] 4. Means of notifying the veterinary medical facility

[1437] Automatic notification system: A system that contacts the veterinary clinic when an abnormality is detected in the pet's health.

[1438] Examples: Twilio, Firebase Cloud Messaging

[1439] 5. Means of generating and playing music

[1440] AI model: Generates music tailored to your pet's preferences.

[1441] Example: Magenta

[1442] Built-in speaker: Ability to play generated music.

[1443] Example: JBL

[1444] 6. Emotion engine that recognizes user emotions

[1445] Camera and microphone: Analyzes the user's facial expressions, tone of voice, and gestures to infer emotions.

[1446] Examples: OpenFace, Praat

[1447] Program processing

[1448] Data collection

[1449] Device: A camera installed by the user records the pet's movements in real time, and sensors capture its sounds and body temperature data, which are periodically sent to a server via the Internet.

[1450] Data analysis

[1451] Server: The received data is input into an AI model, which analyzes the pet's movements and voice patterns to infer its emotions. It also monitors its health from its body temperature and movement patterns, and any abnormalities detected are recorded in a database.

[1452] Emotion analysis

[1453] Server: Uses the user's camera and microphone to analyze the user's facial expressions, tone of voice, and gestures to infer emotions.

[1454] Result notification

[1455] Server: Notifies the user of the analysis results via a smartphone app. For example, if it determines that "your pet is happy," that information is displayed on the user's smartphone. The notification content is also adjusted based on the user's emotions. For example, if the user is feeling stressed, advice such as "Try ways to help your pet relax" is provided.

[1456] Health Monitoring and Notification

[1457] Server: Regularly monitors the pet's health and detects abnormalities in body temperature or behavior. For example, if the body temperature is higher than normal, it determines that the pet may have a fever.

[1458] Server: If an abnormality in health is detected, it automatically sends a notification to the veterinary clinic, which includes detailed health data about the pet.

[1459] Server: At the same time, an alert is sent to the user via the smartphone app, allowing the user to quickly check if there is anything abnormal with their pet's health.

[1460] Relaxation music generation and playback

[1461] Device: The user registers their pet's favorite music through a smartphone app. For example, they can enter information such as "I like classical music."

[1462] Server: Based on the user's music preferences, the AI ​​model generates relaxing music that is customized to suit each pet's individual preferences.

[1463] Device: Plays the received music data to provide a relaxing environment for your pet. For example, playing calming piano music at night can help your pet relax.

[1464] Specific examples

[1465] Data collection

[1466] Device: The camera records the dog's movements and facial expressions, and sensors detect its barks and body temperature. The device captures the dog's wagging tail and lively barks.

[1467] Emotion analysis

[1468] Server: Using the user's camera and microphone, the server analyzes the user's facial expressions and tone of voice to determine whether the user is experiencing stress.

[1469] Data analysis

[1470] Server: The AI ​​model analyzes the received data and infers that the dog is excited. It also determines that the dog is in good health based on the body temperature data.

[1471] Result notification

[1472] Server: The analysis results are sent to the user's smartphone app. A message is displayed stating that the dog is excited, and, taking into consideration that the user is feeling stressed, advice is provided to encourage relaxation.

[1473] Health Monitoring and Notification

[1474] Server: Regularly checks the pet's health and, if the body temperature is abnormally high, determines that the pet may have a fever and notifies the veterinary clinic. An alert is also sent to the user.

[1475] Relaxation music generation and playback

[1476] Device: At night, the camera's built-in speaker plays soothing piano music to help your dog relax.

[1477] Prompt Sentence Examples

[1478] Based on the following data, please generate a program for a system that will infer a pet's emotions and notify the user. Please also clearly describe the system's overview and specific processing steps.

[1479] data:

[1480] Pet movement video data

[1481] Pet sounds and body temperature data

[1482] The user's facial expressions, tone of voice, and gestures

[1483] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1484] Step 1: Collect data

[1485] Device: Captures pet movement, voice, and temperature data in real time through cameras and sensors.

[1486] Input: Pet movement video, bark, body temperature

[1487] Output: Collected motion video data, audio data, and body temperature data

[1488] Specific operation: The camera records the pet's movements, the microphone records the dog's barks, and the temperature sensor measures the dog's temperature. For example, the camera records the dog jumping, the microphone picks up the sound of the dog barking, and the temperature sensor measures the dog's temperature at 37.8°C.

[1489] Step 2: Sending data

[1490] Terminal: Sends collected data to a server via the Internet.

[1491] Input: Collected motion video data, audio data, and body temperature data

[1492] Output: Motion video data, audio data, and body temperature data sent to the server

[1493] Specific operation: Data collected by the camera and sensors is uploaded to a cloud server via Wi-Fi. For example, motion video files, audio files, and body temperature data are sent to the cloud server.

[1494] Step 3: Data analysis

[1495] Server: Inputs the received data into an AI model, analyzes the pet's movements and voice patterns to infer emotions, and monitors its health from its body temperature and movement patterns.

[1496] Input: Motion video data, audio data, and body temperature data received by the server

[1497] Output: Pet emotion prediction results, health status analysis results

[1498] Specific operation: The AI ​​model processes the received video data and, for example, analyzes it as "the dog is wagging its tail" and determines that "the dog is happy." It also determines that "the dog is in good health" based on body temperature data.

[1499] Step 4: Sentiment Analysis

[1500] Server: Uses the user's camera and microphone to analyze the user's facial expressions, tone of voice, and gestures to infer emotions.

[1501] Input: Facial expression data, tone of voice data, and gesture data captured from the user's camera and microphone

[1502] Output: User emotion estimation result

[1503] Specific operation: The system processes data from when the user speaks using facial expressions and voice, and infers that the user is relaxed. For example, it analyzes facial expressions as "smiling" and voice tone as "calm."

[1504] Step 5: Notification of results

[1505] Server: Notifies the user of the analysis results via a smartphone app.

[1506] Input: Pet emotion prediction results, health status analysis results, user emotion prediction results

[1507] Output: Message sent to the user's smartphone app

[1508] Specific operation: Based on the analysis results, a push notification is sent to the user's smartphone saying "your dog is happy." If the user feels stressed, the system provides advice such as "try ways to help your pet relax."

[1509] Step 6: Health monitoring and notification

[1510] Server: If an abnormality is detected in the pet's health condition, the server automatically notifies the veterinary clinic and the user.

[1511] Input: Pet health analysis results

[1512] Output: Alert notifications sent to veterinary clinics and users

[1513] Specific operation: If the temperature data is abnormally high, it is determined that there is a possibility of a fever. An email notification is sent to the veterinary clinic, and a push notification is sent to the user alerting them that their pet has a high temperature.

[1514] Step 7: Generate and play relaxation music

[1515] Device: Based on the user's selection, the AI ​​model generates relaxation music and plays it through the device's speakers.

[1516] Input: User's music preference data

[1517] Output: Generated relaxation music, music to be played

[1518] How it works: The user tells the app that they like classical music. The AI ​​model generates calming piano music and plays it through the device's built-in speaker at night, providing a relaxing environment for pets.

[1519] (Application example 2)

[1520] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1521] Conventional pet monitoring and health management systems provide real-time analysis of pet behavior, voice, and body temperature data, but they are unable to link the emotional state of the pet with the user based on this information, or provide appropriate feedback to the user based on the pet's estimated emotions. This has resulted in a lack of improvement in the quality of pet health management and emotional care. Furthermore, they lack the ability to immediately notify medical institutions when abnormalities are detected in the pet's health.

[1522] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1523] In this invention, the server includes a means for acquiring data on the pet's movements, voice, and body temperature in real time, a means for analyzing the acquired data to infer the pet's emotions, and a means for simultaneously analyzing the emotional states of the user and the pet and providing appropriate feedback, thereby enabling more appropriate communication and health management that takes into account the emotional states of the pet and the user.

[1524] "Behavioral data" refers to information about a pet's movements and behavior, and is primarily data obtained visually using cameras and sensors.

[1525] "Audio data" refers to information about pets' cries and vocalizations, and is data obtained using audio sensors and microphones.

[1526] "Body temperature data" refers to information about a pet's body temperature, and is data obtained using a temperature sensor.

[1527] "Real-time capture" means that data is collected and processed at the present time and in the moment, making it available almost immediately.

[1528] "Analysis" refers to the process of analyzing acquired data using mathematical models and algorithms to extract meaning and trends.

[1529] "Emotion inference" refers to estimating the emotional state of a pet from the analyzed data, including various emotional states such as joy, sadness, excitement, and anxiety.

[1530] "Notifying" refers to the act of informing the user of the analysis results or anomaly detection information, which is usually done via a smartphone or other notification means.

[1531] "Health monitoring" refers to the process of continually checking your pet's health information to ensure there are no irregularities or abnormalities.

[1532] "Notifying a medical institution" refers to the act of communicating information to an appropriate medical provider when an abnormality in a pet's health is detected.

[1533] "Generating preferred music" refers to the process of automatically creating music that will relax your pet using an AI model.

[1534] "Playing" refers to the act of making the generated music audible through speakers or other audio devices.

[1535] "Simultaneous analysis of the emotional states of the user and the pet" refers to the process of collecting the emotional states of both the pet and the user in real time and simultaneously analyzing them based on that data.

[1536] "Providing appropriate feedback" refers to the act of suggesting and notifying users of optimal actions and information based on the analysis results.

[1537] This invention relates to a system that acquires pet behavior, voice, and body temperature data in real time, analyzes this information to infer the pet's emotions, and notifies the user. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it takes into account the emotional states of both the pet and the user, enabling more appropriate communication and health management.

[1538] System Configuration

[1539] The system mainly consists of the following components:

[1540] 1. Device that collects movement, voice, and body temperature data

[1541] Camera: Records your pet's movements as video and captures movement data in real time.

[1542] Sensor: Detects your pet's cries and body temperature and collects data.

[1543] 2. Server that analyzes the data

[1544] AI model: A mathematical model that analyzes the acquired data and predicts the pet's emotions and health condition.

[1545] 3. Means of notifying users

[1546] Smartphone app: An application that notifies users of analysis results and alerts when abnormalities are detected.

[1547] 4. Means of notifying medical institutions

[1548] Automatic notification system: A system that contacts the veterinary clinic when an abnormality is detected in the pet's health.

[1549] 5. Means of generating and playing music

[1550] AI model: Generates music tailored to your pet's preferences.

[1551] Built-in speaker: Ability to play generated music.

[1552] 6. Emotion engine that recognizes user emotions

[1553] Camera and microphone: Analyzes the user's facial expressions, tone of voice, and gestures to infer emotions.

[1554] Program processing

[1555] Data collection and transmission

[1556] The device uses a camera installed by the user to record pet movements in real time, and sensors to capture sounds and body temperature data, which are then periodically sent to a server via the internet.

[1557] Emotion and health analysis

[1558] The server inputs the received data into an AI model and analyzes the pet's movements and voice patterns to infer emotions. It also monitors the pet's health status based on its body temperature and movement patterns, and any abnormalities detected are recorded in a database. Furthermore, the server uses the user's camera and microphone to analyze the user's facial expressions, tone of voice, and gestures to infer emotions.

[1559] Notification of results

[1560] The server notifies the user of the analysis results via a smartphone app. For example, if it determines that "your pet is happy," that information is displayed on the user's smartphone. The notification content is also adjusted based on the user's emotions. For example, if the user is feeling stressed, the server provides advice such as "Try ways to help your pet relax."

[1561] Health Monitoring and Notification

[1562] The server regularly monitors the pet's health and detects any abnormalities in body temperature or behavior. For example, if the body temperature is higher than normal, it determines that the pet may have a fever. If an abnormality in health is detected, a notification is automatically sent to a veterinary clinic. This notification includes detailed health data about the pet. At the same time, an alert is sent to the user via a smartphone app, allowing the user to quickly check if there is anything abnormal with their pet's health.

[1563] Relaxation music generation and playback

[1564] The user registers their pet's preferred music through a smartphone app on the device. For example, they may enter information such as "I like classical music." The server uses an AI model to generate relaxing music based on the music preference information obtained from the user. The generated music is customized to suit each pet's individual preferences. The received music data is played back to provide a relaxing environment for the pet. For example, playing calming piano music at night can promote relaxation for the pet.

[1565] Specific examples

[1566] As a concrete example, let's consider the case of a dog left at a pet hotel. A camera records the dog's movements and facial expressions, and sensors detect its barks and body temperature data. The dog's tail wagging and energetic barks are captured. The server receives this data and uses an AI model to infer that the dog is excited, and based on the body temperature data, determines that the dog is in good health. These analysis results are notified to the user's smartphone app, which displays a message saying that the dog is excited. If the user's emotions indicate stress, the system also provides advice to encourage relaxation.

[1567] Prompt Sentence Examples

[1568] For example, give the generative AI model the following prompt:

[1569] I'd like to create an application for a pet hotel that uses a system to capture and analyze pet behavior, voice, and temperature data in real time, and then predict and notify the pet's emotions and health status. I'm considering a system that notifies the user of detailed analysis results and sends alerts to hotel staff and pet owners in the event of an abnormality. Please provide an example implementation using Python and TensorFlow.

[1570] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1571] Step 1:

[1572] Data collection

[1573] The device collects movement data, audio data, and body temperature data in real time. Specifically, the camera records the pet's movements, the microphone records its cries, and the temperature sensor measures its body temperature. The input is the camera footage, audio files, and body temperature data, and the output is the raw data files.

[1574] Step 2:

[1575] Sending data

[1576] The data collected by the terminal is sent to the server via the Internet. The data is sent directly without any processing. The input is the raw data, and the output is the success status of data transmission to the server.

[1577] Step 3:

[1578] Preparing for analysis

[1579] The server preprocesses the received raw data. Specifically, it extracts the camera footage frame by frame, performs digital signal processing on the audio data, and digitizes the body temperature data. The input is raw data, and the output is preprocessed data.

[1580] Step 4:

[1581] emotion guessing

[1582] The server inputs the preprocessed data into an AI model to infer the pet's emotions. The AI ​​model uses an algorithm to identify emotions from movement patterns, vocal tones, and changes in body temperature. The input is the preprocessed data, and the output is an inferred emotional state. Specifically, it assigns labels such as "happy," "anxious," and "excited."

[1583] Step 5:

[1584] Health monitoring

[1585] The server uses an AI model to analyze the body temperature data and monitor the pet's health, particularly detecting abnormalities when the temperature is outside the normal range. The input is the body temperature data, and the output is the health status assessment result. Specific outputs include labels such as "healthy," "fever," and "low body temperature."

[1586] Step 6:

[1587] Notification of analysis results

[1588] The server notifies the user's smartphone app of the emotion prediction results and health status evaluation results. The input is the analysis results, and the output is a notification message displayed on the user's smartphone app. Specific notification content includes "Your pet is happy" or "Your pet has a fever."

[1589] Step 7:

[1590] Notification to medical institutions

[1591] If the server detects an abnormality, it automatically notifies the veterinary clinic. The input is the health status assessment result, and the output is a notification message sent to the veterinary clinic. Specific notification content might be something like, "Your pet has a fever. Immediate medical examination is required."

[1592] Step 8:

[1593] Music generation and playback

[1594] The device generates music that the pet likes and starts playing it. The server generates the music using an AI model and sends the data to the device. The input is the pet's emotional state and preferred music information, and the output is the generated music data. Specific music examples include "classical music" and "gentle piano pieces."

[1595] Step 9:

[1596] User sentiment analysis

[1597] The server analyzes the user's facial expressions, tone of voice, and gestures using a camera and microphone to infer the user's emotions. The input is the user's video and audio data, and the output is an inferred result of the user's emotional state. Specific inferred results include "stress," "relaxation," and "joy."

[1598] Step 10:

[1599] Providing feedback

[1600] Based on the analysis results, the server provides appropriate feedback to the user. The input is the estimated emotional state of the pet and the user, and the output is a feedback message to the user. Specific feedback content might be, "Spend more time relaxing with your pet."

[1601] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1602] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1603] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1604] [Fourth embodiment]

[1605] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1606] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1607] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1608] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1609] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1610] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1611] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1612] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1613] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1614] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1615] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1616] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1617] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1618] The present invention is a system that acquires pet movement, voice, and body temperature data in real time, analyzes this information to infer the pet's emotions, and notifies the user. It also has the function of monitoring the pet's health condition, notifying a veterinary medical institution if an abnormality is detected, and generating and playing music tailored to the pet's preferences. The following describes in detail an embodiment of this system.

[1619] System Configuration

[1620] The system includes the following main components:

[1621] 1. A device that collects pet movement, voice, and body temperature data

[1622] Camera: Records your pet's movements as video and captures movement data in real time.

[1623] Sensor: Detects your pet's cries and body temperature and collects data.

[1624] 2. Server that analyzes the data

[1625] AI model: A mathematical model that analyzes the acquired data and predicts the pet's emotions and health condition.

[1626] 3. Means of notifying users

[1627] Smartphone app: An application that notifies users of analysis results and alerts when abnormalities are detected.

[1628] 4. Means of notifying the veterinary medical facility

[1629] Automatic notification system: A system that contacts the veterinary clinic when an abnormality is detected in the pet's health.

[1630] 5. Means of generating and playing music

[1631] AI model: Generates music tailored to your pet's preferences.

[1632] Built-in speaker: Ability to play generated music.

[1633] Program processing

[1634] Data collection and transmission

[1635] Device: A camera installed by the user records the pet's movements in real time, and sensors capture sounds and body temperature data. This data is periodically sent to a server via the Internet.

[1636] Emotion and health analysis

[1637] Server: The received data is input into an AI model, which analyzes the pet's movements and voice patterns to infer its emotions. It also monitors its health from its body temperature and movement patterns, and any abnormalities detected are recorded in a database.

[1638] Notification of results

[1639] Server: The analysis results are notified to the user via a smartphone app. For example, if it is determined that the pet is happy, that information is displayed on the user's smartphone. If any abnormalities are detected in the pet's health, a notification is automatically sent to the veterinary clinic, and an alert is also sent to the user.

[1640] Relaxation music generation and playback

[1641] Device: The user registers their pet's favorite music through a smartphone app, and that information is sent to the server.

[1642] Server: Uses an AI model to generate optimal relaxation music for pets and sends the music data to the device.

[1643] Terminal: The received music data is played through the camera's built-in speaker, providing a relaxing environment for your pet.

[1644] Specific examples

[1645] For example, we will explain how a family dog ​​might use this system over the course of a day.

[1646] 1. Data Collection:

[1647] Device: The camera records the dog's movements and facial expressions, and sensors detect its barks and body temperature. The device captures the dog's wagging tail and lively barks.

[1648] 2. Data Analysis:

[1649] Server: The AI ​​model analyzes the received data and infers that the dog is excited. It also determines that the dog is in good health based on the body temperature data.

[1650] 3. Result notification:

[1651] Server: The analysis results are sent to the user's smartphone app. A message saying "The dog is excited" is displayed and the user confirms it.

[1652] 4. Health Monitoring and Notification:

[1653] Server: Regularly checks the pet's health and, if the body temperature is abnormally high, determines that the pet may have a fever and notifies the veterinary clinic. An alert is also sent to the user.

[1654] 5. Generate and play relaxation music:

[1655] Device: At night, the camera's built-in speaker plays soothing piano music to help your dog relax.

[1656] In this way, by using the system of the present invention, it is possible to accurately grasp the emotions and health status of pets, enabling appropriate management and care.

[1657] The processing flow will be explained below.

[1658] Step 1:

[1659] Device: The camera records your pet's movements in real time, and sensors capture sounds and body temperature data, which are then recorded as detailed information about your pet's movements and sounds.

[1660] Step 2:

[1661] Terminal: Collected data is temporarily stored through batch processing and sent to the server at regular intervals (e.g., every 5 minutes). The data is encrypted during transmission to ensure secure transmission.

[1662] Step 3:

[1663] Server: Decompresses and decrypts the data received from the terminal and temporarily stores it in a database. It then checks the data for consistency and eliminates missing or abnormal data.

[1664] Step 4:

[1665] Server: Inputs data into the AI ​​model and infers the pet's emotions. For example, if a large tail wag and a high-pitched bark are detected at the same time, it will determine that the pet is happy.

[1666] Step 5:

[1667] Server: Stores the estimated emotions in a database and prepares to notify the user in real time via a smartphone app.

[1668] Step 6:

[1669] Device: The user's smartphone app receives the notification sent from the server and displays "Your pet is happy." The user can check this information and communicate with their pet.

[1670] Step 7:

[1671] Server: Regularly monitors the pet's health and detects abnormalities in body temperature or behavior. For example, if the body temperature is higher than normal, it determines that the pet may have a fever.

[1672] Step 8:

[1673] Server: If an abnormality in health is detected, it automatically sends a notification to the veterinary clinic, which includes detailed health data about the pet.

[1674] Step 9:

[1675] Server: At the same time, an alert is sent to the user via the smartphone app, allowing the user to quickly check if there is anything abnormal with their pet's health.

[1676] Step 10:

[1677] Device: The user registers their pet's favorite music through a smartphone app. For example, they can enter information such as "I like classical music."

[1678] Step 11:

[1679] Server: Based on the user's music preferences, the AI ​​model generates relaxing music that is customized to suit each pet's individual preferences.

[1680] Step 12:

[1681] Server: The generated music data is sent to the device, which then plays it to the pet through the camera's built-in speaker.

[1682] Step 13:

[1683] Device: Plays the received music data to provide a relaxing environment for your pet. For example, playing calming piano music at night can help your pet relax.

[1684] Example 1

[1685] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1686] Conventional technology has limited methods for accurately understanding a pet's emotions and health condition and communicating this to the user. Furthermore, existing systems lack the functionality to generate and play relaxation music tailored to the pet's preferences. This makes it difficult to manage a pet's emotions and health condition, and for users to understand their pet's condition in real time. Therefore, a system is needed that can analyze a pet's emotions and health condition in real time and quickly notify a veterinary medical facility if there are any abnormalities.

[1687] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1688] In this invention, the server includes means for acquiring data on the pet's movements, voice, and body temperature in real time, means for analyzing the acquired data and inferring the pet's emotions, means for notifying the user of the inferred emotions, means for monitoring the pet's health condition and notifying a veterinary medical institution if an abnormality is detected, means for the user to register the type of music preferred by the pet, means for the generative AI model to generate music based on the registered music preferences, and means for playing the generated music. This makes it possible to grasp the pet's emotions and health condition in real time and provide relaxing music.

[1689] "Movement data" refers to data relating to the pet's physical movements and behavior.

[1690] "Audio data" refers to data related to the cries and sounds made by pets.

[1691] "Body temperature data" refers to data relating to the pet's body temperature.

[1692] "Real-time" refers to data acquisition and analysis occurring almost immediately.

[1693] "Means of acquisition" refers to methods of collecting data using devices such as cameras and sensors.

[1694] "Means of analysis" refers to the way in which acquired data is processed, analyzed, and converted into meaningful information.

[1695] "Means for inferring emotions" refers to a method for determining and inferring a pet's current emotions from the analyzed data.

[1696] "Means of notification" refers to the method of notifying users and related organizations of analysis results and abnormalities.

[1697] "Health monitoring measures" are methods for continuously observing a pet's temperature and behavior to assess its health.

[1698] "Means for detecting abnormalities" are methods for detecting abnormalities in health status when they occur.

[1699] "Means of notifying a veterinary medical institution" refers to a method of notifying a specialized institution such as a veterinary hospital of a detected abnormality.

[1700] The "means for generating preferred music" is a method for creating relaxation music that matches a pet's preferences using a generative AI model.

[1701] "Means for playing" refers to a method for playing the generated music through a device such as a speaker.

[1702] A "generative AI model" is a model that uses artificial intelligence to analyze data and generate music.

[1703] A "prompt sentence" is an input sentence that conveys specific instructions or requests to a generative AI model.

[1704] The present invention is a system that collects pet movement, voice, and body temperature data in real time, analyzes this data to infer the pet's emotions, and notifies the user. It also has the function of monitoring the pet's health and notifying a veterinary clinic if an abnormality is detected. It also has the function of generating and playing music tailored to the pet's preferences.

[1705] Program processing

[1706] Data collection

[1707] Device: First, the user prepares a camera and a sensor. Specifically, the camera (e.g., Logitech C920) records the pet's movements in real time, and the sensor (e.g., DHT22) captures the pet's voice and body temperature. These data are collected by a device placed near the pet.

[1708] Sending data

[1709] Device: The collected data is sent to a server over the internet, packaged in a standard data format such as JSON, and periodically sent to the server via a Wi-Fi connection.

[1710] Emotion and health analysis

[1711] Server: The received data is input into an AI model (e.g., a neural network model using TensorFlow) running on the server. The server analyzes the video data using computer vision technology (e.g., YOLO or OpenPose) to infer the pet's emotions from its movement patterns. The audio data is also analyzed using a voice recognition system (e.g., Google Speech-to-Text API). The body temperature data is compared with pre-set reference values ​​to assess the pet's health.

[1712] Notification of results

[1713] Server: The analysis results are sent to the user's smartphone app. Notifications are sent using a notification service such as Firebase Cloud Messaging. For example, if the analysis result indicates that "the dog is excited," the user's smartphone will be notified.

[1714] Health Monitoring and Notification

[1715] Server: Monitors the pet's health 24 hours a day, and if the pet's body temperature is abnormally high (e.g., above 39°C), an alert is automatically sent to a veterinary clinic. At the same time, the user is notified of the health alert via their smartphone app.

[1716] Relaxation music generation and playback

[1717] User: The user registers their pet's favorite music through a smartphone app. For example, they can specify "classical music."

[1718] Server: Based on the registered information, the server generates music using a generative AI model (e.g., OpenAI's GPT-3). An example prompt is "Generate a calming piano piece that will help my dog ​​relax at night."

[1719] Device: The generated music data is sent to the device and played through the camera's built-in speaker, providing a relaxing environment for your pet.

[1720] Specific examples

[1721] For example, we explain how a family dog ​​uses this system from morning until night.

[1722] Data collection: Cameras capture the dog's movements, and sensors capture barks and body temperature data.

[1723] Data transmission: Collected data is sent to the server every 5 seconds.

[1724] Data analysis: The data received by the server is input into the AI ​​model, which analyzes it as "the dog is excited."

[1725] Result notification: The analysis result "The dog is excited" is notified to the user's smartphone.

[1726] Health monitoring: If the body temperature exceeds 39°C, it is considered a "fever" and an animal medical facility is automatically notified.

[1727] Relaxation Music: At night, gentle piano music generated by the server is played through the camera's built-in speaker to help your dog relax.

[1728] In this way, the system of the present invention can grasp the emotions and health condition of a pet in real time and quickly take appropriate measures.

[1729] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1730] Step 1: Collect data

[1731] Device: A user-installed camera (e.g., Logitech C920) and sensor (e.g., DHT22) collect pet behavior, sound, and temperature data in real time. The camera captures video at 60 fps and acquires behavior data. The sensor periodically reads and stores temperature and sound data.

[1732] Input: Pet movement, voice, and body temperature data

[1733] Output: A package of captured motion, voice, and temperature data

[1734] Specific behavior:

[1735] The camera records the dog's walking and eating, while sensors measure its barking and temperature.

[1736] Step 2: Sending data

[1737] Device: The collected data is converted into JSON format and sent to the server via Wi-Fi connection. The data is set to be sent every 5 seconds.

[1738] Input: Data collected in step 1

[1739] Output: Movement, voice, and temperature data sent to the server

[1740] Specific behavior:

[1741] The device automatically packages the data collected by the cameras and sensors and sends it to a server via the internet.

[1742] Step 3: Emotion and health analysis

[1743] Server: The received data is input into an AI model for analysis. For example, movement data is analyzed using a computer vision model, voice data is analyzed using a voice recognition model, and body temperature data is compared with pre-defined criteria.

[1744] Input: Data sent in step 2

[1745] Output: Analysis results (e.g. dog is excited, body temperature is normal)

[1746] Specific behavior:

[1747] The server analyzes the video data and determines that the dog is wagging its tail. It analyzes the audio data and determines that the dog is barking. It also checks the body temperature and records it as 36.5°C.

[1748] Step 4: Notification of results

[1749] Server: Notifies the user of the analysis results via their smartphone app. Uses services such as Firebase Cloud Messaging to communicate information to the user in real time.

[1750] Input: Analysis results obtained in step 3

[1751] Output: Notification sent to the user

[1752] Specific behavior:

[1753] The server formats the message "The dog is excited" in JSON format and notifies the user using the Firebase Cloud Messaging API.

[1754] Step 5: Health monitoring and notification

[1755] Server: Continuously monitors body temperature and behavior data, and if an abnormality is detected, contacts a veterinary medical facility. If the standard value is exceeded, an alert is automatically issued.

[1756] Input: Body temperature data and movement data obtained in step 3

[1757] Output: Abnormal alert notification

[1758] Specific behavior:

[1759] The server monitors body temperature data 24 hours a day, and if the temperature exceeds 39°C, it sends a "fever" alert to the veterinary clinic and notifies the user at the same time.

[1760] Step 6: Generate and play relaxation music

[1761] User: Register your pet's favorite music through the smartphone app. You can specify genres such as "classical music."

[1762] Server: Generates music using a generative AI model based on the registered information. Enter a prompt such as "Generate classical music" into the AI ​​model.

[1763] Device: The generated music data is played through the camera's built-in speaker.

[1764] Input: Music type (user input), music generation prompt

[1765] Output: Generated music data, music to be played

[1766] Specific behavior:

[1767] The user selects "classical music" and sends it to the server. The server then inputs the prompt "Please generate a gentle piano piece that will help my dog ​​relax at night" into the AI ​​model and sends the generated music to the device. The device then plays the music, providing a relaxing effect to the dog.

[1768] (Application example 1)

[1769] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1770] The present invention aims to accurately grasp a pet's emotions and health status by collecting data on its movements, voice, and body temperature in real time and analyzing this information. Furthermore, there is a need for early detection of abnormalities in a pet's health and providing a means of communication for prompt response, thereby improving pet health management and pet owner peace of mind. Another important issue is providing a means for generating and playing relaxation music to relieve stress in pets.

[1771] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1772] In this invention, the server includes a means for analyzing the pet's monitoring data and notifying the smartphone app of the results, a means for automatically contacting a veterinary clinic if the pet's health is abnormal, and a means for playing relaxation music if the pet is suspected to be stressed. This allows the pet's emotions and health condition to be accurately grasped in real time, enabling early detection of health abnormalities and prompt response. It also provides an environment that relieves pet stress, improving the owner's sense of security.

[1773] "Pet's movement, voice, and body temperature data" is a general term for behavioral information indicating the pet's movements, sound data reflecting vocal expressions, and temperature data for measuring body temperature.

[1774] "Real-time acquisition means" means the technological means for obtaining current information immediately without delay.

[1775] A "means for analyzing and inferring pet emotions" is a system or algorithm designed to analyze the collected data and infer the pet's emotional state.

[1776] "Means for notifying the user" refers to communication means or interfaces for immediately conveying analysis results and important information to the user.

[1777] The "means of notifying veterinary medical institutions when an abnormality is detected" is a system that automatically contacts pre-registered veterinary medical institutions when an abnormality is detected in a pet's health condition.

[1778] "Means for generating and playing music that suits pets' preferences" refers to devices or programs that use AI or algorithms to create music that suits pets' preferences and play it back.

[1779] "Means for analyzing pet monitoring data and notifying the results to a smartphone app" refers to methods or technologies for analyzing acquired monitoring data and notifying the user of the analysis results via a smartphone app.

[1780] "Means for automatically contacting a veterinary medical institution when an abnormality in the health of a pet" refers to a system or mechanism for automatically contacting a designated veterinary medical institution when an abnormality in the health of a pet is discovered.

[1781] The "means for playing relaxation music when it is determined that a pet is feeling stressed" is a mechanism for playing music to relax a pet when it is determined that the pet is in a stressful state.

[1782] To implement the present invention, a smart delivery robot specialized for food delivery services is used. The specific configuration and operation of the system are described below.

[1783] System Configuration

[1784] 1. Device Configuration

[1785] Camera: Used to record your pet's movements in real time.

[1786] Microphone: Used to capture pet sounds.

[1787] Temperature sensor: Used to measure your pet's body temperature.

[1788] Built-in speaker: Used to play the generated relaxation music.

[1789] 2. Server Configuration

[1790] AI model: Used to analyze the acquired data and run a generative AI model to infer the pet's emotions and health status, specifically using machine learning frameworks such as TensorFlow and PyTorch.

[1791] 3. User Interface

[1792] Smartphone app: Used to notify users of analysis results and alerts when anomalies are detected. Notification APIs such as Firebase Cloud Messaging are used.

[1793] How it works

[1794] Data collection

[1795] The device records your pet's movements in real time with a camera, picks up its cries with a microphone, and measures its body temperature with a temperature sensor, all of which are immediately sent to a server.

[1796] Data analysis

[1797] The server inputs the received movement data, voice data, and body temperature data into a generative AI model to analyze the pet's emotions and health condition. For example, by inputting the movement data and voice data into the AI ​​model, it can infer that the pet is "excited," and from the body temperature data, it can determine that the pet is "in good health."

[1798] Result notification

[1799] The server notifies the user of the analysis results via a smartphone app, which displays the results as messages such as "your pet is excited" or "your pet is in good health."

[1800] Health monitoring and abnormality notification

[1801] The server monitors the health of your pet and automatically sends a notification to a veterinary clinic if an abnormality is detected. For example, if your pet's temperature is abnormally high, it will immediately determine that there is a possibility of a fever and contact the veterinary clinic. An alert will also be sent to the user.

[1802] Relaxation music generation and playback

[1803] The device uses a generative AI model to generate relaxing music and plays it through the built-in speaker to provide a relaxing environment for pets, especially at night, if the device detects that your pet is feeling stressed.

[1804] Specific examples

[1805] For example, consider the case where a food delivery robot arrives at a user's home and encounters a pet. At this time, a camera records the pet's movements in real time, a microphone picks up its meows, and a temperature sensor measures its body temperature. The data is immediately sent to a server and analyzed by a generative AI model. The resulting analysis results, such as "the pet is excited" or "its health is good," are notified to the user's smartphone. If the pet appears stressed, relaxing music is played.

[1806] Prompt Sentence Examples

[1807] Example prompt sentence:

[1808] Take your pet's motion and voice data as input and infer its emotions. Then, analyze its temperature data to determine its health status.

[1809] As a result, the system of the present invention can accurately grasp the emotions and health status of pets in real time, enabling early detection of health abnormalities and prompt response, and providing an environment that reduces stress for pets.

[1810] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1811] Step 1:

[1812] Data collection:

[1813] The device records the pet's movements in real time with a camera, picks up its cries with a microphone, and measures its body temperature with a temperature sensor. These data are collected as movement video data, audio data, and body temperature data.

[1814] Input: Pet movement, sound, temperature

[1815] Output: Motion video data, audio data, body temperature data

[1816] Step 2:

[1817] Data transmission:

[1818] The device sends the collected data to a server via the internet, and each data is protected by a secure communication protocol to prevent data loss or tampering.

[1819] Input: motion video data, audio data, body temperature data

[1820] Output: Each data sent to the server

[1821] Step 3:

[1822] Data Analysis:

[1823] The server inputs the received motion video data, audio data, and body temperature data into a generative AI model to analyze the pet's emotions and health condition. The AI ​​model infers the pet's emotions from its motion and audio patterns and evaluates its health condition based on the body temperature data.

[1824] Input: motion video data, audio data, body temperature data

[1825] Output: Emotion prediction result, health status evaluation result

[1826] Step 4:

[1827] Notification of results:

[1828] The server then notifies the user of the analysis results via a smartphone app. The notification content, which is related to the pet's emotional state and health status, is displayed in a user-friendly format.

[1829] Input: Emotion prediction results, health status assessment results

[1830] Output: A notification message to the user (e.g. "Your pet is excited" or "It is in good health")

[1831] Step 5:

[1832] Automatic notification of health abnormalities:

[1833] The server operates a system that automatically contacts animal medical institutions if an abnormality is detected in the pet's health condition. An alert is also sent to the user, enabling a prompt response.

[1834] Input: Health status assessment result (if abnormality detected)

[1835] Output: Notification message to veterinary clinic, alert to user

[1836] Step 6:

[1837] Relaxation music generation and playback:

[1838] The device uses a generative AI model to generate relaxing music for your pet, which is then played through the built-in speaker, providing a relaxing environment for your pet.

[1839] Input: Emotion prediction result (if stress is detected)

[1840] Output: Generated relaxation music data, music playback

[1841] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1842] This invention is a system that acquires pet behavior, voice, and body temperature data in real time, analyzes this information, predicts the pet's emotions, and notifies the user. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it takes into account the emotional states of both the pet and the user, enabling more appropriate communication and health management.

[1843] System Configuration

[1844] The system includes the following main components:

[1845] 1. A device that collects pet movement, voice, and body temperature data

[1846] Camera: Records your pet's movements as video and captures movement data in real time.

[1847] Sensor: Detects your pet's cries and body temperature and collects data.

[1848] 2. Server that analyzes the data

[1849] AI model: A mathematical model that analyzes the acquired data and predicts the pet's emotions and health condition.

[1850] 3. Means of notifying users

[1851] Smartphone app: An application that notifies users of analysis results and alerts when abnormalities are detected.

[1852] 4. Means of notifying the veterinary medical facility

[1853] Automatic notification system: A system that contacts the veterinary clinic when an abnormality is detected in the pet's health.

[1854] 5. Means of generating and playing music

[1855] AI model: Generates music tailored to your pet's preferences.

[1856] Built-in speaker: Ability to play generated music.

[1857] 6. Emotion engine that recognizes user emotions

[1858] Camera and microphone: Analyzes the user's facial expressions, tone of voice, and gestures to infer emotions.

[1859] Program processing

[1860] Data collection and transmission

[1861] Device: A camera installed by the user records the pet's movements in real time, and sensors capture its sounds and body temperature data, which are periodically sent to a server via the Internet.

[1862] Emotion and health analysis

[1863] Server: The received data is input into an AI model, which analyzes the pet's movements and voice patterns to infer its emotions. It also monitors its health from its body temperature and movement patterns, and any abnormalities detected are recorded in a database.

[1864] Server: Additionally, the server uses the user's camera and microphone to analyze the user's facial expressions, tone of voice, and gestures to infer emotions.

[1865] Notification of results

[1866] Server: Notifies the user of the analysis results via a smartphone app. For example, if it determines that "your pet is happy," that information is displayed on the user's smartphone. The notification content is also adjusted based on the user's emotions. For example, if the user is feeling stressed, advice such as "Try ways to help your pet relax" is provided.

[1867] Health Monitoring and Notification

[1868] Server: Regularly monitors the pet's health and detects abnormalities in body temperature or behavior. For example, if the body temperature is higher than normal, it determines that the pet may have a fever.

[1869] Server: If an abnormality in health is detected, it automatically sends a notification to the veterinary clinic, which includes detailed health data about the pet.

[1870] Server: At the same time, an alert is sent to the user via the smartphone app, allowing the user to quickly check if there is anything abnormal with their pet's health.

[1871] Relaxation music generation and playback

[1872] Device: The user registers their pet's favorite music through a smartphone app. For example, they can enter information such as "I like classical music."

[1873] Server: Based on the user's music preferences, the AI ​​model generates relaxing music that is customized to suit each pet's individual preferences.

[1874] Device: Plays the received music data to provide a relaxing environment for your pet. For example, playing calming piano music at night can help your pet relax.

[1875] Specific examples

[1876] For example, we will explain how a dog and its owner in a household might use this system over the course of a day.

[1877] 1. Data Collection:

[1878] Device: The camera records the dog's movements and facial expressions, and sensors detect its barks and body temperature. The device captures the dog's wagging tail and lively barks.

[1879] 2. User sentiment analysis:

[1880] Server: Using the user's camera and microphone, the server analyzes the user's facial expressions and tone of voice to determine whether the user is experiencing stress.

[1881] 3. Data Analysis:

[1882] Server: The AI ​​model analyzes the received data and infers that the dog is excited. It also determines that the dog is in good health based on the body temperature data.

[1883] 4. Result notification:

[1884] Server: The analysis results are sent to the user's smartphone app. A message is displayed stating that the dog is excited, and, taking into consideration that the user is feeling stressed, advice is provided to encourage relaxation.

[1885] 5. Health Monitoring and Notification:

[1886] Server: Regularly checks the pet's health and, if the body temperature is abnormally high, determines that the pet may have a fever and notifies the veterinary clinic. An alert is also sent to the user.

[1887] 6. Generate and play relaxation music:

[1888] Device: At night, the camera's built-in speaker plays soothing piano music to help your dog relax.

[1889] In this way, by using the system of the present invention, the emotions and health conditions of pets and users can be accurately understood, enabling appropriate management and care.

[1890] The processing flow will be explained below.

[1891] Step 1:

[1892] Device: The camera records your pet's movements in real time, while the sensors capture vocalization and temperature data. The camera captures video of your pet's movements and facial expressions, while the sensors record your pet's body temperature and vocalization frequency.

[1893] Step 2:

[1894] The terminal periodically processes the acquired data in batches and transmits them over the internet in encrypted form to a server. This transmission process occurs at regular intervals (e.g., every 5 minutes).

[1895] Step 3:

[1896] Server: The server decompresses and decrypts the data received from the device and temporarily stores it in a database. Here, it performs a data integrity check and eliminates any invalid or missing data.

[1897] Step 4:

[1898] Server: Inputs data into the AI ​​model, analyzes the pet's movements and vocal patterns, and infers its emotions. For example, if a vigorous tail wagging and a high-pitched bark are detected at the same time, it will determine that the pet is happy.

[1899] Step 5:

[1900] Server: Stores the estimated emotions in a database and prepares to notify the user. Notifications are set up to be sent in real time via a smartphone app.

[1901] Step 6:

[1902] Device: The user's smartphone app receives the notification sent from the server and displays the message "Your pet is happy." The user can check this information and communicate with their pet.

[1903] Step 7:

[1904] On the device: The device uses the user's camera and microphone to record the user's facial expressions, tone of voice, and gestures, which then stores the user's emotional data on the device.

[1905] Step 8:

[1906] Terminal: The user's emotional data is periodically sent to the server in encrypted format, similar to the data sent to pets.

[1907] Step 9:

[1908] Server: The received user emotion data is input into the AI ​​model to infer the user's emotions. For example, if the user's facial expression is grim and their voice tone is low, it is determined that the user is feeling stressed.

[1909] Step 10:

[1910] Server: Adjusts the content of notifications sent to pets based on the user's emotional data. For example, if the pet is excited and the user is feeling stressed, the server will provide advice such as "Try to help your pet relax."

[1911] Step 11:

[1912] Server: Regularly monitors the pet's health and detects abnormalities. For example, if the body temperature is higher than normal, it determines that the pet may have a fever.

[1913] Step 12:

[1914] Server: If an abnormality in health status is detected, a notification is automatically sent to a veterinary clinic. The notification contains detailed health data of the pet to aid in diagnosis.

[1915] Step 13:

[1916] Server: At the same time, an alert is sent to the user via the smartphone app, allowing them to quickly check if there is anything abnormal with their pet's health.

[1917] Step 14:

[1918] Device: The user registers their pet's favorite music through a smartphone app. For example, they can enter a preference such as "I like classical music."

[1919] Step 15:

[1920] Server: Based on the user's music preferences, the AI ​​model generates relaxing music that is customized to suit each pet's individual preferences.

[1921] Step 16:

[1922] Server: The generated music data is sent to the device, which then plays it through the device's built-in speaker.

[1923] Step 17:

[1924] Device: Plays the received music data to provide a relaxing environment for your pet. For example, playing calming piano music at night can help your pet relax.

[1925] This series of steps allows accurate monitoring of the emotions and health of pets and users, and allows for appropriate management and countermeasures.

[1926] Example 2

[1927] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1928] Modern pet ownership requires predicting a pet's emotions and health status and notifying the user at the appropriate time. However, current technology makes it difficult to accurately predict a pet's emotions, and responses when abnormalities occur in the pet's health are often delayed. Furthermore, there are no notification systems that take the user's emotions into account. This makes it difficult to provide appropriate care and communication.

[1929] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1930] In this invention, the server includes means for acquiring pet movement, voice, and body temperature data in real time, means for analyzing the acquired data to infer the pet's emotions and health condition, means for notifying the user of the inferred emotions and health condition, means for acquiring the user's facial expressions, tone of voice, and gestures to infer emotions, means for notifying the veterinary clinic and the user if an abnormality in the pet's health condition is detected, and means for generating and playing music preferred by the pet. This makes it possible to accurately grasp the pet's emotions and health condition and quickly notify the user and the veterinary clinic. It also enables appropriate notifications that take the user's emotions into account and the generation and playback of music tailored to the pet's preferences, thereby achieving better communication between the pet and the user and appropriate health management.

[1931] "Movement data" is information that is recorded and collected in real time about your pet's body movements and behavior.

[1932] "Audio data" refers to acoustic information collected using sensors that record the cries and sounds made by pets.

[1933] "Body temperature data" refers to temperature information collected by measuring a pet's body temperature with a sensor.

[1934] "Acquisition means" refers to a device or equipment for acquiring pet movement data, voice data, and body temperature data in real time.

[1935] "Analysis methods" refers to the processes and algorithms used to use AI models to infer a pet's emotions and health status based on the acquired data.

[1936] "Notification means" refers to a device or application for communicating analysis results to users or animal medical facilities.

[1937] "Facial expression" refers to data that refers to facial movements and expressions captured using a camera to assess the user's emotions.

[1938] "Voice tone" is voice information used to infer emotions by analyzing the pitch and tone of the user's voice.

[1939] "Gesture" is movement information that is used to analyze the user's hand movements and body movements and to infer emotions.

[1940] "AI model" refers to a mathematical and computational model that uses artificial intelligence to analyze data and infer emotions and health states.

[1941] "Veterinary medical institution" refers to a medical facility or veterinary hospital that should be contacted if any abnormalities are discovered in the health of a pet.

[1942] "Music Generation Method" refers to the algorithm or process used to create music based on the pet's preferences.

[1943] "Playback means" refers to a device or system for actually playing back the generated music.

[1944] MODE FOR CARRYING OUT THE INVENTION

[1945] The present invention is a system that acquires pet movement, voice, and body temperature data in real time, analyzes this information to estimate the pet's emotions and health condition, and notifies the user. It also includes an emotion engine for recognizing the user's emotions, enabling appropriate communication and health management that takes into account the emotional states of both the pet and the user. Specific embodiments are described below.

[1946] System Configuration

[1947] The system includes the following main components:

[1948] 1. A device that collects pet movement, voice, and body temperature data

[1949] Camera: Records your pet's movements as video and captures movement data in real time.

[1950] Example: Logitech C920

[1951] Sensor: Detects your pet's cries and body temperature and collects data.

[1952] Example: DHT11

[1953] 2. Server that analyzes the data

[1954] AI model: A mathematical model that analyzes the acquired data and predicts the pet's emotions and health condition.

[1955] Examples: TensorFlow, Keras

[1956] Database: Stores analysis results and collected data.

[1957] Example: MySQL

[1958] 3. Means of notifying users

[1959] Smartphone app: An application that notifies users of analysis results and alerts when abnormalities are detected.

[1960] Example: React Native, Flutter

[1961] 4. Means of notifying the veterinary medical facility

[1962] Automatic notification system: A system that contacts the veterinary clinic when an abnormality is detected in the pet's health.

[1963] Examples: Twilio, Firebase Cloud Messaging

[1964] 5. Means of generating and playing music

[1965] AI model: Generates music tailored to your pet's preferences.

[1966] Example: Magenta

[1967] Built-in speaker: Ability to play generated music.

[1968] Example: JBL

[1969] 6. Emotion engine that recognizes user emotions

[1970] Camera and microphone: Analyzes the user's facial expressions, tone of voice, and gestures to infer emotions.

[1971] Examples: OpenFace, Praat

[1972] Program processing

[1973] Data collection

[1974] Device: A camera installed by the user records the pet's movements in real time, and sensors capture its sounds and body temperature data, which are periodically sent to a server via the Internet.

[1975] Data analysis

[1976] Server: The received data is input into an AI model, which analyzes the pet's movements and voice patterns to infer its emotions. It also monitors its health from its body temperature and movement patterns, and any abnormalities detected are recorded in a database.

[1977] Emotion analysis

[1978] Server: Uses the user's camera and microphone to analyze the user's facial expressions, tone of voice, and gestures to infer emotions.

[1979] Result notification

[1980] Server: Notifies the user of the analysis results via a smartphone app. For example, if it determines that "your pet is happy," that information is displayed on the user's smartphone. The notification content is also adjusted based on the user's emotions. For example, if the user is feeling stressed, advice such as "Try ways to help your pet relax" is provided.

[1981] Health Monitoring and Notification

[1982] Server: Regularly monitors the pet's health and detects abnormalities in body temperature or behavior. For example, if the body temperature is higher than normal, it determines that the pet may have a fever.

[1983] Server: If an abnormality in health is detected, it automatically sends a notification to the veterinary clinic, which includes detailed health data about the pet.

[1984] Server: At the same time, an alert is sent to the user via the smartphone app, allowing the user to quickly check if there is anything abnormal with their pet's health.

[1985] Relaxation music generation and playback

[1986] Device: The user registers their pet's favorite music through a smartphone app. For example, they can enter information such as "I like classical music."

[1987] Server: Based on the user's music preferences, the AI ​​model generates relaxing music that is customized to suit each pet's individual preferences.

[1988] Device: Plays the received music data to provide a relaxing environment for your pet. For example, playing calming piano music at night can help your pet relax.

[1989] Specific examples

[1990] Data collection

[1991] Device: The camera records the dog's movements and facial expressions, and sensors detect its barks and body temperature. The device captures the dog's wagging tail and lively barks.

[1992] Emotion analysis

[1993] Server: Using the user's camera and microphone, the server analyzes the user's facial expressions and tone of voice to determine whether the user is experiencing stress.

[1994] Data analysis

[1995] Server: The AI ​​model analyzes the received data and infers that the dog is excited. It also determines that the dog is in good health based on the body temperature data.

[1996] Result notification

[1997] Server: The analysis results are sent to the user's smartphone app. A message is displayed stating that the dog is excited, and, taking into consideration that the user is feeling stressed, advice is provided to encourage relaxation.

[1998] Health Monitoring and Notification

[1999] Server: Regularly checks the pet's health and, if the body temperature is abnormally high, determines that the pet may have a fever and notifies the veterinary clinic. An alert is also sent to the user.

[2000] Relaxation music generation and playback

[2001] Device: At night, the camera's built-in speaker plays soothing piano music to help your dog relax.

[2002] Prompt Sentence Examples

[2003] Based on the following data, please generate a program for a system that will infer a pet's emotions and notify the user. Please also clearly describe the system's overview and specific processing steps.

[2004] data:

[2005] Pet movement video data

[2006] Pet sounds and body temperature data

[2007] The user's facial expressions, tone of voice, and gestures

[2008] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2009] Step 1: Collect data

[2010] Device: Captures pet movement, voice, and temperature data in real time through cameras and sensors.

[2011] Input: Pet movement video, bark, body temperature

[2012] Output: Collected motion video data, audio data, and body temperature data

[2013] Specific operation: The camera records the pet's movements, the microphone records the dog's barks, and the temperature sensor measures the dog's temperature. For example, the camera records the dog jumping, the microphone picks up the sound of the dog barking, and the temperature sensor measures the dog's temperature at 37.8°C.

[2014] Step 2: Sending data

[2015] Terminal: Sends collected data to a server via the Internet.

[2016] Input: Collected motion video data, audio data, and body temperature data

[2017] Output: Motion video data, audio data, and body temperature data sent to the server

[2018] Specific operation: Data collected by the camera and sensors is uploaded to a cloud server via Wi-Fi. For example, motion video files, audio files, and body temperature data are sent to the cloud server.

[2019] Step 3: Data analysis

[2020] Server: Inputs the received data into an AI model, analyzes the pet's movements and voice patterns to infer emotions, and monitors its health from its body temperature and movement patterns.

[2021] Input: Motion video data, audio data, and body temperature data received by the server

[2022] Output: Pet emotion prediction results, health status analysis results

[2023] Specific operation: The AI ​​model processes the received video data and, for example, analyzes it as "the dog is wagging its tail" and determines that "the dog is happy." It also determines that "the dog is in good health" based on body temperature data.

[2024] Step 4: Sentiment Analysis

[2025] Server: Uses the user's camera and microphone to analyze the user's facial expressions, tone of voice, and gestures to infer emotions.

[2026] Input: Facial expression data, tone of voice data, and gesture data captured from the user's camera and microphone

[2027] Output: User emotion estimation result

[2028] Specific operation: The system processes data from when the user speaks using facial expressions and voice, and infers that the user is relaxed. For example, it analyzes facial expressions as "smiling" and voice tone as "calm."

[2029] Step 5: Notification of results

[2030] Server: Notifies the user of the analysis results via a smartphone app.

[2031] Input: Pet emotion prediction results, health status analysis results, user emotion prediction results

[2032] Output: Message sent to the user's smartphone app

[2033] Specific operation: Based on the analysis results, a push notification is sent to the user's smartphone saying "your dog is happy." If the user feels stressed, the system provides advice such as "try ways to help your pet relax."

[2034] Step 6: Health monitoring and notification

[2035] Server: If an abnormality is detected in the pet's health condition, the server automatically notifies the veterinary clinic and the user.

[2036] Input: Pet health analysis results

[2037] Output: Alert notifications sent to veterinary clinics and users

[2038] Specific operation: If the temperature data is abnormally high, it is determined that there is a possibility of a fever. An email notification is sent to the veterinary clinic, and a push notification is sent to the user alerting them that their pet has a high temperature.

[2039] Step 7: Generate and play relaxation music

[2040] Device: Based on the user's selection, the AI ​​model generates relaxation music and plays it through the device's speakers.

[2041] Input: User's music preference data

[2042] Output: Generated relaxation music, music to be played

[2043] How it works: The user tells the app that they like classical music. The AI ​​model generates calming piano music and plays it through the device's built-in speaker at night, providing a relaxing environment for pets.

[2044] (Application example 2)

[2045] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2046] Conventional pet monitoring and health management systems provide real-time analysis of pet behavior, voice, and body temperature data, but they are unable to link the emotional state of the pet with the user based on this information, or provide appropriate feedback to the user based on the pet's estimated emotions. This has resulted in a lack of improvement in the quality of pet health management and emotional care. Furthermore, they lack the ability to immediately notify medical institutions when abnormalities are detected in the pet's health.

[2047] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2048] In this invention, the server includes a means for acquiring data on the pet's movements, voice, and body temperature in real time, a means for analyzing the acquired data to infer the pet's emotions, and a means for simultaneously analyzing the emotional states of the user and the pet and providing appropriate feedback, thereby enabling more appropriate communication and health management that takes into account the emotional states of the pet and the user.

[2049] "Behavioral data" refers to information about a pet's movements and behavior, and is primarily data obtained visually using cameras and sensors.

[2050] "Audio data" refers to information about pets' cries and vocalizations, and is data obtained using audio sensors and microphones.

[2051] "Body temperature data" refers to information about a pet's body temperature, and is data obtained using a temperature sensor.

[2052] "Real-time capture" means that data is collected and processed at the present time and in the moment, making it available almost immediately.

[2053] "Analysis" refers to the process of analyzing acquired data using mathematical models and algorithms to extract meaning and trends.

[2054] "Emotion inference" refers to estimating the emotional state of a pet from the analyzed data, including various emotional states such as joy, sadness, excitement, and anxiety.

[2055] "Notifying" refers to the act of informing the user of the analysis results or anomaly detection information, which is usually done via a smartphone or other notification means.

[2056] "Health monitoring" refers to the process of continually checking your pet's health information to ensure there are no irregularities or abnormalities.

[2057] "Notifying a medical institution" refers to the act of communicating information to an appropriate medical provider when an abnormality in a pet's health is detected.

[2058] "Generating preferred music" refers to the process of automatically creating music that will relax your pet using an AI model.

[2059] "Playing" refers to the act of making the generated music audible through speakers or other audio devices.

[2060] "Simultaneous analysis of the emotional states of the user and the pet" refers to the process of collecting the emotional states of both the pet and the user in real time and simultaneously analyzing them based on that data.

[2061] "Providing appropriate feedback" refers to the act of suggesting and notifying users of optimal actions and information based on the analysis results.

[2062] This invention relates to a system that acquires pet behavior, voice, and body temperature data in real time, analyzes this information to infer the pet's emotions, and notifies the user. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it takes into account the emotional states of both the pet and the user, enabling more appropriate communication and health management.

[2063] System Configuration

[2064] The system mainly consists of the following components:

[2065] 1. Device that collects movement, voice, and body temperature data

[2066] Camera: Records your pet's movements as video and captures movement data in real time.

[2067] Sensor: Detects your pet's cries and body temperature and collects data.

[2068] 2. Server that analyzes the data

[2069] AI model: A mathematical model that analyzes the acquired data and predicts the pet's emotions and health condition.

[2070] 3. Means of notifying users

[2071] Smartphone app: An application that notifies users of analysis results and alerts when abnormalities are detected.

[2072] 4. Means of notifying medical institutions

[2073] Automatic notification system: A system that contacts the veterinary clinic when an abnormality is detected in the pet's health.

[2074] 5. Means of generating and playing music

[2075] AI model: Generates music tailored to your pet's preferences.

[2076] Built-in speaker: Ability to play generated music.

[2077] 6. Emotion engine that recognizes user emotions

[2078] Camera and microphone: Analyzes the user's facial expressions, tone of voice, and gestures to infer emotions.

[2079] Program processing

[2080] Data collection and transmission

[2081] The device uses a camera installed by the user to record pet movements in real time, and sensors to capture sounds and body temperature data, which are then periodically sent to a server via the internet.

[2082] Emotion and health analysis

[2083] The server inputs the received data into an AI model and analyzes the pet's movements and voice patterns to infer emotions. It also monitors the pet's health status based on its body temperature and movement patterns, and any abnormalities detected are recorded in a database. Furthermore, the server uses the user's camera and microphone to analyze the user's facial expressions, tone of voice, and gestures to infer emotions.

[2084] Notification of results

[2085] The server notifies the user of the analysis results via a smartphone app. For example, if it determines that "your pet is happy," that information is displayed on the user's smartphone. The notification content is also adjusted based on the user's emotions. For example, if the user is feeling stressed, the server provides advice such as "Try ways to help your pet relax."

[2086] Health Monitoring and Notification

[2087] The server regularly monitors the pet's health and detects any abnormalities in body temperature or behavior. For example, if the body temperature is higher than normal, it determines that the pet may have a fever. If an abnormality in health is detected, a notification is automatically sent to a veterinary clinic. This notification includes detailed health data about the pet. At the same time, an alert is sent to the user via a smartphone app, allowing the user to quickly check if there is anything abnormal with their pet's health.

[2088] Relaxation music generation and playback

[2089] The user registers their pet's preferred music through a smartphone app on the device. For example, they may enter information such as "I like classical music." The server uses an AI model to generate relaxing music based on the music preference information obtained from the user. The generated music is customized to suit each pet's individual preferences. The received music data is played back to provide a relaxing environment for the pet. For example, playing calming piano music at night can promote relaxation for the pet.

[2090] Specific examples

[2091] As a concrete example, let's consider the case of a dog left at a pet hotel. A camera records the dog's movements and facial expressions, and sensors detect its barks and body temperature data. The dog's tail wagging and energetic barks are captured. The server receives this data and uses an AI model to infer that the dog is excited, and based on the body temperature data, determines that the dog is in good health. These analysis results are notified to the user's smartphone app, which displays a message saying that the dog is excited. If the user's emotions indicate stress, the system also provides advice to encourage relaxation.

[2092] Prompt Sentence Examples

[2093] For example, give the generative AI model the following prompt:

[2094] I'd like to create an application for a pet hotel that uses a system to capture and analyze pet behavior, voice, and temperature data in real time, and then predict and notify the pet's emotions and health status. I'm considering a system that notifies the user of detailed analysis results and sends alerts to hotel staff and pet owners in the event of an abnormality. Please provide an example implementation using Python and TensorFlow.

[2095] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2096] Step 1:

[2097] Data collection

[2098] The device collects movement data, audio data, and body temperature data in real time. Specifically, the camera records the pet's movements, the microphone records its cries, and the temperature sensor measures its body temperature. The input is the camera footage, audio files, and body temperature data, and the output is the raw data files.

[2099] Step 2:

[2100] Sending data

[2101] The data collected by the terminal is sent to the server via the Internet. The data is sent directly without any processing. The input is the raw data, and the output is the success status of data transmission to the server.

[2102] Step 3:

[2103] Preparing for analysis

[2104] The server preprocesses the received raw data. Specifically, it extracts the camera footage frame by frame, performs digital signal processing on the audio data, and digitizes the body temperature data. The input is raw data, and the output is preprocessed data.

[2105] Step 4:

[2106] emotion guessing

[2107] The server inputs the preprocessed data into an AI model to infer the pet's emotions. The AI ​​model uses an algorithm to identify emotions from movement patterns, vocal tones, and changes in body temperature. The input is the preprocessed data, and the output is an inferred emotional state. Specifically, it assigns labels such as "happy," "anxious," and "excited."

[2108] Step 5:

[2109] Health monitoring

[2110] The server uses an AI model to analyze the body temperature data and monitor the pet's health, particularly detecting abnormalities when the temperature is outside the normal range. The input is the body temperature data, and the output is the health status assessment result. Specific outputs include labels such as "healthy," "fever," and "low body temperature."

[2111] Step 6:

[2112] Notification of analysis results

[2113] The server notifies the user's smartphone app of the emotion prediction results and health status evaluation results. The input is the analysis results, and the output is a notification message displayed on the user's smartphone app. Specific notification content includes "Your pet is happy" or "Your pet has a fever."

[2114] Step 7:

[2115] Notification to medical institutions

[2116] If the server detects an abnormality, it automatically notifies the veterinary clinic. The input is the health status assessment result, and the output is a notification message sent to the veterinary clinic. Specific notification content might be something like, "Your pet has a fever. Immediate medical examination is required."

[2117] Step 8:

[2118] Music generation and playback

[2119] The device generates music that the pet likes and starts playing it. The server generates the music using an AI model and sends the data to the device. The input is the pet's emotional state and preferred music information, and the output is the generated music data. Specific music examples include "classical music" and "gentle piano pieces."

[2120] Step 9:

[2121] User sentiment analysis

[2122] The server analyzes the user's facial expressions, tone of voice, and gestures using a camera and microphone to infer the user's emotions. The input is the user's video and audio data, and the output is an inferred result of the user's emotional state. Specific inferred results include "stress," "relaxation," and "joy."

[2123] Step 10:

[2124] Providing feedback

[2125] Based on the analysis results, the server provides appropriate feedback to the user. The input is the estimated emotional state of the pet and the user, and the output is a feedback message to the user. Specific feedback content might be, "Spend more time relaxing with your pet."

[2126] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2127] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2128] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414. ...

Claims

1. A means of capturing pet movement, voice, and temperature data in real time; A method for analyzing the acquired data and inferring the pet's emotions, means for notifying a user of the inferred emotion; A means to monitor the health of pets and notify veterinary clinics if any abnormalities are detected; means for generating and playing music preferred by the pet; A system including:

2. 2. The system of claim 1, wherein the means for acquiring pet movement, sound, and temperature data includes a camera and a sensor.

3. 10. The system of claim 1, wherein the means for monitoring the health of the pet performs analysis using an AI model.

Images