System

Abstract

To provide a system for promoting communication between a pet and an owner.SOLUTION: A system includes: means for receiving voice data; means for collecting characteristics of a pet; means for identifying an animal type; means for collecting characteristics data on general animals; means for generating a prompt sentence for instructing generation of information on at least one of a state and a request of the pet on the basis of the voice data, the characteristics of the pet, the identified animal type, and characteristics data; means for generating information about at least one of the state and the request of the pet on the basis of the generated prompt sentence and a generative AI model; and means for generating a prompt sentence for instructing to translate an instruction to the pet into a form understandable to the pet on the basis of a feeling state of a user identified using a feeling engine, and means for translating and outputting the instruction spoken to the pet into the form understandable to the pet using the generated prompt sentence and the generative AI model.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] Traditionally, pet owners have tried to understand their pets' condition from their cries, but the cries alone are limited and it is difficult to accurately understand their pets' condition. In addition, communication between humans and pets is one-way, making it difficult to understand the pets' intentions. [Means for solving the problem]

[0005] The present invention collects voice data (animal cries), characteristics of pets (personality, cries, etc.), types of animals, and general animal characteristic data, and based on this data, understands the condition of pets from their cries. Furthermore, it utilizes AI to realize conversations between people and pets, and understands the health and emotional state of pets. This allows for a more accurate understanding of the pet's condition and improves communication between people and pets. [Brief description of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Diagram 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. [Diagram 3] FIG. 11 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Diagram 5] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a process flow of the data processing system in the first embodiment of the first form example. [Figure 12] FIG. 11 is a sequence diagram showing a process flow of the data processing system in the application example 1 of the first embodiment. [Figure 13] FIG. 11 is a sequence diagram showing a process flow of the data processing system in the second embodiment of the second embodiment. [Figure 14] FIG. 13 is a sequence diagram showing a process flow of the data processing system in application example 2 of embodiment 2. [Figure 15]FIG. 13 is a sequence diagram showing a process flow of a data processing system in a third embodiment of the present invention. [Figure 16] FIG. 13 is a sequence diagram showing a process flow of the data processing system in the third application example of the third embodiment. [Figure 17] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when combined with an emotion engine. [Figure 19] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system according to the embodiment 2 of the second embodiment in the case where an emotion engine is combined. [Figure 20] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2 when combined with an emotion engine. [Figure 21] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in the embodiment 3 of the form example 3 in the case where an emotion engine is combined. [Figure 22] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3 when combined with an emotion engine. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by the processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (e.g., voice and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs voice according to instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of main functions of the data processing device 12 and the smart device 14.

[0023] As shown in Fig. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32. The specific process program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific process program 56 from the storage 32, and executes the read specific process program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific process program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores a reception output program 60. The reception output program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads out the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] In one embodiment of the present invention, the cries of a pet are collected using a sound data collection device such as a microphone. The collected sound data is sent to a sound analysis device and identified as the cry of a specific animal. In addition, the characteristics of the pet animal (personality, cries, etc.) are collected using input from the owner or a sensor that observes the behavior of the pet. The type of animal is identified using input from the owner or image recognition technology. General animal characteristic data is obtained from a database or the like. This data is analyzed by AI to understand the condition of the pet.

[0029] "Example 2"

[0030] As another embodiment of the present invention, a system is provided that utilizes AI to realize conversation between humans and pets. Specifically, AI analyzes the cries of pets and converts them into words that humans can understand. Conversely, AI analyzes human words and converts them into cries and actions that pets can understand. For example, if a pet expresses its "hungry" state with a cry, the AI ​​analyzes it and conveys the message "your pet is hungry" to the owner. Also, if the owner says "let's play," the AI ​​analyzes it and conveys the message in a form that the pet can understand.

[0031] "Example 3"

[0032] As a further embodiment of the present invention, a system for understanding a pet's health and emotional state is provided. Specifically, an AI analyzes a pet's health and emotional state from its cries and behavior. For example, if a pet is suffering from an illness, its cries and behavior may be different from normal. The AI ​​detects these changes and issues a warning to the owner. Also, if a pet is happy, its cries and behavior may become more active. The AI ​​detects these changes and conveys the happiness to the owner.

[0033] The process flow of each embodiment will be described below.

[0034] "Example 1"

[0035] Step 1: Collect the pet's cries using an audio data collection device such as a microphone.

[0036] Step 2: The collected audio data is sent to an audio analyzer to identify it as the sound of a specific animal.

[0037] Step 3: Collect information about the pet's characteristics (personality, sounds, etc.) using input from the owner and sensors that observe the pet's behavior.

[0038] Step 4: Identify the type of animal using input from the owner and image recognition technology.

[0039] Step 5: Obtain general animal characteristic data from a database, etc.

[0040] Step 6: This data is analyzed using AI to understand the pet's condition.

[0041] "Example 2"

[0042] Step 1: AI analyzes your pet's cries and converts them into words that humans can understand. Step 2: AI analyzes human words and converts them into cries and actions that your pet can understand.

[0043] Step 3: For example, if a pet expresses its hunger state through a cry, the AI ​​will analyze it and convey the message "your pet is hungry" to the owner.

[0044] Step 4: Also, if the owner says "let's play," the AI ​​will analyze it and convey the message in a way that the pet can understand.

[0045] "Example 3"

[0046] Step 1: AI analyzes your pet's health and emotional state based on its sounds and behavior.

[0047] Step 2: For example, if a pet is suffering from an illness, its sounds and behavior may be different from normal. AI will detect these changes and alert the owner.

[0048] Step 3: Also, if a pet is happy, its vocalizations and behavior may become more active. AI will detect these changes and communicate its happiness to the owner.

[0049] Example 1

[0050] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal".

[0051] Conventional pet condition monitoring systems mainly collect and analyze voice data and animal characteristic data separately, making it difficult to integrate this data and understand the pet's condition comprehensively. In addition, there was a lack of means to accurately grasp the health and emotional state of pets, making it difficult for owners to appropriately understand and respond to their pet's condition.

[0052] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0053] In this invention, the server includes a means for collecting voice data, a means for transmitting the collected voice data, a means for analyzing the voice data, a means for collecting characteristics of the pet animal, a means for identifying the type of animal, a means for collecting general animal characteristic data, and a means for grasping the condition of the pet from its cry based on the collected data. This makes it possible to comprehensively grasp the condition of the pet by integrating the voice data and the animal characteristic data.

[0054] "Audio data" refers to audio signals, such as animal cries, recorded in digital format.

[0055] "Collecting means" refers to a device or method for acquiring audio data and animal characteristic data.

[0056] A "transmitting means" is a device or method for transferring collected data to another device or server.

[0057] An "analyzing means" is a device or method for processing collected data and extracting specific information.

[0058] "Animal characteristics" refers to individual attributes of pet animals, such as their personalities or the sounds they make.

[0059] An "identification means" is a device or method for determining the type or characteristics of an animal.

[0060] "General animal characteristic data" refers to data such as characteristics and behavioral patterns common to a specific animal species.

[0061] "Pet's condition" refers to the pet's overall condition, including its health and emotional state.

[0062] "AI" is a technology that uses artificial intelligence to analyze data and make specific judgments and predictions.

[0063] "Means for enabling conversation" refers to devices or methods that allow people and pets to communicate.

[0064] "Health Status" refers to information regarding the physical health of a pet.

[0065] "Emotional state" refers to a pet's psychological state or mood.

[0066] The present invention is a system for collecting sounds made by pets and analyzing the data to grasp the state of the pets. A specific embodiment of this system will be described below.

[0067] First, a user uses the smartphone's built-in microphone or a dedicated external microphone to collect the sound of their pet. For example, the user starts a recording app on the smartphone and records the sound of their pet. This sound data is sent to the server by the device. Specifically, the device uses an HTTP POST request to send the sound data to the server's API endpoint.

[0068] The server analyzes the received voice data using a voice analyzer (e.g., Google® Cloud Speech-to-Text API). The server sends the voice data to the API, analyzes the returned text data, and identifies it as the sound of a specific animal.

[0069] Next, the user inputs the characteristics of the pet (personality, cries, etc.) into the app. For example, the user inputs information such as "personality: active, cries: high-pitched" into the app's form. The app also uses sensors connected to the device (e.g., acceleration sensor and camera) to observe the pet's behavior and collect data. The device periodically transmits the data from the sensors to the server.

[0070] To identify the type of animal, the user inputs the type of pet into the app, or the device uses image recognition technology (e.g., TENSORFLOW (registered trademark) or OpenCV) to identify the type of animal. For example, the user inputs "dog" or the device sends an image taken with the camera to the server, and the server uses image recognition technology to identify it as a "dog."

[0071] The server retrieves general animal characteristics data from a database (e.g., AWS® RDS or Google BigQuery). The server executes SQL queries to retrieve the required data from the database.

[0072] Finally, the server uses AI (for example, TensorFlow or PyTorch) to analyze the collected voice data, pet characteristic data, animal type data, and general animal characteristic data. The server integrates this data and inputs it into an AI model to understand the pet's condition. For example, the server analyzes data such as "the dog is barking loudly and is behaving actively" and determines that "the dog is excited."

[0073] Examples and prompts

[0074] As a concrete example, the barking sound of a user's dog is collected by the microphone on the smartphone and sent to the server. The server uses the Google Cloud Speech-to-Text API to analyze the sound and identify it as a dog's bark. The user inputs the dog's personality and characteristics of its bark into the app, and observes the dog's behavior with a camera connected to the device. The server obtains general dog characteristic data from AWS RDS, analyzes the data using TensorFlow, and understands the dog's condition.

[0075] An example of a prompt sentence is as follows:

[0076] "Please analyze my dog's barks and understand his condition. My dog ​​is very active and barks a lot. Please analyze the following audio data."

[0077] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0078] Step 1:

[0079] A user collects the sounds of their pet.

[0080] Specifically, the user launches a recording app on their smartphone and records their pet's cries.

[0081] Input: Pet sounds

[0082] Output: Audio data file (e.g. .wav format)

[0083] Step 2:

[0084] The terminal transmits the collected voice data to a server.

[0085] Specifically, the device uses an HTTP POST request to send audio data to the server's API endpoint.

[0086] Input: Audio data file

[0087] Output: Audio data sent to the server

[0088] Step 3:

[0089] The server analyzes the received audio data.

[0090] Specifically, the server uses a voice analysis device (e.g., Google Cloud Speech-to-Text API) to convert the voice data into text data and identify it as the sound of a specific animal.

[0091] Input: Audio data sent to the server

[0092] Output: Parsed text data (e.g. "dog barking")

[0093] Step 4:

[0094] Collect characteristics of the animals kept by the user.

[0095] Specifically, the user enters information such as "personality: active, bark: high-pitched" into the app's form. In addition, the app uses sensors connected to the device (e.g., acceleration sensor and camera) to observe the pet's behavior and collect data.

[0096] Input: User-entered animal feature data, behavioral data from sensors

[0097] Output: Collected animal trait data

[0098] Step 5:

[0099] Identify types of animals.

[0100] Specifically, the user inputs the type of animal they own into the app, or the device uses image recognition technology (e.g. TensorFlow or OpenCV) to identify the type of animal.

[0101] Input: Animal species data or image data entered by the user

[0102] Output: Identified animal type data (e.g. "dog")

[0103] Step 6:

[0104] A server obtains general animal characteristic data.

[0105] Specifically, the server executes an SQL query from a database (e.g. AWS RDS or Google BigQuery) to retrieve the required data.

[0106] Input: Animal species data

[0107] Output: General animal traits data

[0108] Step 7:

[0109] The server analyzes the collected data and determines the pet's condition.

[0110] Specifically, the server uses AI (e.g. TensorFlow or PyTorch) to analyze collected voice data, animal feature data, animal species data, and general animal feature data. The server integrates this data and inputs it into an AI model to determine the pet's condition.

[0111] Input: Audio data, animal feature data, animal species data, general animal feature data

[0112] Output: Analysis results (e.g. "The dog is excited")

[0113] (Application example 1)

[0114] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0115] There is a problem in that it is difficult for pet owners to grasp abnormal behavior or health conditions of their pets in real time while they are out or not watching over them. In addition, there is a lack of means to detect abnormalities in pets early and respond quickly, making it difficult to ensure the health and safety of pets.

[0116] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0117] In this invention, the server includes a means for collecting voice data (animal cries), a means for collecting characteristics of pet animals (personality, cries, etc.), a means for identifying the type of animal, a means for collecting general animal characteristic data, a means for grasping the pet's condition from the cries based on the collected data, and a means for sending a notification when an abnormality is detected. This makes it possible to grasp the pet's abnormal behavior and health condition in real time, and to promptly notify the owner when an abnormality is detected.

[0118] "Audio data" refers to information collected in digital format from audio signals such as animal cries.

[0119] "Collecting means" refers to a device or method for acquiring audio data and animal characteristic data.

[0120] "Animal characteristics" refers to information that refers to the unique characteristics of an animal, such as its personality or the sound it makes.

[0121] An "identification means" is a device or method for determining the type or characteristics of an animal.

[0122] "General animal characteristic data" refers to information such as personality and behavioral patterns common to a specific animal species.

[0123] "Means of understanding" refers to devices and methods used to analyze collected data and understand the condition of animals.

[0124] "Means for detecting abnormalities" refers to devices or methods for identifying abnormalities in the behavior or sounds of animals and detecting the occurrence of an abnormality.

[0125] A "means for sending a notification" is a device or method for conveying that information to an owner or other person when an abnormality is detected.

[0126] The system for carrying out the present invention collects and analyzes the cries of pets to grasp the state of the pets, and sends a notification to the pet's owner if an abnormality is detected. A specific embodiment of the system will be described below.

[0127] Hardware Configuration

[0128] The system includes the following hardware:

[0129] Microphone: Used to collect pet sounds.

[0130] Smartphone or smart glasses: Used to analyze collected voice data and detect anomalies.

[0131] Server: Analyzes voice data and sends notifications.

[0132] Software Configuration

[0133] The system includes the following software:

[0134] TensorFlow: Runs AI models to analyze audio data.

[0135] sounddevice: Used to collect audio data.

[0136] smtplib: Used to send notifications if an anomaly is detected.

[0137] Data processing and calculation

[0138] 1. Audio data collection: The device (smartphone or smart glasses) uses a microphone to collect the pet's cries. The collected audio data is stored in digital format.

[0139] 2. Audio data analysis: The server inputs the collected audio data into the TensorFlow model and performs analysis. Based on the analysis results, the pet's condition is understood.

[0140] 3. Anomaly detection: The server determines whether an anomaly has been detected based on the analysis results. If an anomaly is detected, a notification is sent to the owner.

[0141] 4. Send notification: The server uses smtplib to send a notification to the pet owner, containing information about the pet's abnormal behavior and health status.

[0142] Examples

[0143] For example, if a pet makes an abnormal sound while the owner is out, the system will collect and analyze the sound. If an abnormality is detected based on the analysis results, a notification will be sent to the owner's smartphone, allowing the owner to take prompt action.

[0144] Examples of prompt statements

[0145] "Collect the sounds of pets' cries and analyze them using an AI model. If an abnormality is detected, create a program to notify the pet owner."

[0146] In this way, the present invention makes it possible to grasp a pet's abnormal behavior and health condition in real time and quickly notify the owner if an abnormality is detected.

[0147] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0148] Step 1:

[0149] The terminal (smartphone or smart glasses) collects the pet's cries using a microphone. The input is the pet's cries, and the output is digital audio data. Specifically, the terminal uses software for collecting audio data (sounddevice library) to record audio for a certain period of time and save it as digital data.

[0150] Step 2:

[0151] The device transmits the collected voice data to the server. The input is the digital voice data, and the output is the voice data transmitted to the server. Specifically, the device uses an Internet connection to upload the voice data to the server.

[0152] Step 3:

[0153] The server inputs the received voice data into the TensorFlow model for analysis. The input is digital voice data, and the output is the analysis result (the state of the pet). Specifically, the server uses the TensorFlow library to input the voice data into the AI ​​model and analyze the characteristics of the pet's cries.

[0154] Step 4:

[0155] The server determines whether an anomaly has been detected based on the analysis results. The input is the analysis results, and the output is information about the presence or absence of an anomaly. Specifically, the server compares the analysis results with a threshold, and sets a flag if an anomaly is detected.

[0156] Step 5:

[0157] If an abnormality is detected, the server will send a notification to the pet owner. The input is information about the presence or absence of an abnormality, and the output is a notification to the pet owner. Specifically, the server uses the smtplib library to send an abnormality notification to the pet owner's email address. The notification contains detailed information about the pet's abnormal behavior and health condition.

[0158] Step 6:

[0159] The user (owner) checks the received notification and takes action to check the pet's status if necessary. The input is the notification from the server, and the output is the user's response. Specifically, the user checks the notification on their smartphone and takes action such as returning home to check the pet's status or contacting the pet sitter.

[0160] Example 2

[0161] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0162] Conventional means of communication between pets and humans are limited, making it difficult to accurately grasp a pet's intentions and state from its cries and actions. In addition, there was a lack of means to communicate human words to pets in a form that pets could understand. This made it difficult to properly manage a pet's health and emotional state, and communication between pets and owners was insufficient.

[0163] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for collecting voice data, a means for collecting animal characteristics, a means for identifying the type of animal, a means for collecting general animal characteristic data, a means for grasping the state of the pet from the cry based on the collected data, a means for converting voice data into text data using voice recognition software, a means for analyzing text data using a generative AI model to understand the intention of the pet, a means for generating a message based on the analysis result and notifying the user, a means for collecting the user's words and converting them into text data using voice recognition software, a means for analyzing text data using a generative AI model to generate a message to be conveyed to the pet, and a means for converting and conveying a message into a cry or behavior that the pet can understand using voice synthesis software. This makes it possible to accurately grasp the intention and state of the pet from its cry or behavior, and to convey human words in a form that the pet can understand.

[0164] "Audio data" refers to data that has been recorded in digital format from sounds made by animals or humans.

[0165] "Animal characteristics" refers to characteristics unique to each individual animal, such as their personality, sounds, and behavioral patterns.

[0166] "Type of animal" refers to the classification of animals, such as dog, cat, bird, etc.

[0167] "General animal characteristic data" refers to data such as personality and behavioral patterns common to a particular animal species.

[0168] "Voice recognition software" is software for analyzing voice data and converting it into text data.

[0169] A "generative AI model" is a model that uses artificial intelligence to analyze text data and understand its meaning.

[0170] "Text data" is textual information converted from voice data by voice recognition software.

[0171] "Message" refers to the content of notifications or instructions generated based on the analysis results.

[0172] "Speech synthesis software" refers to software for converting text data into speech.

[0173] "User" refers to a person who uses this system.

[0174] "Pet" refers to animals kept as pets in the home.

[0175] This invention is a system that uses AI to realize conversations between people and pets. This system has the functions of analyzing pet cries and converting them into words that humans can understand, and analyzing human words and converting them into cries and actions that pets can understand.

[0176] Hardware and Software Used

[0177] Hardware

[0178] Microphone: Used to capture pet sounds and user speech.

[0179] Speaker: Used to transmit generated sounds to your pet.

[0180] Camera: Used to monitor pet behavior if necessary.

[0181] Server: Used to analyze and process data.

[0182] software

[0183] Speech recognition software: Used to convert voice data into text data (e.g. Google Speech-to-Text).

[0184] Generative AI models: Analyze text data and use it to understand your pet's intent (e.g., GPT-4®).

[0185] Speech synthesis software: Used to convert text data into audio (e.g. Google Text-to-Speech).

[0186] Specific operation of the system

[0187] Pet sound analysis

[0188] The microphone on the device captures the sound of your pet barking. For example, if your dog barks, the audio data is collected through the microphone. The collected audio data is then sent to the server in real time.

[0189] The server analyzes the captured voice data using voice recognition software and converts it into text data, for example converting the voice "woof woof" into the text "woof woof." The converted text data is sent to the next analysis step.

[0190] The server uses a generative AI model to analyze the text data and understand the pet's intention. For example, the text "woof woof" is analyzed to mean "I'm hungry." The analysis result is sent to the message generation step.

[0191] The server generates a message that says "your pet is hungry" based on the analysis results and notifies the user. For example, the notification is sent to the user via a smartphone app. The user can check the notification on the app and understand the status of the pet.

[0192] Human Language Analysis

[0193] The microphone on the device captures the user's words. For example, when the user says "Let's play," the voice data is collected through the microphone. The collected voice data is sent to the server in real time.

[0194] The server analyzes the captured voice data using voice recognition software and converts it into text data. For example, the voice "Let's play" is converted into the text "Let's play." The converted text data is sent to the next analysis step.

[0195] The server uses a generative AI model to analyze the text data and generate a message to be sent to the pet. For example, the text "Let's play" is analyzed to mean "It's play time." The analysis result is sent to the message generation step.

[0196] The server uses speech synthesis software to convert the generated message into sounds and actions that the pet can understand, and communicates it to the pet through the speaker. For example, the message "It's playtime" is converted into sounds and actions that the dog can understand, and played back through the speaker. The pet understands the sounds and actions, and gets ready to play with the user.

[0197] Examples of concrete examples and prompts

[0198] Examples

[0199] Pet Sound Analysis:

[0200] If a pet barks "woof woof," a microphone will capture the sound and the server will notify the user that "your pet is hungry."

[0201] Human Language Analysis:

[0202] When a user says "let's play," a microphone captures the audio and the server communicates through the speaker with sounds and actions that the pet understands to mean "it's play time."

[0203] Examples of prompt statements

[0204] Pet Sound Analysis:

[0205] If a pet barks "woof woof", explain the process of analyzing that sound and informing the user that their pet is hungry.

[0206] Human Language Analysis:

[0207] When the user says "let's play," describe the process that analyzes that speech and communicates to your pet "it's play time" in a way that they can understand.

[0208] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0209] Step 1: Capture your pet's sounds

[0210] The microphone on the device captures the sound of the pet barking. For example, if a dog barks "woof woof," the audio data is collected through the microphone. The input is the pet's bark, and the output is audio data. The collected audio data is sent to the server in real time.

[0211] Step 2: Analyzing bird calls and converting them into text

[0212] The server analyzes the captured voice data using voice recognition software and converts it into text data. For example, the voice "woof woof" is converted into the text "woof woof." The input is voice data and the output is text data. The converted text data is sent to the next analysis step.

[0213] Step 3: Semantic analysis of the text

[0214] The server uses a generative AI model to analyze the text data and understand the pet's intention. For example, the text "woof woof" is analyzed to mean "I'm hungry." The input is the text data, and the output is the analysis result that indicates the pet's intention. The analysis result is sent to the message generation step.

[0215] Step 4: Message generation and notification

[0216] The server generates a message saying "your pet is hungry" based on the analysis results and notifies the user. For example, the notification is sent to the user via a smartphone app. The input is the analysis results, and the output is a notification message to the user. The user can check the notification on the app to understand the status of their pet.

[0217] Step 5: Capturing human language

[0218] The microphone on the device captures the user's words. For example, when a user says "Let's play," the voice data is collected through the microphone. The input is the user's words, and the output is voice data. The collected voice data is sent to the server in real time.

[0219] Step 6: Word analysis and text conversion

[0220] The server analyzes the captured voice data using voice recognition software and converts it into text data. For example, the voice "Let's play" is converted into the text "Let's play". The input is voice data and the output is text data. The converted text data is sent to the next analysis step.

[0221] Step 7: Semantic analysis of the text

[0222] The server uses a generative AI model to analyze the text data and generate a message to be conveyed to the pet. For example, the text "Let's play" is analyzed to mean "It's play time." The input is the text data, and the output is the message to be conveyed to the pet. The analysis result is sent to the message generation step.

[0223] Step 8: Create a message and send it to your pet

[0224] The server uses speech synthesis software to convert the generated message into sounds and actions that the pet can understand, and communicates them to the pet through the speaker. For example, the message "It's playtime" is converted into sounds and actions that the dog can understand, and played back from the speaker. The input is the message to be communicated to the pet, and the output is sounds and actions that the pet can understand. The pet understands the sounds and actions, and gets ready to play with the user.

[0225] (Application example 2)

[0226] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0227] Conventional pet care systems have problems in that it is difficult to accurately understand a pet's cries and behavior and take appropriate action. They also lack a means to notify the owner that the pet is hungry and to quickly order pet food. Furthermore, they lack an effective means to facilitate communication between the owner and the pet.

[0228] The identification process by the identification processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means. In this invention, the server includes a means for collecting voice data (animal cries), a means for collecting characteristics of pet animals (personality, cries, etc.), a means for identifying the type of animal, a means for collecting general animal characteristic data, a means for grasping the pet's condition from the cries based on the collected data, a means for notifying the owner based on the pet's condition, and a means for ordering pet food. This makes it possible to accurately grasp the pet's condition and respond quickly. In addition, it is possible to facilitate communication between the owner and the pet and automate the ordering of pet food.

[0229] "Audio data" refers to sound information, including animal cries.

[0230] "Characteristics of pets" refers to individual characteristics such as a pet's personality or the sounds it makes.

[0231] "Means for identifying the type of animal" refers to a method or device for identifying the type of pet.

[0232] "General animal characteristic data" refers to information about the widely known characteristics of animals.

[0233] "Means for understanding a pet's condition from its cries" refers to a method or device for analyzing a pet's cries to understand its condition.

[0234] "Means for notifying an owner based on the condition of the pet" refers to a method or device for notifying an owner of the condition of the pet.

[0235] "Pet food ordering means" refers to a method or device for purchasing pet food online.

[0236] "Means of using AI to enable conversation between people and pets" refers to methods and devices that use artificial intelligence to exchange information between humans and pets.

[0237] "Means for understanding a pet's health and emotional state" refers to a method or device for understanding a pet's physical condition and emotions.

[0238] The following system configuration will be described as an embodiment of the present invention.

[0239] System Configuration

[0240] This system includes means for collecting voice data (animal cries), means for collecting characteristics of pet animals (personality, cries, etc.), means for identifying the type of animal, means for collecting general data on animal characteristics, means for determining the condition of pets from their cries based on the collected data, means for notifying owners based on the pet's condition, and means for ordering pet food.

[0241] Hardware and Software Used

[0242] Hardware: Smartphone, microphone

[0243] Software: Python, SpeechRecognition library, Requests library

[0244] Data processing and calculation

[0245] Acquisition and analysis of audio data

[0246] The server records the pet's cries using the smartphone's microphone. The recorded voice data is converted to text data using the SpeechRecognition library. It is then analyzed using a generative AI model to understand the pet's condition.

[0247] Collecting and analyzing owner comments

[0248] The server records the owner's words using the smartphone's microphone. The recorded voice data is converted to text data using the SpeechRecognition library. It is then analyzed using a generative AI model to generate a message to be conveyed to the pet.

[0249] Pet food order

[0250] The server notifies the owner based on the pet's condition. For example, if the pet cries "I'm hungry," the server generates a prompt to order pet food and notifies the owner. When the owner orders pet food through the app, recommended products are displayed taking into account the pet's preferences and allergy information.

[0251] Examples

[0252] Analyzing pet cries: If a pet cries "I'm hungry," the server analyzes the cry and notifies the owner with a message that "your pet is hungry."

[0253] Analyzing owner's words: When the owner says "let's play", the server analyzes the words and conveys the message in a way that the pet can understand.

[0254] Examples of prompt statements

[0255] "Analyze your pet's cries and let us know their condition."

[0256] "Analyze the owner's words and generate a message to convey to the pet."

[0257] In this way, the present invention makes it possible to accurately grasp the condition of a pet and respond quickly, facilitate communication between the owner and the pet, and automate the ordering of pet food.

[0258] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0259] Step 1:

[0260] The server records the pet's cry using the smartphone's microphone. The input is the pet's cry, and the output is an audio data file. This audio data file is used for subsequent analysis processing.

[0261] Step 2:

[0262] The server converts the recorded voice data into text data using the SpeechRecognition library. The input is the voice data file, and the output is text data. This text data is input to the generative AI model.

[0263] Step 3:

[0264] The server analyzes the text data using a generative AI model to understand the pet's state. The input is text data, and the output is information that indicates the pet's state. For example, the state "hungry" is analyzed.

[0265] Step 4:

[0266] The server notifies the owner based on the pet's status. The input is information indicating the pet's status, and the output is a notification message to the owner. For example, a message "your pet is hungry" is sent to the owner.

[0267] Step 5:

[0268] The server records the owner's words using the smartphone's microphone. The input is the owner's words, and the output is an audio data file. This audio data file is used for subsequent analysis processing.

[0269] Step 6:

[0270] The server converts the recorded voice data of the owner into text data using the SpeechRecognition library. The input is a voice data file, and the output is text data. This text data is input to the generative AI model.

[0271] Step 7:

[0272] The server uses a generative AI model to analyze the text data and generate a message to be conveyed to the pet. The input is text data, and the output is a message to be conveyed to the pet. For example, the command "Let's play" is analyzed.

[0273] Step 8:

[0274] The server provides a means for ordering pet food based on the pet's condition. The input is information indicating the pet's condition, and the output is pet food ordering information. For example, if the pet cries "I'm hungry," an order for pet food is automatically placed.

[0275] Step 9:

[0276] The server displays recommended products to the pet owner, taking into account the pet's preferences and allergy information. The input is the pet's preferences and allergy information, and the output is a list of recommended products. This allows the pet owner to select an appropriate pet food.

[0277] Step 10:

[0278] The server notifies the pet owner that the pet food order is complete. The input is the order completion information, and the output is a notification message to the pet owner. For example, a message saying "Your pet food order is complete" is sent to the pet owner.

[0279] Example 3

[0280] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0281] Conventional systems for understanding the health and emotional state of pets only analyze voice data and do not take into account behavioral data, making it difficult to accurately understand the condition. In addition, there is a lack of a way to quickly notify the owner of the analysis results, making it difficult to discover abnormalities in pets at an early stage.

[0282] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0283] In this invention, the server includes a means for collecting voice data, a means for collecting animal characteristics, a means for identifying the type of animal, a means for collecting general animal characteristic data, a means for grasping the state of the pet from the cries based on the collected data, a means for collecting animal behavior data, a means for analyzing the collected behavior data, and a means for notifying the analysis result. By analyzing both the voice data and the behavior data, it becomes possible to grasp the health and emotional state of the pet more accurately and promptly notify the owner.

[0284] "Audio Data" refers to animal sounds and other audio information.

[0285] "Animal characteristics" refers to individual characteristics such as an animal's personality, sounds, behavior patterns, etc.

[0286] "Type of animal" refers to a classification of animals, such as dogs, cats, birds, etc.

[0287] "General animal characteristic data" refers to data such as personality and behavioral patterns common to a particular type of animal.

[0288] "Animal condition" refers to the animal's physical and emotional state.

[0289] "Behavioral Data" refers to information regarding an animal's movements and behavior patterns.

[0290] "Analysis results" refers to the diagnosis and evaluation output by the AI ​​model based on the collected voice data and behavioral data.

[0291] "Notification means" refers to a method or device for communicating the analysis results to the owner.

[0292] The present invention relates to a system for understanding the health and emotional state of a pet. A specific embodiment of the system will be described below.

[0293] First, the user uses a device such as a smartphone or tablet to collect the sounds and behavior of their pet. The device is equipped with a microphone and camera, and these devices are used to capture voice data and behavior data in real time. For example, when a user starts the smartphone app, points it at their pet and presses the "start recording" button, the microphone starts collecting the pet's sounds. At the same time, the camera captures the pet's movements.

[0294] The device then transmits the collected data to a server, where it is sent in real time over Wi-Fi or mobile data, and the data is encrypted to ensure security.

[0295] The server inputs the received voice and behavior data into an AI model, which is built using machine learning frameworks such as TensorFlow and PyTorch, to analyze the frequency and volume of the pet's cries, as well as the speed and direction of its movements, to detect abnormal patterns and patterns of delight.

[0296] The analysis results are sent from the server to the device, and the user can check the results through a smartphone app. For example, the server will send a warning message such as "Your pet may be sick." If the pet is happy, it will send a message such as "Your pet is happy."

[0297] As a concrete example, the following prompt sentence could be input to a generative AI model:

[0298] "Analyze your pet's vocalizations and behavioral data to determine their health and emotional state. Alert you if your pet is sick and let you know if they're happy."

[0299] Using this prompt sentence, the AI ​​model can accurately analyze the pet's condition and provide appropriate feedback to the user.

[0300] As described above, the present invention makes it possible to grasp the health and emotional state of a pet more accurately and promptly notify the owner by analyzing both the voice data and the behavior data. The flow of the identification process in the third embodiment will be described with reference to FIG.

[0301] Step 1:

[0302] The user starts the smartphone app and collects the sounds and behavior of the pet. Specifically, when the user presses the "Start Recording" button, the device's microphone starts collecting the pet's sounds. At the same time, the camera captures the pet's movements. The input is the pet's sounds and behavior, and the output is the collected voice data and behavior data.

[0303] Step 2:

[0304] The device transmits the collected voice data and behavioral data to the server. Specifically, the device transmits data in real time using Wi-Fi or mobile data communication. The input is the collected voice data and behavioral data, and the output is the data transmitted to the server.

[0305] Step 3:

[0306] The server inputs the received voice data and behavioral data into an AI model. Specifically, the server inputs data into an AI model built using machine learning frameworks such as TensorFlow and PyTorch. The input is the voice data and behavioral data sent to the server, and the output is the analysis result by the AI ​​model.

[0307] Step 4:

[0308] The server judges the pet's health and emotional state based on the analysis results of the AI ​​model. Specifically, the AI ​​model analyzes the frequency and volume of the pet's cries, the speed and direction of its movements, etc., to detect abnormal patterns and patterns of joy. The input is the analysis results of the AI ​​model, and the output is the judgment of the pet's health and emotional state.

[0309] Step 5:

[0310] The server transmits the analysis results to the terminal. Specifically, the server transmits a message generated based on the analysis results to the terminal. The input is the judgment result of the pet's health condition and emotional state, and the output is the message transmitted to the terminal.

[0311] Step 6:

[0312] The user checks the analysis results through the terminal. Specifically, the user opens the smartphone app and checks the notified message. For example, a warning message such as "Your pet may be sick" or a message such as "Your pet is happy" may be displayed. The input is the message sent to the terminal, and the output is the analysis result checked by the user.

[0313] (Application example 3)

[0314] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0315] It is important for pet owners to know their pet's health and emotional state, but conventional methods make it difficult to detect abnormalities in pets at an early stage. In addition, there are limited means to accurately know whether a pet is happy. This can lead to insufficient health management and understanding of the pet's emotions, which can lead to a decline in the quality of the pet's life. Furthermore, there is no means to quickly notify owners of abnormalities in their pets, which can delay emergency response.

[0316] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0317] In this invention, the server includes means for collecting voice data (animal cries), means for collecting characteristics of pet animals (personality, cries, etc.), means for identifying the type of animal, means for collecting general animal characteristic data, means for grasping the pet's condition from the cries based on the collected data, means for analyzing the pet's condition and sending a notification when an abnormality is detected, means for grasping the pet's health and emotional state in real time, means for issuing a warning to the owner when an abnormality is detected, and means for conveying information about the pet to the owner when the pet is happy. This makes it possible to quickly and accurately grasp the pet's health and emotional state, and to immediately notify the owner when an abnormality occurs.

[0318] "Audio data" refers to data that records sounds such as animal cries in digital format.

[0319] "Animal characteristics" refers to the animal's unique characteristics, such as personality or sounds.

[0320] "Means for identifying animal species" refers to methods or techniques for identifying the species of an animal.

[0321] "General animal characteristic data" refers to data about widely known characteristics of animals.

[0322] "Means of understanding a pet's condition" refers to methods of analyzing a pet's health and emotional state based on collected data.

[0323] "Means for analyzing a pet's condition" refers to a method for analyzing a pet's cries and behavior to assess its condition.

[0324] "Means for sending a notification when an abnormality is detected" refers to a method for issuing a warning to the pet owner when an abnormality is detected in the pet.

[0325] "Means of understanding a pet's health and emotional state in real time" refers to a method of instantly analyzing a pet's condition and providing the results in real time.

[0326] "Means for issuing a warning to the pet owner when an abnormality is detected" refers to a method for issuing a warning to the pet owner when an abnormality is detected in the pet.

[0327] The term "means for conveying information about a happy pet to its owner" refers to a method for detecting a happy state of a pet and conveying the information to its owner.

[0328] As an embodiment of the present invention, a pet security alert system will be described as an example. This system analyzes the cries and behavior of pets in real time and notifies the owner if an abnormality is detected.

[0329] System Configuration

[0330] Hardware:

[0331] Smartphone

[0332] microphone

[0333] software:

[0334] TensorFlow

[0335] librosa

[0336] smtplib

[0337] Data processing and calculation

[0338] Audio data collection:

[0339] The device (smartphone) uses a microphone to collect the sounds of pets, and the audio data is recorded in digital format.

[0340] Analysis of audio data:

[0341] The server converts the collected voice data into MFCC (Mel-Frequency Cepstral Coefficients) using librosa, which allows the extraction of features from the voice data.

[0342] Analysis by AI model:

[0343] The server feeds the voice data into a generative AI model pre-trained with TensorFlow to analyze the pet's health and emotional state, which is then used to detect abnormalities in the pet.

[0344] Sending notifications:

[0345] If an abnormality is detected, the server uses smtplib to send an email notification to the owner, containing detailed information about the pet's abnormal condition.

[0346] Examples

[0347] For example, you can record your pet's cries and input the audio data into the application. The AI ​​model analyzes the audio data, and if it detects an abnormality, it will notify the owner by email. The notification will include a message such as, "An abnormality has been detected with your pet. Please check immediately."

[0348] Examples of prompt statements

[0349] Examples of prompt sentences to be input to a generative AI model include the following:

[0350] Write a Python program that analyzes pet sound data and notifies owners if an abnormality is detected. Use TensorFlow and librosa to analyze the sound data, and smtplib to send email notifications.

[0351] In this way, a system can be provided that can grasp the health and emotional state of a pet in real time and quickly respond if an abnormality occurs.

[0352] The flow of the specific process in the application example 3 will be described with reference to FIG.

[0353] Step 1:

[0354] Audio data collection

[0355] The terminal (smartphone) collects the pet's cries using a microphone. The input is the pet's cries, and the output is digital audio data. This audio data is used in the subsequent analysis steps.

[0356] Step 2:

[0357] Audio data preprocessing

[0358] The server converts the collected voice data into MFCC (Mel-Frequency Cepstral Coefficients) using librosa. The input is digital voice data, and the output is MFCC features. Through this conversion, the features of the voice data are extracted.

[0359] Step 3:

[0360] Analysis by AI model

[0361] The server inputs the MFCC features into a generative AI model pre-trained using TensorFlow to analyze the pet's health and emotional state. The input is the MFCC features, and the output is a prediction result about the pet's condition. This analysis evaluates the pet's abnormalities and emotional state.

[0362] Step 4:

[0363] Anomaly detection

[0364] The server judges whether an abnormality has been detected in the pet based on the prediction result of the AI ​​model. The input is the prediction result of the AI ​​model, and the output is information on the presence or absence of an abnormality. If an abnormality is detected, it proceeds to the next step.

[0365] Step 5:

[0366] Sending notifications

[0367] The server uses smtplib to send notifications to the pet owner via email. The input is the pet owner's email address and information about the pet's abnormal condition. The output is the notification email sent, which contains detailed information about the pet's abnormal condition.

[0368] Step 6:

[0369] Real-time status

[0370] The server monitors the pet's health and emotional state in real time and provides the information to the owner as needed. The input is continuously collected voice data, and the output is real-time status information, allowing the owner to always be aware of the pet's condition.

[0371] In this way, it is possible to quickly and accurately grasp the pet's health and emotional state, and to immediately notify the owner if any abnormality occurs.

[0372] Furthermore, an emotion engine that estimates the emotion of the user may be combined. That is, the identification processing unit 290 may estimate the emotion of the user using the emotion identification model 59, and perform identification processing using the emotion of the user.

[0373] "Example 1"

[0374] The system according to claim 4 includes an emotion engine that recognizes the emotion of the user. The emotion engine analyzes the emotion of the user from the tone of voice, facial expression, behavior, etc. of the user. For example, when the user is angry with a pet, the emotion engine recognizes the anger and adjusts the instruction to the pet. When the user is happy, the emotion engine recognizes the joy and adjusts the instruction to the pet. This allows smoother communication between the pet and the user.

[0375] "Example 2"

[0376] The system described in claim 5 includes a means for utilizing AI to realize a conversation between a person and a pet, and an emotion engine for recognizing the user's emotions. This emotion engine analyzes the user's emotions from the user's tone of voice, facial expressions, behavior, etc., and feeds the results back to the AI. The AI ​​adjusts instructions to the pet based on this feedback. For example, when the user is angry at the pet, the AI ​​recognizes the anger and adjusts instructions to the pet. Also, when the user is happy, the AI ​​recognizes the joy and adjusts instructions to the pet. This makes communication between the pet and the user smoother.

[0377] "Example 3"

[0378] The system according to claim 6 includes a means for grasping the state of a pet and an emotion engine for recognizing the emotion of a user. The emotion engine analyzes the emotion of a user from the tone of voice, facial expression, behavior, etc. of the user, and feeds back the result to the means for grasping the state of a pet. The means for grasping the state of a pet adjusts the state of the pet based on this feedback. For example, when the user is angry at the pet, the means for grasping the state of a pet recognizes the anger and adjusts the behavior of the pet. Also, when the user is happy, the means for grasping the state of a pet recognizes the joy and adjusts the behavior of the pet. This allows smoother communication between the pet and the user.

[0379] The process flow of each embodiment will be described below.

[0380] "Example 1"

[0381] Step 1: The emotion engine analyzes the user's emotions based on their tone of voice, facial expressions, behavior, etc.

[0382] Step 2: Adjust instructions to your pet based on the emotion engine's analysis.

[0383] Step 3: The system operates to facilitate smooth communication between the pet and the user.

[0384] "Example 2"

[0385] Step 1: The emotion engine analyzes the user's emotions based on their tone of voice, facial expressions, behavior, etc.

[0386] Step 2: The emotion engine's analysis results are fed back to the AI.

[0387] Step 3: The AI ​​uses this feedback to adjust its instructions to the pet.

[0388] Step 4: The system operates to facilitate smooth communication between the pet and the user.

[0389] "Example 3"

[0390] Step 1: The emotion engine analyzes the user's emotions based on their tone of voice, facial expressions, behavior, etc.

[0391] Step 2: The results of the emotion engine's analysis are fed back into a means of understanding the pet's condition.

[0392] Step 3: A means to understand the pet's condition will be used to adjust the pet's condition based on this feedback.

[0393] Step 4: The system operates to facilitate smooth communication between the pet and the user.

[0394] Example 1

[0395] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal".

[0396] Conventional pet management systems simply record the sounds and behavior of pets, and have the problem that it is difficult to respond appropriately based on the pet's condition or the user's emotions. In addition, there is a lack of means for smooth communication between the user and the pet. This makes it difficult to properly grasp the pet's health condition and emotional state and to give instructions to the pet according to the user's emotions.

[0397] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0398] In this invention, the server includes a means for collecting voice data, a means for collecting characteristics of the pet animal, a means for identifying the type of animal, a means for collecting general characteristic data of the animal, a means for grasping the state of the pet from the cry based on the collected data, a means for recognizing the user's emotions, and a means for adjusting instructions to the pet based on the user's emotions. This makes it possible to grasp the state of the pet and the user's emotions in real time and to give appropriate instructions to the pet.

[0399] "Audio data" refers to data that records sounds, such as animal cries or the user's voice, in digital format.

[0400] "Characteristics of pets" refers to specific information about pets, such as their personalities, sounds, and behavioral patterns.

[0401] "Type of animal" is information indicating the type of pet, such as dog, cat, bird, etc.

[0402] "General animal characteristic data" is data such as personality and behavior patterns common to a specific animal species.

[0403] "Pet status" is information that indicates the current state of the pet, such as the pet's health condition or emotional state.

[0404] "User emotion" refers to the user's emotional state as analyzed from the user's tone of voice, facial expression, behavior, etc.

[0405] "Adjusting instructions" means changing the instructions given to the pet based on the user's emotions and the state of the pet.

[0406] "Collecting means" refers to devices and methods for acquiring voice data, animal characteristic data, etc.

[0407] An "identification means" is a device or method for identifying the type or characteristics of an animal based on the collected data.

[0408] "Analyzing means" refers to a device or method for analyzing collected data and understanding the pet's condition and the user's emotions.

[0409] The present invention is a system that collects and analyzes data on the cries and behavior of a pet to grasp the state of the pet and adjusts instructions to the pet based on the user's emotions. A specific embodiment of this system will be described below.

[0410] Audio data collection

[0411] Users collect their pet's cries using the microphone on their smartphone. The collected audio data is sent from the device to the server in real time. Specifically, the user starts the smartphone app and presses the "Start Recording" button to start collecting audio data.

[0412] Analysis of audio data

[0413] The server uses voice analysis software to analyze the voice data. Specifically, it converts the collected voice data into text data using a common voice analysis API. For example, Google Cloud Speech-to-Text API can be used. Based on the analysis results, it identifies whether the voice is the sound of a specific animal.

[0414] Collecting animal characteristic data

[0415] The user inputs the pet's personality and the characteristics of the voice it makes into the smartphone app. The device also uses sensors to observe the pet's behavior. Specifically, the device uses behavior observation sensors such as Nest Cam to collect the pet's behavior data. The collected data is then sent to a server.

[0416] Identifying Animal Species

[0417] The user inputs the type of pet into the app. The terminal then uses image recognition technology to identify the type of pet. Specifically, a general image recognition API can be used. For example, Google Cloud Vision API can be used to analyze the image of the pet and identify the type of animal.

[0418] Analyzing data and understanding your pet's condition

[0419] The server uses AI models to analyze the collected data, specifically machine learning frameworks such as TensorFlow to predict the pet's condition, and then understands the pet's health and emotional state based on the analysis results.

[0420] User Emotion Recognition

[0421] The device uses emotion analysis software to recognize the user's emotions. Specifically, it uses the Emotion API to analyze the user's tone of voice and facial expressions. Based on the analysis results, the server recognizes whether the user is angry or happy with the pet.

[0422] Adjusting pet instructions

[0423] The server adjusts the instructions to the pet based on the user's emotions: if the user is angry, the server generates a gentle instruction, and if the user is happy, the server generates a reinforced instruction. The generated instruction is transmitted to the pet through the terminal.

[0424] Examples

[0425] Audio data collection: The user launches the app on their smartphone and presses the “start recording” button to record the sound of their dog barking.

[0426] Voice data analysis: The server uses the Google Cloud Speech-to-Text API to analyze the recorded voice data and obtain the text data "Woof woof."

[0427] Animal characteristic data collection: The user enters "My dog's personality is gentle" into the app's input form, and the device uses Nest Cam to collect the dog's behavior data.

[0428] Animal type identification: A user uploads a photo of their dog to the app, and the device uses the Google Cloud Vision API to identify it as a "Golden Retriever."

[0429] Data analysis and pet condition understanding: The server uses TensorFlow to analyze the collected data and determine whether the dog is feeling stressed.

[0430] Recognizing user emotions: The device uses the Emotion API to analyze the user's tone of voice and facial expressions and recognize that the user is angry.

[0431] Pet command adjustment: The server analyzes the user's emotional data and generates gentle commands such as "sit," which the device then relays to the dog via voice.

[0432] Examples of prompt statements

[0433] "Collect dog barks and analyze them with a common voice analysis API. Based on the analysis results, use a machine learning framework to understand the dog's condition, and use emotion analysis software to recognize the user's emotions and adjust instructions for the pet."

[0434] In this way, the system specifically executes each processing step to facilitate smooth communication between the pet and the user.

[0435] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0436] Step 1:

[0437] Audio data collection

[0438] The user collects the sounds of their pet using the microphone on their smartphone. Specifically, the user starts the smartphone app and presses the "Start Recording" button to start collecting audio data.

[0439] Input: Pet noises

[0440] Output: Recorded audio data

[0441] Step 2:

[0442] Sending audio data

[0443] The device transmits the collected voice data to a server in real time. Specifically, the smartphone app uploads the recorded voice data to the server via the Internet.

[0444] Input: Recorded audio data

[0445] Output: Audio data sent to the server

[0446] Step 3:

[0447] Analysis of audio data

[0448] The server uses voice analysis software to analyze the voice data. Specifically, the server converts the collected voice data into text data using a common voice analysis API. For example, the Google Cloud Speech-to-Text API can be used.

[0449] Input: Audio data sent to the server

[0450] Output: Parsed text data

[0451] Step 4:

[0452] Collecting animal characteristic data

[0453] The user inputs the pet's personality and the characteristics of the voice it makes into the smartphone app. The device also uses sensors to observe the pet's behavior. Specifically, the device uses behavior observation sensors such as Nest Cam to collect the pet's behavior data. The collected data is then sent to a server.

[0454] Input: Pet's personality, vocalizations, and behavior data

[0455] Output: Feature and behavioral data sent to the server.

[0456] Step 5:

[0457] Identifying Animal Species

[0458] The user inputs the type of pet into the app. The terminal then uses image recognition technology to identify the type of pet. Specifically, a general image recognition API can be used. For example, Google Cloud Vision API can be used to analyze the image of the pet and identify the type of animal.

[0459] Input: Pet picture

[0460] Output: Analyzed animal species

[0461] Step 6:

[0462] Analyzing data and understanding your pet's condition

[0463] The server uses AI models to analyze the collected data, specifically machine learning frameworks such as TensorFlow to predict the pet's condition, and then understands the pet's health and emotional state based on the analysis results.

[0464] Input: feature data, behavior data, animal type

[0465] Output: Analyzed pet status

[0466] Step 7:

[0467] User Emotion Recognition

[0468] The device uses emotion analysis software to recognize the user's emotions. Specifically, it uses the Emotion API to analyze the user's tone of voice and facial expressions. Based on the analysis results, the server recognizes whether the user is angry or happy with the pet.

[0469] Input: The user's tone of voice and facial expressions

[0470] Output: Parsed user sentiment

[0471] Step 8:

[0472] Adjusting pet instructions

[0473] The server adjusts the instructions to the pet based on the user's emotions: if the user is angry, the server generates a gentle instruction, and if the user is happy, the server generates a reinforced instruction. The generated instruction is transmitted to the pet through the terminal.

[0474] Input: Analyzed pet state, user emotion

[0475] Output: Adjusted instructions

[0476] In this way, the system specifically executes each processing step to facilitate smooth communication between the pet and the user.

[0477] (Application example 1)

[0478] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0479] Conventional pet care systems only analyzed the pet's cries and behavior, and were unable to provide advice or services that took the user's emotions into account. This resulted in poor communication between the pet and the user, making it difficult to properly manage the pet's health and emotional state. In addition, it was difficult to provide appropriate services according to the pet's condition in brick-and-mortar stores.

[0480] The identification process by the identification processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means. In this invention, the server includes a means for collecting voice data (animal cries), a means for collecting characteristics of pet animals (personality, cries, etc.), a means for identifying the type of animal, a means for collecting general animal characteristic data, a means for grasping the pet's condition from its cries based on the collected data, a means for recognizing the user's emotions, a means for generating advice based on the pet's condition and the user's emotions, and a means for providing the generated advice to the user. This makes it possible to comprehensively analyze the pet's condition and the user's emotions and provide appropriate advice and services.

[0481] "Audio data" refers to audio information such as animal cries.

[0482] "Characteristics of pet animals" refers to information about the characteristics of the animals kept by the owner, such as their personalities and the sounds they make.

[0483] "Means for identifying the type of animal" refers to a method for identifying the type of animal using image recognition technology or input from the owner.

[0484] "General animal characteristic data" refers to characteristic information about general animals obtained from a database or the like.

[0485] "Means of understanding a pet's condition" refers to methods of analyzing a pet's health and emotional state based on collected voice data and animal characteristic data.

[0486] "Means for recognizing a user's emotions" refers to a method for identifying a user's emotions by analyzing the user's tone of voice, facial expression, behavior, etc.

[0487] The "means for generating advice" refers to a method for generating appropriate advice or services based on the pet's condition and the user's emotions.

[0488] The term "means for providing the generated advice to the user" refers to a method for communicating the generated advice or service to the user.

[0489] A system for implementing this invention includes means for collecting audio data (animal cries), means for collecting characteristics of pet animals (personality, cries, etc.), means for identifying the type of animal, means for collecting general animal characteristic data, means for understanding the condition of the pet from its cries based on the collected data, means for recognizing the user's emotions, means for generating advice based on the pet's condition and the user's emotions, and means for providing the generated advice to the user.

[0490] Hardware and Software Used

[0491] Hardware:

[0492] Microphone: Used to collect audio data.

[0493] Camera: Used to collect video data of the user.

[0494] software:

[0495] speech_recognition library: Used to perform speech recognition.

[0496] cv2 (OpenCV): Used to collect and analyze video data.

[0497] emotion_recognition: An emotion recognition engine for recognizing user emotions.

[0498] pet_behavior_analysis: A pet behavior analysis engine for analyzing pet sounds and behavior.

[0499] advice_generator: An advice generation engine for generating advice based on the pet's state and the user's emotions.

[0500] System processing overview

[0501] 1. Audio data collection:

[0502] The server collects the pet's cries using a microphone, and the collected audio data is analyzed using the speech_recognition library.

[0503] 2. Video data collection:

[0504] The server collects the user's video data using a camera, and the collected video data is analyzed using cv2 (OpenCV).

[0505] 3. Pet Status Analysis:

[0506] The server analyzes the collected voice data using the pet_behavior_analysis engine to determine the pet's condition (e.g., hunger, stress, happiness, etc.).

[0507] 4. User emotion recognition:

[0508] The server analyzes the collected video data using the emotion_recognition engine and recognizes the user's emotions (e.g., anger, joy, sadness, etc.).

[0509] 5. Generating Advice:

[0510] The server generates appropriate advice using the advice_generator engine based on the pet's condition and the user's emotions.

[0511] 6. Providing advice:

[0512] The server provides the generated advice to the user, for example, suggesting items to help the pet relax if the pet is stressed.

[0513] Examples

[0514] If your pet is stressed:

[0515] The server generates advice such as "Your pet is stressed. Try using a toy that will relax your pet" and provides it to the user.

[0516] If the user is angry:

[0517] The server generates advice such as "The user is angry. Please be kind to your pet" and provides it to the user.

[0518] Examples of prompt statements

[0519] Analyze the pet's cry data and the user's video data, and generate appropriate advice based on the pet's condition and the user's emotions. If the pet is stressed, suggest items that will help the user relax, and if the user is angry, advise the user to be gentle with the pet.

[0520] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0521] Step 1:

[0522] The server collects the pet's cries using a microphone. The collected voice data is analyzed using the speech_recognition library. The input is the pet's cries, and the output is the analyzed voice data. Through this analysis, the features of the pet's cries are extracted.

[0523] Step 2:

[0524] The server collects the user's video data using a camera. The collected video data is analyzed using cv2 (OpenCV). The input is the user's video data, and the output is the analyzed video data. Through this analysis, the user's facial expressions and behavioral features are extracted.

[0525] Step 3:

[0526] The server analyzes the collected voice data with the pet_behavior_analysis engine to understand the pet's state. The input is the analyzed voice data, and the output is the pet's state (e.g., hunger, stress, joy, etc.). This analysis identifies the pet's health and emotional state.

[0527] Step 4:

[0528] The server analyzes the collected video data with the emotion_recognition engine to recognize the user's emotions. The input is the analyzed video data, and the output is the user's emotion (e.g., anger, joy, sadness, etc.). This analysis identifies the user's emotional state.

[0529] Step 5:

[0530] The server generates appropriate advice using the advice_generator engine based on the pet's state and the user's emotion. The input is the pet's state and the user's emotion, and the output is the generated advice. This generation creates specific advice according to the pet and user's situation.

[0531] Step 6:

[0532] The server provides the generated advice to the user. The input is the generated advice, and the output is the advice provided to the user. This allows the user to take appropriate action according to the state of the pet.

[0533] Example 2

[0534] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0535] Conventional methods of communication with pets have the problem that it is difficult to accurately understand the cries and behavior of pets, and it takes time to grasp the pet's condition and requests. In addition, when a user gives instructions to a pet, the pet often does not understand the instructions, which makes it difficult to communicate smoothly.

[0536] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for collecting voice data, a means for collecting animal characteristics, a means for identifying the type of animal, a means for collecting general animal characteristic data, a means for grasping the state of the pet from the cry based on the collected data, a means for analyzing the collected voice data, a means for notifying the user of the analysis result, a means for collecting the user's voice instruction, a means for analyzing the voice instruction data, and a means for notifying the pet of the analysis result. This allows the pet's cry and the user's voice instruction to be accurately analyzed, enabling smooth communication between the pet and the user.

[0537] "Audio data" refers to data that is a digital recording of sounds made by a pet or a user.

[0538] "Animal characteristics" refers to information that refers to characteristics unique to each individual animal, such as a pet's personality or the sound it makes.

[0539] "Animal type" refers to the biological classification to which the pet belongs, such as dog or cat.

[0540] "General animal characteristic data" is data that includes information such as behaviors and sounds common to a particular type of animal.

[0541] The "means for understanding the state of a pet from its cry" refers to a method or device for analyzing the cry of a pet and identifying the state or request of the pet as indicated by the cry.

[0542] The "means for analyzing collected voice data" refers to a method or device for converting collected voice data into text data and analyzing the content thereof.

[0543] The "means for notifying the user of the analysis results" refers to a method or device for conveying the analyzed information to the user.

[0544] The "means for collecting user's voice instructions" refers to a method or device for recording voice instructions given by a user and collecting the data.

[0545] The "means for analyzing voice instruction data" refers to a method or device for converting a user's voice instruction into text data and analyzing the content of the text data.

[0546] The "means for communicating the analysis results to the pet" refers to a method or device for communicating the analyzed user's instructions in a form that the pet can understand.

[0547] The present invention is a system for realizing smooth communication between a pet and a user. This system includes a series of processes for collecting and analyzing voice data and notifying the results to the user and the pet.

[0548] Hardware and Software Used

[0549] Subject: Server

[0550] The server uses a speech recognition API and a natural language processing (NLP) model to analyze the pet's cries and the user's voice instructions. Specifically, it uses Google Cloud's Speech-to-Text API to convert the voice data into text data, and then analyzes the text data using the NLP model. As a result of the analysis, it identifies the pet's status and requests, as well as the user's instructions, and conveys them to the user and the pet in an appropriate manner.

[0551] Subject: Terminal

[0552] The terminal is equipped with a microphone to collect the pet's cries and the user's voice instructions. The collected voice data is sent to a server via the Internet. It also has a display and speaker to notify the user of the analysis results received from the server. For example, a smartphone or tablet can play this role.

[0553] Subject: User

[0554] The user inputs the sound of the pet's cry into the terminal and also issues his or her own voice instructions to the terminal. For example, if the pet barks "woof woof," the user inputs the sound into the terminal, and the terminal transmits the voice data to the server. Similarly, if the user says "let's play," the voice instruction is transmitted to the server through the terminal.

[0555] Examples

[0556] Example 1: Pet sound analysis

[0557] 1. My pet barks "woof woof."

[0558] 2. The device's microphone captures the barking sound.

[0559] 3. The device sends the bird's cry data to the server.

[0560] 4. The server converts the audio data into text data using Google Cloud's Speech-to-Text API.

[0561] 5. The server uses an NLP model to analyze the text data and generate a message saying "Your pet is hungry."

[0562] 6. The server generates a message and sends it to the terminal.

[0563] 7. The device displays a notification to the user that "Pet is hungry."

[0564] Example 2: User voice command analysis

[0565] 1. The user says "Let's play."

[0566] 2. The device's microphone captures the audio.

[0567] 3. The device sends the voice data to the server.

[0568] 4. The server converts the audio data into text data using Google Cloud's Speech-to-Text API.

[0569] 5. The server uses an NLP model to analyze the text data and convert the command "let's play" into a form the pet can understand.

[0570] 6. The server sends the converted instructions to the terminal.

[0571] 7. The device will play a specific sound and tell your pet to "play."

[0572] Examples of prompt statements

[0573] "If your pet barks 'woof woof', we can analyze the sound and tell you what condition your pet is in."

[0574] "When the user says 'Let's play,' analyze that voice and generate commands for the pet."

[0575] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0576] Step 1: Collect your pet's sounds

[0577] Subject: Terminal

[0578] The device collects the pet's barking sounds through a microphone. For example, if the pet barks "woof woof," the device's microphone captures the sound. The input is the pet's barking sound, and the output is digital audio data.

[0579] Step 2: Send the bird sound data to the server

[0580] Subject: Terminal

[0581] The terminal transmits the collected bird call data to a server via the Internet. The data is transmitted in audio file format (e.g., WAV format). The input is digital audio data, and the output is the audio data transmitted to the server.

[0582] Step 3: Analyze bird call data on the server

[0583] Subject: Server

[0584] The server converts the received bark data into text data using a speech recognition API. Specifically, it uses Google Cloud's Speech-to-Text API. The input is voice data, and the output is text data. The text data is then analyzed using an NLP model to identify the pet's status and requests. For example, it may analyze the bark "woof woof" to mean "I'm hungry." The input is text data, and the output is the analysis result.

[0585] Step 4: Notify the user of the analysis results

[0586] Subject: Server

[0587] The server sends the analysis result to the user's device. For example, it generates a message such as "your pet is hungry" and notifies the user's smartphone. The input is the analysis result, and the output is the notification message sent to the user's device.

[0588] Step 5: Collect user's voice commands

[0589] Subject: Terminal

[0590] The terminal collects the user's voice instructions through a microphone. For example, when the user says "Let's play," the device captures the voice. The input is the user's voice instruction, and the output is the digital voice data.

[0591] Step 6: Send voice command data to the server

[0592] Subject: Terminal

[0593] The terminal transmits the collected voice instruction data to a server via the Internet. The data is transmitted in audio file format (e.g., WAV format). The input is audio data in digital format, and the output is the audio data transmitted to the server.

[0594] Step 7: Parse the voice command data on the server

[0595] Subject: Server

[0596] The server converts the received voice instruction data into text data using a voice recognition API. Specifically, it uses Google Cloud's Speech-to-Text API. The input is voice data, and the output is text data. The text data is then analyzed using an NLP model and converted into a form that the pet can understand. For example, the instruction "Let's play" is converted into a sound or action instruction that the pet can understand. The input is text data, and the output is the analysis result.

[0597] Step 8: Communicate the results to your pet

[0598] Subject: Terminal

[0599] Based on the analysis results received from the server, the device generates sounds and behavioral instructions that the pet can understand and communicates them to the pet. For example, the device can play a specific sound to communicate the command to the pet "Let's play." The input is the analysis results, and the output is the command to be communicated to the pet.

[0600] (Application example 2)

[0601] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0602] Conventional methods of communication between pets and owners rely on the owner's intuitive understanding of the pet's cries and behavior, which makes accurate communication difficult. It is also difficult to accurately grasp the pet's health and emotional state, which can delay appropriate responses. Furthermore, in physical stores such as pet shops, staff are required to quickly grasp the pet's condition and take appropriate action, but there is currently a lack of effective means to do so.

[0603] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0604] In this invention, the server includes a means for collecting voice data (animal cries), a means for collecting characteristics of pet animals (personality, cries, etc.), a means for identifying the type of animal, a means for collecting general animal characteristic data, a means for grasping the state of the pet from the cries based on the collected data, a means including an emotion engine for recognizing the user's emotions, a means for adjusting instructions to the pet based on the user's emotions, a means for analyzing the pet's cries and grasping the state and requests of the pet, a means for converting human words into a form that the pet can understand, and a means including an application to be installed on a smartphone. This enables accurate communication between the pet and the owner, and allows the pet's health and emotional state to be quickly grasped and appropriate measures to be taken. Also, in a physical store, a store clerk can quickly grasp the state of the pet and take appropriate measures.

[0605] "Audio data" refers to sound information, including animal cries, recorded in digital format.

[0606] "Animal characteristics" refers to the individual characteristics of pet animals, such as their personalities and the sounds they make.

[0607] "Means for identifying animal species" refers to methods or techniques for distinguishing between specific animal species.

[0608] "General animal characteristic data" refers to information about the widely known characteristics of animals.

[0609] "Means for understanding a pet's condition from its cries" refers to methods and technologies for analyzing a pet's cries to understand its health and emotional state.

[0610] An "emotion engine that recognizes the user's emotions" refers to technology for analyzing emotions from the user's tone of voice, facial expressions, behavior, etc.

[0611] "Means for adjusting instructions to a pet based on the user's emotions" refers to a method or technology for changing the instructions to a pet depending on the user's emotional state.

[0612] "Means for analyzing a pet's cries and understanding the pet's condition and requests" refers to a method or technology for analyzing a pet's cries and understanding the pet's condition and requests.

[0613] "Means of converting human language into a form that pets can understand" refers to methods or technologies for converting human language into sounds and actions that pets can easily understand.

[0614] An "application installed on a smartphone" refers to a software program that runs on a smartphone.

[0615] A system for implementing this invention includes means for collecting voice data (animal cries), means for collecting characteristics of pet animals (personality, cries, etc.), means for identifying the type of animal, means for collecting general animal characteristic data, means for understanding the condition of a pet from its cries based on the collected data, means including an emotion engine for recognizing the user's emotions, means for adjusting instructions to the pet based on the user's emotions, means for analyzing the pet's cries and understanding the pet's condition and requests, means for converting human words into a form that the pet can understand, and means including an application to be installed on a smartphone.

[0616] The server has a database for collecting voice data and identifying the characteristics and type of animal. This allows it to analyze the cries of pets and understand their status and requests. Furthermore, using an emotion engine that recognizes the user's emotions, it analyzes the user's emotions from the tone of their voice, facial expressions, behavior, etc., and adjusts instructions to the pet based on the results.

[0617] The terminal (smartphone) sends the collected voice data to the server and receives the analysis results. The analysis results are displayed to inform the user of the pet's status and requests. When the user gives instructions to the pet, the instructions are converted into a form that the pet can understand and communicated to the pet.

[0618] As a concrete example, there is a scenario in which a pet shop clerk uses a smartphone to analyze the cries of pets and understand that the pet is saying "I'm hungry." Another scenario includes when the clerk says "Let's play," the command is conveyed to the pet in a way that is easy for the pet to understand.

[0619] Examples of prompts to be input to a generative AI model include the following:

[0620] "Analyze your pet's cries and let us know their condition."

[0621] "Translate human language into a form your pet can understand."

[0622] This system enables accurate communication between pets and their owners, allowing them to quickly grasp the health and emotional state of the pet and take appropriate action. It also enables store staff in physical stores to quickly grasp the pet's condition and take appropriate action.

[0623] The flow of the identification process in the application example 2 will be described with reference to FIG.

[0624] Step 1:

[0625] The server receives audio data (animal cries) sent from the device (smartphone). The input is audio data, and the output is data ready for analysis. In concrete terms, the server stores the audio data in digital format and passes it to the analysis engine.

[0626] Step 2:

[0627] The server analyzes the voice data and identifies the characteristics and type of animal. The input is the voice data received in step 1, and the output is information about the animal's type and characteristics. Specifically, the server uses a voice recognition algorithm to convert the sound into text, and then searches a database for the animal's type and characteristics based on that text.

[0628] Step 3:

[0629] The server understands the pet's condition and requests based on the collected animal characteristic data. The input is the animal characteristic data obtained in step 2, and the output is information about the pet's condition and requests. In concrete terms, the server runs an algorithm to estimate the pet's health condition and emotional state based on the analysis results.

[0630] Step 4:

[0631] The server uses an emotion engine that recognizes the user's emotions to analyze the user's emotions from the user's tone of voice, facial expressions, behavior, etc. The input is the user's voice data and video data, and the output is information about the user's emotional state. In concrete terms, the server executes an emotion recognition algorithm to analyze the user's emotions.

[0632] Step 5:

[0633] The server adjusts the instructions to the pet based on the user's emotional state. The input is the information about the user's emotional state obtained in step 4, and the output is the adjusted instructions. Specifically, the server executes an algorithm that changes the instructions depending on the user's emotions.

[0634] Step 6:

[0635] The server analyzes the pet's cries and understands the pet's status and requests. The input is the information about the pet's status and requests obtained in step 3, and the output is a message to notify the user. In concrete terms, the server generates a message to notify the user based on the analysis results.

[0636] Step 7:

[0637] The server converts human speech into a form the pet can understand. The input is the user's words, and the output is instructions converted into sounds and actions the pet can understand. In concrete terms, the server uses a generative AI model to translate the user's words into a form the pet can understand.

[0638] Step 8:

[0639] The terminal (smartphone) displays the analysis results and instructions received from the server to the user. The input is the message sent from the server, and the output is the information displayed to the user. In concrete terms, the terminal displays the received message on the screen and notifies the user.

[0640] Step 9:

[0641] The user checks the pet's status and requests through the terminal and takes necessary action. The input is the information displayed on the terminal, and the output is the user's actions. In concrete terms, the user takes appropriate action toward the pet based on the information displayed on the terminal.

[0642] Example 3

[0643] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0644] Conventional systems for understanding the health and emotional state of pets mainly analyze only the pet's cries and behavior, and have the problem of being unable to adjust the pet's behavior in consideration of the user's emotions. In addition, there is a lack of means to facilitate smooth communication between the pet and the user. This has led to problems such as insufficient health management and understanding of the pet's emotions, making it difficult to communicate smoothly with the user.

[0645] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0646] In this invention, the server includes means for collecting voice data, means for collecting animal characteristics, means for identifying animal species, means for collecting general animal characteristic data, means for grasping the state of the pet from the cries based on the collected data, means for collecting the user's emotions, means for analyzing the user's emotions, and means for adjusting the behavior of the pet based on the results of the user's emotion analysis, thereby enabling the health and emotional state of the pet to be grasped more accurately and the behavior of the pet to be adjusted according to the user's emotions.

[0647] "Means for collecting voice data" refers to a function for collecting a pet's cries using a voice input device such as a microphone.

[0648] "Means of collecting animal characteristics" refers to the function of collecting individual characteristics such as a pet's personality or cries using sensors, cameras, etc.

[0649] "Means for identifying animal species" refers to algorithms or identification technologies that identify the species of pet based on collected data.

[0650] "Means for collecting general animal characteristic data" refers to the functionality for collecting data on general animal characteristics from databases or external sources.

[0651] "Means of understanding a pet's condition from its cries based on the collected data" refers to an algorithm that analyzes the collected voice data and characteristic data to determine the pet's health and emotional state.

[0652] "Means for collecting user emotions" refers to a function for collecting the user's voice and facial expressions using a microphone or camera.

[0653] "Means for analyzing user emotions" refers to algorithms and analytical technologies that analyze collected user voice and facial expression data to determine the user's emotional state.

[0654] "Means for adjusting the behavior of the pet based on the results of the user's emotion analysis" refers to a control algorithm or instruction function for appropriately adjusting the behavior of the pet based on the results of the user's emotion analysis.

[0655] A "generative AI model" refers to an artificial intelligence model that analyzes a pet's condition and the user's emotions based on collected data.

[0656] The present invention is a system for grasping the health and emotional state of a pet and adjusting the behavior of the pet according to the emotion of a user. A specific embodiment of this system will be described below.

[0657] Hardware and Software Used

[0658] Hardware: Microphone, camera, and sensors (to detect pet movements)

[0659] Software: Generative AI models (e.g., general artificial intelligence models), sentiment engines (e.g., general sentiment analysis APIs)

[0660] System Configuration

[0661] 1. The terminal is equipped with a microphone to collect the pet's cries and a camera to capture the pet's behavior. These devices collect the pet's cries and behavior data in real time.

[0662] 2. The terminal temporarily stores the collected data and transmits it to the server at regular intervals.

[0663] 3. The server inputs the received data into the generative AI model to analyze the pet's health and emotional state. The generative AI model determines the pet's condition based on the collected voice and behavioral data.

[0664] 4. The server sends the analysis results to the device, which notifies the user. For example, the device can display a message on a smartphone app saying, "Your pet may be unwell."

[0665] 5. The terminal is equipped with a microphone to collect the user's voice and a camera to capture the user's facial expressions. These devices collect the user's voice and facial expression data.

[0666] 6. The device sends the collected user data to the server, and the server uses an emotion engine to analyze the user's emotions.

[0667] 7. The server sends the user's emotion analysis results to the device, which then adjusts the pet's behavior. For example, if the user is angry, the device will instruct the pet to "calm down."

[0668] Examples

[0669] Pet Health Analysis:

[0670] Example: If a pet is making an unusual sound, the generative AI model can detect the change and notify the owner that their pet may be unwell.

[0671] User emotion recognition and pet behavior adjustment:

[0672] Example: If a user is angry at their pet, the emotion engine will recognize that anger and the means of understanding the pet's state will adjust the pet's behavior to calm it down.

[0673] Examples of prompt statements

[0674] Pet Health Analysis:

[0675] "Analyze your pet's sounds and behavior to determine their health condition."

[0676] User emotion recognition and pet behavior adjustment:

[0677] "Analyze the user's voice and facial expressions to recognize emotions and adjust your pet's behavior based on the results."

[0678] This system makes it possible to grasp the status of the pet and the user in real time and take appropriate measures. The flow of the identification process in the third embodiment will be described with reference to FIG.

[0679] Step 1:

[0680] The device collects the sounds your pet makes.

[0681] Input: Pet noises

[0682] How it works: A microphone on the device collects your pet's sounds in real time.

[0683] Output: Collected audio data

[0684] Step 2:

[0685] The device captures your pet's behavior.

[0686] Input: pet behavior

[0687] How it works: The camera on the device captures your pet's movements in real time.

[0688] Output: Collected video data

[0689] Step 3:

[0690] The data collected by the terminal is temporarily stored and then sent to the server at regular intervals.

[0691] Input: Audio data, video data

[0692] Operation: The data collected by the terminal is temporarily stored in the internal memory and then sent to the server in batches at regular intervals.

[0693] Output: Data sent to the server

[0694] Step 4:

[0695] The data received by the server is input into a generative AI model to analyze the pet's health and emotional state.

[0696] Input: Audio data, video data

[0697] How it works: The server feeds data into a generative AI model that analyzes the pet's health and emotional state.

[0698] Output: Analysis results (pet's health and emotional state)

[0699] Step 5:

[0700] The server transmits the analysis results to the terminal, and the terminal notifies the user.

[0701] Input: Analysis results

[0702] How it works: The server sends the analysis results to the device, and the device sends a push notification to the user's smartphone.

[0703] Output: Analysis results reported to the user

[0704] Step 6:

[0705] The terminal collects user feedback.

[0706] Input: User's voice

[0707] How it works: A microphone on the device collects the user's voice.

[0708] Output: Collected audio data

[0709] Step 7:

[0710] The device captures the user's facial expression.

[0711] Input: User's facial expression

[0712] How it works: The device's camera captures the user's facial expressions.

[0713] Output: Collected video data

[0714] Step 8:

[0715] The terminal transmits the collected user data to the server.

[0716] Input: Audio data, video data

[0717] Operation: The device sends the collected data to the server.

[0718] Output: Data sent to the server

[0719] Step 9:

[0720] The server uses an emotion engine to analyze the user's emotions.

[0721] Input: Audio data, video data

[0722] How it works: The server inputs data into the emotion engine to analyze the user's emotional state.

[0723] Output: Analysis results (user's emotional state)

[0724] Step 10:

[0725] The server sends the results of the user's emotion analysis to the terminal, and the terminal adjusts the pet's behavior.

[0726] Input: Analysis results

[0727] How it works: The server sends the analysis results to the device, and the device gives the appropriate instructions to the pet.

[0728] Output: Adjusted pet behavior

[0729] In this way, the system can grasp the status of the pet and the user in real time and take appropriate action.

[0730] (Application example 3)

[0731] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0732] Conventional pet care systems have difficulty in accurately grasping the health and emotional state of pets, making it difficult for owners and store staff to properly manage the condition of pets. In addition, they lack the function to adjust the behavior of pets based on the user's emotions, which makes it difficult for smooth communication between pets and users.

[0733] The identification process by the identification processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means. In this invention, the server includes a means for collecting voice data (animal cries), a means for collecting characteristics of pet animals (personality, cries, etc.), a means for identifying the type of animal, a means for collecting general animal characteristic data, a means for grasping the state of the pet from the cries based on the collected data, a means for collecting and analyzing image data, a means for recognizing the user's emotions, and a means for adjusting the behavior of the pet based on the user's emotions. This makes it possible to accurately grasp the health and emotional state of the pet and adjust the behavior of the pet according to the user's emotions.

[0734] "Audio data" refers to audio information such as animal cries, and is data collected to analyze the condition of pets.

[0735] "Animal characteristics" refers to characteristics unique to each animal, such as personality and cries, and is information collected to understand the condition of your pet.

[0736] "Means for identifying animal type" refers to technology for identifying what type of animal a pet is based on collected data.

[0737] "General animal characteristic data" is data regarding widely known characteristics of animals, and is information used as a standard when analyzing the condition of pets.

[0738] "A means of understanding a pet's condition from its cries" is a technology that analyzes collected cries data to determine a pet's health and emotional state.

[0739] "Means for collecting and analyzing image data" refers to technology for collecting images of a pet and analyzing the images to understand the pet's condition.

[0740] "Means for recognizing the user's emotions" refers to technology for analyzing the user's tone of voice, facial expressions, etc. to determine the user's emotional state.

[0741] The "means for adjusting the behavior of a pet based on the user's emotions" refers to a technique for appropriately changing the behavior of a pet in response to the recognized emotions of the user.

[0742] A pet care assistant system will be described as an embodiment of the present invention. This system is intended for use in actual stores such as pet shops and pet cafes. The system is installed on a smartphone, smart glasses, or a head-mounted display, and analyzes the health and emotional state of pets in real time and notifies store staff and owners.

[0743] Hardware and Software Used

[0744] Hardware

[0745] Smartphone

[0746] Smart Glasses

[0747] Head-mounted display

[0748] software

[0749] OpenCV: Image processing library

[0750] Keras: A deep learning library

[0751] librosa: Audio processing library

[0752] soundfile: Sound file reading library

[0753] Data processing and calculation

[0754] Analysis of audio data

[0755] The server collects audio data (animal cries) and extracts MFCC features using librosa, which converts the audio data into numerical data and feeds it into a model trained using Keras. The model predicts the health and emotional state of the pet.

[0756] Analysis of image data

[0757] The server collects image data, resizes and preprocesses it using OpenCV, and feeds the preprocessed image data into a model trained using Keras to analyze the pet's condition.

[0758] User Emotion Recognition

[0759] The server collects the user's video data and extracts frames using OpenCV, which are then fed into an emotion recognition model trained using Keras to predict the user's emotional state.

[0760] Pet behavior adjustment

[0761] The server adjusts the pet's behavior based on the user's emotional state, for example adjusting the pet's behavior to be calmer if the user is angry and increasing the pet's activity if the user is happy.

[0762] Examples

[0763] Example of use in a pet shop

[0764] If a pet in a pet shop becomes stressed, the system will analyze voice and image data to detect the pet's stress state, and will alert store staff to take appropriate measures to care for the pet.

[0765] Example of use at a pet cafe

[0766] In the pet cafe, while a user is playing with a pet, the system analyzes the user's emotional state. If the user is happy, the system will make the pet more active and promote communication with the user.

[0767] Examples of prompt statements

[0768] Develop a system that analyzes your pet's sounds and behavior, and notifies you of your pet's health and emotional state. Include the ability to recognize the user's emotions and adjust your pet's behavior accordingly.

[0769] The flow of the specific process in the application example 3 will be described with reference to FIG.

[0770] Step 1:

[0771] The server collects the pet's voice data. Specifically, it records the pet's cries using a microphone installed in a smartphone, smart glasses, or a head-mounted display. The input is the voice data, and the output is the recorded voice file.

[0772] Step 2:

[0773] The server analyzes the collected voice data. It uses librosa to extract MFCC features from the voice data and convert them into numerical data. The input is the recorded voice file, and the output is numerical data including MFCC features.

[0774] Step 3:

[0775] The server inputs the MFCC features into a voice analysis model trained using Keras to predict the pet's health condition and emotional state. The input is numerical data including the MFCC features, and the output is a prediction result indicating the pet's health condition and emotional state.

[0776] Step 4:

[0777] The server collects image data of pets. Images of pets are taken using a camera installed in a smartphone, smart glasses, or a head-mounted display. The input is image data, and the output is a captured image file.

[0778] Step 5:

[0779] The server analyzes the collected image data. It resizes and preprocesses the images using OpenCV and inputs them into an image analysis model trained using Keras. The input is the captured image file and the output is the preprocessed image data.

[0780] Step 6:

[0781] The server inputs the preprocessed image data into a model trained with Keras to analyze the pet's state. The input is the preprocessed image data, and the output is a prediction result indicating the pet's state.

[0782] Step 7:

[0783] The server collects the user's video data. It captures the user's facial expressions using a camera installed in a smartphone, smart glasses, or a head-mounted display. The input is the video data, and the output is the captured video file.

[0784] Step 8:

[0785] The server analyzes the collected video data. It uses OpenCV to extract frames from the video and inputs them into an emotion recognition model trained with Keras. The input is the captured video file and the output is the extracted frame data.

[0786] Step 9:

[0787] The server inputs the extracted frame data into an emotion recognition model trained with Keras to predict the user's emotional state. The input is the extracted frame data, and the output is a prediction result indicating the user's emotional state.

[0788] Step 10:

[0789] The server adjusts the pet's behavior based on the user's emotional state. For example, if the user is angry, the server adjusts the pet's behavior to be calmer, and if the user is happy, the server adjusts the pet's behavior to be more active. The input is a prediction result indicating the user's emotional state, and the output is the adjusted pet's behavior.

[0790] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0791] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0792] Another example of generative AI is Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0793] In the above embodiment, an example was given in which the specific process was performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0794] [Second embodiment]

[0795] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0796] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0797] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0798] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0799] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0800] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0801] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0802] Fig. 4 shows an example of main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0803] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0804] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0805] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0806] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0807] "Example 1"

[0808] In one embodiment of the present invention, the cries of a pet are collected using a sound data collection device such as a microphone. The collected sound data is sent to a sound analysis device and identified as the cry of a specific animal. In addition, the characteristics of the pet animal (personality, cries, etc.) are collected using input from the owner or a sensor that observes the behavior of the pet. The type of animal is identified using input from the owner or image recognition technology. General animal characteristic data is obtained from a database or the like. This data is analyzed by AI to understand the condition of the pet.

[0809] "Example 2"

[0810] As another embodiment of the present invention, a system is provided that utilizes AI to realize conversation between humans and pets. Specifically, AI analyzes the cries of pets and converts them into words that humans can understand. Conversely, AI analyzes human words and converts them into cries and actions that pets can understand. For example, if a pet expresses its "hungry" state with a cry, the AI ​​analyzes it and conveys the message "your pet is hungry" to the owner. Also, if the owner says "let's play," the AI ​​analyzes it and conveys the message in a form that the pet can understand.

[0811] "Example 3"

[0812] As a further embodiment of the present invention, a system for understanding a pet's health and emotional state is provided. Specifically, an AI analyzes a pet's health and emotional state from its cries and behavior. For example, if a pet is suffering from an illness, its cries and behavior may be different from normal. The AI ​​detects these changes and issues a warning to the owner. Also, if a pet is happy, its cries and behavior may become more active. The AI ​​detects these changes and conveys the happiness to the owner.

[0813] The process flow of each embodiment will be described below.

[0814] "Example 1"

[0815] Step 1: Collect the pet's cries using an audio data collection device such as a microphone.

[0816] Step 2: The collected audio data is sent to an audio analyzer to identify it as the sound of a specific animal.

[0817] Step 3: Collect information about the pet's characteristics (personality, sounds, etc.) using input from the owner and sensors that observe the pet's behavior.

[0818] Step 4: Identify the type of animal using input from the owner and image recognition technology.

[0819] Step 5: Obtain general animal characteristic data from a database, etc.

[0820] Step 6: This data is analyzed using AI to understand the pet's condition.

[0821] "Example 2"

[0822] Step 1: AI analyzes your pet's cries and converts them into words that humans can understand. Step 2: AI analyzes human words and converts them into cries and actions that your pet can understand.

[0823] Step 3: For example, if a pet expresses its hunger state through a cry, the AI ​​will analyze it and convey the message "your pet is hungry" to the owner.

[0824] Step 4: Also, if the owner says "let's play," the AI ​​will analyze it and convey the message in a way that the pet can understand.

[0825] "Example 3"

[0826] Step 1: AI analyzes your pet's health and emotional state based on its sounds and behavior.

[0827] Step 2: For example, if a pet is suffering from an illness, its sounds and behavior may be different from normal. AI will detect these changes and alert the owner.

[0828] Step 3: Also, if a pet is happy, its vocalizations and behavior may become more active. AI will detect these changes and communicate its happiness to the owner.

[0829] Example 1

[0830] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0831] Conventional pet condition monitoring systems mainly collect and analyze voice data and animal characteristic data separately, making it difficult to integrate this data and understand the pet's condition comprehensively. In addition, there was a lack of means to accurately grasp the health and emotional state of pets, making it difficult for owners to appropriately understand and respond to their pet's condition.

[0832] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0833] In this invention, the server includes a means for collecting voice data, a means for transmitting the collected voice data, a means for analyzing the voice data, a means for collecting characteristics of the pet animal, a means for identifying the type of animal, a means for collecting general animal characteristic data, and a means for grasping the condition of the pet from its cry based on the collected data. This makes it possible to comprehensively grasp the condition of the pet by integrating the voice data and the animal characteristic data.

[0834] "Audio data" refers to audio signals, such as animal cries, recorded in digital format.

[0835] "Collecting means" refers to a device or method for acquiring audio data and animal characteristic data.

[0836] A "transmitting means" is a device or method for transferring collected data to another device or server.

[0837] An "analyzing means" is a device or method for processing collected data and extracting specific information.

[0838] "Animal characteristics" refers to individual attributes of pet animals, such as their personalities or the sounds they make.

[0839] An "identification means" is a device or method for determining the type or characteristics of an animal.

[0840] "General animal characteristic data" refers to data such as characteristics and behavioral patterns common to a specific animal species.

[0841] "Pet's condition" refers to the pet's overall condition, including its health and emotional state.

[0842] "AI" is a technology that uses artificial intelligence to analyze data and make specific judgments and predictions.

[0843] "Means for enabling conversation" refers to devices or methods that allow people and pets to communicate.

[0844] "Health Status" refers to information regarding the physical health of a pet.

[0845] "Emotional state" refers to a pet's psychological state or mood.

[0846] The present invention is a system for collecting sounds made by pets and analyzing the data to grasp the state of the pets. A specific embodiment of this system will be described below.

[0847] First, a user uses the smartphone's built-in microphone or a dedicated external microphone to collect the sound of their pet. For example, the user starts a recording app on the smartphone and records the sound of their pet. This sound data is sent to the server by the device. Specifically, the device uses an HTTP POST request to send the sound data to the server's API endpoint.

[0848] The server analyzes the received voice data using a voice analyzer (for example, Google Cloud Speech-to-Text API). The server sends the voice data to the API, analyzes the returned text data, and identifies it as the sound of a specific animal.

[0849] Next, the user inputs the characteristics of the pet (personality, cries, etc.) into the app. For example, the user inputs information such as "personality: active, cries: high-pitched" into the app's form. The app also uses sensors connected to the device (e.g., acceleration sensor and camera) to observe the pet's behavior and collect data. The device periodically transmits the data from the sensors to the server.

[0850] To identify the type of animal, the user can input the type of pet into the app, or the device can use image recognition technology (e.g., TensorFlow or OpenCV) to identify the type of animal. For example, the user can input "dog" or the device can send an image taken with the camera to the server, and the server can use image recognition technology to identify it as a "dog."

[0851] The server retrieves general animal trait data from a database (e.g. AWS RDS or Google BigQuery). The server runs SQL queries to retrieve the required data from the database.

[0852] Finally, the server uses AI (for example, TensorFlow or PyTorch) to analyze the collected voice data, pet characteristic data, animal type data, and general animal characteristic data. The server integrates this data and inputs it into an AI model to understand the pet's condition. For example, the server analyzes data such as "the dog is barking loudly and is behaving actively" and determines that "the dog is excited."

[0853] Examples and prompts

[0854] As a concrete example, the barking sound of a user's dog is collected by the microphone on the smartphone and sent to the server. The server uses the Google Cloud Speech-to-Text API to analyze the sound and identify it as a dog's bark. The user inputs the dog's personality and characteristics of its bark into the app, and observes the dog's behavior with a camera connected to the device. The server obtains general dog characteristic data from AWS RDS, analyzes the data using TensorFlow, and understands the dog's condition.

[0855] An example of a prompt sentence is as follows:

[0856] "Please analyze my dog's barks and understand his condition. My dog ​​is very active and barks a lot. Please analyze the following audio data."

[0857] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0858] Step 1:

[0859] A user collects the sounds of their pet.

[0860] Specifically, the user launches a recording app on their smartphone and records their pet's cries.

[0861] Input: Pet sounds

[0862] Output: Audio data file (e.g. .wav format)

[0863] Step 2:

[0864] The terminal transmits the collected voice data to a server.

[0865] Specifically, the device uses an HTTP POST request to send audio data to the server's API endpoint.

[0866] Input: Audio data file

[0867] Output: Audio data sent to the server

[0868] Step 3:

[0869] The server analyzes the received audio data.

[0870] Specifically, the server uses a voice analysis device (e.g., Google Cloud Speech-to-Text API) to convert the voice data into text data and identify it as the sound of a specific animal.

[0871] Input: Audio data sent to the server

[0872] Output: Parsed text data (e.g. "dog barking")

[0873] Step 4:

[0874] Collect characteristics of the animals kept by the user.

[0875] Specifically, the user enters information such as "personality: active, bark: high-pitched" into the app's form. In addition, the app uses sensors connected to the device (e.g., acceleration sensor and camera) to observe the pet's behavior and collect data.

[0876] Input: User-entered animal feature data, behavioral data from sensors

[0877] Output: Collected animal trait data

[0878] Step 5:

[0879] Identify types of animals.

[0880] Specifically, the user inputs the type of animal they own into the app, or the device uses image recognition technology (e.g. TensorFlow or OpenCV) to identify the type of animal.

[0881] Input: Animal species data or image data entered by the user

[0882] Output: Identified animal type data (e.g. "dog")

[0883] Step 6:

[0884] A server obtains general animal characteristic data.

[0885] Specifically, the server executes an SQL query from a database (e.g. AWS RDS or Google BigQuery) to retrieve the required data.

[0886] Input: Animal species data

[0887] Output: General animal traits data

[0888] Step 7:

[0889] The server analyzes the collected data and determines the pet's condition.

[0890] Specifically, the server uses AI (e.g. TensorFlow or PyTorch) to analyze collected voice data, animal feature data, animal species data, and general animal feature data. The server integrates this data and inputs it into an AI model to determine the pet's condition.

[0891] Input: Audio data, animal feature data, animal species data, general animal feature data

[0892] Output: Analysis results (e.g. "The dog is excited")

[0893] (Application example 1)

[0894] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0895] There is a problem in that it is difficult for pet owners to grasp abnormal behavior or health conditions of their pets in real time while they are out or not watching over them. In addition, there is a lack of means to detect abnormalities in pets early and respond quickly, making it difficult to ensure the health and safety of pets.

[0896] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0897] In this invention, the server includes a means for collecting voice data (animal cries), a means for collecting characteristics of pet animals (personality, cries, etc.), a means for identifying the type of animal, a means for collecting general animal characteristic data, a means for grasping the pet's condition from the cries based on the collected data, and a means for sending a notification when an abnormality is detected. This makes it possible to grasp the pet's abnormal behavior and health condition in real time, and to promptly notify the owner when an abnormality is detected.

[0898] "Audio data" refers to information collected in digital format from audio signals such as animal cries.

[0899] "Collecting means" refers to a device or method for acquiring audio data and animal characteristic data.

[0900] "Animal characteristics" refers to information that refers to the unique characteristics of an animal, such as its personality or the sound it makes.

[0901] An "identification means" is a device or method for determining the type or characteristics of an animal.

[0902] "General animal characteristic data" refers to information such as personality and behavioral patterns common to a specific animal species.

[0903] "Means of understanding" refers to devices and methods used to analyze collected data and understand the condition of animals.

[0904] "Means for detecting abnormalities" refers to devices or methods for identifying abnormalities in the behavior or sounds of animals and detecting the occurrence of an abnormality.

[0905] A "means for sending a notification" is a device or method for conveying that information to an owner or other person when an abnormality is detected.

[0906] The system for carrying out the present invention collects and analyzes the cries of pets to grasp the state of the pets, and sends a notification to the pet's owner if an abnormality is detected. A specific embodiment of the system will be described below.

[0907] Hardware Configuration

[0908] The system includes the following hardware:

[0909] Microphone: Used to collect pet sounds.

[0910] Smartphone or smart glasses: Used to analyze collected voice data and detect anomalies.

[0911] Server: Analyzes voice data and sends notifications.

[0912] Software Configuration

[0913] The system includes the following software:

[0914] TensorFlow: Runs AI models to analyze audio data.

[0915] sounddevice: Used to collect audio data.

[0916] smtplib: Used to send notifications if an anomaly is detected.

[0917] Data processing and calculation

[0918] 1. Audio data collection: The device (smartphone or smart glasses) uses a microphone to collect the pet's cries. The collected audio data is stored in digital format.

[0919] 2. Audio data analysis: The server inputs the collected audio data into the TensorFlow model and performs analysis. Based on the analysis results, the pet's condition is understood.

[0920] 3. Anomaly detection: The server determines whether an anomaly has been detected based on the analysis results. If an anomaly is detected, a notification is sent to the owner.

[0921] 4. Send notification: The server uses smtplib to send a notification to the pet owner, containing information about the pet's abnormal behavior and health status.

[0922] Examples

[0923] For example, if a pet makes an abnormal sound while the owner is out, the system will collect and analyze the sound. If an abnormality is detected based on the analysis results, a notification will be sent to the owner's smartphone, allowing the owner to take prompt action.

[0924] Examples of prompt statements

[0925] "Collect the sounds of pets' cries and analyze them using an AI model. If an abnormality is detected, create a program to notify the pet owner."

[0926] In this way, the present invention makes it possible to grasp a pet's abnormal behavior and health condition in real time and quickly notify the owner if an abnormality is detected.

[0927] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0928] Step 1:

[0929] The terminal (smartphone or smart glasses) collects the pet's cries using a microphone. The input is the pet's cries, and the output is digital audio data. Specifically, the terminal uses software for collecting audio data (sounddevice library) to record audio for a certain period of time and save it as digital data.

[0930] Step 2:

[0931] The device transmits the collected voice data to the server. The input is the digital voice data, and the output is the voice data transmitted to the server. Specifically, the device uses an Internet connection to upload the voice data to the server.

[0932] Step 3:

[0933] The server inputs the received voice data into the TensorFlow model for analysis. The input is digital voice data, and the output is the analysis result (the state of the pet). Specifically, the server uses the TensorFlow library to input the voice data into the AI ​​model and analyze the characteristics of the pet's cries.

[0934] Step 4:

[0935] The server determines whether an anomaly has been detected based on the analysis results. The input is the analysis results, and the output is information about the presence or absence of an anomaly. Specifically, the server compares the analysis results with a threshold, and sets a flag if an anomaly is detected.

[0936] Step 5:

[0937] If an abnormality is detected, the server will send a notification to the pet owner. The input is information about the presence or absence of an abnormality, and the output is a notification to the pet owner. Specifically, the server uses the smtplib library to send an abnormality notification to the pet owner's email address. The notification contains detailed information about the pet's abnormal behavior and health condition.

[0938] Step 6:

[0939] The user (owner) checks the received notification and takes action to check the pet's status if necessary. The input is the notification from the server, and the output is the user's response. Specifically, the user checks the notification on their smartphone and takes action such as returning home to check the pet's status or contacting the pet sitter.

[0940] Example 2

[0941] Next, a description will be given of Example 2 of the embodiment 2. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[0942] Conventional means of communication between pets and humans are limited, making it difficult to accurately grasp a pet's intentions and state from its cries and actions. In addition, there was a lack of means to communicate human words to pets in a form that pets could understand. This made it difficult to properly manage a pet's health and emotional state, and communication between pets and owners was insufficient.

[0943] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for collecting voice data, a means for collecting animal characteristics, a means for identifying the type of animal, a means for collecting general animal characteristic data, a means for grasping the state of the pet from the cry based on the collected data, a means for converting voice data into text data using voice recognition software, a means for analyzing text data using a generative AI model to understand the intention of the pet, a means for generating a message based on the analysis result and notifying the user, a means for collecting the user's words and converting them into text data using voice recognition software, a means for analyzing text data using a generative AI model to generate a message to be conveyed to the pet, and a means for converting and conveying a message into a cry or behavior that the pet can understand using voice synthesis software. This makes it possible to accurately grasp the intention and state of the pet from its cry or behavior, and to convey human words in a form that the pet can understand.

[0944] "Audio data" refers to data that has been recorded in digital format from sounds made by animals or humans.

[0945] "Animal characteristics" refers to characteristics unique to each individual animal, such as their personality, sounds, and behavioral patterns.

[0946] "Type of animal" refers to the classification of animals, such as dog, cat, bird, etc.

[0947] "General animal characteristic data" refers to data such as personality and behavioral patterns common to a particular animal species.

[0948] "Voice recognition software" is software for analyzing voice data and converting it into text data.

[0949] A "generative AI model" is a model that uses artificial intelligence to analyze text data and understand its meaning.

[0950] "Text data" is textual information converted from voice data by voice recognition software.

[0951] "Message" refers to the content of notifications or instructions generated based on the analysis results.

[0952] "Speech synthesis software" refers to software for converting text data into speech.

[0953] "User" refers to a person who uses this system.

[0954] "Pet" refers to animals kept as pets in the home.

[0955] This invention is a system that uses AI to realize conversations between people and pets. This system has the functions of analyzing pet cries and converting them into words that humans can understand, and analyzing human words and converting them into cries and actions that pets can understand.

[0956] Hardware and Software Used

[0957] Hardware

[0958] Microphone: Used to capture pet sounds and user speech.

[0959] Speaker: Used to transmit generated sounds to your pet.

[0960] Camera: Used to monitor pet behavior if necessary.

[0961] Server: Used to analyze and process data.

[0962] software

[0963] Speech recognition software: Used to convert voice data into text data (e.g. Google Speech-to-Text).

[0964] Generative AI models: Analyze text data and use it to understand your pet's intent (e.g. GPT-4).

[0965] Speech synthesis software: Used to convert text data into audio (e.g. Google Text-to-Speech).

[0966] Specific operation of the system

[0967] Pet sound analysis

[0968] The microphone on the device captures the sound of your pet barking. For example, if your dog barks, the audio data is collected through the microphone. The collected audio data is then sent to the server in real time.

[0969] The server analyzes the captured voice data using voice recognition software and converts it into text data, for example converting the voice "woof woof" into the text "woof woof." The converted text data is sent to the next analysis step.

[0970] The server uses a generative AI model to analyze the text data and understand the pet's intention. For example, the text "woof woof" is analyzed to mean "I'm hungry." The analysis result is sent to the message generation step.

[0971] The server generates a message that says "your pet is hungry" based on the analysis results and notifies the user. For example, the notification is sent to the user via a smartphone app. The user can check the notification on the app and understand the status of the pet.

[0972] Human Language Analysis

[0973] The microphone on the device captures the user's words. For example, when the user says "Let's play," the voice data is collected through the microphone. The collected voice data is sent to the server in real time.

[0974] The server analyzes the captured voice data using voice recognition software and converts it into text data. For example, the voice "Let's play" is converted into the text "Let's play." The converted text data is sent to the next analysis step.

[0975] The server uses a generative AI model to analyze the text data and generate a message to be sent to the pet. For example, the text "Let's play" is analyzed to mean "It's play time." The analysis result is sent to the message generation step.

[0976] The server uses speech synthesis software to convert the generated message into sounds and actions that the pet can understand, and communicates it to the pet through the speaker. For example, the message "It's playtime" is converted into sounds and actions that the dog can understand, and played back through the speaker. The pet understands the sounds and actions, and gets ready to play with the user.

[0977] Examples of concrete examples and prompts

[0978] Examples

[0979] Pet Sound Analysis:

[0980] If a pet barks "woof woof," a microphone will capture the sound and the server will notify the user that "your pet is hungry."

[0981] Human Language Analysis:

[0982] When a user says "let's play," a microphone captures the audio and the server communicates through the speaker with sounds and actions that the pet understands to mean "it's play time."

[0983] Examples of prompt statements

[0984] Pet Sound Analysis:

[0985] If a pet barks "woof woof", explain the process of analyzing that sound and informing the user that their pet is hungry.

[0986] Human Language Analysis:

[0987] When the user says "let's play," describe the process that analyzes that speech and communicates to your pet "it's play time" in a way that they can understand.

[0988] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0989] Step 1: Capture your pet's sounds

[0990] The microphone on the device captures the sound of the pet barking. For example, if a dog barks "woof woof," the audio data is collected through the microphone. The input is the pet's bark, and the output is audio data. The collected audio data is sent to the server in real time.

[0991] Step 2: Analyzing bird calls and converting them into text

[0992] The server analyzes the captured voice data using voice recognition software and converts it into text data. For example, the voice "woof woof" is converted into the text "woof woof." The input is voice data and the output is text data. The converted text data is sent to the next analysis step.

[0993] Step 3: Semantic analysis of the text

[0994] The server uses a generative AI model to analyze the text data and understand the pet's intention. For example, the text "woof woof" is analyzed to mean "I'm hungry." The input is the text data, and the output is the analysis result that indicates the pet's intention. The analysis result is sent to the message generation step.

[0995] Step 4: Message generation and notification

[0996] The server generates a message saying "your pet is hungry" based on the analysis results and notifies the user. For example, the notification is sent to the user via a smartphone app. The input is the analysis results, and the output is a notification message to the user. The user can check the notification on the app to understand the status of their pet.

[0997] Step 5: Capturing human language

[0998] The microphone on the device captures the user's words. For example, when a user says "Let's play," the voice data is collected through the microphone. The input is the user's words, and the output is voice data. The collected voice data is sent to the server in real time.

[0999] Step 6: Word analysis and text conversion

[1000] The server analyzes the captured voice data using voice recognition software and converts it into text data. For example, the voice "Let's play" is converted into the text "Let's play". The input is voice data and the output is text data. The converted text data is sent to the next analysis step.

[1001] Step 7: Semantic analysis of the text

[1002] The server uses a generative AI model to analyze the text data and generate a message to be conveyed to the pet. For example, the text "Let's play" is analyzed to mean "It's play time." The input is the text data, and the output is the message to be conveyed to the pet. The analysis result is sent to the message generation step.

[1003] Step 8: Create a message and send it to your pet

[1004] The server uses speech synthesis software to convert the generated message into sounds and actions that the pet can understand, and communicates them to the pet through the speaker. For example, the message "It's playtime" is converted into sounds and actions that the dog can understand, and played back from the speaker. The input is the message to be communicated to the pet, and the output is sounds and actions that the pet can understand. The pet understands the sounds and actions, and gets ready to play with the user.

[1005] (Application example 2)

[1006] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[1007] Conventional pet care systems have problems in that it is difficult to accurately understand a pet's cries and behavior and take appropriate action. They also lack a means to notify the owner that the pet is hungry and to quickly order pet food. Furthermore, they lack an effective means to facilitate communication between the owner and the pet.

[1008] The identification process by the identification processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means. In this invention, the server includes a means for collecting voice data (animal cries), a means for collecting characteristics of pet animals (personality, cries, etc.), a means for identifying the type of animal, a means for collecting general animal characteristic data, a means for grasping the pet's condition from the cries based on the collected data, a means for notifying the owner based on the pet's condition, and a means for ordering pet food. This makes it possible to accurately grasp the pet's condition and respond quickly. In addition, it is possible to facilitate communication between the owner and the pet and automate the ordering of pet food.

[1009] "Audio data" refers to sound information, including animal cries.

[1010] "Characteristics of pets" refers to individual characteristics such as a pet's personality or the sounds it makes.

[1011] "Means for identifying the type of animal" refers to a method or device for identifying the type of pet.

[1012] "General animal characteristic data" refers to information about the widely known characteristics of animals.

[1013] "Means for understanding a pet's condition from its cries" refers to a method or device for analyzing a pet's cries to understand its condition.

[1014] "Means for notifying an owner based on the condition of the pet" refers to a method or device for notifying an owner of the condition of the pet.

[1015] "Pet food ordering means" refers to a method or device for purchasing pet food online.

[1016] "Means of using AI to enable conversation between people and pets" refers to methods and devices that use artificial intelligence to exchange information between humans and pets.

[1017] "Means for understanding a pet's health and emotional state" refers to a method or device for understanding a pet's physical condition and emotions.

[1018] The following system configuration will be described as an embodiment of the present invention.

[1019] System Configuration

[1020] This system includes means for collecting voice data (animal cries), means for collecting characteristics of pet animals (personality, cries, etc.), means for identifying the type of animal, means for collecting general data on animal characteristics, means for determining the condition of pets from their cries based on the collected data, means for notifying owners based on the pet's condition, and means for ordering pet food.

[1021] Hardware and Software Used

[1022] Hardware: Smartphone, microphone

[1023] Software: Python, SpeechRecognition library, Requests library

[1024] Data processing and calculation

[1025] Acquisition and analysis of audio data

[1026] The server records the pet's cries using the smartphone's microphone. The recorded voice data is converted to text data using the SpeechRecognition library. It is then analyzed using a generative AI model to understand the pet's condition.

[1027] Collecting and analyzing owner comments

[1028] The server records the owner's words using the smartphone's microphone. The recorded voice data is converted to text data using the SpeechRecognition library. It is then analyzed using a generative AI model to generate a message to be conveyed to the pet.

[1029] Pet food order

[1030] The server notifies the owner based on the pet's condition. For example, if the pet cries "I'm hungry," the server generates a prompt to order pet food and notifies the owner. When the owner orders pet food through the app, recommended products are displayed taking into account the pet's preferences and allergy information.

[1031] Examples

[1032] Analyzing pet cries: If a pet cries "I'm hungry," the server analyzes the cry and notifies the owner with a message that "your pet is hungry."

[1033] Analyzing owner's words: When the owner says "let's play", the server analyzes the words and conveys the message in a way that the pet can understand.

[1034] Examples of prompt statements

[1035] "Analyze your pet's cries and let us know their condition."

[1036] "Analyze the owner's words and generate a message to convey to the pet."

[1037] In this way, the present invention makes it possible to accurately grasp the condition of a pet and respond quickly, facilitate communication between the owner and the pet, and automate the ordering of pet food.

[1038] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1039] Step 1:

[1040] The server records the pet's cry using the smartphone's microphone. The input is the pet's cry, and the output is an audio data file. This audio data file is used for subsequent analysis processing.

[1041] Step 2:

[1042] The server converts the recorded voice data into text data using the SpeechRecognition library. The input is the voice data file, and the output is text data. This text data is input to the generative AI model.

[1043] Step 3:

[1044] The server analyzes the text data using a generative AI model to understand the pet's state. The input is text data, and the output is information that indicates the pet's state. For example, the state "hungry" is analyzed.

[1045] Step 4:

[1046] The server notifies the owner based on the pet's status. The input is information indicating the pet's status, and the output is a notification message to the owner. For example, a message "your pet is hungry" is sent to the owner.

[1047] Step 5:

[1048] The server records the owner's words using the smartphone's microphone. The input is the owner's words, and the output is an audio data file. This audio data file is used for subsequent analysis processing.

[1049] Step 6:

[1050] The server converts the recorded voice data of the owner into text data using the SpeechRecognition library. The input is a voice data file, and the output is text data. This text data is input to the generative AI model.

[1051] Step 7:

[1052] The server uses a generative AI model to analyze the text data and generate a message to be conveyed to the pet. The input is text data, and the output is a message to be conveyed to the pet. For example, the command "Let's play" is analyzed.

[1053] Step 8:

[1054] The server provides a means for ordering pet food based on the pet's condition. The input is information indicating the pet's condition, and the output is pet food ordering information. For example, if the pet cries "I'm hungry," an order for pet food is automatically placed.

[1055] Step 9:

[1056] The server displays recommended products to the pet owner, taking into account the pet's preferences and allergy information. The input is the pet's preferences and allergy information, and the output is a list of recommended products. This allows the pet owner to select an appropriate pet food.

[1057] Step 10:

[1058] The server notifies the pet owner that the pet food order is complete. The input is the order completion information, and the output is a notification message to the pet owner. For example, a message saying "Your pet food order is complete" is sent to the pet owner.

[1059] Example 3

[1060] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[1061] Conventional systems for understanding the health and emotional state of pets only analyze voice data and do not take into account behavioral data, making it difficult to accurately understand the condition. In addition, there is a lack of a way to quickly notify the owner of the analysis results, making it difficult to discover abnormalities in pets at an early stage.

[1062] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1063] In this invention, the server includes a means for collecting voice data, a means for collecting animal characteristics, a means for identifying the type of animal, a means for collecting general animal characteristic data, a means for grasping the state of the pet from the cries based on the collected data, a means for collecting animal behavior data, a means for analyzing the collected behavior data, and a means for notifying the analysis result. By analyzing both the voice data and the behavior data, it becomes possible to grasp the health and emotional state of the pet more accurately and promptly notify the owner.

[1064] "Audio Data" refers to animal sounds and other audio information.

[1065] "Animal characteristics" refers to individual characteristics such as an animal's personality, sounds, behavior patterns, etc.

[1066] "Type of animal" refers to a classification of animals, such as dogs, cats, birds, etc.

[1067] "General animal characteristic data" refers to data such as personality and behavioral patterns common to a particular type of animal.

[1068] "Animal condition" refers to the animal's physical and emotional state.

[1069] "Behavioral Data" refers to information regarding an animal's movements and behavior patterns.

[1070] "Analysis results" refers to the diagnosis and evaluation output by the AI ​​model based on the collected voice data and behavioral data.

[1071] "Notification means" refers to a method or device for communicating the analysis results to the owner.

[1072] The present invention relates to a system for understanding the health and emotional state of a pet. A specific embodiment of the system will be described below.

[1073] First, the user uses a device such as a smartphone or tablet to collect the sounds and behavior of their pet. The device is equipped with a microphone and camera, and these devices are used to capture voice data and behavior data in real time. For example, when a user starts the smartphone app, points it at their pet and presses the "start recording" button, the microphone starts collecting the pet's sounds. At the same time, the camera captures the pet's movements.

[1074] The device then transmits the collected data to a server, where it is sent in real time over Wi-Fi or mobile data, and the data is encrypted to ensure security.

[1075] The server inputs the received voice and behavior data into an AI model, which is built using machine learning frameworks such as TensorFlow and PyTorch, to analyze the frequency and volume of the pet's cries, as well as the speed and direction of its movements, to detect abnormal patterns and patterns of delight.

[1076] The analysis results are sent from the server to the device, and the user can check the results through a smartphone app. For example, the server will send a warning message such as "Your pet may be sick." If the pet is happy, it will send a message such as "Your pet is happy."

[1077] As a concrete example, the following prompt sentence could be input to a generative AI model:

[1078] "Analyze your pet's vocalizations and behavioral data to determine their health and emotional state. Alert you if your pet is sick and let you know if they're happy."

[1079] Using this prompt sentence, the AI ​​model can accurately analyze the pet's condition and provide appropriate feedback to the user.

[1080] As described above, the present invention makes it possible to grasp the health and emotional state of a pet more accurately and promptly notify the owner by analyzing both the voice data and the behavior data. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1081] Step 1:

[1082] The user starts the smartphone app and collects the sounds and behavior of the pet. Specifically, when the user presses the "Start Recording" button, the device's microphone starts collecting the pet's sounds. At the same time, the camera captures the pet's movements. The input is the pet's sounds and behavior, and the output is the collected voice data and behavior data.

[1083] Step 2:

[1084] The device transmits the collected voice data and behavioral data to the server. Specifically, the device transmits data in real time using Wi-Fi or mobile data communication. The input is the collected voice data and behavioral data, and the output is the data transmitted to the server.

[1085] Step 3:

[1086] The server inputs the received voice data and behavioral data into an AI model. Specifically, the server inputs data into an AI model built using machine learning frameworks such as TensorFlow and PyTorch. The input is the voice data and behavioral data sent to the server, and the output is the analysis result by the AI ​​model.

[1087] Step 4:

[1088] The server judges the pet's health and emotional state based on the analysis results of the AI ​​model. Specifically, the AI ​​model analyzes the frequency and volume of the pet's cries, the speed and direction of its movements, etc., to detect abnormal patterns and patterns of joy. The input is the analysis results of the AI ​​model, and the output is the judgment of the pet's health and emotional state.

[1089] Step 5:

[1090] The server transmits the analysis results to the terminal. Specifically, the server transmits a message generated based on the analysis results to the terminal. The input is the judgment result of the pet's health condition and emotional state, and the output is the message transmitted to the terminal.

[1091] Step 6:

[1092] The user checks the analysis results through the terminal. Specifically, the user opens the smartphone app and checks the notified message. For example, a warning message such as "Your pet may be sick" or a message such as "Your pet is happy" may be displayed. The input is the message sent to the terminal, and the output is the analysis result checked by the user.

[1093] (Application example 3)

[1094] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[1095] It is important for pet owners to know their pet's health and emotional state, but conventional methods make it difficult to detect abnormalities in pets at an early stage. In addition, there are limited means to accurately know whether a pet is happy. This can lead to insufficient health management and understanding of the pet's emotions, which can lead to a decline in the quality of the pet's life. Furthermore, there is no means to quickly notify owners of abnormalities in their pets, which can delay emergency response.

[1096] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1097] In this invention, the server includes means for collecting voice data (animal cries), means for collecting characteristics of pet animals (personality, cries, etc.), means for identifying the type of animal, means for collecting general animal characteristic data, means for grasping the pet's condition from the cries based on the collected data, means for analyzing the pet's condition and sending a notification when an abnormality is detected, means for grasping the pet's health and emotional state in real time, means for issuing a warning to the owner when an abnormality is detected, and means for conveying information about the pet to the owner when the pet is happy. This makes it possible to quickly and accurately grasp the pet's health and emotional state, and to immediately notify the owner when an abnormality occurs.

[1098] "Audio data" refers to data that records sounds such as animal cries in digital format.

[1099] "Animal characteristics" refers to the animal's unique characteristics, such as personality or sounds.

[1100] "Means for identifying animal species" refers to methods or techniques for identifying the species of an animal.

[1101] "General animal characteristic data" refers to data about widely known characteristics of animals.

[1102] "Means of understanding a pet's condition" refers to methods of analyzing a pet's health and emotional state based on collected data.

[1103] "Means for analyzing a pet's condition" refers to a method for analyzing a pet's cries and behavior to assess its condition.

[1104] "Means for sending a notification when an abnormality is detected" refers to a method for issuing a warning to the pet owner when an abnormality is detected in the pet.

[1105] "Means of understanding a pet's health and emotional state in real time" refers to a method of instantly analyzing a pet's condition and providing the results in real time.

[1106] "Means for issuing a warning to the pet owner when an abnormality is detected" refers to a method for issuing a warning to the pet owner when an abnormality is detected in the pet.

[1107] The term "means for conveying information about a happy pet to its owner" refers to a method for detecting a happy state of a pet and conveying the information to its owner.

[1108] As an embodiment of the present invention, a pet security alert system will be described as an example. This system analyzes the cries and behavior of pets in real time and notifies the owner if an abnormality is detected.

[1109] System Configuration

[1110] Hardware:

[1111] Smartphone

[1112] microphone

[1113] software:

[1114] TensorFlow

[1115] librosa

[1116] smtplib

[1117] Data processing and calculation

[1118] Audio data collection:

[1119] The device (smartphone) uses a microphone to collect the sounds of pets, and the audio data is recorded in digital format.

[1120] Analysis of audio data:

[1121] The server converts the collected voice data into MFCC (Mel-Frequency Cepstral Coefficients) using librosa, which allows the extraction of features from the voice data.

[1122] Analysis by AI model:

[1123] The server feeds the voice data into a generative AI model pre-trained with TensorFlow to analyze the pet's health and emotional state, which is then used to detect abnormalities in the pet.

[1124] Sending notifications:

[1125] If an abnormality is detected, the server uses smtplib to send an email notification to the owner, containing detailed information about the pet's abnormal condition.

[1126] Examples

[1127] For example, you can record your pet's cries and input the audio data into the application. The AI ​​model analyzes the audio data, and if it detects an abnormality, it will notify the owner by email. The notification will include a message such as, "An abnormality has been detected with your pet. Please check immediately."

[1128] Examples of prompt statements

[1129] Examples of prompt sentences to be input to a generative AI model include the following:

[1130] Write a Python program that analyzes pet sound data and notifies owners if an abnormality is detected. Use TensorFlow and librosa to analyze the sound data, and smtplib to send email notifications.

[1131] In this way, a system can be provided that can grasp the health and emotional state of a pet in real time and quickly respond if an abnormality occurs.

[1132] The flow of the specific process in the application example 3 will be described with reference to FIG.

[1133] Step 1:

[1134] Audio data collection

[1135] The terminal (smartphone) collects the pet's cries using a microphone. The input is the pet's cries, and the output is digital audio data. This audio data is used in the subsequent analysis steps.

[1136] Step 2:

[1137] Audio data preprocessing

[1138] The server converts the collected voice data into MFCC (Mel-Frequency Cepstral Coefficients) using librosa. The input is digital voice data, and the output is MFCC features. Through this conversion, the features of the voice data are extracted.

[1139] Step 3:

[1140] Analysis by AI model

[1141] The server inputs the MFCC features into a generative AI model pre-trained using TensorFlow to analyze the pet's health and emotional state. The input is the MFCC features, and the output is a prediction result about the pet's condition. This analysis evaluates the pet's abnormalities and emotional state.

[1142] Step 4:

[1143] Anomaly detection

[1144] The server judges whether an abnormality has been detected in the pet based on the prediction result of the AI ​​model. The input is the prediction result of the AI ​​model, and the output is information on the presence or absence of an abnormality. If an abnormality is detected, it proceeds to the next step.

[1145] Step 5:

[1146] Sending notifications

[1147] The server uses smtplib to send notifications to the pet owner via email. The input is the pet owner's email address and information about the pet's abnormal condition. The output is the notification email sent, which contains detailed information about the pet's abnormal condition.

[1148] Step 6:

[1149] Real-time status

[1150] The server monitors the pet's health and emotional state in real time and provides the information to the owner as needed. The input is continuously collected voice data, and the output is real-time status information, allowing the owner to always be aware of the pet's condition.

[1151] In this way, it is possible to quickly and accurately grasp the pet's health and emotional state, and to immediately notify the owner if any abnormality occurs.

[1152] In addition, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1153] "Example 1"

[1154] The system according to claim 4 includes an emotion engine that recognizes the emotion of the user. The emotion engine analyzes the emotion of the user from the tone of voice, facial expression, behavior, etc. of the user. For example, when the user is angry with a pet, the emotion engine recognizes the anger and adjusts the instruction to the pet. When the user is happy, the emotion engine recognizes the joy and adjusts the instruction to the pet. This allows smoother communication between the pet and the user.

[1155] "Example 2"

[1156] The system described in claim 5 includes a means for utilizing AI to realize a conversation between a person and a pet, and an emotion engine for recognizing the user's emotions. This emotion engine analyzes the user's emotions from the user's tone of voice, facial expressions, behavior, etc., and feeds the results back to the AI. The AI ​​adjusts instructions to the pet based on this feedback. For example, when the user is angry at the pet, the AI ​​recognizes the anger and adjusts instructions to the pet. Also, when the user is happy, the AI ​​recognizes the joy and adjusts instructions to the pet. This makes communication between the pet and the user smoother.

[1157] "Example 3"

[1158] The system according to claim 6 includes a means for grasping the state of a pet and an emotion engine for recognizing the emotion of a user. The emotion engine analyzes the emotion of a user from the tone of voice, facial expression, behavior, etc. of the user, and feeds back the result to the means for grasping the state of a pet. The means for grasping the state of a pet adjusts the state of the pet based on this feedback. For example, when the user is angry at the pet, the means for grasping the state of a pet recognizes the anger and adjusts the behavior of the pet. Also, when the user is happy, the means for grasping the state of a pet recognizes the joy and adjusts the behavior of the pet. This allows smoother communication between the pet and the user.

[1159] The process flow of each embodiment will be described below.

[1160] "Example 1"

[1161] Step 1: The emotion engine analyzes the user's emotions based on their tone of voice, facial expressions, behavior, etc.

[1162] Step 2: Adjust instructions to your pet based on the emotion engine's analysis.

[1163] Step 3: The system operates to facilitate smooth communication between the pet and the user.

[1164] "Example 2"

[1165] Step 1: The emotion engine analyzes the user's emotions based on their tone of voice, facial expressions, behavior, etc.

[1166] Step 2: The emotion engine's analysis results are fed back to the AI.

[1167] Step 3: The AI ​​uses this feedback to adjust its instructions to the pet.

[1168] Step 4: The system operates to facilitate smooth communication between the pet and the user.

[1169] "Example 3"

[1170] Step 1: The emotion engine analyzes the user's emotions based on their tone of voice, facial expressions, behavior, etc.

[1171] Step 2: The results of the emotion engine's analysis are fed back into a means of understanding the pet's condition.

[1172] Step 3: A means to understand the pet's condition will be used to adjust the pet's condition based on this feedback.

[1173] Step 4: The system operates to facilitate smooth communication between the pet and the user.

[1174] Example 1

[1175] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[1176] Conventional pet management systems simply record the sounds and behavior of pets, and have the problem that it is difficult to respond appropriately based on the pet's condition or the user's emotions. In addition, there is a lack of means for smooth communication between the user and the pet. This makes it difficult to properly grasp the pet's health condition and emotional state and to give instructions to the pet according to the user's emotions.

[1177] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1178] In this invention, the server includes a means for collecting voice data, a means for collecting characteristics of the pet animal, a means for identifying the type of animal, a means for collecting general characteristic data of the animal, a means for grasping the state of the pet from the cry based on the collected data, a means for recognizing the user's emotions, and a means for adjusting instructions to the pet based on the user's emotions. This makes it possible to grasp the state of the pet and the user's emotions in real time and to give appropriate instructions to the pet.

[1179] "Audio data" refers to data that records sounds, such as animal cries or the user's voice, in digital format.

[1180] "Characteristics of pets" refers to specific information about pets, such as their personalities, sounds, and behavioral patterns.

[1181] "Type of animal" is information indicating the type of pet, such as dog, cat, bird, etc.

[1182] "General animal characteristic data" is data such as personality and behavior patterns common to a specific animal species.

[1183] "Pet status" is information that indicates the current state of the pet, such as the pet's health condition or emotional state.

[1184] "User emotion" refers to the user's emotional state as analyzed from the user's tone of voice, facial expression, behavior, etc.

[1185] "Adjusting instructions" means changing the instructions given to the pet based on the user's emotions and the state of the pet.

[1186] "Collecting means" refers to devices and methods for acquiring voice data, animal characteristic data, etc.

[1187] An "identification means" is a device or method for identifying the type or characteristics of an animal based on the collected data.

[1188] "Analyzing means" refers to a device or method for analyzing collected data and understanding the pet's condition and the user's emotions.

[1189] The present invention is a system that collects and analyzes data on the cries and behavior of a pet to grasp the state of the pet and adjusts instructions to the pet based on the user's emotions. A specific embodiment of this system will be described below.

[1190] Audio data collection

[1191] Users collect their pet's cries using the microphone on their smartphone. The collected audio data is sent from the device to the server in real time. Specifically, the user starts the smartphone app and presses the "Start Recording" button to start collecting audio data.

[1192] Analysis of audio data

[1193] The server uses voice analysis software to analyze the voice data. Specifically, it converts the collected voice data into text data using a common voice analysis API. For example, Google Cloud Speech-to-Text API can be used. Based on the analysis results, it identifies whether the voice is the sound of a specific animal.

[1194] Collecting animal characteristic data

[1195] The user inputs the pet's personality and the characteristics of the voice it makes into the smartphone app. The device also uses sensors to observe the pet's behavior. Specifically, the device uses behavior observation sensors such as Nest Cam to collect the pet's behavior data. The collected data is then sent to a server.

[1196] Identifying Animal Species

[1197] The user inputs the type of pet into the app. The terminal then uses image recognition technology to identify the type of pet. Specifically, a general image recognition API can be used. For example, Google Cloud Vision API can be used to analyze the image of the pet and identify the type of animal.

[1198] Analyzing data and understanding your pet's condition

[1199] The server uses AI models to analyze the collected data, specifically machine learning frameworks such as TensorFlow to predict the pet's condition, and then understands the pet's health and emotional state based on the analysis results.

[1200] User Emotion Recognition

[1201] The device uses emotion analysis software to recognize the user's emotions. Specifically, it uses the Emotion API to analyze the user's tone of voice and facial expressions. Based on the analysis results, the server recognizes whether the user is angry or happy with the pet.

[1202] Adjusting pet instructions

[1203] The server adjusts the instructions to the pet based on the user's emotions: if the user is angry, the server generates a gentle instruction, and if the user is happy, the server generates a reinforced instruction. The generated instruction is transmitted to the pet through the terminal.

[1204] Examples

[1205] Audio data collection: The user launches the app on their smartphone and presses the “start recording” button to record the sound of their dog barking.

[1206] Voice data analysis: The server uses the Google Cloud Speech-to-Text API to analyze the recorded voice data and obtain the text data "Woof woof."

[1207] Animal characteristic data collection: The user enters "My dog's personality is gentle" into the app's input form, and the device uses Nest Cam to collect the dog's behavior data.

[1208] Animal type identification: A user uploads a photo of their dog to the app, and the device uses the Google Cloud Vision API to identify it as a "Golden Retriever."

[1209] Data analysis and pet condition understanding: The server uses TensorFlow to analyze the collected data and determine whether the dog is feeling stressed.

[1210] Recognizing user emotions: The device uses the Emotion API to analyze the user's tone of voice and facial expressions and recognize that the user is angry.

[1211] Pet command adjustment: The server analyzes the user's emotional data and generates gentle commands such as "sit," which the device then relays to the dog via voice.

[1212] Examples of prompt statements

[1213] "Collect dog barks and analyze them with a common voice analysis API. Based on the analysis results, use a machine learning framework to understand the dog's condition, and use emotion analysis software to recognize the user's emotions and adjust instructions for the pet."

[1214] In this way, the system specifically executes each processing step to facilitate smooth communication between the pet and the user.

[1215] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1216] Step 1:

[1217] Audio data collection

[1218] The user collects the sounds of their pet using the microphone on their smartphone. Specifically, the user starts the smartphone app and presses the "Start Recording" button to start collecting audio data.

[1219] Input: Pet noises

[1220] Output: Recorded audio data

[1221] Step 2:

[1222] Sending audio data

[1223] The device transmits the collected voice data to a server in real time. Specifically, the smartphone app uploads the recorded voice data to the server via the Internet.

[1224] Input: Recorded audio data

[1225] Output: Audio data sent to the server

[1226] Step 3:

[1227] Analysis of audio data

[1228] The server uses voice analysis software to analyze the voice data. Specifically, the server converts the collected voice data into text data using a common voice analysis API. For example, the Google Cloud Speech-to-Text API can be used.

[1229] Input: Audio data sent to the server

[1230] Output: Parsed text data

[1231] Step 4:

[1232] Collecting animal characteristic data

[1233] The user inputs the pet's personality and the characteristics of the voice it makes into the smartphone app. The device also uses sensors to observe the pet's behavior. Specifically, the device uses behavior observation sensors such as Nest Cam to collect the pet's behavior data. The collected data is then sent to a server.

[1234] Input: Pet's personality, vocalizations, and behavior data

[1235] Output: Feature and behavioral data sent to the server.

[1236] Step 5:

[1237] Identifying Animal Species

[1238] The user inputs the type of pet into the app. The terminal then uses image recognition technology to identify the type of pet. Specifically, a general image recognition API can be used. For example, Google Cloud Vision API can be used to analyze the image of the pet and identify the type of animal.

[1239] Input: Pet picture

[1240] Output: Analyzed animal species

[1241] Step 6:

[1242] Analyzing data and understanding your pet's condition

[1243] The server uses AI models to analyze the collected data, specifically machine learning frameworks such as TensorFlow to predict the pet's condition, and then understands the pet's health and emotional state based on the analysis results.

[1244] Input: feature data, behavior data, animal type

[1245] Output: Analyzed pet status

[1246] Step 7:

[1247] User Emotion Recognition

[1248] The device uses emotion analysis software to recognize the user's emotions. Specifically, it uses the Emotion API to analyze the user's tone of voice and facial expressions. Based on the analysis results, the server recognizes whether the user is angry or happy with the pet.

[1249] Input: The user's tone of voice and facial expressions

[1250] Output: Parsed user sentiment

[1251] Step 8:

[1252] Adjusting pet instructions

[1253] The server adjusts the instructions to the pet based on the user's emotions: if the user is angry, the server generates a gentle instruction, and if the user is happy, the server generates a reinforced instruction. The generated instruction is transmitted to the pet through the terminal.

[1254] Input: Analyzed pet state, user emotion

[1255] Output: Adjusted instructions

[1256] In this way, the system specifically executes each processing step to facilitate smooth communication between the pet and the user.

[1257] (Application example 1)

[1258] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[1259] Conventional pet care systems only analyzed the pet's cries and behavior, and were unable to provide advice or services that took the user's emotions into account. This resulted in poor communication between the pet and the user, making it difficult to properly manage the pet's health and emotional state. In addition, it was difficult to provide appropriate services according to the pet's condition in brick-and-mortar stores.

[1260] The identification process by the identification processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means. In this invention, the server includes a means for collecting voice data (animal cries), a means for collecting characteristics of pet animals (personality, cries, etc.), a means for identifying the type of animal, a means for collecting general animal characteristic data, a means for grasping the pet's condition from its cries based on the collected data, a means for recognizing the user's emotions, a means for generating advice based on the pet's condition and the user's emotions, and a means for providing the generated advice to the user. This makes it possible to comprehensively analyze the pet's condition and the user's emotions and provide appropriate advice and services.

[1261] "Audio data" refers to audio information such as animal cries.

[1262] "Characteristics of pet animals" refers to information about the characteristics of the animals kept by the owner, such as their personalities and the sounds they make.

[1263] "Means for identifying the type of animal" refers to a method for identifying the type of animal using image recognition technology or input from the owner.

[1264] "General animal characteristic data" refers to characteristic information about general animals obtained from a database or the like.

[1265] "Means of understanding a pet's condition" refers to methods of analyzing a pet's health and emotional state based on collected voice data and animal characteristic data.

[1266] "Means for recognizing a user's emotions" refers to a method for identifying a user's emotions by analyzing the user's tone of voice, facial expression, behavior, etc.

[1267] The "means for generating advice" refers to a method for generating appropriate advice or services based on the pet's condition and the user's emotions.

[1268] The term "means for providing the generated advice to the user" refers to a method for communicating the generated advice or service to the user.

[1269] A system for implementing this invention includes means for collecting audio data (animal cries), means for collecting characteristics of pet animals (personality, cries, etc.), means for identifying the type of animal, means for collecting general animal characteristic data, means for understanding the condition of the pet from its cries based on the collected data, means for recognizing the user's emotions, means for generating advice based on the pet's condition and the user's emotions, and means for providing the generated advice to the user.

[1270] Hardware and Software Used

[1271] Hardware:

[1272] Microphone: Used to collect audio data.

[1273] Camera: Used to collect video data of the user.

[1274] software:

[1275] speech_recognition library: Used to perform speech recognition.

[1276] cv2 (OpenCV): Used to collect and analyze video data.

[1277] emotion_recognition: An emotion recognition engine for recognizing user emotions.

[1278] pet_behavior_analysis: A pet behavior analysis engine for analyzing pet sounds and behavior.

[1279] advice_generator: An advice generation engine for generating advice based on the pet's state and the user's emotions.

[1280] System processing overview

[1281] 1. Audio data collection:

[1282] The server collects the pet's cries using a microphone, and the collected audio data is analyzed using the speech_recognition library.

[1283] 2. Video data collection:

[1284] The server collects the user's video data using a camera, and the collected video data is analyzed using cv2 (OpenCV).

[1285] 3. Pet Status Analysis:

[1286] The server analyzes the collected voice data using the pet_behavior_analysis engine to determine the pet's condition (e.g., hunger, stress, happiness, etc.).

[1287] 4. User emotion recognition:

[1288] The server analyzes the collected video data using the emotion_recognition engine and recognizes the user's emotions (e.g., anger, joy, sadness, etc.).

[1289] 5. Generating Advice:

[1290] The server generates appropriate advice using the advice_generator engine based on the pet's condition and the user's emotions.

[1291] 6. Providing advice:

[1292] The server provides the generated advice to the user, for example, suggesting items to help the pet relax if the pet is stressed.

[1293] Examples

[1294] If your pet is stressed:

[1295] The server generates advice such as "Your pet is stressed. Try using a toy that will relax your pet" and provides it to the user.

[1296] If the user is angry:

[1297] The server generates advice such as "The user is angry. Please be kind to your pet" and provides it to the user.

[1298] Examples of prompt statements

[1299] Analyze the pet's cry data and the user's video data, and generate appropriate advice based on the pet's condition and the user's emotions. If the pet is stressed, suggest items that will help the user relax, and if the user is angry, advise the user to be gentle with the pet.

[1300] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1301] Step 1:

[1302] The server collects the pet's cries using a microphone. The collected voice data is analyzed using the speech_recognition library. The input is the pet's cries, and the output is the analyzed voice data. Through this analysis, the features of the pet's cries are extracted.

[1303] Step 2:

[1304] The server collects the user's video data using a camera. The collected video data is analyzed using cv2 (OpenCV). The input is the user's video data, and the output is the analyzed video data. Through this analysis, the user's facial expressions and behavioral features are extracted.

[1305] Step 3:

[1306] The server analyzes the collected voice data with the pet_behavior_analysis engine to understand the pet's state. The input is the analyzed voice data, and the output is the pet's state (e.g., hunger, stress, joy, etc.). This analysis identifies the pet's health and emotional state.

[1307] Step 4:

[1308] The server analyzes the collected video data with the emotion_recognition engine to recognize the user's emotions. The input is the analyzed video data, and the output is the user's emotion (e.g., anger, joy, sadness, etc.). This analysis identifies the user's emotional state.

[1309] Step 5:

[1310] The server generates appropriate advice using the advice_generator engine based on the pet's state and the user's emotion. The input is the pet's state and the user's emotion, and the output is the generated advice. This generation creates specific advice according to the pet and user's situation.

[1311] Step 6:

[1312] The server provides the generated advice to the user. The input is the generated advice, and the output is the advice provided to the user. This allows the user to take appropriate action according to the state of the pet.

[1313] Example 2

[1314] Next, a description will be given of Example 2 of the embodiment 2. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[1315] Conventional methods of communication with pets have the problem that it is difficult to accurately understand the cries and behavior of pets, and it takes time to grasp the pet's condition and requests. In addition, when a user gives instructions to a pet, the pet often does not understand the instructions, which makes it difficult to communicate smoothly.

[1316] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for collecting voice data, a means for collecting animal characteristics, a means for identifying the type of animal, a means for collecting general animal characteristic data, a means for grasping the state of the pet from the cry based on the collected data, a means for analyzing the collected voice data, a means for notifying the user of the analysis result, a means for collecting the user's voice instruction, a means for analyzing the voice instruction data, and a means for notifying the pet of the analysis result. This allows the pet's cry and the user's voice instruction to be accurately analyzed, enabling smooth communication between the pet and the user.

[1317] "Audio data" refers to data that is a digital recording of sounds made by a pet or a user.

[1318] "Animal characteristics" refers to information that refers to characteristics unique to each individual animal, such as a pet's personality or the sound it makes.

[1319] "Animal type" refers to the biological classification to which the pet belongs, such as dog or cat.

[1320] "General animal characteristic data" is data that includes information such as behaviors and sounds common to a particular type of animal.

[1321] The "means for understanding the state of a pet from its cry" refers to a method or device for analyzing the cry of a pet and identifying the state or request of the pet as indicated by the cry.

[1322] The "means for analyzing collected voice data" refers to a method or device for converting collected voice data into text data and analyzing the content thereof.

[1323] The "means for notifying the user of the analysis results" refers to a method or device for conveying the analyzed information to the user.

[1324] The "means for collecting user's voice instructions" refers to a method or device for recording voice instructions given by a user and collecting the data.

[1325] The "means for analyzing voice instruction data" refers to a method or device for converting a user's voice instruction into text data and analyzing the content of the text data.

[1326] The "means for communicating the analysis results to the pet" refers to a method or device for communicating the analyzed user's instructions in a form that the pet can understand.

[1327] The present invention is a system for realizing smooth communication between a pet and a user. This system includes a series of processes for collecting and analyzing voice data and notifying the results to the user and the pet.

[1328] Hardware and Software Used

[1329] Subject: Server

[1330] The server uses a speech recognition API and a natural language processing (NLP) model to analyze the pet's cries and the user's voice instructions. Specifically, it uses Google Cloud's Speech-to-Text API to convert the voice data into text data, and then analyzes the text data using the NLP model. As a result of the analysis, it identifies the pet's status and requests, as well as the user's instructions, and conveys them to the user and the pet in an appropriate manner.

[1331] Subject: Terminal

[1332] The terminal is equipped with a microphone to collect the pet's cries and the user's voice instructions. The collected voice data is sent to a server via the Internet. It also has a display and speaker to notify the user of the analysis results received from the server. For example, a smartphone or tablet can play this role.

[1333] Subject: User

[1334] The user inputs the sound of the pet's cry into the terminal and also issues his or her own voice instructions to the terminal. For example, if the pet barks "woof woof," the user inputs the sound into the terminal, and the terminal transmits the voice data to the server. Similarly, if the user says "let's play," the voice instruction is transmitted to the server through the terminal.

[1335] Examples

[1336] Example 1: Pet sound analysis

[1337] 1. My pet barks "woof woof."

[1338] 2. The device's microphone captures the barking sound.

[1339] 3. The device sends the bird's cry data to the server.

[1340] 4. The server converts the audio data into text data using Google Cloud's Speech-to-Text API.

[1341] 5. The server uses an NLP model to analyze the text data and generate a message saying "Your pet is hungry."

[1342] 6. The server generates a message and sends it to the terminal.

[1343] 7. The device displays a notification to the user that "Pet is hungry."

[1344] Example 2: User voice command analysis

[1345] 1. The user says "Let's play."

[1346] 2. The device's microphone captures the audio.

[1347] 3. The device sends the voice data to the server.

[1348] 4. The server converts the audio data into text data using Google Cloud's Speech-to-Text API.

[1349] 5. The server uses an NLP model to analyze the text data and convert the command "let's play" into a form the pet can understand.

[1350] 6. The server sends the converted instructions to the terminal.

[1351] 7. The device will play a specific sound and tell your pet to "play."

[1352] Examples of prompt statements

[1353] "If your pet barks 'woof woof', we can analyze the sound and tell you what condition your pet is in."

[1354] "When the user says 'Let's play,' analyze that voice and generate commands for the pet."

[1355] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1356] Step 1: Collect your pet's sounds

[1357] Subject: Terminal

[1358] The device collects the pet's barking sounds through a microphone. For example, if the pet barks "woof woof," the device's microphone captures the sound. The input is the pet's barking sound, and the output is digital audio data.

[1359] Step 2: Send the bird sound data to the server

[1360] Subject: Terminal

[1361] The terminal transmits the collected bird call data to a server via the Internet. The data is transmitted in audio file format (e.g., WAV format). The input is digital audio data, and the output is the audio data transmitted to the server.

[1362] Step 3: Analyze bird call data on the server

[1363] Subject: Server

[1364] The server converts the received bark data into text data using a speech recognition API. Specifically, it uses Google Cloud's Speech-to-Text API. The input is voice data, and the output is text data. The text data is then analyzed using an NLP model to identify the pet's status and requests. For example, it may analyze the bark "woof woof" to mean "I'm hungry." The input is text data, and the output is the analysis result.

[1365] Step 4: Notify the user of the analysis results

[1366] Subject: Server

[1367] The server sends the analysis result to the user's device. For example, it generates a message such as "your pet is hungry" and notifies the user's smartphone. The input is the analysis result, and the output is the notification message sent to the user's device.

[1368] Step 5: Collect user's voice commands

[1369] Subject: Terminal

[1370] The terminal collects the user's voice instructions through a microphone. For example, when the user says "Let's play," the device captures the voice. The input is the user's voice instruction, and the output is the digital voice data.

[1371] Step 6: Send voice command data to the server

[1372] Subject: Terminal

[1373] The terminal transmits the collected voice instruction data to a server via the Internet. The data is transmitted in audio file format (e.g., WAV format). The input is audio data in digital format, and the output is the audio data transmitted to the server.

[1374] Step 7: Parse the voice command data on the server

[1375] Subject: Server

[1376] The server converts the received voice instruction data into text data using a voice recognition API. Specifically, it uses Google Cloud's Speech-to-Text API. The input is voice data, and the output is text data. The text data is then analyzed using an NLP model and converted into a form that the pet can understand. For example, the instruction "Let's play" is converted into a sound or action instruction that the pet can understand. The input is text data, and the output is the analysis result.

[1377] Step 8: Communicate the results to your pet

[1378] Subject: Terminal

[1379] Based on the analysis results received from the server, the device generates sounds and behavioral instructions that the pet can understand and communicates them to the pet. For example, the device can play a specific sound to communicate the command to the pet "Let's play." The input is the analysis results, and the output is the command to be communicated to the pet.

[1380] (Application example 2)

[1381] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[1382] Conventional methods of communication between pets and owners rely on the owner's intuitive understanding of the pet's cries and behavior, which makes accurate communication difficult. It is also difficult to accurately grasp the pet's health and emotional state, which can delay appropriate responses. Furthermore, in brick-and-mortar stores such as pet shops, staff are required to quickly grasp the pet's condition and take appropriate action, but currently there is a lack of effective means to do so.

[1383] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1384] In this invention, the server includes a means for collecting voice data (animal cries), a means for collecting characteristics of pet animals (personality, cries, etc.), a means for identifying the type of animal, a means for collecting general animal characteristic data, a means for grasping the state of the pet from the cries based on the collected data, a means including an emotion engine for recognizing the user's emotions, a means for adjusting instructions to the pet based on the user's emotions, a means for analyzing the pet's cries and grasping the state and requests of the pet, a means for converting human words into a form that the pet can understand, and a means including an application to be installed on a smartphone. This enables accurate communication between the pet and the owner, and allows the pet's health and emotional state to be quickly grasped and appropriate measures to be taken. Also, in a physical store, a store clerk can quickly grasp the state of the pet and take appropriate measures.

[1385] "Audio data" refers to sound information, including animal cries, recorded in digital format.

[1386] "Animal characteristics" refers to the individual characteristics of pet animals, such as their personalities and the sounds they make.

[1387] "Means for identifying animal species" refers to methods or techniques for distinguishing between specific animal species.

[1388] "General animal characteristic data" refers to information about the widely known characteristics of animals.

[1389] "Means for understanding a pet's condition from its cries" refers to methods and technologies for analyzing a pet's cries to understand its health and emotional state.

[1390] An "emotion engine that recognizes the user's emotions" refers to technology for analyzing emotions from the user's tone of voice, facial expressions, behavior, etc.

[1391] "Means for adjusting instructions to a pet based on the user's emotions" refers to a method or technology for changing the instructions to a pet depending on the user's emotional state.

[1392] "Means for analyzing a pet's cries and understanding the pet's condition and requests" refers to a method or technology for analyzing a pet's cries and understanding the pet's condition and requests.

[1393] "Means of converting human language into a form that pets can understand" refers to methods or technologies for converting human language into sounds and actions that pets can easily understand.

[1394] An "application installed on a smartphone" refers to a software program that runs on a smartphone.

[1395] A system for implementing this invention includes means for collecting voice data (animal cries), means for collecting characteristics of pet animals (personality, cries, etc.), means for identifying the type of animal, means for collecting general animal characteristic data, means for understanding the condition of a pet from its cries based on the collected data, means including an emotion engine for recognizing the user's emotions, means for adjusting instructions to the pet based on the user's emotions, means for analyzing the pet's cries and understanding the pet's condition and requests, means for converting human words into a form that the pet can understand, and means including an application to be installed on a smartphone.

[1396] The server has a database for collecting voice data and identifying the characteristics and type of animal. This allows it to analyze the cries of pets and understand their status and requests. Furthermore, using an emotion engine that recognizes the user's emotions, it analyzes the user's emotions from the tone of their voice, facial expressions, behavior, etc., and adjusts instructions to the pet based on the results.

[1397] The terminal (smartphone) sends the collected voice data to the server and receives the analysis results. The analysis results are displayed to inform the user of the pet's status and requests. When the user gives instructions to the pet, the instructions are converted into a form that the pet can understand and communicated to the pet.

[1398] As a concrete example, there is a scenario in which a pet shop clerk uses a smartphone to analyze the cries of pets and understand that the pet is saying "I'm hungry." Another scenario includes when the clerk says "Let's play," the command is conveyed to the pet in a way that is easy for the pet to understand.

[1399] Examples of prompts to be input to a generative AI model include the following:

[1400] "Analyze your pet's cries and let us know their condition."

[1401] "Translate human language into a form your pet can understand."

[1402] This system enables accurate communication between pets and their owners, allowing them to quickly grasp the health and emotional state of the pet and take appropriate action. It also enables store staff in physical stores to quickly grasp the pet's condition and take appropriate action.

[1403] The flow of the identification process in the application example 2 will be described with reference to FIG.

[1404] Step 1:

[1405] The server receives audio data (animal cries) sent from the device (smartphone). The input is audio data, and the output is data ready for analysis. In concrete terms, the server stores the audio data in digital format and passes it to the analysis engine.

[1406] Step 2:

[1407] The server analyzes the voice data and identifies the characteristics and type of animal. The input is the voice data received in step 1, and the output is information about the animal's type and characteristics. Specifically, the server uses a voice recognition algorithm to convert the sound into text, and then searches a database for the animal's type and characteristics based on that text.

[1408] Step 3:

[1409] The server understands the pet's condition and requests based on the collected animal characteristic data. The input is the animal characteristic data obtained in step 2, and the output is information about the pet's condition and requests. In concrete terms, the server runs an algorithm to estimate the pet's health condition and emotional state based on the analysis results.

[1410] Step 4:

[1411] The server uses an emotion engine that recognizes the user's emotions to analyze the user's emotions from the user's tone of voice, facial expressions, behavior, etc. The input is the user's voice data and video data, and the output is information about the user's emotional state. In concrete terms, the server executes an emotion recognition algorithm to analyze the user's emotions.

[1412] Step 5:

[1413] The server adjusts the instructions to the pet based on the user's emotional state. The input is the information about the user's emotional state obtained in step 4, and the output is the adjusted instructions. Specifically, the server executes an algorithm that changes the instructions depending on the user's emotions.

[1414] Step 6:

[1415] The server analyzes the pet's cries and understands the pet's status and requests. The input is the information about the pet's status and requests obtained in step 3, and the output is a message to notify the user. In concrete terms, the server generates a message to notify the user based on the analysis results.

[1416] Step 7:

[1417] The server converts human speech into a form the pet can understand. The input is the user's words, and the output is instructions converted into sounds and actions the pet can understand. In concrete terms, the server uses a generative AI model to translate the user's words into a form the pet can understand.

[1418] Step 8:

[1419] The terminal (smartphone) displays the analysis results and instructions received from the server to the user. The input is the message sent from the server, and the output is the information displayed to the user. In concrete terms, the terminal displays the received message on the screen and notifies the user.

[1420] Step 9:

[1421] The user checks the pet's status and requests through the terminal and takes necessary action. The input is the information displayed on the terminal, and the output is the user's actions. In concrete terms, the user takes appropriate action toward the pet based on the information displayed on the terminal.

[1422] Example 3

[1423] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[1424] Conventional systems for understanding the health and emotional state of pets mainly analyze only the pet's cries and behavior, and have the problem of being unable to adjust the pet's behavior in consideration of the user's emotions. In addition, there is a lack of means to facilitate smooth communication between the pet and the user. This has led to problems such as insufficient health management and understanding of the pet's emotions, making it difficult to communicate smoothly with the user.

[1425] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1426] In this invention, the server includes means for collecting voice data, means for collecting animal characteristics, means for identifying animal species, means for collecting general animal characteristic data, means for grasping the state of the pet from the cries based on the collected data, means for collecting the user's emotions, means for analyzing the user's emotions, and means for adjusting the behavior of the pet based on the results of the user's emotion analysis, thereby enabling the health and emotional state of the pet to be grasped more accurately and the behavior of the pet to be adjusted according to the user's emotions.

[1427] "Means for collecting voice data" refers to a function for collecting a pet's cries using a voice input device such as a microphone.

[1428] "Means of collecting animal characteristics" refers to the function of collecting individual characteristics such as a pet's personality or cries using sensors, cameras, etc.

[1429] "Means for identifying animal species" refers to algorithms or identification technologies that identify the species of pet based on collected data.

[1430] "Means for collecting general animal characteristic data" refers to the functionality for collecting data on general animal characteristics from databases or external sources.

[1431] "Means of understanding a pet's condition from its cries based on the collected data" refers to an algorithm that analyzes the collected voice data and characteristic data to determine the pet's health and emotional state.

[1432] "Means for collecting user emotions" refers to a function for collecting the user's voice and facial expressions using a microphone or camera.

[1433] "Means for analyzing user emotions" refers to algorithms and analytical technologies that analyze collected user voice and facial expression data to determine the user's emotional state.

[1434] "Means for adjusting the behavior of the pet based on the results of the user's emotion analysis" refers to a control algorithm or instruction function for appropriately adjusting the behavior of the pet based on the results of the user's emotion analysis.

[1435] A "generative AI model" refers to an artificial intelligence model that analyzes a pet's condition and the user's emotions based on collected data.

[1436] The present invention is a system for grasping the health and emotional state of a pet and adjusting the behavior of the pet according to the emotion of a user. A specific embodiment of this system will be described below.

[1437] Hardware and Software Used

[1438] Hardware: Microphone, camera, and sensors (to detect pet movements)

[1439] Software: Generative AI models (e.g., general artificial intelligence models), sentiment engines (e.g., general sentiment analysis APIs)

[1440] System Configuration

[1441] 1. The terminal is equipped with a microphone to collect the pet's cries and a camera to capture the pet's behavior. These devices collect the pet's cries and behavior data in real time.

[1442] 2. The terminal temporarily stores the collected data and transmits it to the server at regular intervals.

[1443] 3. The server inputs the received data into the generative AI model to analyze the pet's health and emotional state. The generative AI model determines the pet's condition based on the collected voice and behavioral data.

[1444] 4. The server sends the analysis results to the device, which notifies the user. For example, the device can display a message on a smartphone app saying, "Your pet may be unwell."

[1445] 5. The terminal is equipped with a microphone to collect the user's voice and a camera to capture the user's facial expressions. These devices collect the user's voice and facial expression data.

[1446] 6. The device sends the collected user data to the server, and the server uses an emotion engine to analyze the user's emotions.

[1447] 7. The server sends the user's emotion analysis results to the device, which then adjusts the pet's behavior. For example, if the user is angry, the device will instruct the pet to "calm down."

[1448] Examples

[1449] Pet Health Analysis:

[1450] Example: If a pet is making an unusual sound, the generative AI model can detect the change and notify the owner that their pet may be unwell.

[1451] User emotion recognition and pet behavior adjustment:

[1452] Example: If a user is angry at their pet, the emotion engine will recognize that anger and the means of understanding the pet's state will adjust the pet's behavior to calm it down.

[1453] Examples of prompt statements

[1454] Pet Health Analysis:

[1455] "Analyze your pet's sounds and behavior to determine their health condition."

[1456] User emotion recognition and pet behavior adjustment:

[1457] "Analyze the user's voice and facial expressions to recognize emotions and adjust your pet's behavior based on the results."

[1458] This system makes it possible to grasp the status of the pet and the user in real time and take appropriate measures. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1459] Step 1:

[1460] The device collects the sounds your pet makes.

[1461] Input: Pet noises

[1462] How it works: A microphone on the device collects your pet's sounds in real time.

[1463] Output: Collected audio data

[1464] Step 2:

[1465] The device captures your pet's behavior.

[1466] Input: pet behavior

[1467] How it works: The camera on the device captures your pet's movements in real time.

[1468] Output: Collected video data

[1469] Step 3:

[1470] The data collected by the terminal is temporarily stored and then sent to the server at regular intervals.

[1471] Input: Audio data, video data

[1472] Operation: The data collected by the terminal is temporarily stored in the internal memory and then sent to the server in batches at regular intervals.

[1473] Output: Data sent to the server

[1474] Step 4:

[1475] The data received by the server is input into a generative AI model to analyze the pet's health and emotional state.

[1476] Input: Audio data, video data

[1477] How it works: The server feeds data into a generative AI model that analyzes the pet's health and emotional state.

[1478] Output: Analysis results (pet's health and emotional state)

[1479] Step 5:

[1480] The server transmits the analysis results to the terminal, and the terminal notifies the user.

[1481] Input: Analysis results

[1482] How it works: The server sends the analysis results to the device, and the device sends a push notification to the user's smartphone.

[1483] Output: Analysis results reported to the user

[1484] Step 6:

[1485] The terminal collects user feedback.

[1486] Input: User's voice

[1487] How it works: A microphone on the device collects the user's voice.

[1488] Output: Collected audio data

[1489] Step 7:

[1490] The device captures the user's facial expression.

[1491] Input: User's facial expression

[1492] How it works: The device's camera captures the user's facial expressions.

[1493] Output: Collected video data

[1494] Step 8:

[1495] The terminal transmits the collected user data to the server.

[1496] Input: Audio data, video data

[1497] Operation: The device sends the collected data to the server.

[1498] Output: Data sent to the server

[1499] Step 9:

[1500] The server uses an emotion engine to analyze the user's emotions.

[1501] Input: Audio data, video data

[1502] How it works: The server inputs data into the emotion engine to analyze the user's emotional state.

[1503] Output: Analysis results (user's emotional state)

[1504] Step 10:

[1505] The server sends the results of the user's emotion analysis to the terminal, and the terminal adjusts the pet's behavior.

[1506] Input: Analysis results

[1507] How it works: The server sends the analysis results to the device, and the device gives the appropriate instructions to the pet.

[1508] Output: Adjusted pet behavior

[1509] In this way, the system can grasp the status of the pet and the user in real time and take appropriate action.

[1510] (Application example 3)

[1511] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[1512] Conventional pet care systems have difficulty in accurately grasping the health and emotional state of pets, making it difficult for owners and store staff to properly manage the condition of pets. In addition, they lack the function to adjust the behavior of pets based on the user's emotions, which makes it difficult for smooth communication between pets and users.

[1513] The identification process by the identification processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means. In this invention, the server includes a means for collecting voice data (animal cries), a means for collecting characteristics of pet animals (personality, cries, etc.), a means for identifying the type of animal, a means for collecting general animal characteristic data, a means for grasping the state of the pet from the cries based on the collected data, a means for collecting and analyzing image data, a means for recognizing the user's emotions, and a means for adjusting the behavior of the pet based on the user's emotions. This makes it possible to accurately grasp the health and emotional state of the pet and adjust the behavior of the pet according to the user's emotions.

[1514] "Audio data" refers to audio information such as animal cries, and is data collected to analyze the condition of pets.

[1515] "Animal characteristics" refers to characteristics unique to each animal, such as personality and cries, and is information collected to understand the condition of your pet.

[1516] "Means for identifying animal type" refers to technology for identifying what type of animal a pet is based on collected data.

[1517] "General animal characteristic data" is data regarding widely known characteristics of animals, and is information used as a standard when analyzing the condition of pets.

[1518] "A means of understanding a pet's condition from its cries" is a technology that analyzes collected cries data to determine a pet's health and emotional state.

[1519] "Means for collecting and analyzing image data" refers to technology for collecting images of a pet and analyzing the images to understand the pet's condition.

[1520] "Means for recognizing the user's emotions" refers to technology for analyzing the user's tone of voice, facial expressions, etc. to determine the user's emotional state.

[1521] The "means for adjusting the behavior of a pet based on the user's emotions" refers to a technique for appropriately changing the behavior of a pet in response to the recognized emotions of the user.

[1522] A pet care assistant system will be described as an embodiment of the present invention. This system is intended for use in actual stores such as pet shops and pet cafes. The system is installed on a smartphone, smart glasses, or a head-mounted display, and analyzes the health and emotional state of pets in real time and notifies store staff and owners.

[1523] Hardware and Software Used

[1524] Hardware

[1525] Smartphone

[1526] Smart Glasses

[1527] Head-mounted display

[1528] software

[1529] OpenCV: Image processing library

[1530] Keras: A deep learning library

[1531] librosa: Audio processing library

[1532] soundfile: Sound file reading library

[1533] Data processing and calculation

[1534] Analysis of audio data

[1535] The server collects audio data (animal cries) and extracts MFCC features using librosa, which converts the audio data into numerical data and feeds it into a model trained using Keras. The model predicts the health and emotional state of the pet.

[1536] Analysis of image data

[1537] The server collects image data, resizes and preprocesses it using OpenCV, and feeds the preprocessed image data into a model trained using Keras to analyze the pet's condition.

[1538] User Emotion Recognition

[1539] The server collects the user's video data and extracts frames using OpenCV, which are then fed into an emotion recognition model trained using Keras to predict the user's emotional state.

[1540] Pet behavior adjustment

[1541] The server adjusts the pet's behavior based on the user's emotional state, for example adjusting the pet's behavior to be calmer if the user is angry and increasing the pet's activity if the user is happy.

[1542] Examples

[1543] Example of use in a pet shop

[1544] If a pet in a pet shop becomes stressed, the system will analyze voice and image data to detect the pet's stress state, and will alert store staff to take appropriate measures to care for the pet.

[1545] Example of use at a pet cafe

[1546] In the pet cafe, while a user is playing with a pet, the system analyzes the user's emotional state. If the user is happy, the system will make the pet more active and promote communication with the user.

[1547] Examples of prompt statements

[1548] Develop a system that analyzes your pet's sounds and behavior, and notifies you of your pet's health and emotional state. Include the ability to recognize the user's emotions and adjust your pet's behavior accordingly.

[1549] The flow of the specific process in the application example 3 will be described with reference to FIG.

[1550] Step 1:

[1551] The server collects the pet's voice data. Specifically, it records the pet's cries using a microphone installed in a smartphone, smart glasses, or a head-mounted display. The input is the voice data, and the output is the recorded voice file.

[1552] Step 2:

[1553] The server analyzes the collected voice data. It uses librosa to extract MFCC features from the voice data and convert them into numerical data. The input is the recorded voice file, and the output is numerical data including MFCC features.

[1554] Step 3:

[1555] The server inputs the MFCC features into a voice analysis model trained using Keras to predict the pet's health condition and emotional state. The input is numerical data including the MFCC features, and the output is a prediction result indicating the pet's health condition and emotional state.

[1556] Step 4:

[1557] The server collects image data of pets. Images of pets are taken using a camera installed in a smartphone, smart glasses, or a head-mounted display. The input is image data, and the output is a captured image file.

[1558] Step 5:

[1559] The server analyzes the collected image data. It resizes and preprocesses the images using OpenCV and inputs them into an image analysis model trained using Keras. The input is the captured image file and the output is the preprocessed image data.

[1560] Step 6:

[1561] The server inputs the preprocessed image data into a model trained with Keras to analyze the pet's state. The input is the preprocessed image data, and the output is a prediction result indicating the pet's state.

[1562] Step 7:

[1563] The server collects the user's video data. It captures the user's facial expressions using a camera installed in a smartphone, smart glasses, or a head-mounted display. The input is the video data, and the output is the captured video file.

[1564] Step 8:

[1565] The server analyzes the collected video data. It uses OpenCV to extract frames from the video and inputs them into an emotion recognition model trained with Keras. The input is the captured video file and the output is the extracted frame data.

[1566] Step 9:

[1567] The server inputs the extracted frame data into an emotion recognition model trained with Keras to predict the user's emotional state. The input is the extracted frame data, and the output is a prediction result indicating the user's emotional state.

[1568] Step 10:

[1569] The server adjusts the pet's behavior based on the user's emotional state. For example, if the user is angry, the server adjusts the pet's behavior to be calmer, and if the user is happy, the server adjusts the pet's behavior to be more active. The input is a prediction result indicating the user's emotional state, and the output is the adjusted pet's behavior.

[1570] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1571] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1572] Another example of generative AI is Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1573] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1574] [Third embodiment]

[1575] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1576] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1577] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[1578] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1579] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[1580] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[1581] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[1582] Fig. 6 shows an example of main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1583] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1584] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1585] In the headset type terminal 314, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1586] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1587] "Example 1"

[1588] In one embodiment of the present invention, the cries of a pet are collected using a sound data collection device such as a microphone. The collected sound data is sent to a sound analysis device and identified as the cry of a specific animal. In addition, the characteristics of the pet animal (personality, cries, etc.) are collected using input from the owner or a sensor that observes the behavior of the pet. The type of animal is identified using input from the owner or image recognition technology. General animal characteristic data is obtained from a database or the like. This data is analyzed by AI to understand the condition of the pet.

[1589] "Example 2"

[1590] As another embodiment of the present invention, a system is provided that utilizes AI to realize conversation between humans and pets. Specifically, AI analyzes the cries of pets and converts them into words that humans can understand. Conversely, AI analyzes human words and converts them into cries and actions that pets can understand. For example, if a pet expresses its "hungry" state with a cry, the AI ​​analyzes it and conveys the message "your pet is hungry" to the owner. Also, if the owner says "let's play," the AI ​​analyzes it and conveys the message in a form that the pet can understand.

[1591] "Example 3"

[1592] As a further embodiment of the present invention, a system for understanding a pet's health and emotional state is provided. Specifically, an AI analyzes a pet's health and emotional state from its cries and behavior. For example, if a pet is suffering from an illness, its cries and behavior may be different from normal. The AI ​​detects these changes and issues a warning to the owner. Also, if a pet is happy, its cries and behavior may become more active. The AI ​​detects these changes and conveys the happiness to the owner.

[1593] The process flow of each embodiment will be described below.

[1594] "Example 1"

[1595] Step 1: Collect the pet's cries using an audio data collection device such as a microphone.

[1596] Step 2: The collected audio data is sent to an audio analyzer to identify it as the sound of a specific animal.

[1597] Step 3: Collect information about the pet's characteristics (personality, sounds, etc.) using input from the owner and sensors that observe the pet's behavior.

[1598] Step 4: Identify the type of animal using input from the owner and image recognition technology.

[1599] Step 5: Obtain general animal characteristic data from a database, etc.

[1600] Step 6: This data is analyzed using AI to understand the pet's condition.

[1601] "Example 2"

[1602] Step 1: AI analyzes your pet's cries and converts them into words that humans can understand. Step 2: AI analyzes human words and converts them into cries and actions that your pet can understand.

[1603] Step 3: For example, if a pet expresses its hunger state through a cry, the AI ​​will analyze it and convey the message "your pet is hungry" to the owner.

[1604] Step 4: Also, if the owner says "let's play," the AI ​​will analyze it and convey the message in a way that the pet can understand.

[1605] "Example 3"

[1606] Step 1: AI analyzes your pet's health and emotional state based on its sounds and behavior.

[1607] Step 2: For example, if a pet is suffering from an illness, its sounds and behavior may be different from normal. AI will detect these changes and alert the owner.

[1608] Step 3: Also, if a pet is happy, its vocalizations and behavior may become more active. AI will detect these changes and communicate its happiness to the owner.

[1609] Example 1

[1610] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1611] Conventional pet condition monitoring systems mainly collect and analyze voice data and animal characteristic data separately, making it difficult to integrate this data and understand the pet's condition comprehensively. In addition, there was a lack of means to accurately grasp the health and emotional state of pets, making it difficult for owners to appropriately understand and respond to their pet's condition.

[1612] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1613] In this invention, the server includes a means for collecting voice data, a means for transmitting the collected voice data, a means for analyzing the voice data, a means for collecting characteristics of the pet animal, a means for identifying the type of animal, a means for collecting general animal characteristic data, and a means for grasping the condition of the pet from its cry based on the collected data. This makes it possible to comprehensively grasp the condition of the pet by integrating the voice data and the animal characteristic data.

[1614] "Audio data" refers to audio signals, such as animal cries, recorded in digital format.

[1615] "Collecting means" refers to a device or method for acquiring audio data and animal characteristic data.

[1616] A "transmitting means" is a device or method for transferring collected data to another device or server.

[1617] An "analyzing means" is a device or method for processing collected data and extracting specific information.

[1618] "Animal characteristics" refers to individual attributes of pet animals, such as their personalities or the sounds they make.

[1619] An "identification means" is a device or method for determining the type or characteristics of an animal.

[1620] "General animal characteristic data" refers to data such as characteristics and behavioral patterns common to a specific animal species.

[1621] "Pet's condition" refers to the pet's overall condition, including its health and emotional state.

[1622] "AI" is a technology that uses artificial intelligence to analyze data and make specific judgments and predictions.

[1623] "Means for enabling conversation" refers to devices or methods that allow people and pets to communicate.

[1624] "Health Status" refers to information regarding the physical health of a pet.

[1625] "Emotional state" refers to a pet's psychological state or mood.

[1626] The present invention is a system for collecting sounds made by pets and analyzing the data to grasp the state of the pets. A specific embodiment of this system will be described below.

[1627] First, a user uses the smartphone's built-in microphone or a dedicated external microphone to collect the sound of their pet. For example, the user starts a recording app on the smartphone and records the sound of their pet. This sound data is sent to the server by the device. Specifically, the device uses an HTTP POST request to send the sound data to the server's API endpoint.

[1628] The server analyzes the received voice data using a voice analyzer (for example, Google Cloud Speech-to-Text API). The server sends the voice data to the API, analyzes the returned text data, and identifies it as the sound of a specific animal.

[1629] Next, the user inputs the characteristics of the pet (personality, cries, etc.) into the app. For example, the user inputs information such as "personality: active, cries: high-pitched" into the app's form. The app also uses sensors connected to the device (e.g., acceleration sensor and camera) to observe the pet's behavior and collect data. The device periodically transmits the data from the sensors to the server.

[1630] To identify the type of animal, the user can input the type of pet into the app, or the device can use image recognition technology (e.g., TensorFlow or OpenCV) to identify the type of animal. For example, the user can input "dog" or the device can send an image taken with the camera to the server, and the server can use image recognition technology to identify it as a "dog."

[1631] The server retrieves general animal trait data from a database (e.g. AWS RDS or Google BigQuery). The server runs SQL queries to retrieve the required data from the database.

[1632] Finally, the server uses AI (for example, TensorFlow or PyTorch) to analyze the collected voice data, pet characteristic data, animal type data, and general animal characteristic data. The server integrates this data and inputs it into an AI model to understand the pet's condition. For example, the server analyzes data such as "the dog is barking loudly and is behaving actively" and determines that "the dog is excited."

[1633] Examples and prompts

[1634] As a concrete example, the barking sound of a user's dog is collected by the microphone on the smartphone and sent to the server. The server uses the Google Cloud Speech-to-Text API to analyze the sound and identify it as a dog's bark. The user inputs the dog's personality and characteristics of its bark into the app, and observes the dog's behavior with a camera connected to the device. The server obtains general dog characteristic data from AWS RDS, analyzes the data using TensorFlow, and understands the dog's condition.

[1635] An example of a prompt sentence is as follows:

[1636] "Please analyze my dog's barks and understand his condition. My dog ​​is very active and barks a lot. Please analyze the following audio data."

[1637] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1638] Step 1:

[1639] A user collects the sounds of their pet.

[1640] Specifically, the user launches a recording app on their smartphone and records their pet's cries.

[1641] Input: Pet sounds

[1642] Output: Audio data file (e.g. .wav format)

[1643] Step 2:

[1644] The terminal transmits the collected voice data to a server.

[1645] Specifically, the device uses an HTTP POST request to send audio data to the server's API endpoint.

[1646] Input: Audio data file

[1647] Output: Audio data sent to the server

[1648] Step 3:

[1649] The server analyzes the received audio data.

[1650] Specifically, the server uses a voice analysis device (e.g., Google Cloud Speech-to-Text API) to convert the voice data into text data and identify it as the sound of a specific animal.

[1651] Input: Audio data sent to the server

[1652] Output: Parsed text data (e.g. "dog barking")

[1653] Step 4:

[1654] Collect characteristics of the animals kept by the user.

[1655] Specifically, the user enters information such as "personality: active, bark: high-pitched" into the app's form. In addition, the app uses sensors connected to the device (e.g., acceleration sensor and camera) to observe the pet's behavior and collect data.

[1656] Input: User-entered animal feature data, behavioral data from sensors

[1657] Output: Collected animal trait data

[1658] Step 5:

[1659] Identify types of animals.

[1660] Specifically, the user inputs the type of animal they own into the app, or the device uses image recognition technology (e.g. TensorFlow or OpenCV) to identify the type of animal.

[1661] Input: Animal species data or image data entered by the user

[1662] Output: Identified animal type data (e.g. "dog")

[1663] Step 6:

[1664] A server obtains general animal characteristic data.

[1665] Specifically, the server executes an SQL query from a database (e.g. AWS RDS or Google BigQuery) to retrieve the required data.

[1666] Input: Animal species data

[1667] Output: General animal traits data

[1668] Step 7:

[1669] The server analyzes the collected data and determines the pet's condition.

[1670] Specifically, the server uses AI (e.g. TensorFlow or PyTorch) to analyze collected voice data, animal feature data, animal species data, and general animal feature data. The server integrates this data and inputs it into an AI model to determine the pet's condition.

[1671] Input: Audio data, animal feature data, animal species data, general animal feature data

[1672] Output: Analysis results (e.g. "The dog is excited")

[1673] (Application example 1)

[1674] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1675] There is a problem in that it is difficult for pet owners to grasp abnormal behavior or health conditions of their pets in real time while they are out or not watching over them. In addition, there is a lack of means to detect abnormalities in pets early and respond quickly, making it difficult to ensure the health and safety of pets.

[1676] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1677] In this invention, the server includes a means for collecting voice data (animal cries), a means for collecting characteristics of pet animals (personality, cries, etc.), a means for identifying the type of animal, a means for collecting general animal characteristic data, a means for grasping the pet's condition from the cries based on the collected data, and a means for sending a notification when an abnormality is detected. This makes it possible to grasp the pet's abnormal behavior and health condition in real time, and to promptly notify the owner when an abnormality is detected.

[1678] "Audio data" refers to information collected in digital format from audio signals such as animal cries.

[1679] "Collecting means" refers to a device or method for acquiring audio data and animal characteristic data.

[1680] "Animal characteristics" refers to information that refers to the unique characteristics of an animal, such as its personality or the sound it makes.

[1681] An "identification means" is a device or method for determining the type or characteristics of an animal.

[1682] "General animal characteristic data" refers to information such as personality and behavioral patterns common to a specific animal species.

[1683] "Means of understanding" refers to devices and methods used to analyze collected data and understand the condition of animals.

[1684] "Means for detecting abnormalities" refers to devices or methods for identifying abnormalities in the behavior or sounds of animals and detecting the occurrence of an abnormality.

[1685] A "means for sending a notification" is a device or method for conveying that information to an owner or other person when an abnormality is detected.

[1686] The system for carrying out the present invention collects and analyzes the cries of pets to grasp the state of the pets, and sends a notification to the pet's owner if an abnormality is detected. A specific embodiment of the system will be described below.

[1687] Hardware Configuration

[1688] The system includes the following hardware:

[1689] Microphone: Used to collect pet sounds.

[1690] Smartphone or smart glasses: Used to analyze collected voice data and detect anomalies.

[1691] Server: Analyzes voice data and sends notifications.

[1692] Software Configuration

[1693] The system includes the following software:

[1694] TensorFlow: Runs AI models to analyze audio data.

[1695] sounddevice: Used to collect audio data.

[1696] smtplib: Used to send notifications if an anomaly is detected.

[1697] Data processing and calculation

[1698] 1. Audio data collection: The device (smartphone or smart glasses) uses a microphone to collect the pet's cries. The collected audio data is stored in digital format.

[1699] 2. Audio data analysis: The server inputs the collected audio data into the TensorFlow model and performs analysis. Based on the analysis results, the pet's condition is understood.

[1700] 3. Anomaly detection: The server determines whether an anomaly has been detected based on the analysis results. If an anomaly is detected, a notification is sent to the owner.

[1701] 4. Send notification: The server uses smtplib to send a notification to the pet owner, containing information about the pet's abnormal behavior and health status.

[1702] Examples

[1703] For example, if a pet makes an abnormal sound while the owner is out, the system will collect and analyze the sound. If an abnormality is detected based on the analysis results, a notification will be sent to the owner's smartphone, allowing the owner to take prompt action.

[1704] Examples of prompt statements

[1705] "Collect the sounds of pets' cries and analyze them using an AI model. If an abnormality is detected, create a program to notify the pet owner."

[1706] In this way, the present invention makes it possible to grasp a pet's abnormal behavior and health condition in real time and quickly notify the owner if an abnormality is detected.

[1707] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1708] Step 1:

[1709] The terminal (smartphone or smart glasses) collects the pet's cries using a microphone. The input is the pet's cries, and the output is digital audio data. Specifically, the terminal uses software for collecting audio data (sounddevice library) to record audio for a certain period of time and save it as digital data.

[1710] Step 2:

[1711] The device transmits the collected voice data to the server. The input is the digital voice data, and the output is the voice data transmitted to the server. Specifically, the device uses an Internet connection to upload the voice data to the server.

[1712] Step 3:

[1713] The server inputs the received voice data into the TensorFlow model for analysis. The input is digital voice data, and the output is the analysis result (the state of the pet). Specifically, the server uses the TensorFlow library to input the voice data into the AI ​​model and analyze the characteristics of the pet's cries.

[1714] Step 4:

[1715] The server determines whether an anomaly has been detected based on the analysis results. The input is the analysis results, and the output is information about the presence or absence of an anomaly. Specifically, the server compares the analysis results with a threshold, and sets a flag if an anomaly is detected.

[1716] Step 5:

[1717] If an abnormality is detected, the server will send a notification to the pet owner. The input is information about the presence or absence of an abnormality, and the output is a notification to the pet owner. Specifically, the server uses the smtplib library to send an abnormality notification to the pet owner's email address. The notification contains detailed information about the pet's abnormal behavior and health condition.

[1718] Step 6:

[1719] The user (owner) checks the received notification and takes action to check the pet's status if necessary. The input is the notification from the server, and the output is the user's response. Specifically, the user checks the notification on their smartphone and takes action such as returning home to check the pet's status or contacting the pet sitter.

[1720] Example 2

[1721] Next, a second embodiment of the second form will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1722] Conventional means of communication between pets and humans are limited, making it difficult to accurately grasp a pet's intentions and state from its cries and actions. In addition, there was a lack of means to communicate human words to pets in a form that pets could understand. This made it difficult to properly manage a pet's health and emotional state, and communication between pets and owners was insufficient.

[1723] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for collecting voice data, a means for collecting animal characteristics, a means for identifying the type of animal, a means for collecting general animal characteristic data, a means for grasping the state of the pet from the cry based on the collected data, a means for converting voice data into text data using voice recognition software, a means for analyzing text data using a generative AI model to understand the intention of the pet, a means for generating a message based on the analysis result and notifying the user, a means for collecting the user's words and converting them into text data using voice recognition software, a means for analyzing text data using a generative AI model to generate a message to be conveyed to the pet, and a means for converting and conveying a message into a cry or behavior that the pet can understand using voice synthesis software. This makes it possible to accurately grasp the intention and state of the pet from its cry or behavior, and to convey human words in a form that the pet can understand.

[1724] "Audio data" refers to data that has been recorded in digital format from sounds made by animals or humans.

[1725] "Animal characteristics" refers to characteristics unique to each individual animal, such as their personality, sounds, and behavioral patterns.

[1726] "Type of animal" refers to the classification of animals, such as dog, cat, bird, etc.

[1727] "General animal characteristic data" refers to data such as personality and behavioral patterns common to a particular animal species.

[1728] "Voice recognition software" is software for analyzing voice data and converting it into text data.

[1729] A "generative AI model" is a model that uses artificial intelligence to analyze text data and understand its meaning.

[1730] "Text data" is textual information converted from voice data by voice recognition software.

[1731] "Message" refers to the content of notifications or instructions generated based on the analysis results.

[1732] "Speech synthesis software" refers to software for converting text data into speech.

[1733] "User" refers to a person who uses this system.

[1734] "Pet" refers to animals kept as pets in the home.

[1735] This invention is a system that uses AI to realize conversations between people and pets. This system has the functions of analyzing pet cries and converting them into words that humans can understand, and analyzing human words and converting them into cries and actions that pets can understand.

[1736] Hardware and Software Used

[1737] Hardware

[1738] Microphone: Used to capture pet sounds and user speech.

[1739] Speaker: Used to transmit generated sounds to your pet.

[1740] Camera: Used to monitor pet behavior if necessary.

[1741] Server: Used to analyze and process data.

[1742] software

[1743] Speech recognition software: Used to convert voice data into text data (e.g. Google Speech-to-Text).

[1744] Generative AI models: Analyze text data and use it to understand your pet's intent (e.g. GPT-4).

[1745] Speech synthesis software: Used to convert text data into audio (e.g. Google Text-to-Speech).

[1746] Specific operation of the system

[1747] Pet sound analysis

[1748] The microphone on the device captures the sound of your pet barking. For example, if your dog barks, the audio data is collected through the microphone. The collected audio data is then sent to the server in real time.

[1749] The server analyzes the captured voice data using voice recognition software and converts it into text data, for example converting the voice "woof woof" into the text "woof woof." The converted text data is sent to the next analysis step.

[1750] The server uses a generative AI model to analyze the text data and understand the pet's intention. For example, the text "woof woof" is analyzed to mean "I'm hungry." The analysis result is sent to the message generation step.

[1751] The server generates a message that says "your pet is hungry" based on the analysis results and notifies the user. For example, the notification is sent to the user via a smartphone app. The user can check the notification on the app and understand the status of the pet.

[1752] Human Language Analysis

[1753] The microphone on the device captures the user's words. For example, when the user says "Let's play," the voice data is collected through the microphone. The collected voice data is sent to the server in real time.

[1754] The server analyzes the captured voice data using voice recognition software and converts it into text data. For example, the voice "Let's play" is converted into the text "Let's play." The converted text data is sent to the next analysis step.

[1755] The server uses a generative AI model to analyze the text data and generate a message to be sent to the pet. For example, the text "Let's play" is analyzed to mean "It's play time." The analysis result is sent to the message generation step.

[1756] The server uses speech synthesis software to convert the generated message into sounds and actions that the pet can understand, and communicates it to the pet through the speaker. For example, the message "It's playtime" is converted into sounds and actions that the dog can understand, and played back through the speaker. The pet understands the sounds and actions, and gets ready to play with the user.

[1757] Examples of concrete examples and prompts

[1758] Examples

[1759] Pet Sound Analysis:

[1760] If a pet barks "woof woof," a microphone will capture the sound and the server will notify the user that "your pet is hungry."

[1761] Human Language Analysis:

[1762] When a user says "let's play," a microphone captures the audio and the server communicates through the speaker with sounds and actions that the pet understands to mean "it's play time."

[1763] Examples of prompt statements

[1764] Pet Sound Analysis:

[1765] If a pet barks "woof woof", explain the process of analyzing that sound and informing the user that their pet is hungry.

[1766] Human Language Analysis:

[1767] When the user says "let's play," describe the process that analyzes that speech and communicates to your pet "it's play time" in a way that they can understand.

[1768] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1769] Step 1: Capture your pet's sounds

[1770] The microphone on the device captures the sound of the pet barking. For example, if a dog barks "woof woof," the audio data is collected through the microphone. The input is the pet's bark, and the output is audio data. The collected audio data is sent to the server in real time.

[1771] Step 2: Analyzing bird calls and converting them into text

[1772] The server analyzes the captured voice data using voice recognition software and converts it into text data. For example, the voice "woof woof" is converted into the text "woof woof." The input is voice data and the output is text data. The converted text data is sent to the next analysis step.

[1773] Step 3: Semantic analysis of the text

[1774] The server uses a generative AI model to analyze the text data and understand the pet's intention. For example, the text "woof woof" is analyzed to mean "I'm hungry." The input is the text data, and the output is the analysis result that indicates the pet's intention. The analysis result is sent to the message generation step.

[1775] Step 4: Message generation and notification

[1776] The server generates a message saying "your pet is hungry" based on the analysis results and notifies the user. For example, the notification is sent to the user via a smartphone app. The input is the analysis results, and the output is a notification message to the user. The user can check the notification on the app to understand the status of their pet.

[1777] Step 5: Capturing human language

[1778] The microphone on the device captures the user's words. For example, when a user says "Let's play," the voice data is collected through the microphone. The input is the user's words, and the output is voice data. The collected voice data is sent to the server in real time.

[1779] Step 6: Word analysis and text conversion

[1780] The server analyzes the captured voice data using voice recognition software and converts it into text data. For example, the voice "Let's play" is converted into the text "Let's play". The input is voice data and the output is text data. The converted text data is sent to the next analysis step.

[1781] Step 7: Semantic analysis of the text

[1782] The server uses a generative AI model to analyze the text data and generate a message to be conveyed to the pet. For example, the text "Let's play" is analyzed to mean "It's play time." The input is the text data, and the output is the message to be conveyed to the pet. The analysis result is sent to the message generation step.

[1783] Step 8: Create a message and send it to your pet

[1784] The server uses speech synthesis software to convert the generated message into sounds and actions that the pet can understand, and communicates them to the pet through the speaker. For example, the message "It's playtime" is converted into sounds and actions that the dog can understand, and played back from the speaker. The input is the message to be communicated to the pet, and the output is sounds and actions that the pet can understand. The pet understands the sounds and actions, and gets ready to play with the user.

[1785] (Application example 2)

[1786] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1787] Conventional pet care systems have problems in that it is difficult to accurately understand a pet's cries and behavior and take appropriate action. They also lack a means to notify the owner that the pet is hungry and to quickly order pet food. Furthermore, they lack an effective means to facilitate communication between the owner and the pet.

[1788] The identification process by the identification processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means. In this invention, the server includes a means for collecting voice data (animal cries), a means for collecting characteristics of pet animals (personality, cries, etc.), a means for identifying the type of animal, a means for collecting general animal characteristic data, a means for grasping the pet's condition from the cries based on the collected data, a means for notifying the owner based on the pet's condition, and a means for ordering pet food. This makes it possible to accurately grasp the pet's condition and respond quickly. In addition, it is possible to facilitate communication between the owner and the pet and automate the ordering of pet food.

[1789] "Audio data" refers to sound information, including animal cries.

[1790] "Characteristics of pets" refers to individual characteristics such as a pet's personality or the sounds it makes.

[1791] "Means for identifying the type of animal" refers to a method or device for identifying the type of pet.

[1792] "General animal characteristic data" refers to information about the widely known characteristics of animals.

[1793] "Means for understanding a pet's condition from its cries" refers to a method or device for analyzing a pet's cries to understand its condition.

[1794] "Means for notifying an owner based on the condition of the pet" refers to a method or device for notifying an owner of the condition of the pet.

[1795] "Pet food ordering means" refers to a method or device for purchasing pet food online.

[1796] "Means of using AI to enable conversation between people and pets" refers to methods and devices that use artificial intelligence to exchange information between humans and pets.

[1797] "Means for understanding a pet's health and emotional state" refers to a method or device for understanding a pet's physical condition and emotions.

[1798] The following system configuration will be described as an embodiment of the present invention.

[1799] System Configuration

[1800] This system includes means for collecting voice data (animal cries), means for collecting characteristics of pet animals (personality, cries, etc.), means for identifying the type of animal, means for collecting general data on animal characteristics, means for determining the condition of pets from their cries based on the collected data, means for notifying owners based on the pet's condition, and means for ordering pet food.

[1801] Hardware and Software Used

[1802] Hardware: Smartphone, microphone

[1803] Software: Python, SpeechRecognition library, Requests library

[1804] Data processing and calculation

[1805] Acquisition and analysis of audio data

[1806] The server records the pet's cries using the smartphone's microphone. The recorded voice data is converted to text data using the SpeechRecognition library. It is then analyzed using a generative AI model to understand the pet's condition.

[1807] Collecting and analyzing owner comments

[1808] The server records the owner's words using the smartphone's microphone. The recorded voice data is converted to text data using the SpeechRecognition library. It is then analyzed using a generative AI model to generate a message to be conveyed to the pet.

[1809] Pet food order

[1810] The server notifies the owner based on the pet's condition. For example, if the pet cries "I'm hungry," the server generates a prompt to order pet food and notifies the owner. When the owner orders pet food through the app, recommended products are displayed taking into account the pet's preferences and allergy information.

[1811] Examples

[1812] Analyzing pet cries: If a pet cries "I'm hungry," the server analyzes the cry and notifies the owner with a message that "your pet is hungry."

[1813] Analyzing owner's words: When the owner says "let's play", the server analyzes the words and conveys the message in a way that the pet can understand.

[1814] Examples of prompt statements

[1815] "Analyze your pet's cries and let us know their condition."

[1816] "Analyze the owner's words and generate a message to convey to the pet."

[1817] In this way, the present invention makes it possible to accurately grasp the condition of a pet and respond quickly, facilitate communication between the owner and the pet, and automate the ordering of pet food.

[1818] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1819] Step 1:

[1820] The server records the pet's cry using the smartphone's microphone. The input is the pet's cry, and the output is an audio data file. This audio data file is used for subsequent analysis processing.

[1821] Step 2:

[1822] The server converts the recorded voice data into text data using the SpeechRecognition library. The input is the voice data file, and the output is text data. This text data is input to the generative AI model.

[1823] Step 3:

[1824] The server analyzes the text data using a generative AI model to understand the pet's state. The input is text data, and the output is information that indicates the pet's state. For example, the state "hungry" is analyzed.

[1825] Step 4:

[1826] The server notifies the owner based on the pet's status. The input is information indicating the pet's status, and the output is a notification message to the owner. For example, a message "your pet is hungry" is sent to the owner.

[1827] Step 5:

[1828] The server records the owner's words using the smartphone's microphone. The input is the owner's words, and the output is an audio data file. This audio data file is used for subsequent analysis processing.

[1829] Step 6:

[1830] The server converts the recorded voice data of the owner into text data using the SpeechRecognition library. The input is a voice data file, and the output is text data. This text data is input to the generative AI model.

[1831] Step 7:

[1832] The server uses a generative AI model to analyze the text data and generate a message to be conveyed to the pet. The input is text data, and the output is a message to be conveyed to the pet. For example, the command "Let's play" is analyzed.

[1833] Step 8:

[1834] The server provides a means for ordering pet food based on the pet's condition. The input is information indicating the pet's condition, and the output is pet food ordering information. For example, if the pet cries "I'm hungry," an order for pet food is automatically placed.

[1835] Step 9:

[1836] The server displays recommended products to the pet owner, taking into account the pet's preferences and allergy information. The input is the pet's preferences and allergy information, and the output is a list of recommended products. This allows the pet owner to select an appropriate pet food.

[1837] Step 10:

[1838] The server notifies the pet owner that the pet food order is complete. The input is the order completion information, and the output is a notification message to the pet owner. For example, a message saying "Your pet food order is complete" is sent to the pet owner.

[1839] Example 3

[1840] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1841] Conventional systems for understanding the health and emotional state of pets only analyze voice data and do not take into account behavioral data, making it difficult to accurately understand the condition. In addition, there is a lack of a way to quickly notify the owner of the analysis results, making it difficult to discover abnormalities in pets at an early stage.

[1842] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1843] In this invention, the server includes a means for collecting voice data, a means for collecting animal characteristics, a means for identifying the type of animal, a means for collecting general animal characteristic data, a means for grasping the state of the pet from the cries based on the collected data, a means for collecting animal behavior data, a means for analyzing the collected behavior data, and a means for notifying the analysis result. By analyzing both the voice data and the behavior data, it becomes possible to grasp the health and emotional state of the pet more accurately and promptly notify the owner.

[1844] "Audio Data" refers to animal sounds and other audio information.

[1845] "Animal characteristics" refers to individual characteristics such as an animal's personality, sounds, behavior patterns, etc.

[1846] "Type of animal" refers to a classification of animals, such as dogs, cats, birds, etc.

[1847] "General animal characteristic data" refers to data such as personality and behavioral patterns common to a particular type of animal.

[1848] "Animal condition" refers to the animal's physical and emotional state.

[1849] "Behavioral Data" refers to information regarding an animal's movements and behavior patterns.

[1850] "Analysis results" refers to the diagnosis and evaluation output by the AI ​​model based on the collected voice data and behavioral data.

[1851] "Notification means" refers to a method or device for communicating the analysis results to the owner.

[1852] The present invention relates to a system for understanding the health and emotional state of a pet. A specific embodiment of the system will be described below.

[1853] First, the user uses a device such as a smartphone or tablet to collect the sounds and behavior of their pet. The device is equipped with a microphone and camera, and these devices are used to capture voice data and behavior data in real time. For example, when a user starts the smartphone app, points it at their pet and presses the "start recording" button, the microphone starts collecting the pet's sounds. At the same time, the camera captures the pet's movements.

[1854] The device then transmits the collected data to a server, where it is sent in real time over Wi-Fi or mobile data, and the data is encrypted to ensure security.

[1855] The server inputs the received voice and behavior data into an AI model, which is built using machine learning frameworks such as TensorFlow and PyTorch, to analyze the frequency and volume of the pet's cries, as well as the speed and direction of its movements, to detect abnormal patterns and patterns of delight.

[1856] The analysis results are sent from the server to the device, and the user can check the results through a smartphone app. For example, the server will send a warning message such as "Your pet may be sick." If the pet is happy, it will send a message such as "Your pet is happy."

[1857] As a concrete example, the following prompt sentence could be input to a generative AI model:

[1858] "Analyze your pet's vocalizations and behavioral data to determine their health and emotional state. Alert you if your pet is sick and let you know if they're happy."

[1859] Using this prompt sentence, the AI ​​model can accurately analyze the pet's condition and provide appropriate feedback to the user.

[1860] As described above, the present invention makes it possible to grasp the health and emotional state of a pet more accurately and promptly notify the owner by analyzing both the voice data and the behavior data. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1861] Step 1:

[1862] The user starts the smartphone app and collects the sounds and behavior of the pet. Specifically, when the user presses the "Start Recording" button, the device's microphone starts collecting the pet's sounds. At the same time, the camera captures the pet's movements. The input is the pet's sounds and behavior, and the output is the collected voice data and behavior data.

[1863] Step 2:

[1864] The device transmits the collected voice data and behavioral data to the server. Specifically, the device transmits data in real time using Wi-Fi or mobile data communication. The input is the collected voice data and behavioral data, and the output is the data transmitted to the server.

[1865] Step 3:

[1866] The server inputs the received voice data and behavioral data into an AI model. Specifically, the server inputs data into an AI model built using machine learning frameworks such as TensorFlow and PyTorch. The input is the voice data and behavioral data sent to the server, and the output is the analysis result by the AI ​​model.

[1867] Step 4:

[1868] The server judges the pet's health and emotional state based on the analysis results of the AI ​​model. Specifically, the AI ​​model analyzes the frequency and volume of the pet's cries, the speed and direction of its movements, etc., to detect abnormal patterns and patterns of joy. The input is the analysis results of the AI ​​model, and the output is the judgment of the pet's health and emotional state.

[1869] Step 5:

[1870] The server transmits the analysis results to the terminal. Specifically, the server transmits a message generated based on the analysis results to the terminal. The input is the judgment result of the pet's health condition and emotional state, and the output is the message transmitted to the terminal.

[1871] Step 6:

[1872] The user checks the analysis results through the terminal. Specifically, the user opens the smartphone app and checks the notified message. For example, a warning message such as "Your pet may be sick" or a message such as "Your pet is happy" may be displayed. The input is the message sent to the terminal, and the output is the analysis result checked by the user.

[1873] (Application example 3)

[1874] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1875] It is important for pet owners to know their pet's health and emotional state, but conventional methods make it difficult to detect abnormalities in pets at an early stage. In addition, there are limited means to accurately know whether a pet is happy. This can lead to insufficient health management and understanding of the pet's emotions, which can lead to a decline in the quality of the pet's life. Furthermore, there is no means to quickly notify owners of abnormalities in their pets, which can delay emergency response.

[1876] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1877] In this invention, the server includes means for collecting voice data (animal cries), means for collecting characteristics of pet animals (personality, cries, etc.), means for identifying the type of animal, means for collecting general animal characteristic data, means for grasping the pet's condition from the cries based on the collected data, means for analyzing the pet's condition and sending a notification when an abnormality is detected, means for grasping the pet's health and emotional state in real time, means for issuing a warning to the owner when an abnormality is detected, and means for conveying information about the pet to the owner when the pet is happy. This makes it possible to quickly and accurately grasp the pet's health and emotional state, and to immediately notify the owner when an abnormality occurs.

[1878] "Audio data" refers to data that records sounds such as animal cries in digital format.

[1879] "Animal characteristics" refers to the animal's unique characteristics, such as personality or sounds.

[1880] "Means for identifying animal species" refers to methods or techniques for identifying the species of an animal.

[1881] "General animal characteristic data" refers to data about widely known characteristics of animals.

[1882] "Means of understanding a pet's condition" refers to methods of analyzing a pet's health and emotional state based on collected data.

[1883] "Means for analyzing a pet's condition" refers to a method for analyzing a pet's cries and behavior to assess its condition.

[1884] "Means for sending a notification when an abnormality is detected" refers to a method for issuing a warning to the pet owner when an abnormality is detected in the pet.

[1885] "Means of understanding a pet's health and emotional state in real time" refers to a method of instantly analyzing a pet's condition and providing the results in real time.

[1886] "Means for issuing a warning to the pet owner when an abnormality is detected" refers to a method for issuing a warning to the pet owner when an abnormality is detected in the pet.

[1887] The term "means for conveying information about a happy pet to its owner" refers to a method for detecting a happy state of a pet and conveying the information to its owner.

[1888] As an embodiment of the present invention, a pet security alert system will be described as an example. This system analyzes t...

Claims

1. A means for receiving audio data including animal sounds and user speech; A means for collecting characteristics, including personality, of an animal kept by a user; A means for identifying the type of animal; means for collecting trait data about general animal characteristics; a means for generating a prompt sentence instructing the generation of information regarding at least one of the status and requirements of the animal kept by the user, based on the voice data, the characteristics of the personality of the animal kept by the user, the identified type of animal, and the collected characteristic data; means for generating information regarding at least one of the status and requirements of an animal kept by the user using the generated prompt sentence and a generative AI model; means for determining an emotional state of a user using an emotion engine; a means for generating a prompt sentence based on the identified emotional state of the user, the prompt sentence instructing the animal to translate an instruction spoken by the user to the animal into a form that the animal can understand; A means for translating an instruction spoken by the user to an animal being kept into a form understandable by the animal using the generated prompt sentence and a generative AI model; means for outputting the translated voice instructions; A system including:

2. The system according to claim 1 , wherein the means for translating the instruction uttered by the user to the animal into a form understandable by the animal outputs the instruction as an image understandable by the animal.

3. The system of claim 1 , wherein the means for generating information regarding at least one of the condition and needs of the animal kept by the user identifies at least one of a health state and an emotional state of the animal.

Images