System

Abstract

A system is provided.SOLUTION: A system, comprising: means for capturing an image of a pet; means for preprocessing the captured image; means for transmitting the preprocessed image to a server; means for analyzing a health condition of the pet by a AI model on the server; means for generating an action suggestion for a user based on an analysis result; and means for displaying the generated action suggestion.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Pet owners lack the expertise to properly assess their pets' health, making it difficult to decide whether to take their pets to the veterinarian when they are ill or injured. Furthermore, veterinary consultations are expensive, and time is limited for pet owners to take their pets to the veterinarian. Because managing pet health is such a difficult task, there is a need for a method that allows pet owners to easily assess their pets' health and learn appropriate treatment. [Means for solving the problem]

[0005] The present invention solves the above problems by the following means.

[0006] A system is provided that includes a means for capturing images of a pet, a means for preprocessing the captured images, a means for sending the preprocessed images to a server, a means for analyzing the pet's health condition using an AI model on the server, a means for generating suggested actions for the user based on the analysis results, and a means for displaying the generated suggested actions. The system further includes a means for displaying detailed explanations of high-risk areas and a means for providing links to schedule medical appointments or purchase care products. This allows pet owners to easily understand their pet's health condition and take appropriate measures. Furthermore, the accuracy of the analysis can be improved by including a means for resizing or noise reduction of images in the preprocessing step.

[0007] "Pets" refers to all animals kept at home, such as dogs, cats, birds, and fish.

[0008] "Images" refers to photographic data of pets taken using a digital camera, smartphone, etc.

[0009] "Capture means" refers to the function for taking a picture of your pet with a camera or selecting an existing image file.

[0010] "Preprocessing" refers to performing improvement processes such as resizing and noise removal on a captured or selected image.

[0011] "Server" refers to a computer system that receives image data over a network and analyzes the data using an AI model.

[0012] An "AI model" refers to an algorithm that uses machine learning and deep learning technologies to analyze a pet's health condition.

[0013] "Health analysis" refers to the process of evaluating each part of your pet's body that appears in the image to determine its level of risk for illness or injury.

[0014] "Action suggestions" refers to presenting specific countermeasures and recommended actions to the user based on the results of health status analysis.

[0015] "Means for displaying" refers to the function of displaying analysis results and suggested actions on the screen of a user device such as a smartphone or tablet.

[0016] "Detailed explanation" refers to providing additional information about high-risk areas, causes of symptoms, and how to deal with them.

[0017] "Appointment Booking" refers to a feature that allows pet owners to easily book veterinary appointments online.

[0018] "Purchase Care Products" refers to the function for purchasing medicines and care products necessary for pet health management online.

[0019] "Resizing" refers to the process of adjusting the size and resolution of an image to change it to a size suitable for analysis.

[0020] "Noise reduction" refers to the process of removing unnecessary noise and artifacts from an image to create a clear image. [Brief explanation of the drawings]

[0021] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6]FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0022] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0023] First, the terms used in the following description will be explained.

[0024] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0025] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0026] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0027] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0028] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0029] [First embodiment]

[0030] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0031] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0032] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0033] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0034] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0035] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0036] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0037] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0038] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0039] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0040] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0041] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0042] This invention is a system that allows pet owners to grasp the health condition of their pets at an early stage and take appropriate measures. Specifically, it assesses the health risks of pets through a series of processes that involves capturing images of the pet, preprocessing the images, and analyzing them based on an AI model, and then provides users with suggested actions.

[0043] System configuration

[0044] 1. Image capture method

[0045] User: Uses a device such as a smartphone or tablet to take a picture of their pet or select an existing image.

[0046] 2. Image preprocessing methods

[0047] Device: Performs preprocessing such as resizing and noise removal on captured or selected images, making the image data suitable for AI analysis.

[0048] 3. Image transmission method

[0049] Terminal: Sends the pre-processed image to the server.

[0050] 4. AI model for analyzing health status

[0051] Server: The received image data is input into the AI ​​model to analyze the pet's health condition. Specifically, it detects the pet's face and body parts and determines the health risk of each part (e.g., normal, requires observation, emergency).

[0052] 5. Action suggestion generation means

[0053] Server: Based on the analysis results of the AI ​​model, it generates specific action suggestions for the user (e.g., recommendations for daily care, the need for a veterinarian).

[0054] 6. Action suggestion display means

[0055] Device: The generated action proposals are displayed in an easy-to-understand manner to the user. Specifically, risk areas and countermeasures are displayed on the screen.

[0056] 7. Detailed explanation display means

[0057] Device: When the user taps on a high-risk area, a detailed explanation and countermeasures for that area will be displayed.

[0058] 8. How to book appointments and purchase care products

[0059] Terminal: Provides links to schedule appointments and purchase care products. This feature allows users to easily schedule veterinary appointments and purchase necessary care products.

[0060] Specific examples

[0061] 1. The user launches the app and takes a full-body photo of their dog.

[0062] User: Use the app's camera function to take a full-body photo of your pet (e.g., dog).

[0063] Device: Temporarily saves the captured image and displays the "Next" button.

[0064] 2. The device preprocesses the image

[0065] Device: Resize and denoise the image to make it optimal for analysis.

[0066] Terminal: Sends the image data after preprocessing to the server.

[0067] 3. The server analyzes the image and generates action suggestions.

[0068] Server: The received images are input into the AI ​​model to analyze the pet's health condition. For example, risk areas such as bloodshot eyes and abdominal swelling are identified.

[0069] Server: Based on the analysis results, it generates appropriate action suggestions to the user (e.g., "Your eyes are bloodshot. Consider using eye drops. If symptoms persist, consult your veterinarian").

[0070] 4. The device displays the diagnosis results and suggested actions to the user.

[0071] Terminal: The diagnostic results are displayed intuitively to the user, clearly indicating risk areas and providing specific countermeasures.

[0072] 5. Users access more information and additional features

[0073] User: Tap on high-risk areas to see detailed explanations and solutions.

[0074] Terminal: Users can access links to book appointments and purchase care products as needed.

[0075] This system allows pet owners to easily understand their pet's health status and quickly take necessary measures, which is expected to make pet health management more efficient and effective.

[0076] The processing flow will be explained below.

[0077] Step 1:

[0078] The user launches the app on their smartphone and takes a picture of their pet or selects an existing image, providing the system with up-to-date information about their pet's health.

[0079] Step 2:

[0080] The device temporarily stores the captured or selected image, which is then preprocessed.

[0081] Step 3:

[0082] The device resizes the image to make it easier for the AI ​​model to analyze high-resolution images.

[0083] Step 4:

[0084] The device performs noise reduction, which improves image quality and increases the accuracy of analysis.

[0085] Step 5:

[0086] The device sends the preprocessed image to the server, which forwards the image to the server for analysis.

[0087] Step 6:

[0088] The server inputs the received image into the AI ​​model, which then detects each part of the pet based on the image.

[0089] Step 7:

[0090] The server uses an AI model to analyze the pet's health condition, identifying risk areas such as bloodshot eyes or abdominal swelling, and determining the risk level (e.g., normal, requires observation, or emergency).

[0091] Step 8:

[0092] The server generates action suggestions for the user based on the analysis results, such as "Your eyes are bloodshot. Consider using eye drops. If the symptoms persist, consult a veterinarian."

[0093] Step 9:

[0094] The device receives the diagnostic results and action suggestions from the server and displays them to the user. Risk areas and solutions are presented through an intuitive interface.

[0095] Step 10:

[0096] When the user taps on a high-risk area, the device will display a detailed explanation and solutions, such as the cause of "bloodshot eyes" and how to care for them at home.

[0097] Step 11:

[0098] The device provides links to schedule appointments and purchase care products, allowing users to schedule veterinary appointments and purchase medications as needed.

[0099] This series of steps creates a system that allows pet owners to accurately and quickly understand their pet's health condition and take appropriate measures.

[0100] Example 1

[0101] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0102] For modern pet owners, it is becoming increasingly important to understand their pet's health condition early and take appropriate measures. However, traditional methods often result in late detection of abnormalities in pets or in failure to take appropriate action. Furthermore, there is a lack of means to obtain specific action suggestions and detailed explanations in real time based on a pet's health condition, making it difficult for pet owners to manage their pet's health efficiently and effectively.

[0103] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0104] In this invention, the server includes means for capturing images of the pet, means for preprocessing the captured images, means for transmitting the preprocessed images to the server, means for analyzing the health condition of the pet using a generative AI model on the server, means for generating action suggestions for the user based on the analysis results, means for displaying the generated action suggestions, means for displaying detailed explanations of high-risk areas when the user taps on those areas, and means for providing links to make appointments for medical examinations and purchase care products. This enables pet owners to grasp the health condition of their pets at an early stage and take prompt and appropriate measures.

[0105] "Means for capturing images" refers to a means by which a user can take an image of a pet using a device such as a smartphone or tablet, or select an existing image.

[0106] "Preprocessing means" refers to the process of resizing and removing noise from the captured image, making the image data suitable for AI analysis.

[0107] The "means for transmitting to the server" is a means for transmitting the image preprocessed on the terminal to the server via communication.

[0108] The "analysis means using a generative AI model" is a means of analyzing received pet image data using an AI model placed on a server and evaluating the pet's health condition.

[0109] The "means for generating behavioral suggestions" refers to a means for creating specific behavioral suggestions according to the health condition of a pet based on the analysis results of the generative AI model.

[0110] The "means for displaying suggested actions" is a means for displaying the content of suggested actions on the user's terminal in an easily understandable manner.

[0111] The "means for displaying a detailed explanation" is a means for displaying a detailed explanation and a method for dealing with a high-risk area when the user taps on that area.

[0112] "Means for providing links to schedule appointments and purchase care products" refers to means for providing links that allow a user to easily schedule an appointment with a veterinarian or purchase necessary care products.

[0113] This invention is a system that allows pet owners to grasp the health condition of their pets at an early stage and take prompt and appropriate measures to efficiently and effectively manage the health of their pets. A specific implementation method of this system is described in detail below.

[0114] Hardware and software used

[0115] 1. User Device

[0116] Hardware: Smartphones, tablets

[0117] Software: Dedicated application (e.g., pet health management app)

[0118] 2. Server

[0119] Hardware: High-performance servers (e.g., servers with Intel Xeon processors)

[0120] Software: A software environment for running generative AI models (e.g., Python, TensorFlow)

[0121] System configuration and data processing

[0122] 1. Image capture method

[0123] User: Launches the application on their smartphone or tablet to take a picture of their pet. The user can use the camera function within the application to take a picture of their pet or select an existing image.

[0124] 2. Image preprocessing methods

[0125] On-device: Captured or selected images are resized and pre-processed on-device, including resizing and noise reduction. This involves scaling the image resolution to the appropriate size and applying Gaussian filters to reduce noise.

[0126] 3. Image transmission method

[0127] Terminal: After preprocessing, the image data is sent to the server using an HTTP POST request.

[0128] 4. AI model for analyzing health status

[0129] Server: The server inputs the received image data into a generative AI model to detect the pet's face and body parts and assess health risks. This analysis uses a TensorFlow-based deep learning model.

[0130] 5. Action suggestion generation means

[0131] Server: Based on the analysis results, the server generates specific action suggestions for the user. For example, if it determines that a pet's eyes are bloodshot, it generates a suggestion such as "Consider using eye drops. If the symptoms persist, consult a veterinarian."

[0132] 6. Action suggestion display means

[0133] Device: The generated action proposals are displayed on the user's device, highlighting risk areas and showing specific countermeasures.

[0134] 7. Detailed explanation display means

[0135] On the device: When the user taps on a high-risk area, a detailed explanation and solutions for that area are displayed.

[0136] 8. How to book appointments and purchase care products

[0137] Terminal: Links for booking appointments and purchasing care products are displayed on the terminal, allowing the user to book an appointment with a veterinarian or purchase any necessary care products.

[0138] Specific examples

[0139] The user launches a dedicated application and takes a full-body photo of their pet (e.g., a dog) using their smartphone's camera. The application resizes the image, removes noise, and then sends it to a server. The server uses a generative AI model to analyze the health status of the image. Based on the analysis results, the server generates a suggested action, such as "Your eyes are bloodshot, so consider using eye drops. If symptoms persist, consult a veterinarian," and displays it on the user's device. When the user taps on high-risk areas based on the results, detailed explanations and countermeasures are displayed. Links to make appointments and purchase care products are also provided.

[0140] Examples of prompt statements

[0141] Take or select an image of your pet. After pre-processing, the image will be analyzed to assess your pet's health. For example, if it determines that your dog's eyes are bloodshot, it will display specific action suggestions such as "Consider using eye drops. If symptoms persist, consult a veterinarian."

[0142] This system is expected to enable pet owners to detect health conditions in their pets early and respond quickly and accurately, making pet health management more efficient and effective.

[0143] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0144] Step 1: Importing an image

[0145] User: The user launches a dedicated application on their smartphone or tablet and takes a picture of their pet or selects an existing image.

[0146] Input: A photo of your pet (either a new or existing photo)

[0147] What happens: The user taps the app's "Take Image" button and takes a full-body photo of their pet with the camera, or taps the "Select Image" button to choose an existing image from the gallery.

[0148] Output: Image data of the pet taken or selected by the user

[0149] Step 2: Image preprocessing

[0150] Device: The device performs pre-processing such as resizing and noise reduction on the captured or selected image.

[0151] Input: Image data of the pet taken or selected by the user

[0152] What it does: Resizes the image to a resolution of 600x800 pixels and reduces noise using a Gaussian filter. The application performs these steps automatically.

[0153] Output: Preprocessed image data

[0154] Step 3: Sending images

[0155] Terminal: Sends the image data after preprocessing to the server.

[0156] Input: Preprocessed image data

[0157] Specific operation: The device uses an HTTP POST request to upload the preprocessed image to the server.

[0158] Output: Image data sent to the server

[0159] Step 4: Analyze health status using AI models

[0160] Server: The server inputs the received image data into a generative AI model to detect the pet's face and body parts and assess health risks.

[0161] Input: Image data sent to the server

[0162] How it works: A Python script on the server receives the image and analyzes it using a TensorFlow-based generative AI model, which detects features such as bloodshot eyes and abnormal coat condition and assesses health risks.

[0163] Output: Analysis results on pet health risks

[0164] Step 5: Generate action suggestions

[0165] Server: Based on the analysis results, the server generates specific action suggestions for the user.

[0166] Input: Analysis results on pet health risks

[0167] Specific operation: Based on the analysis results obtained by the server, the server generates specific suggestions by inserting them into a pre-prepared text template. For example, it generates a suggestion such as, "Your eyes are bloodshot. Consider using eye drops. If the symptoms persist, consult a veterinarian."

[0168] Output: Suggested actions for the user

[0169] Step 6: View suggested actions

[0170] Terminal: The generated action suggestions are displayed on the user's terminal.

[0171] Input: Suggested actions for the user

[0172] Specific actions: Highlight risk areas on the device screen and provide text explaining specific ways to deal with the problem. For example, a bloodshot eye is highlighted in a red frame with suggested actions displayed below.

[0173] Output: Action suggestions displayed on the user's device

[0174] Step 7: View detailed instructions

[0175] User: When the user taps on a high-risk area, a detailed explanation and solutions for that area will be displayed.

[0176] Input: User tap operation, information on high-risk areas

[0177] Specific operation: When the user taps the red frame of the risk area, a detailed information screen will open, displaying an explanation of the eye symptoms and how to deal with them.

[0178] Output: Detailed explanation and solutions

[0179] Step 8: Book a consultation and purchase care products

[0180] Device: Links to schedule appointments and purchase care products will be displayed on the device.

[0181] Input: Relevant links based on identified health risks

[0182] Specific behavior: Link buttons to a medical appointment booking page and a care product online store are displayed on the screen. When the user taps these buttons, the respective pages open.

[0183] Output: Links to book appointments and purchase care products

[0184] (Application example 1)

[0185] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0186] Properly managing the health of pets is an important issue for pet owners. With current technology, it is difficult to grasp a pet's health condition early and take specific measures. This increases the risk that pet owners without specialized veterinary knowledge may overlook abnormalities in their pets. Furthermore, the lack of a simple and rapid means of assessing a pet's health raises concerns that pet care may be neglected in today's busy society. Therefore, to solve these problems, a system is needed that allows users to easily assess a pet's health condition and take specific measures.

[0187] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0188] In this invention, the server includes a means for capturing images of pets, a means for preprocessing the captured images, and a means for transmitting the preprocessed images to the server. This allows the pet images to be analyzed using an AI model, and specific action suggestions for the user to be automatically generated and displayed. Additionally, by adding a means for displaying detailed explanations and countermeasures by tapping on risk areas and a means for providing links to make medical appointments or purchase care products, pet owners can immediately take appropriate measures. Furthermore, the accuracy of the analysis is improved by adding a means for resizing and noise reduction to the image preprocessing and normalizing the resized images before inputting the image data into the AI ​​model. This enables early detection of pet health conditions and appropriate measures.

[0189] "Means for capturing an image of a pet" refers to a function that allows a user to take an image of a pet or select an existing image using a device such as a smartphone or tablet.

[0190] "Means for preprocessing captured images" refers to a function that processes captured or selected images, such as resizing and noise removal, to prepare the image data in a state suitable for AI analysis.

[0191] The "means for transmitting preprocessed images to a server" is a function for transmitting preprocessed images to a server via the Internet.

[0192] "Means for analyzing pet health conditions using an AI model on a server" refers to a function that inputs image data stored on a server into an AI model, analyzes the pet's health conditions, and identifies risk areas.

[0193] "Means for generating action suggestions for users based on analysis results" refers to a function for generating appropriate action suggestions for users based on the analysis results of the AI ​​model.

[0194] The "means for displaying the generated action proposal" is a function for displaying the generated action proposal in an easily understandable manner to the user.

[0195] "A means of displaying detailed explanations and solutions by tapping on risk areas" is a function that allows the user to tap on risk areas and display detailed explanations and solutions for that area.

[0196] "Means for providing links to schedule appointments or purchase care products" refers to a function that provides links for users to schedule appointments or purchase care products.

[0197] "Resizing" is a process of changing the size of an image.

[0198] "Noise removal" is a process that removes unnecessary information (noise) from an image.

[0199] "Normalization" is the process of aligning the range of data to a certain scale.

[0200] The present invention provides a system that enables pet owners to grasp the health condition of their pets at an early stage and take appropriate measures. Specific embodiments for carrying out the present invention will be described in detail below.

[0201] System configuration

[0202] 1. Image capture method

[0203] Users can take pictures of their pets using a smartphone, tablet, or other device, and the images are temporarily saved in the app.

[0204] 2. Image preprocessing methods

[0205] The device resizes and denoises the captured images. This preprocessing is performed using OpenCV. Resizing adjusts the image size appropriately, and denoising removes unnecessary data that may affect analysis.

[0206] 3. Image transmission method

[0207] After preprocessing, the image data is sent from the device to the server via Firebase or AWS.

[0208] 4. AI model for analyzing health status

[0209] The server runs an AI model using TensorFlow or PyTorch to analyze the received image data, specifically detecting pet faces and body parts and assessing their health risks.

[0210] 5. Action suggestion generation means

[0211] Based on the analysis results of the AI ​​model, the server generates specific action suggestions for the user, such as "Your eyes are bloodshot. Consider using eye drops. If the symptoms persist, consult a veterinarian."

[0212] 6. Action suggestion display means

[0213] The device displays the generated action suggestions in an easy-to-understand manner to the user, visually indicating risk areas and presenting specific countermeasures.

[0214] 7. Detailed explanation display means

[0215] When the user taps on a risk area, the device displays a detailed explanation and instructions on how to deal with that area, allowing pet owners to take appropriate measures with confidence.

[0216] 8. How to book appointments and purchase care products

[0217] The device provides links to help users easily schedule appointments and purchase care products, helping them get the care they need faster.

[0218] Specific examples

[0219] Below we introduce some specific scenarios in which this system can be used.

[0220] 1. The user launches the app and takes a full-body photo of their dog.

[0221] The user uses the app's camera function to take a full-body photo of their pet (e.g., dog). The device temporarily saves the image and displays a "Next" button.

[0222] 2. The device preprocesses the image

[0223] The device resizes the image and performs noise reduction. This preprocessing is done using OpenCV. The preprocessed image data is then sent to the server.

[0224] 3. The server analyzes the image and generates action suggestions.

[0225] The server inputs the received images into an AI model using TensorFlow or PyTorch to analyze the pet's health. For example, it identifies risk areas such as bloodshot eyes and abdominal swelling. Based on the analysis results, it then generates appropriate action suggestions for the user.

[0226] 4. The device displays the diagnosis results and suggested actions to the user.

[0227] The generated action suggestions are displayed on the device in an easy-to-understand manner to the user, visually indicating risk areas and presenting specific countermeasures.

[0228] Example prompts to input to the generative AI model

[0229] "Evaluate the condition of this dog's face and determine its health risks."

[0230] "Please use an AI model to analyze whether there is anything abnormal about this cat's abdomen."

[0231] This invention allows pet owners to easily understand the health status of their pets and quickly take necessary measures, which is expected to make pet health management more efficient and effective.

[0232] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0233] Step 1:

[0234] The user launches the app and captures an image of their pet. The user can either take a picture of their pet using the camera function of their smartphone or tablet, or import an existing image into the app. The input is a newly taken image or an existing image, which is temporarily stored on the device. The output is image data that awaits preprocessing.

[0235] Step 2:

[0236] The device performs preprocessing. Image resizing and noise removal are performed within the device. Specifically, OpenCV is used to appropriately adjust the image size and remove noise. The input is image data awaiting preprocessing, and the output is preprocessed image data suitable for AI analysis.

[0237] Step 3:

[0238] The device sends the preprocessed image to the server. The preprocessed image data is uploaded to the server via Firebase or AWS. The preprocessed image data is the input, and the image data stored on the server is the output.

[0239] Step 4:

[0240] The server analyzes the image using an AI model. Using TensorFlow or PyTorch on the server, the received image data is input into the AI ​​model. The AI ​​model detects the pet's face and body parts and evaluates health risks. The image data stored on the server is used as input, and the analysis results of the pet's health condition are obtained as output.

[0241] Step 5:

[0242] The server generates action suggestions. Based on the analysis results of the AI ​​model, the server generates specific action suggestions for the user. For example, an action suggestion might be something like, "Your eyes are bloodshot. Consider using eye drops. If the symptoms persist, consult a veterinarian." The input is the analysis result data of the pet's health condition, and the output is action suggestion data.

[0243] Step 6:

[0244] The terminal displays the action proposal. The terminal displays the generated action proposal to the user in a visually easy-to-understand manner. Specifically, it highlights risk areas and shows how to deal with them. The input is the action proposal data, and the output is the action proposal displayed to the user.

[0245] Step 7:

[0246] The user taps on a risky area. When the user taps on a risky area on the action suggestion display screen, a detailed explanation and countermeasures for that area are displayed. Specifically, a detailed screen is displayed that describes the symptoms and describes countermeasures. The user's tap operation is the input, and detailed information is displayed as the output.

[0247] Step 8:

[0248] The terminal provides links for making appointments and purchasing care products. When the user taps the links to make appointments or purchase care products as needed, they are redirected to an external site and the procedure is completed. The input is the action suggestion data and the user's link tap operation, and the output is the completion of the appointment or product purchase.

[0249] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0250] This invention combines a system that allows pet owners to quickly understand the health condition of their pets and take appropriate measures with an emotion engine that recognizes the user's emotions. Specifically, the system captures images of the pet, preprocesses the images, analyzes them based on an AI model, and evaluates the pet's health risks. In addition, the system recognizes the user's emotions and suggests actions based on the emotions.

[0251] System configuration

[0252] 1. Image capture method

[0253] User: Uses a device such as a smartphone or tablet to take a picture of their pet or select an existing image.

[0254] 2. Image preprocessing methods

[0255] Device: Performs preprocessing such as resizing and noise removal on captured or selected images, making the image data suitable for AI analysis.

[0256] 3. Image transmission method

[0257] Terminal: Sends the pre-processed image to the server.

[0258] 4. AI model for analyzing health status

[0259] Server: The received image data is input into the AI ​​model to analyze the pet's health condition. Specifically, it detects the pet's face and body parts and determines the health risk of each part (e.g., normal, requires observation, emergency).

[0260] 5. Action suggestion generation means

[0261] Server: Based on the analysis results of the AI ​​model, it generates specific action suggestions for the user (e.g., recommendations for daily care, the need for a veterinarian).

[0262] 6. Action suggestion display means

[0263] Device: The generated action proposals are displayed in an easy-to-understand manner to the user. Specifically, risk areas and countermeasures are displayed on the screen.

[0264] 7. Emotion Engine

[0265] Terminal or server: Equipped with an emotion engine that recognizes the user's emotions, it determines the user's emotions by analyzing the user's voice data and image data.

[0266] 8. Emotion-based behavioral adjustment measures

[0267] Server: Adjusts suggested actions and provides appropriate feedback based on the user's emotions recognized by the emotion engine. For example, if the user is feeling stressed, the server provides more helpful and specific explanations.

[0268] 9. Detailed explanation display means

[0269] Device: When the user taps on a high-risk area, a detailed explanation and countermeasures for that area will be displayed.

[0270] 10. How to book appointments and purchase care products

[0271] Terminal: Provides links to schedule appointments and purchase care products. This feature allows users to easily schedule veterinary appointments and purchase necessary care products.

[0272] Specific examples

[0273] 1. The user launches the app and takes a full-body photo of their dog.

[0274] User: Use the app's camera function to take a full-body photo of your pet (e.g., dog).

[0275] Device: Temporarily saves the captured image and displays the "Next" button.

[0276] 2. The device preprocesses the image

[0277] Device: Resize and denoise the image to make it optimal for analysis.

[0278] Terminal: Sends the image data after preprocessing to the server.

[0279] 3. The server analyzes the image and generates action suggestions.

[0280] Server: The received images are input into the AI ​​model to analyze the pet's health condition. For example, risk areas such as bloodshot eyes and abdominal swelling are identified.

[0281] Server: Based on the analysis results, it generates appropriate action suggestions to the user (e.g., "Your eyes are bloodshot. Consider using eye drops. If symptoms persist, consult your veterinarian").

[0282] 4. The device displays the diagnosis results and suggested actions to the user.

[0283] Terminal: The diagnostic results are displayed intuitively to the user, clearly indicating risk areas and providing specific countermeasures.

[0284] 5. Users access more information and additional features

[0285] User: Tap on high-risk areas to see detailed explanations and solutions.

[0286] Terminal: Users can access links to book appointments and purchase care products as needed.

[0287] 6. Use of Emotion Engines

[0288] Device or server: While the user is using the app, it analyzes voice data and facial expressions to recognize the user's emotions (e.g., joy, stress, anxiety).

[0289] Server: Adjust the content of the suggested action based on the user's emotions. For example, if the user expresses anxiety, add a detailed and helpful explanation.

[0290] This system not only enables pet owners to accurately and quickly understand their pet's health condition and take appropriate measures, but also enables more effective care by recognizing the user's emotions and providing support accordingly.

[0291] The processing flow will be explained below.

[0292] Step 1:

[0293] The user launches the smartphone app and takes a picture of their pet or selects an existing image, providing the pet's health status as input data.

[0294] Step 2:

[0295] The device temporarily stores the captured or selected image, which is then used in the next processing step.

[0296] Step 3:

[0297] Your device will resize the image to the appropriate resolution, reducing data usage and improving processing speed.

[0298] Step 4:

[0299] The device will then perform noise reduction to improve image quality, which will improve the accuracy of the analysis by the AI ​​model.

[0300] Step 5:

[0301] The device sends the preprocessed image data to the server, which forwards the image data to the server for analysis.

[0302] Step 6:

[0303] The server inputs the received image data into the AI ​​model, which analyzes the image and begins the process of assessing the pet's health.

[0304] Step 7:

[0305] The server uses an AI model to detect the pet's face and body parts, such as the eyes, ears, mouth, and abdomen, and assesses risk areas.

[0306] Step 8:

[0307] The server evaluates the pet's health risk and determines the risk level (e.g., normal, requires observation, emergency). Based on the analysis results, it generates action suggestions for the user.

[0308] Step 9:

[0309] The terminal displays the diagnosis results and action suggestions received from the server to the user. An interface including risk areas and solutions is displayed.

[0310] Step 10:

[0311] When a user uses the app, the device or server inputs the user's voice data and image data into the emotion engine, which analyzes this data and recognizes the user's emotions.

[0312] Step 11:

[0313] The server analyzes the user's emotional data and adjusts the suggested actions, for example, adding specific and helpful explanations if the user is feeling stressed.

[0314] Step 12:

[0315] The device displays behavior suggestions tailored to the user's emotions, providing appropriate feedback according to the user's emotional state.

[0316] Step 13:

[0317] When the user taps on a high-risk area, the device displays a detailed explanation and how to deal with it. The user can check the cause of the specific symptom and how to deal with it.

[0318] Step 14:

[0319] The device provides links to schedule appointments and purchase care products. Users can schedule appointments with veterinarians or purchase care products as needed.

[0320] This system allows pet owners to accurately understand their pet's health condition and take appropriate measures. It also recognizes the user's emotions and provides support accordingly, enabling more effective care.

[0321] Example 2

[0322] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0323] There are few systems that can quickly identify a pet's health condition and provide appropriate treatment, and the suggested actions provided by these systems often do not take the user's feelings into consideration. Furthermore, due to a lack of preprocessing functions to improve the accuracy of pet image data analysis, the reliability of analysis results is often low. This makes it difficult for pet owners to manage their pet's health with peace of mind.

[0324] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0325] In this invention, the server includes means for capturing an image of the pet, means for preprocessing the captured image, means for transmitting the preprocessed image to the server, means for analyzing the health condition of the pet using an AI model on the server, means for generating suggested actions for the user based on the analysis results, means for displaying the generated suggested actions, means for recognizing the user's emotions, and means for adjusting the suggested actions based on the recognized emotions. This makes it possible to quickly analyze the health condition of the pet with high accuracy and provide specific and appropriate suggested actions that take the user's emotions into consideration.

[0326] A "means for capturing pet images" is a device or software that allows a user to take an image of a pet or select an existing image using a device such as a smartphone or tablet, and then register that information in the system.

[0327] The "means for pre-processing captured images" refers to devices or software that perform processes such as resizing and noise removal on images captured on the terminal.

[0328] The "means for transmitting the preprocessed image to the server" refers to a communication function or software for transferring the preprocessed image to the server via a network.

[0329] "Means for analyzing a pet's health condition using an AI model on a server" refers to devices or software that use an AI model running on a server to analyze images of a pet and evaluate its health condition.

[0330] "Means for generating suggested actions for users based on analysis results" refers to devices or software that suggest specific ways of dealing with issues or actions to users based on the results of analysis by the AI ​​model.

[0331] The "means for displaying the generated action suggestions" refers to a device or software for visually displaying the action suggestions to the user on the terminal.

[0332] The "means for recognizing the user's emotions" refers to a device or software that analyzes the user's voice data or image data and identifies the user's emotional state.

[0333] The "means for adjusting suggested actions based on recognized emotions" refers to a device or software that optimizes the content of suggested actions provided based on the results of analyzing the user's emotions using an emotion engine.

[0334] The "means for displaying detailed explanations for high-risk areas" refers to a device or software that displays detailed explanations related to a particular high-risk area when a user requests detailed information about that area.

[0335] A "means for providing links to schedule an appointment or purchase a care product" is a device or software that provides an online link for a user to schedule an appointment or purchase a care product.

[0336] "Means for resizing or denoising images in preprocessing" refers to devices or software for changing the size of images to a resolution suitable for analysis or for reducing noise contained in images.

[0337] The present invention relates to a system that allows pet owners to grasp the health condition of their pets at an early stage and take appropriate measures, and further has the function of recognizing the user's emotions and providing appropriate action suggestions.

[0338] Basic system configuration

[0339] This system consists of the following components: Users use devices such as smartphones and tablets.

[0340] 1. Image capture method

[0341] User: Use the camera function on your smartphone or tablet to take a picture of your pet or select an existing image.

[0342] 2. Image preprocessing methods

[0343] On-device: Perform preprocessing such as resizing and noise reduction on the captured or selected image. For example, you can use an image processing library such as OpenCV.

[0344] 3. Image transmission method

[0345] Terminal: The preprocessed image is sent to the server using an HTTP request.

[0346] 4. AI model for analyzing health status

[0347] Server: The received image data is input into a deep learning model (e.g., a convolutional neural network) to analyze the pet's health condition. Deep learning libraries such as TensorFlow and PyTorch can be used. Specific analysis involves detecting faces and body parts and assessing the health risk of each part.

[0348] 5. Action suggestion generation means

[0349] Server: Based on the analysis results of the AI ​​model, the server generates specific action suggestions for the user. For example, an action suggestion might be, "Your eyes are bloodshot. Consider using eye drops. If the symptoms persist, consult a veterinarian."

[0350] 6. Action suggestion display means

[0351] On-device: The generated action suggestions are displayed to the user. Risk areas are marked on the image, and specific countermeasures are displayed as text. This can be implemented using a UI framework (e.g., Flutter or React Native).

[0352] 7. Emotion Engine

[0353] Terminal or server: Equipped with an emotion engine that analyzes the user's voice data and image data and recognizes the user's emotions. Watson or Azure Cognitive Services can be used for voice recognition, and OpenFace or FaceReader can be used for emotion recognition.

[0354] 8. Emotion-based behavioral adjustment measures

[0355] Server: Adjusts the suggested actions based on the user's emotions recognized by the emotion engine. Specifically, if the user is feeling stressed, it adds detailed and helpful explanations.

[0356] 9. Detailed explanation display means

[0357] Device: When the user taps on a high-risk area, detailed explanations and countermeasures for that area are displayed. For example, if the lead user taps on the eye area, specific instructions on how to use eye drops are displayed.

[0358] 10. How to book appointments and purchase care products

[0359] Terminal: Provides links for users to schedule appointments and purchase care products. With one click, users can schedule a veterinary appointment or purchase the care products they need online.

[0360] Specific examples

[0361] A specific example of use is shown below.

[0362] 1. The user launches the app and takes a full-body photo of their dog.

[0363] User: Use the app's camera function to take a full-body photo of your pet (e.g., your dog).

[0364] Device: Temporarily saves the captured image and displays the "Next" button.

[0365] 2. The device preprocesses the image

[0366] On the device: Resize the image and remove noise to optimize it for analysis. For example, resize the image to 1280x720 pixels, and use a Gaussian filter to remove noise.

[0367] Terminal: Sends the image data after preprocessing to the server.

[0368] 3. The server analyzes the image and generates action suggestions.

[0369] Server: The received images are input into the AI ​​model to analyze the pet's health condition. For example, risk areas such as bloodshot eyes and abdominal swelling are identified.

[0370] Server: Based on the analysis results, it generates appropriate action suggestions to the user (e.g., "Your eyes are bloodshot. Consider using eye drops. If symptoms persist, consult your veterinarian").

[0371] 4. The device displays the diagnosis results and suggested actions to the user.

[0372] Terminal: The diagnostic results are displayed intuitively to the user, clearly indicating risk areas and providing specific countermeasures.

[0373] 5. Users access more information and additional features

[0374] User: Tap on high-risk areas to see detailed explanations and solutions.

[0375] Terminal: Users can access links to book appointments and purchase care products as needed.

[0376] 6. Use of Emotion Engines

[0377] Device or server: While the user is using the app, it analyzes voice data and facial expressions to recognize the user's emotions (e.g., joy, stress, anxiety).

[0378] Server: Adjust the content of the suggested action based on the user's emotions. For example, if the user expresses anxiety, add a detailed and helpful explanation.

[0379] Specific prompt examples:

[0380] "Send a user a picture of their pet and have the AI ​​model analyze its health condition. Also, explain how it can recognize the user's emotions and provide appropriate behavioral suggestions."

[0381] The system of the present invention not only allows pet owners to accurately and quickly understand their pet's health condition and take appropriate measures, but also enables more effective care by the system recognizing the user's emotions and providing support accordingly.

[0382] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0383] Step 1:

[0384] Image capture

[0385] User: Use the device's camera to take a picture of your pet or select an existing image.

[0386] Input: New image data from the camera or existing image data from the photo gallery.

[0387] Output: Temporarily stored image data.

[0388] Step 2:

[0389] Image preprocessing

[0390] Terminal: Preprocess the image, such as resizing and denoising. For example, resize the image to 1280x720 pixels and denoise it using a Gaussian filter.

[0391] Input: Temporarily stored image data.

[0392] Output: Preprocessed image data.

[0393] Step 3:

[0394] Sending image data

[0395] Terminal: Send the preprocessed image to the server using an HTTP request.

[0396] Input: Preprocessed image data.

[0397] Output: Image data sent to the server.

[0398] Step 4:

[0399] Health status analysis

[0400] Server: The received image data is input into an AI model to analyze the pet's health. Specifically, it detects the face and body parts and evaluates the health risks associated with each part. For example, it identifies bloodshot eyes and swollen abdomen.

[0401] Input: Image data sent to the server.

[0402] Output: Analysis result data (risk assessment results).

[0403] Step 5:

[0404] Generate action suggestions

[0405] Server: Generates specific action suggestions for the user based on the analysis results. For example, it creates suggestions such as "Your eyes are bloodshot. Consider using eye drops. If the symptoms persist, consult a veterinarian."

[0406] Input: Analysis result data.

[0407] Output: Action suggestion data.

[0408] Step 6:

[0409] Displaying suggested actions

[0410] Device: The generated action suggestions are displayed in an easy-to-understand manner to the user. Specifically, risk areas are marked on the image and specific countermeasures are displayed in text.

[0411] Input: Action suggestion data.

[0412] Output: Action suggestions displayed to the user.

[0413] Step 7:

[0414] Emotion recognition

[0415] Terminal or server: Analyzes the user's voice data and image data and recognizes the user's emotions using an emotion engine. For example, it uses a voice recognition engine or facial expression recognition software.

[0416] Input: User's voice or image data.

[0417] Output: Recognized emotion data.

[0418] Step 8:

[0419] Adjusting behavioral suggestions according to emotions

[0420] Server: Optimizes the content of action suggestions based on the recognized emotion. For example, if the user is recognized as feeling stressed, it adds detailed and helpful explanations.

[0421] Input: Recognized emotion data and action suggestion data.

[0422] Output: Adjusted action suggestion data.

[0423] Step 9:

[0424] View detailed description

[0425] Device: When a user taps on a high-risk area, a detailed explanation and countermeasures for that area are displayed. For example, if a user taps on a high-risk area in the eyes, specific instructions on how to use eye drops are displayed.

[0426] Input: User taps on the risk area.

[0427] Output: Display detailed descriptive data.

[0428] Step 10:

[0429] Make appointments and purchase care products

[0430] Terminal: Provides links for scheduling appointments and purchasing care products. Users can use this to schedule veterinary appointments and purchase care products online.

[0431] Input: User clicks.

[0432] Output: Display of appointment booking form and care product purchase page.

[0433] (Application example 2)

[0434] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0435] There is a lack of systems that allow pet owners to effectively and quickly understand their pet's health condition and take appropriate measures. As a result, there is a risk that pet health problems will not be detected early and appropriate care will be delayed. In addition, the lack of action suggestions that take into account the pet owner's emotions may make users feel stressed, which could lead to a decline in the quality of care.

[0436] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for capturing an image of the pet, means for preprocessing the captured image, means for transmitting the preprocessed image to the server, means for analyzing the health condition of the pet using a generative AI model, means for generating action suggestions for the user based on the analysis results, means for recognizing the user's emotions, means for adjusting the action suggestions based on the recognized emotions, means for displaying the generated action suggestions, means for suggesting appropriate pet food, and means for generating a prompt message. This enables pet owners to grasp the health condition of their pets at an early stage and receive appropriate care and food suggestions based on the user's emotions.

[0437] A "means for capturing an image of a pet" is a device or software that allows a user to take or select an image of a pet.

[0438] The "means for pre-processing captured images" refers to devices or software that perform processes such as resizing and noise removal on captured or selected images.

[0439] The "means for transmitting preprocessed images to a server" refers to a device or software that transmits preprocessed image data to a remote server via a network.

[0440] A "generative AI model" is an artificial intelligence model that runs on a computer to analyze the health of pets.

[0441] "Means for analyzing the health condition of a pet" refers to a device or software that uses a generative AI model on a server to analyze image data and determine the health condition of a pet.

[0442] The "means for generating suggested actions for the user based on the analysis results" refers to a device or software that generates specific suggested actions for the pet owner based on the results of the analyzed health condition.

[0443] The "means for recognizing user's emotions" refers to a device or software that determines the user's emotions by analyzing the user's voice data and facial expression data.

[0444] The "means for adjusting suggested actions based on recognized emotions" refers to a device or software that adjusts the content of suggested actions taking into account the user's emotional state.

[0445] The "means for displaying the generated action suggestions" refers to a device or software that visually displays the generated action suggestions on the user's terminal.

[0446] The "means for suggesting appropriate pet food" refers to a device or software that suggests appropriate food for a pet, taking into consideration the pet's health condition and the user's emotions.

[0447] A "means for generating prompt sentences" is a device or software that generates questions or instructions that a user can use to assess the health of their pet.

[0448] This invention provides a system that allows pet owners to quickly understand their pet's health condition and take appropriate measures. This system includes a function that captures images of the pet, analyzes them using an AI model, recognizes the user's emotions, and adjusts behavior suggestions accordingly. The main components of this system and their functions are as follows:

[0449] 1. How to capture images of your pet

[0450] Users can take pictures of their pets using the camera function of their smartphones, tablets, or other devices. They can also select existing images. This image capture method allows users to accurately capture their pet's current state.

[0451] 2. A means of preprocessing the captured images

[0452] The device performs preprocessing such as resizing and noise removal on the captured image, making the image data suitable for AI analysis. Specifically, this preprocessing is performed using an image processing library such as OpenCV.

[0453] 3. A means to send preprocessed images to the server

[0454] Once preprocessed, the image data is sent from the device to a server over a network, where it can be analyzed using advanced AI models.

[0455] 4. A means of analyzing your pet's health

[0456] The server uses generative AI models such as TensorFlow and Keras to analyze the received image data. It detects the pet's face and body parts and determines the health risk of each part (normal, requires observation, emergency). Based on the analysis results, the appropriate countermeasures are determined.

[0457] 5. How to Recognize User Emotions

[0458] The device or server analyzes the user's voice data and facial expressions to perform emotion recognition, using emotion analysis tools such as EmotionRecognizer, to determine how the user feels about their pet's health.

[0459] 6. A way to tailor suggested actions based on emotions

[0460] The server then adjusts the suggested actions based on the user's perceived emotions: for example, if the user expresses anxiety, it adds more specific and helpful instructions, and also suggests appropriate pet food and care products.

[0461] 7. Means for displaying generated action suggestions

[0462] The device visually displays the action suggestions received from the server to the user, highlighting risk areas and providing specific solutions to help the user take action quickly.

[0463] Specific examples

[0464] The user launches the app and takes a full-body photo of their dog. The device preprocesses the image and sends it to the server. The server analyzes the image, detects "bloodshot eyes," and suggests to the user, "Consider using eye drops. If symptoms persist, consult a veterinarian." The user then uses the system's camera function to take a photo of their own face and performs emotion analysis. The analysis results identify the user's emotion as "anxiety," so the server makes a suggested course of action with a detailed and helpful explanation.

[0465] Prompt Sentence Examples

[0466] "Take a photo of your dog's face and body with this app and analyze its health condition. Then, take a photo of your face and analyze its emotions. Based on the analysis results, the app will make appropriate food and care recommendations."

[0467] This system allows pet owners to grasp their pet's health condition early on and receive appropriate care and food suggestions based on the user's emotions.

[0468] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0469] Step 1:

[0470] The user takes a picture of their pet. The user uses the camera function of their smartphone or tablet to capture the image of their pet. The input is the "pet image" and the output is the image data stored on the device.

[0471] Step 2:

[0472] The device preprocesses the image. Specifically, it resizes and removes noise using an image processing library such as OpenCV. The input is the image data obtained in step 1, and the output is the preprocessed image data. Resizing changes the resolution to a level suitable for analysis, and noise removal increases the clarity of the image.

[0473] Step 3:

[0474] Send the preprocessed image data to the server. The device sends the image data to the remote server via the network. The input is the preprocessed image data obtained in step 2, and the output is the image data sent to the server.

[0475] Step 4:

[0476] The server analyzes the pet's health condition. The image data is analyzed using a generative AI model (using TensorFlow and Keras) on the server. The input is the image data sent in step 3, and the output is the pet's health assessment results (e.g., "bloodshot eyes" or "swollen abdomen"). The AI ​​model detects specific parts of the face and body and determines the health risk of each.

[0477] Step 5:

[0478] Recognize user emotions. The user uses a device to capture their own voice or facial image. The device or server performs emotion analysis using EmotionRecognizer. The input is the user's voice data or facial image data, and the output is the user's emotional assessment result (e.g., "joy," "stress," or "anxiety").

[0479] Step 6:

[0480] The server adjusts the action suggestions based on the user's emotions. The action suggestions generated from the analysis results are adjusted taking into account the recognized user's emotions. The input is the health assessment result from step 4 and the emotion assessment result from step 5, and the output is the final action suggestions. For example, for users who show anxiety, a detailed and friendly explanation is added.

[0481] Step 7:

[0482] The server recommends the appropriate pet food based on the pet's health condition and the user's emotions. The input is the health assessment result in step 4 and the emotion assessment result in step 5, and the output is the food recommendation.

[0483] Step 8:

[0484] Display the generated action suggestions. The device visually displays the action suggestions received from the server to the user, providing specific solutions and links to care products. The input is the suggestion results from steps 6 and 7, and the output is the action suggestions displayed on the user's device.

[0485] Step 9:

[0486] The user accesses detailed information and additional features. The user taps on a high-risk area to see detailed explanations and solutions, and then accesses links to schedule an appointment or purchase care products. The input is the action suggestion link provided by the server, and the output is the user's action (e.g., schedule an appointment, purchase a product).

[0487] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0488] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0489] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0490] [Second embodiment]

[0491] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0492] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0493] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0494] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0495] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0496] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0497] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0498] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0499] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0500] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0501] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0502] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0503] This invention is a system that allows pet owners to grasp the health condition of their pets at an early stage and take appropriate measures. Specifically, it assesses the health risks of pets through a series of processes that involves capturing images of the pet, preprocessing the images, and analyzing them based on an AI model, and then provides users with suggested actions.

[0504] System configuration

[0505] 1. Image capture method

[0506] User: Uses a device such as a smartphone or tablet to take a picture of their pet or select an existing image.

[0507] 2. Image preprocessing methods

[0508] Device: Performs preprocessing such as resizing and noise removal on captured or selected images, making the image data suitable for AI analysis.

[0509] 3. Image transmission method

[0510] Terminal: Sends the pre-processed image to the server.

[0511] 4. AI model for analyzing health status

[0512] Server: The received image data is input into the AI ​​model to analyze the pet's health condition. Specifically, it detects the pet's face and body parts and determines the health risk of each part (e.g., normal, requires observation, emergency).

[0513] 5. Action suggestion generation means

[0514] Server: Based on the analysis results of the AI ​​model, it generates specific action suggestions for the user (e.g., recommendations for daily care, the need for a veterinarian).

[0515] 6. Action suggestion display means

[0516] Device: The generated action proposals are displayed in an easy-to-understand manner to the user. Specifically, risk areas and countermeasures are displayed on the screen.

[0517] 7. Detailed explanation display means

[0518] Device: When the user taps on a high-risk area, a detailed explanation and countermeasures for that area will be displayed.

[0519] 8. How to book appointments and purchase care products

[0520] Terminal: Provides links to schedule appointments and purchase care products. This feature allows users to easily schedule veterinary appointments and purchase necessary care products.

[0521] Specific examples

[0522] 1. The user launches the app and takes a full-body photo of their dog.

[0523] User: Use the app's camera function to take a full-body photo of your pet (e.g., dog).

[0524] Device: Temporarily saves the captured image and displays the "Next" button.

[0525] 2. The device preprocesses the image

[0526] Device: Resize and denoise the image to make it optimal for analysis.

[0527] Terminal: Sends the image data after preprocessing to the server.

[0528] 3. The server analyzes the image and generates action suggestions.

[0529] Server: The received images are input into the AI ​​model to analyze the pet's health condition. For example, risk areas such as bloodshot eyes and abdominal swelling are identified.

[0530] Server: Based on the analysis results, it generates appropriate action suggestions to the user (e.g., "Your eyes are bloodshot. Consider using eye drops. If symptoms persist, consult your veterinarian").

[0531] 4. The device displays the diagnosis results and suggested actions to the user.

[0532] Terminal: The diagnostic results are displayed intuitively to the user, clearly indicating risk areas and providing specific countermeasures.

[0533] 5. Users access more information and additional features

[0534] User: Tap on high-risk areas to see detailed explanations and solutions.

[0535] Terminal: Users can access links to book appointments and purchase care products as needed.

[0536] This system allows pet owners to easily understand their pet's health status and quickly take necessary measures, which is expected to make pet health management more efficient and effective.

[0537] The processing flow will be explained below.

[0538] Step 1:

[0539] The user launches the app on their smartphone and takes a picture of their pet or selects an existing image, providing the system with up-to-date information about their pet's health.

[0540] Step 2:

[0541] The device temporarily stores the captured or selected image, which is then preprocessed.

[0542] Step 3:

[0543] The device resizes the image to make it easier for the AI ​​model to analyze high-resolution images.

[0544] Step 4:

[0545] The device performs noise reduction, which improves image quality and increases the accuracy of analysis.

[0546] Step 5:

[0547] The device sends the preprocessed image to the server, which forwards the image to the server for analysis.

[0548] Step 6:

[0549] The server inputs the received image into the AI ​​model, which then detects each part of the pet based on the image.

[0550] Step 7:

[0551] The server uses an AI model to analyze the pet's health condition, identifying risk areas such as bloodshot eyes or abdominal swelling, and determining the risk level (e.g., normal, requires observation, or emergency).

[0552] Step 8:

[0553] The server generates action suggestions for the user based on the analysis results, such as "Your eyes are bloodshot. Consider using eye drops. If the symptoms persist, consult a veterinarian."

[0554] Step 9:

[0555] The device receives the diagnostic results and action suggestions from the server and displays them to the user. Risk areas and solutions are presented through an intuitive interface.

[0556] Step 10:

[0557] When the user taps on a high-risk area, the device will display a detailed explanation and solutions, such as the cause of "bloodshot eyes" and how to care for them at home.

[0558] Step 11:

[0559] The device provides links to schedule appointments and purchase care products, allowing users to schedule veterinary appointments and purchase medications as needed.

[0560] This series of steps creates a system that allows pet owners to accurately and quickly understand their pet's health condition and take appropriate measures.

[0561] Example 1

[0562] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0563] For modern pet owners, it is becoming increasingly important to understand their pet's health condition early and take appropriate measures. However, traditional methods often result in late detection of abnormalities in pets or in failure to take appropriate action. Furthermore, there is a lack of means to obtain specific action suggestions and detailed explanations in real time based on a pet's health condition, making it difficult for pet owners to manage their pet's health efficiently and effectively.

[0564] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0565] In this invention, the server includes means for capturing images of the pet, means for preprocessing the captured images, means for transmitting the preprocessed images to the server, means for analyzing the health condition of the pet using a generative AI model on the server, means for generating action suggestions for the user based on the analysis results, means for displaying the generated action suggestions, means for displaying detailed explanations of high-risk areas when the user taps on those areas, and means for providing links to make appointments for medical examinations and purchase care products. This enables pet owners to grasp the health condition of their pets at an early stage and take prompt and appropriate measures.

[0566] "Means for capturing images" refers to a means by which a user can take an image of a pet using a device such as a smartphone or tablet, or select an existing image.

[0567] "Preprocessing means" refers to the process of resizing and removing noise from the captured image, making the image data suitable for AI analysis.

[0568] The "means for transmitting to the server" is a means for transmitting the image preprocessed on the terminal to the server via communication.

[0569] The "analysis means using a generative AI model" is a means of analyzing received pet image data using an AI model placed on a server and evaluating the pet's health condition.

[0570] The "means for generating behavioral suggestions" refers to a means for creating specific behavioral suggestions according to the health condition of a pet based on the analysis results of the generative AI model.

[0571] The "means for displaying suggested actions" is a means for displaying the content of suggested actions on the user's terminal in an easily understandable manner.

[0572] The "means for displaying a detailed explanation" is a means for displaying a detailed explanation and a method for dealing with a high-risk area when the user taps on that area.

[0573] "Means for providing links to schedule appointments and purchase care products" refers to means for providing links that allow a user to easily schedule an appointment with a veterinarian or purchase necessary care products.

[0574] This invention is a system that allows pet owners to grasp the health condition of their pets at an early stage and take prompt and appropriate measures to efficiently and effectively manage the health of their pets. A specific implementation method of this system is described in detail below.

[0575] Hardware and software used

[0576] 1. User Device

[0577] Hardware: Smartphones, tablets

[0578] Software: Dedicated application (e.g., pet health management app)

[0579] 2. Server

[0580] Hardware: High-performance servers (e.g., servers with Intel Xeon processors)

[0581] Software: A software environment for running generative AI models (e.g., Python, TensorFlow)

[0582] System configuration and data processing

[0583] 1. Image capture method

[0584] User: Launches the application on their smartphone or tablet to take a picture of their pet. The user can use the camera function within the application to take a picture of their pet or select an existing image.

[0585] 2. Image preprocessing methods

[0586] On-device: Captured or selected images are resized and pre-processed on-device, including resizing and noise reduction. This involves scaling the image resolution to the appropriate size and applying Gaussian filters to reduce noise.

[0587] 3. Image transmission method

[0588] Terminal: After preprocessing, the image data is sent to the server using an HTTP POST request.

[0589] 4. AI model for analyzing health status

[0590] Server: The server inputs the received image data into a generative AI model to detect the pet's face and body parts and assess health risks. This analysis uses a TensorFlow-based deep learning model.

[0591] 5. Action suggestion generation means

[0592] Server: Based on the analysis results, the server generates specific action suggestions for the user. For example, if it determines that a pet's eyes are bloodshot, it generates a suggestion such as "Consider using eye drops. If the symptoms persist, consult a veterinarian."

[0593] 6. Action suggestion display means

[0594] Device: The generated action proposals are displayed on the user's device, highlighting risk areas and showing specific countermeasures.

[0595] 7. Detailed explanation display means

[0596] On the device: When the user taps on a high-risk area, a detailed explanation and solutions for that area are displayed.

[0597] 8. How to book appointments and purchase care products

[0598] Terminal: Links for booking appointments and purchasing care products are displayed on the terminal, allowing the user to book an appointment with a veterinarian or purchase any necessary care products.

[0599] Specific examples

[0600] The user launches a dedicated application and takes a full-body photo of their pet (e.g., a dog) using their smartphone's camera. The application resizes the image, removes noise, and then sends it to a server. The server uses a generative AI model to analyze the health status of the image. Based on the analysis results, the server generates a suggested action, such as "Your eyes are bloodshot, so consider using eye drops. If symptoms persist, consult a veterinarian," and displays it on the user's device. When the user taps on high-risk areas based on the results, detailed explanations and countermeasures are displayed. Links to make appointments and purchase care products are also provided.

[0601] Examples of prompt statements

[0602] Take or select an image of your pet. After pre-processing, the image will be analyzed to assess your pet's health. For example, if it determines that your dog's eyes are bloodshot, it will display specific action suggestions such as "Consider using eye drops. If symptoms persist, consult a veterinarian."

[0603] This system is expected to enable pet owners to detect health conditions in their pets early and respond quickly and accurately, making pet health management more efficient and effective.

[0604] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0605] Step 1: Importing an image

[0606] User: The user launches a dedicated application on their smartphone or tablet and takes a picture of their pet or selects an existing image.

[0607] Input: A photo of your pet (either a new or existing photo)

[0608] What happens: The user taps the app's "Take Image" button and takes a full-body photo of their pet with the camera, or taps the "Select Image" button to choose an existing image from the gallery.

[0609] Output: Image data of the pet taken or selected by the user

[0610] Step 2: Image preprocessing

[0611] Device: The device performs pre-processing such as resizing and noise reduction on the captured or selected image.

[0612] Input: Image data of the pet taken or selected by the user

[0613] What it does: Resizes the image to a resolution of 600x800 pixels and reduces noise using a Gaussian filter. The application performs these steps automatically.

[0614] Output: Preprocessed image data

[0615] Step 3: Sending images

[0616] Terminal: Sends the image data after preprocessing to the server.

[0617] Input: Preprocessed image data

[0618] Specific operation: The device uses an HTTP POST request to upload the preprocessed image to the server.

[0619] Output: Image data sent to the server

[0620] Step 4: Analyze health status using AI models

[0621] Server: The server inputs the received image data into a generative AI model to detect the pet's face and body parts and assess health risks.

[0622] Input: Image data sent to the server

[0623] How it works: A Python script on the server receives the image and analyzes it using a TensorFlow-based generative AI model, which detects features such as bloodshot eyes and abnormal coat condition and assesses health risks.

[0624] Output: Analysis results on pet health risks

[0625] Step 5: Generate action suggestions

[0626] Server: Based on the analysis results, the server generates specific action suggestions for the user.

[0627] Input: Analysis results on pet health risks

[0628] Specific operation: Based on the analysis results obtained by the server, the server generates specific suggestions by inserting them into a pre-prepared text template. For example, it generates a suggestion such as, "Your eyes are bloodshot. Consider using eye drops. If the symptoms persist, consult a veterinarian."

[0629] Output: Suggested actions for the user

[0630] Step 6: View suggested actions

[0631] Terminal: The generated action suggestions are displayed on the user's terminal.

[0632] Input: Suggested actions for the user

[0633] Specific actions: Highlight risk areas on the device screen and provide text explaining specific ways to deal with the problem. For example, a bloodshot eye is highlighted in a red frame with suggested actions displayed below.

[0634] Output: Action suggestions displayed on the user's device

[0635] Step 7: View detailed instructions

[0636] User: When the user taps on a high-risk area, a detailed explanation and solutions for that area will be displayed.

[0637] Input: User tap operation, information on high-risk areas

[0638] Specific operation: When the user taps the red frame of the risk area, a detailed information screen will open, displaying an explanation of the eye symptoms and how to deal with them.

[0639] Output: Detailed explanation and solutions

[0640] Step 8: Book a consultation and purchase care products

[0641] Device: Links to schedule appointments and purchase care products will be displayed on the device.

[0642] Input: Relevant links based on identified health risks

[0643] Specific behavior: Link buttons to a medical appointment booking page and a care product online store are displayed on the screen. When the user taps these buttons, the respective pages open.

[0644] Output: Links to book appointments and purchase care products

[0645] (Application example 1)

[0646] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0647] Properly managing the health of pets is an important issue for pet owners. With current technology, it is difficult to grasp a pet's health condition early and take specific measures. This increases the risk that pet owners without specialized veterinary knowledge may overlook abnormalities in their pets. Furthermore, the lack of a simple and rapid means of assessing a pet's health raises concerns that pet care may be neglected in today's busy society. Therefore, to solve these problems, a system is needed that allows users to easily assess a pet's health condition and take specific measures.

[0648] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0649] In this invention, the server includes a means for capturing images of pets, a means for preprocessing the captured images, and a means for transmitting the preprocessed images to the server. This allows the pet images to be analyzed using an AI model, and specific action suggestions for the user to be automatically generated and displayed. Additionally, by adding a means for displaying detailed explanations and countermeasures by tapping on risk areas and a means for providing links to make medical appointments or purchase care products, pet owners can immediately take appropriate measures. Furthermore, the accuracy of the analysis is improved by adding a means for resizing and noise reduction to the image preprocessing and normalizing the resized images before inputting the image data into the AI ​​model. This enables early detection of pet health conditions and appropriate measures.

[0650] "Means for capturing an image of a pet" refers to a function that allows a user to take an image of a pet or select an existing image using a device such as a smartphone or tablet.

[0651] "Means for preprocessing captured images" refers to a function that processes captured or selected images, such as resizing and noise removal, to prepare the image data in a state suitable for AI analysis.

[0652] The "means for transmitting preprocessed images to a server" is a function for transmitting preprocessed images to a server via the Internet.

[0653] "Means for analyzing pet health conditions using an AI model on a server" refers to a function that inputs image data stored on a server into an AI model, analyzes the pet's health conditions, and identifies risk areas.

[0654] "Means for generating action suggestions for users based on analysis results" refers to a function for generating appropriate action suggestions for users based on the analysis results of the AI ​​model.

[0655] The "means for displaying the generated action proposal" is a function for displaying the generated action proposal in an easily understandable manner to the user.

[0656] "A means of displaying detailed explanations and solutions by tapping on risk areas" is a function that allows the user to tap on risk areas and display detailed explanations and solutions for that area.

[0657] "Means for providing links to schedule appointments or purchase care products" refers to a function that provides links for users to schedule appointments or purchase care products.

[0658] "Resizing" is a process of changing the size of an image.

[0659] "Noise removal" is a process that removes unnecessary information (noise) from an image.

[0660] "Normalization" is the process of aligning the range of data to a certain scale.

[0661] The present invention provides a system that enables pet owners to grasp the health condition of their pets at an early stage and take appropriate measures. Specific embodiments for carrying out the present invention will be described in detail below.

[0662] System configuration

[0663] 1. Image capture method

[0664] Users can take pictures of their pets using a smartphone, tablet, or other device, and the images are temporarily saved in the app.

[0665] 2. Image preprocessing methods

[0666] The device resizes and denoises the captured images. This preprocessing is performed using OpenCV. Resizing adjusts the image size appropriately, and denoising removes unnecessary data that may affect analysis.

[0667] 3. Image transmission method

[0668] After preprocessing, the image data is sent from the device to the server via Firebase or AWS.

[0669] 4. AI model for analyzing health status

[0670] The server runs an AI model using TensorFlow or PyTorch to analyze the received image data, specifically detecting pet faces and body parts and assessing their health risks.

[0671] 5. Action suggestion generation means

[0672] Based on the analysis results of the AI ​​model, the server generates specific action suggestions for the user, such as "Your eyes are bloodshot. Consider using eye drops. If the symptoms persist, consult a veterinarian."

[0673] 6. Action suggestion display means

[0674] The device displays the generated action suggestions in an easy-to-understand manner to the user, visually indicating risk areas and presenting specific countermeasures.

[0675] 7. Detailed explanation display means

[0676] When the user taps on a risk area, the device displays a detailed explanation and instructions on how to deal with that area, allowing pet owners to take appropriate measures with confidence.

[0677] 8. How to book appointments and purchase care products

[0678] The device provides links to help users easily schedule appointments and purchase care products, helping them get the care they need faster.

[0679] Specific examples

[0680] Below we introduce some specific scenarios in which this system can be used.

[0681] 1. The user launches the app and takes a full-body photo of their dog.

[0682] The user uses the app's camera function to take a full-body photo of their pet (e.g., dog). The device temporarily saves the image and displays a "Next" button.

[0683] 2. The device preprocesses the image

[0684] The device resizes the image and performs noise reduction. This preprocessing is done using OpenCV. The preprocessed image data is then sent to the server.

[0685] 3. The server analyzes the image and generates action suggestions.

[0686] The server inputs the received images into an AI model using TensorFlow or PyTorch to analyze the pet's health. For example, it identifies risk areas such as bloodshot eyes and abdominal swelling. Based on the analysis results, it then generates appropriate action suggestions for the user.

[0687] 4. The device displays the diagnosis results and suggested actions to the user.

[0688] The generated action suggestions are displayed on the device in an easy-to-understand manner to the user, visually indicating risk areas and presenting specific countermeasures.

[0689] Example prompts to input to the generative AI model

[0690] "Evaluate the condition of this dog's face and determine its health risks."

[0691] "Please use an AI model to analyze whether there is anything abnormal about this cat's abdomen."

[0692] This invention allows pet owners to easily understand the health status of their pets and quickly take necessary measures, which is expected to make pet health management more efficient and effective.

[0693] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0694] Step 1:

[0695] The user launches the app and captures an image of their pet. The user can either take a picture of their pet using the camera function of their smartphone or tablet, or import an existing image into the app. The input is a newly taken image or an existing image, which is temporarily stored on the device. The output is image data that awaits preprocessing.

[0696] Step 2:

[0697] The device performs preprocessing. Image resizing and noise removal are performed within the device. Specifically, OpenCV is used to appropriately adjust the image size and remove noise. The input is image data awaiting preprocessing, and the output is preprocessed image data suitable for AI analysis.

[0698] Step 3:

[0699] The device sends the preprocessed image to the server. The preprocessed image data is uploaded to the server via Firebase or AWS. The preprocessed image data is the input, and the image data stored on the server is the output.

[0700] Step 4:

[0701] The server analyzes the image using an AI model. Using TensorFlow or PyTorch on the server, the received image data is input into the AI ​​model. The AI ​​model detects the pet's face and body parts and evaluates health risks. The image data stored on the server is used as input, and the analysis results of the pet's health condition are obtained as output.

[0702] Step 5:

[0703] The server generates action suggestions. Based on the analysis results of the AI ​​model, the server generates specific action suggestions for the user. For example, an action suggestion might be something like, "Your eyes are bloodshot. Consider using eye drops. If the symptoms persist, consult a veterinarian." The input is the analysis result data of the pet's health condition, and the output is action suggestion data.

[0704] Step 6:

[0705] The terminal displays the action proposal. The terminal displays the generated action proposal to the user in a visually easy-to-understand manner. Specifically, it highlights risk areas and shows how to deal with them. The input is the action proposal data, and the output is the action proposal displayed to the user.

[0706] Step 7:

[0707] The user taps on a risky area. When the user taps on a risky area on the action suggestion display screen, a detailed explanation and countermeasures for that area are displayed. Specifically, a detailed screen is displayed that describes the symptoms and describes countermeasures. The user's tap operation is the input, and detailed information is displayed as the output.

[0708] Step 8:

[0709] The terminal provides links for making appointments and purchasing care products. When the user taps the links to make appointments or purchase care products as needed, they are redirected to an external site and the procedure is completed. The input is the action suggestion data and the user's link tap operation, and the output is the completion of the appointment or product purchase.

[0710] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0711] This invention combines a system that allows pet owners to quickly understand the health condition of their pets and take appropriate measures with an emotion engine that recognizes the user's emotions. Specifically, the system captures images of the pet, preprocesses the images, analyzes them based on an AI model, and evaluates the pet's health risks. In addition, the system recognizes the user's emotions and suggests actions based on the emotions.

[0712] System configuration

[0713] 1. Image capture method

[0714] User: Uses a device such as a smartphone or tablet to take a picture of their pet or select an existing image.

[0715] 2. Image preprocessing methods

[0716] Device: Performs preprocessing such as resizing and noise removal on captured or selected images, making the image data suitable for AI analysis.

[0717] 3. Image transmission method

[0718] Terminal: Sends the pre-processed image to the server.

[0719] 4. AI model for analyzing health status

[0720] Server: The received image data is input into the AI ​​model to analyze the pet's health condition. Specifically, it detects the pet's face and body parts and determines the health risk of each part (e.g., normal, requires observation, emergency).

[0721] 5. Action suggestion generation means

[0722] Server: Based on the analysis results of the AI ​​model, it generates specific action suggestions for the user (e.g., recommendations for daily care, the need for a veterinarian).

[0723] 6. Action suggestion display means

[0724] Device: The generated action proposals are displayed in an easy-to-understand manner to the user. Specifically, risk areas and countermeasures are displayed on the screen.

[0725] 7. Emotion Engine

[0726] Terminal or server: Equipped with an emotion engine that recognizes the user's emotions, it determines the user's emotions by analyzing the user's voice data and image data.

[0727] 8. Emotion-based behavioral adjustment measures

[0728] Server: Adjusts suggested actions and provides appropriate feedback based on the user's emotions recognized by the emotion engine. For example, if the user is feeling stressed, the server provides more helpful and specific explanations.

[0729] 9. Detailed explanation display means

[0730] Device: When the user taps on a high-risk area, a detailed explanation and countermeasures for that area will be displayed.

[0731] 10. How to book appointments and purchase care products

[0732] Terminal: Provides links to schedule appointments and purchase care products. This feature allows users to easily schedule veterinary appointments and purchase necessary care products.

[0733] Specific examples

[0734] 1. The user launches the app and takes a full-body photo of their dog.

[0735] User: Use the app's camera function to take a full-body photo of your pet (e.g., dog).

[0736] Device: Temporarily saves the captured image and displays the "Next" button.

[0737] 2. The device preprocesses the image

[0738] Device: Resize and denoise the image to make it optimal for analysis.

[0739] Terminal: Sends the image data after preprocessing to the server.

[0740] 3. The server analyzes the image and generates action suggestions.

[0741] Server: The received images are input into the AI ​​model to analyze the pet's health condition. For example, risk areas such as bloodshot eyes and abdominal swelling are identified.

[0742] Server: Based on the analysis results, it generates appropriate action suggestions to the user (e.g., "Your eyes are bloodshot. Consider using eye drops. If symptoms persist, consult your veterinarian").

[0743] 4. The device displays the diagnosis results and suggested actions to the user.

[0744] Terminal: The diagnostic results are displayed intuitively to the user, clearly indicating risk areas and providing specific countermeasures.

[0745] 5. Users access more information and additional features

[0746] User: Tap on high-risk areas to see detailed explanations and solutions.

[0747] Terminal: Users can access links to book appointments and purchase care products as needed.

[0748] 6. Use of Emotion Engines

[0749] Device or server: While the user is using the app, it analyzes voice data and facial expressions to recognize the user's emotions (e.g., joy, stress, anxiety).

[0750] Server: Adjust the content of the suggested action based on the user's emotions. For example, if the user expresses anxiety, add a detailed and helpful explanation.

[0751] This system not only enables pet owners to accurately and quickly understand their pet's health condition and take appropriate measures, but also enables more effective care by recognizing the user's emotions and providing support accordingly.

[0752] The processing flow will be explained below.

[0753] Step 1:

[0754] The user launches the smartphone app and takes a picture of their pet or selects an existing image, providing the pet's health status as input data.

[0755] Step 2:

[0756] The device temporarily stores the captured or selected image, which is then used in the next processing step.

[0757] Step 3:

[0758] Your device will resize the image to the appropriate resolution, reducing data usage and improving processing speed.

[0759] Step 4:

[0760] The device will then perform noise reduction to improve image quality, which will improve the accuracy of the analysis by the AI ​​model.

[0761] Step 5:

[0762] The device sends the preprocessed image data to the server, which forwards the image data to the server for analysis.

[0763] Step 6:

[0764] The server inputs the received image data into the AI ​​model, which analyzes the image and begins the process of assessing the pet's health.

[0765] Step 7:

[0766] The server uses an AI model to detect the pet's face and body parts, such as the eyes, ears, mouth, and abdomen, and assesses risk areas.

[0767] Step 8:

[0768] The server evaluates the pet's health risk and determines the risk level (e.g., normal, requires observation, emergency). Based on the analysis results, it generates action suggestions for the user.

[0769] Step 9:

[0770] The terminal displays the diagnosis results and action suggestions received from the server to the user. An interface including risk areas and solutions is displayed.

[0771] Step 10:

[0772] When a user uses the app, the device or server inputs the user's voice data and image data into the emotion engine, which analyzes this data and recognizes the user's emotions.

[0773] Step 11:

[0774] The server analyzes the user's emotional data and adjusts the suggested actions, for example, adding specific and helpful explanations if the user is feeling stressed.

[0775] Step 12:

[0776] The device displays behavior suggestions tailored to the user's emotions, providing appropriate feedback according to the user's emotional state.

[0777] Step 13:

[0778] When the user taps on a high-risk area, the device displays a detailed explanation and how to deal with it. The user can check the cause of the specific symptom and how to deal with it.

[0779] Step 14:

[0780] The device provides links to schedule appointments and purchase care products. Users can schedule appointments with veterinarians or purchase care products as needed.

[0781] This system allows pet owners to accurately understand their pet's health condition and take appropriate measures. It also recognizes the user's emotions and provides support accordingly, enabling more effective care.

[0782] Example 2

[0783] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0784] There are few systems that can quickly identify a pet's health condition and provide appropriate treatment, and the suggested actions provided by these systems often do not take the user's feelings into consideration. Furthermore, due to a lack of preprocessing functions to improve the accuracy of pet image data analysis, the reliability of analysis results is often low. This makes it difficult for pet owners to manage their pet's health with peace of mind.

[0785] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0786] In this invention, the server includes means for capturing an image of the pet, means for preprocessing the captured image, means for transmitting the preprocessed image to the server, means for analyzing the health condition of the pet using an AI model on the server, means for generating suggested actions for the user based on the analysis results, means for displaying the generated suggested actions, means for recognizing the user's emotions, and means for adjusting the suggested actions based on the recognized emotions. This makes it possible to quickly analyze the health condition of the pet with high accuracy and provide specific and appropriate suggested actions that take the user's emotions into consideration.

[0787] A "means for capturing pet images" is a device or software that allows a user to take an image of a pet or select an existing image using a device such as a smartphone or tablet, and then register that information in the system.

[0788] The "means for pre-processing captured images" refers to devices or software that perform processes such as resizing and noise removal on images captured on the terminal.

[0789] The "means for transmitting the preprocessed image to the server" refers to a communication function or software for transferring the preprocessed image to the server via a network.

[0790] "Means for analyzing a pet's health condition using an AI model on a server" refers to devices or software that use an AI model running on a server to analyze images of a pet and evaluate its health condition.

[0791] "Means for generating suggested actions for users based on analysis results" refers to devices or software that suggest specific ways of dealing with issues or actions to users based on the results of analysis by the AI ​​model.

[0792] The "means for displaying the generated action suggestions" refers to a device or software for visually displaying the action suggestions to the user on the terminal.

[0793] The "means for recognizing the user's emotions" refers to a device or software that analyzes the user's voice data or image data and identifies the user's emotional state.

[0794] The "means for adjusting suggested actions based on recognized emotions" refers to a device or software that optimizes the content of suggested actions provided based on the results of analyzing the user's emotions using an emotion engine.

[0795] The "means for displaying detailed explanations for high-risk areas" refers to a device or software that displays detailed explanations related to a particular high-risk area when a user requests detailed information about that area.

[0796] A "means for providing links to schedule an appointment or purchase a care product" is a device or software that provides an online link for a user to schedule an appointment or purchase a care product.

[0797] "Means for resizing or denoising images in preprocessing" refers to devices or software for changing the size of images to a resolution suitable for analysis or for reducing noise contained in images.

[0798] The present invention relates to a system that allows pet owners to grasp the health condition of their pets at an early stage and take appropriate measures, and further has the function of recognizing the user's emotions and providing appropriate action suggestions.

[0799] Basic system configuration

[0800] This system consists of the following components: Users use devices such as smartphones and tablets.

[0801] 1. Image capture method

[0802] User: Use the camera function on your smartphone or tablet to take a picture of your pet or select an existing image.

[0803] 2. Image preprocessing methods

[0804] On-device: Perform preprocessing such as resizing and noise reduction on the captured or selected image. For example, you can use an image processing library such as OpenCV.

[0805] 3. Image transmission method

[0806] Terminal: The preprocessed image is sent to the server using an HTTP request.

[0807] 4. AI model for analyzing health status

[0808] Server: The received image data is input into a deep learning model (e.g., a convolutional neural network) to analyze the pet's health condition. Deep learning libraries such as TensorFlow and PyTorch can be used. Specific analysis involves detecting faces and body parts and assessing the health risk of each part.

[0809] 5. Action suggestion generation means

[0810] Server: Based on the analysis results of the AI ​​model, the server generates specific action suggestions for the user. For example, an action suggestion might be, "Your eyes are bloodshot. Consider using eye drops. If the symptoms persist, consult a veterinarian."

[0811] 6. Action suggestion display means

[0812] On-device: The generated action suggestions are displayed to the user. Risk areas are marked on the image, and specific countermeasures are displayed as text. This can be implemented using a UI framework (e.g., Flutter or React Native).

[0813] 7. Emotion Engine

[0814] Terminal or server: Equipped with an emotion engine that analyzes the user's voice data and image data and recognizes the user's emotions. Watson or Azure Cognitive Services can be used for voice recognition, and OpenFace or FaceReader can be used for emotion recognition.

[0815] 8. Emotion-based behavioral adjustment measures

[0816] Server: Adjusts the suggested actions based on the user's emotions recognized by the emotion engine. Specifically, if the user is feeling stressed, it adds detailed and helpful explanations.

[0817] 9. Detailed explanation display means

[0818] Device: When the user taps on a high-risk area, detailed explanations and countermeasures for that area are displayed. For example, if the lead user taps on the eye area, specific instructions on how to use eye drops are displayed.

[0819] 10. How to book appointments and purchase care products

[0820] Terminal: Provides links for users to schedule appointments and purchase care products. With one click, users can schedule a veterinary appointment or purchase the care products they need online.

[0821] Specific examples

[0822] A specific example of use is shown below.

[0823] 1. The user launches the app and takes a full-body photo of their dog.

[0824] User: Use the app's camera function to take a full-body photo of your pet (e.g., your dog).

[0825] Device: Temporarily saves the captured image and displays the "Next" button.

[0826] 2. The device preprocesses the image

[0827] On the device: Resize the image and remove noise to optimize it for analysis. For example, resize the image to 1280x720 pixels, and use a Gaussian filter to remove noise.

[0828] Terminal: Sends the image data after preprocessing to the server.

[0829] 3. The server analyzes the image and generates action suggestions.

[0830] Server: The received images are input into the AI ​​model to analyze the pet's health condition. For example, risk areas such as bloodshot eyes and abdominal swelling are identified.

[0831] Server: Based on the analysis results, it generates appropriate action suggestions to the user (e.g., "Your eyes are bloodshot. Consider using eye drops. If symptoms persist, consult your veterinarian").

[0832] 4. The device displays the diagnosis results and suggested actions to the user.

[0833] Terminal: The diagnostic results are displayed intuitively to the user, clearly indicating risk areas and providing specific countermeasures.

[0834] 5. Users access more information and additional features

[0835] User: Tap on high-risk areas to see detailed explanations and solutions.

[0836] Terminal: Users can access links to book appointments and purchase care products as needed.

[0837] 6. Use of Emotion Engines

[0838] Device or server: While the user is using the app, it analyzes voice data and facial expressions to recognize the user's emotions (e.g., joy, stress, anxiety).

[0839] Server: Adjust the content of the suggested action based on the user's emotions. For example, if the user expresses anxiety, add a detailed and helpful explanation.

[0840] Specific prompt examples:

[0841] "Send a user a picture of their pet and have the AI ​​model analyze its health condition. Also, explain how it can recognize the user's emotions and provide appropriate behavioral suggestions."

[0842] The system of the present invention not only allows pet owners to accurately and quickly understand their pet's health condition and take appropriate measures, but also enables more effective care by the system recognizing the user's emotions and providing support accordingly.

[0843] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0844] Step 1:

[0845] Image capture

[0846] User: Use the device's camera to take a picture of your pet or select an existing image.

[0847] Input: New image data from the camera or existing image data from the photo gallery.

[0848] Output: Temporarily stored image data.

[0849] Step 2:

[0850] Image preprocessing

[0851] Terminal: Preprocess the image, such as resizing and denoising. For example, resize the image to 1280x720 pixels and denoise it using a Gaussian filter.

[0852] Input: Temporarily stored image data.

[0853] Output: Preprocessed image data.

[0854] Step 3:

[0855] Sending image data

[0856] Terminal: Send the preprocessed image to the server using an HTTP request.

[0857] Input: Preprocessed image data.

[0858] Output: Image data sent to the server.

[0859] Step 4:

[0860] Health status analysis

[0861] Server: The received image data is input into an AI model to analyze the pet's health. Specifically, it detects the face and body parts and evaluates the health risks associated with each part. For example, it identifies bloodshot eyes and swollen abdomen.

[0862] Input: Image data sent to the server.

[0863] Output: Analysis result data (risk assessment results).

[0864] Step 5:

[0865] Generate action suggestions

[0866] Server: Generates specific action suggestions for the user based on the analysis results. For example, it creates suggestions such as "Your eyes are bloodshot. Consider using eye drops. If the symptoms persist, consult a veterinarian."

[0867] Input: Analysis result data.

[0868] Output: Action suggestion data.

[0869] Step 6:

[0870] Displaying suggested actions

[0871] Device: The generated action suggestions are displayed in an easy-to-understand manner to the user. Specifically, risk areas are marked on the image and specific countermeasures are displayed in text.

[0872] Input: Action suggestion data.

[0873] Output: Action suggestions displayed to the user.

[0874] Step 7:

[0875] Emotion recognition

[0876] Terminal or server: Analyzes the user's voice data and image data and recognizes the user's emotions using an emotion engine. For example, it uses a voice recognition engine or facial expression recognition software.

[0877] Input: User's voice or image data.

[0878] Output: Recognized emotion data.

[0879] Step 8:

[0880] Adjusting behavioral suggestions according to emotions

[0881] Server: Optimizes the content of action suggestions based on the recognized emotion. For example, if the user is recognized as feeling stressed, it adds detailed and helpful explanations.

[0882] Input: Recognized emotion data and action suggestion data.

[0883] Output: Adjusted action suggestion data.

[0884] Step 9:

[0885] View detailed description

[0886] Device: When a user taps on a high-risk area, a detailed explanation and countermeasures for that area are displayed. For example, if a user taps on a high-risk area in the eyes, specific instructions on how to use eye drops are displayed.

[0887] Input: User taps on the risk area.

[0888] Output: Display detailed descriptive data.

[0889] Step 10:

[0890] Make appointments and purchase care products

[0891] Terminal: Provides links for scheduling appointments and purchasing care products. Users can use this to schedule veterinary appointments and purchase care products online.

[0892] Input: User clicks.

[0893] Output: Display of appointment booking form and care product purchase page.

[0894] (Application example 2)

[0895] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0896] There is a lack of systems that allow pet owners to effectively and quickly understand their pet's health condition and take appropriate measures. As a result, there is a risk that pet health problems will not be detected early and appropriate care will be delayed. In addition, the lack of action suggestions that take into account the pet owner's emotions may make users feel stressed, which could lead to a decline in the quality of care.

[0897] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for capturing an image of the pet, means for preprocessing the captured image, means for transmitting the preprocessed image to the server, means for analyzing the health condition of the pet using a generative AI model, means for generating action suggestions for the user based on the analysis results, means for recognizing the user's emotions, means for adjusting the action suggestions based on the recognized emotions, means for displaying the generated action suggestions, means for suggesting appropriate pet food, and means for generating a prompt message. This enables pet owners to grasp the health condition of their pets at an early stage and receive appropriate care and food suggestions based on the user's emotions.

[0898] A "means for capturing an image of a pet" is a device or software that allows a user to take or select an image of a pet.

[0899] The "means for pre-processing captured images" refers to devices or software that perform processes such as resizing and noise removal on captured or selected images.

[0900] The "means for transmitting preprocessed images to a server" refers to a device or software that transmits preprocessed image data to a remote server via a network.

[0901] A "generative AI model" is an artificial intelligence model that runs on a computer to analyze the health of pets.

[0902] "Means for analyzing the health condition of a pet" refers to a device or software that uses a generative AI model on a server to analyze image data and determine the health condition of a pet.

[0903] The "means for generating suggested actions for the user based on the analysis results" refers to a device or software that generates specific suggested actions for the pet owner based on the results of the analyzed health condition.

[0904] The "means for recognizing user's emotions" refers to a device or software that determines the user's emotions by analyzing the user's voice data and facial expression data.

[0905] The "means for adjusting suggested actions based on recognized emotions" refers to a device or software that adjusts the content of suggested actions taking into account the user's emotional state.

[0906] The "means for displaying the generated action suggestions" refers to a device or software that visually displays the generated action suggestions on the user's terminal.

[0907] The "means for suggesting appropriate pet food" refers to a device or software that suggests appropriate food for a pet, taking into consideration the pet's health condition and the user's emotions.

[0908] A "means for generating prompt sentences" is a device or software that generates questions or instructions that a user can use to assess the health of their pet.

[0909] This invention provides a system that allows pet owners to quickly understand their pet's health condition and take appropriate measures. This system includes a function that captures images of the pet, analyzes them using an AI model, recognizes the user's emotions, and adjusts behavior suggestions accordingly. The main components of this system and their functions are as follows:

[0910] 1. How to capture images of your pet

[0911] Users can take pictures of their pets using the camera function of their smartphones, tablets, or other devices. They can also select existing images. This image capture method allows users to accurately capture their pet's current state.

[0912] 2. A means of preprocessing the captured images

[0913] The device performs preprocessing such as resizing and noise removal on the captured image, making the image data suitable for AI analysis. Specifically, this preprocessing is performed using an image processing library such as OpenCV.

[0914] 3. A means to send preprocessed images to the server

[0915] Once preprocessed, the image data is sent from the device to a server over a network, where it can be analyzed using advanced AI models.

[0916] 4. A means of analyzing your pet's health

[0917] The server uses generative AI models such as TensorFlow and Keras to analyze the received image data. It detects the pet's face and body parts and determines the health risk of each part (normal, requires observation, emergency). Based on the analysis results, the appropriate countermeasures are determined.

[0918] 5. How to Recognize User Emotions

[0919] The device or server analyzes the user's voice data and facial expressions to perform emotion recognition, using emotion analysis tools such as EmotionRecognizer, to determine how the user feels about their pet's health.

[0920] 6. A way to tailor suggested actions based on emotions

[0921] The server then adjusts the suggested actions based on the user's perceived emotions: for example, if the user expresses anxiety, it adds more specific and helpful instructions, and also suggests appropriate pet food and care products.

[0922] 7. Means for displaying generated action suggestions

[0923] The device visually displays the action suggestions received from the server to the user, highlighting risk areas and providing specific solutions to help the user take action quickly.

[0924] Specific examples

[0925] The user launches the app and takes a full-body photo of their dog. The device preprocesses the image and sends it to the server. The server analyzes the image, detects "bloodshot eyes," and suggests to the user, "Consider using eye drops. If symptoms persist, consult a veterinarian." The user then uses the system's camera function to take a photo of their own face and performs emotion analysis. The analysis results identify the user's emotion as "anxiety," so the server makes a suggested course of action with a detailed and helpful explanation.

[0926] Prompt Sentence Examples

[0927] "Take a photo of your dog's face and body with this app and analyze its health condition. Then, take a photo of your face and analyze its emotions. Based on the analysis results, the app will make appropriate food and care recommendations."

[0928] This system allows pet owners to grasp their pet's health condition early on and receive appropriate care and food suggestions based on the user's emotions.

[0929] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0930] Step 1:

[0931] The user takes a picture of their pet. The user uses the camera function of their smartphone or tablet to capture the image of their pet. The input is the "pet image" and the output is the image data stored on the device.

[0932] Step 2:

[0933] The device preprocesses the image. Specifically, it resizes and removes noise using an image processing library such as OpenCV. The input is the image data obtained in step 1, and the output is the preprocessed image data. Resizing changes the resolution to a level suitable for analysis, and noise removal increases the clarity of the image.

[0934] Step 3:

[0935] Send the preprocessed image data to the server. The device sends the image data to the remote server via the network. The input is the preprocessed image data obtained in step 2, and the output is the image data sent to the server.

[0936] Step 4:

[0937] The server analyzes the pet's health condition. The image data is analyzed using a generative AI model (using TensorFlow and Keras) on the server. The input is the image data sent in step 3, and the output is the pet's health assessment results (e.g., "bloodshot eyes" or "swollen abdomen"). The AI ​​model detects specific parts of the face and body and determines the health risk of each.

[0938] Step 5:

[0939] Recognize user emotions. The user uses a device to capture their own voice or facial image. The device or server performs emotion analysis using EmotionRecognizer. The input is the user's voice data or facial image data, and the output is the user's emotional assessment result (e.g., "joy," "stress," or "anxiety").

[0940] Step 6:

[0941] The server adjusts the action suggestions based on the user's emotions. The action suggestions generated from the analysis results are adjusted taking into account the recognized user's emotions. The input is the health assessment result from step 4 and the emotion assessment result from step 5, and the output is the final action suggestions. For example, for users who show anxiety, a detailed and friendly explanation is added.

[0942] Step 7:

[0943] The server recommends the appropriate pet food based on the pet's health condition and the user's emotions. The input is the health assessment result in step 4 and the emotion assessment result in step 5, and the output is the food recommendation.

[0944] Step 8:

[0945] Display the generated action suggestions. The device visually displays the action suggestions received from the server to the user, providing specific solutions and links to care products. The input is the suggestion results from steps 6 and 7, and the output is the action suggestions displayed on the user's device.

[0946] Step 9:

[0947] The user accesses detailed information and additional features. The user taps on a high-risk area to see detailed explanations and solutions, and then accesses links to schedule an appointment or purchase care products. The input is the action suggestion link provided by the server, and the output is the user's action (e.g., schedule an appointment, purchase a product).

[0948] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0949] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0950] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0951] [Third embodiment]

[0952] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0953] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0954] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0955] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0956] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0957] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0958] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0959] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0960] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0961] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0962] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0963] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0964] This invention is a system that allows pet owners to grasp the health condition of their pets at an early stage and take appropriate measures. Specifically, it assesses the health risks of pets through a series of processes that involves capturing images of the pet, preprocessing the images, and analyzing them based on an AI model, and then provides users with suggested actions.

[0965] System configuration

[0966] 1. Image capture method

[0967] User: Uses a device such as a smartphone or tablet to take a picture of their pet or select an existing image.

[0968] 2. Image preprocessing methods

[0969] Device: Performs preprocessing such as resizing and noise removal on captured or selected images, making the image data suitable for AI analysis.

[0970] 3. Image transmission method

[0971] Terminal: Sends the pre-processed image to the server.

[0972] 4. AI model for analyzing health status

[0973] Server: The received image data is input into the AI ​​model to analyze the pet's health condition. Specifically, it detects the pet's face and body parts and determines the health risk of each part (e.g., normal, requires observation, emergency).

[0974] 5. Action suggestion generation means

[0975] Server: Based on the analysis results of the AI ​​model, it generates specific action suggestions for the user (e.g., recommendations for daily care, the need for a veterinarian).

[0976] 6. Action suggestion display means

[0977] Device: The generated action proposals are displayed in an easy-to-understand manner to the user. Specifically, risk areas and countermeasures are displayed on the screen.

[0978] 7. Detailed explanation display means

[0979] Device: When the user taps on a high-risk area, a detailed explanation and countermeasures for that area will be displayed.

[0980] 8. How to book appointments and purchase care products

[0981] Terminal: Provides links to schedule appointments and purchase care products. This feature allows users to easily schedule veterinary appointments and purchase necessary care products.

[0982] Specific examples

[0983] 1. The user launches the app and takes a full-body photo of their dog.

[0984] User: Use the app's camera function to take a full-body photo of your pet (e.g., dog).

[0985] Device: Temporarily saves the captured image and displays the "Next" button.

[0986] 2. The device preprocesses the image

[0987] Device: Resize and denoise the image to make it optimal for analysis.

[0988] Terminal: Sends the image data after preprocessing to the server.

[0989] 3. The server analyzes the image and generates action suggestions.

[0990] Server: The received images are input into the AI ​​model to analyze the pet's health condition. For example, risk areas such as bloodshot eyes and abdominal swelling are identified.

[0991] Server: Based on the analysis results, it generates appropriate action suggestions to the user (e.g., "Your eyes are bloodshot. Consider using eye drops. If symptoms persist, consult your veterinarian").

[0992] 4. The device displays the diagnosis results and suggested actions to the user.

[0993] Terminal: The diagnostic results are displayed intuitively to the user, clearly indicating risk areas and providing specific countermeasures.

[0994] 5. Users access more information and additional features

[0995] User: Tap on high-risk areas to see detailed explanations and solutions.

[0996] Terminal: Users can access links to book appointments and purchase care products as needed.

[0997] This system allows pet owners to easily understand their pet's health status and quickly take necessary measures, which is expected to make pet health management more efficient and effective.

[0998] The processing flow will be explained below.

[0999] Step 1:

[1000] The user launches the app on their smartphone and takes a picture of their pet or selects an existing image, providing the system with up-to-date information about their pet's health.

[1001] Step 2:

[1002] The device temporarily stores the captured or selected image, which is then preprocessed.

[1003] Step 3:

[1004] The device resizes the image to make it easier for the AI ​​model to analyze high-resolution images.

[1005] Step 4:

[1006] The device performs noise reduction, which improves image quality and increases the accuracy of analysis.

[1007] Step 5:

[1008] The device sends the preprocessed image to the server, which forwards the image to the server for analysis.

[1009] Step 6:

[1010] The server inputs the received image into the AI ​​model, which then detects each part of the pet based on the image.

[1011] Step 7:

[1012] The server uses an AI model to analyze the pet's health condition, identifying risk areas such as bloodshot eyes or abdominal swelling, and determining the risk level (e.g., normal, requires observation, or emergency).

[1013] Step 8:

[1014] The server generates action suggestions for the user based on the analysis results, such as "Your eyes are bloodshot. Consider using eye drops. If the symptoms persist, consult a veterinarian."

[1015] Step 9:

[1016] The device receives the diagnostic results and action suggestions from the server and displays them to the user. Risk areas and solutions are presented through an intuitive interface.

[1017] Step 10:

[1018] When the user taps on a high-risk area, the device will display a detailed explanation and solutions, such as the cause of "bloodshot eyes" and how to care for them at home.

[1019] Step 11:

[1020] The device provides links to schedule appointments and purchase care products, allowing users to schedule veterinary appointments and purchase medications as needed.

[1021] This series of steps creates a system that allows pet owners to accurately and quickly understand their pet's health condition and take appropriate measures.

[1022] Example 1

[1023] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1024] For modern pet owners, it is becoming increasingly important to understand their pet's health condition early and take appropriate measures. However, traditional methods often result in late detection of abnormalities in pets or in failure to take appropriate action. Furthermore, there is a lack of means to obtain specific action suggestions and detailed explanations in real time based on a pet's health condition, making it difficult for pet owners to manage their pet's health efficiently and effectively.

[1025] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1026] In this invention, the server includes means for capturing images of the pet, means for preprocessing the captured images, means for transmitting the preprocessed images to the server, means for analyzing the health condition of the pet using a generative AI model on the server, means for generating action suggestions for the user based on the analysis results, means for displaying the generated action suggestions, means for displaying detailed explanations of high-risk areas when the user taps on those areas, and means for providing links to make appointments for medical examinations and purchase care products. This enables pet owners to grasp the health condition of their pets at an early stage and take prompt and appropriate measures.

[1027] "Means for capturing images" refers to a means by which a user can take an image of a pet using a device such as a smartphone or tablet, or select an existing image.

[1028] "Preprocessing means" refers to the process of resizing and removing noise from the captured image, making the image data suitable for AI analysis.

[1029] The "means for transmitting to the server" is a means for transmitting the image preprocessed on the terminal to the server via communication.

[1030] The "analysis means using a generative AI model" is a means of analyzing received pet image data using an AI model placed on a server and evaluating the pet's health condition.

[1031] The "means for generating behavioral suggestions" refers to a means for creating specific behavioral suggestions according to the health condition of a pet based on the analysis results of the generative AI model.

[1032] The "means for displaying suggested actions" is a means for displaying the content of suggested actions on the user's terminal in an easily understandable manner.

[1033] The "means for displaying a detailed explanation" is a means for displaying a detailed explanation and a method for dealing with a high-risk area when the user taps on that area.

[1034] "Means for providing links to schedule appointments and purchase care products" refers to means for providing links that allow a user to easily schedule an appointment with a veterinarian or purchase necessary care products.

[1035] This invention is a system that allows pet owners to grasp the health condition of their pets at an early stage and take prompt and appropriate measures to efficiently and effectively manage the health of their pets. A specific implementation method of this system is described in detail below.

[1036] Hardware and software used

[1037] 1. User Device

[1038] Hardware: Smartphones, tablets

[1039] Software: Dedicated application (e.g., pet health management app)

[1040] 2. Server

[1041] Hardware: High-performance servers (e.g., servers with Intel Xeon processors)

[1042] Software: A software environment for running generative AI models (e.g., Python, TensorFlow)

[1043] System configuration and data processing

[1044] 1. Image capture method

[1045] User: Launches the application on their smartphone or tablet to take a picture of their pet. The user can use the camera function within the application to take a picture of their pet or select an existing image.

[1046] 2. Image preprocessing methods

[1047] On-device: Captured or selected images are resized and pre-processed on-device, including resizing and noise reduction. This involves scaling the image resolution to the appropriate size and applying Gaussian filters to reduce noise.

[1048] 3. Image transmission method

[1049] Terminal: After preprocessing, the image data is sent to the server using an HTTP POST request.

[1050] 4. AI model for analyzing health status

[1051] Server: The server inputs the received image data into a generative AI model to detect the pet's face and body parts and assess health risks. This analysis uses a TensorFlow-based deep learning model.

[1052] 5. Action suggestion generation means

[1053] Server: Based on the analysis results, the server generates specific action suggestions for the user. For example, if it determines that a pet's eyes are bloodshot, it generates a suggestion such as "Consider using eye drops. If the symptoms persist, consult a veterinarian."

[1054] 6. Action suggestion display means

[1055] Device: The generated action proposals are displayed on the user's device, highlighting risk areas and showing specific countermeasures.

[1056] 7. Detailed explanation display means

[1057] On the device: When the user taps on a high-risk area, a detailed explanation and solutions for that area are displayed.

[1058] 8. How to book appointments and purchase care products

[1059] Terminal: Links for booking appointments and purchasing care products are displayed on the terminal, allowing the user to book an appointment with a veterinarian or purchase any necessary care products.

[1060] Specific examples

[1061] The user launches a dedicated application and takes a full-body photo of their pet (e.g., a dog) using their smartphone's camera. The application resizes the image, removes noise, and then sends it to a server. The server uses a generative AI model to analyze the health status of the image. Based on the analysis results, the server generates a suggested action, such as "Your eyes are bloodshot, so consider using eye drops. If symptoms persist, consult a veterinarian," and displays it on the user's device. When the user taps on high-risk areas based on the results, detailed explanations and countermeasures are displayed. Links to make appointments and purchase care products are also provided.

[1062] Examples of prompt statements

[1063] Take or select an image of your pet. After pre-processing, the image will be analyzed to assess your pet's health. For example, if it determines that your dog's eyes are bloodshot, it will display specific action suggestions such as "Consider using eye drops. If symptoms persist, consult a veterinarian."

[1064] This system is expected to enable pet owners to detect health conditions in their pets early and respond quickly and accurately, making pet health management more efficient and effective.

[1065] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1066] Step 1: Importing an image

[1067] User: The user launches a dedicated application on their smartphone or tablet and takes a picture of their pet or selects an existing image.

[1068] Input: A photo of your pet (either a new or existing photo)

[1069] What happens: The user taps the app's "Take Image" button and takes a full-body photo of their pet with the camera, or taps the "Select Image" button to choose an existing image from the gallery.

[1070] Output: Image data of the pet taken or selected by the user

[1071] Step 2: Image preprocessing

[1072] Device: The device performs pre-processing such as resizing and noise reduction on the captured or selected image.

[1073] Input: Image data of the pet taken or selected by the user

[1074] What it does: Resizes the image to a resolution of 600x800 pixels and reduces noise using a Gaussian filter. The application performs these steps automatically.

[1075] Output: Preprocessed image data

[1076] Step 3: Sending images

[1077] Terminal: Sends the image data after preprocessing to the server.

[1078] Input: Preprocessed image data

[1079] Specific operation: The device uses an HTTP POST request to upload the preprocessed image to the server.

[1080] Output: Image data sent to the server

[1081] Step 4: Analyze health status using AI models

[1082] Server: The server inputs the received image data into a generative AI model to detect the pet's face and body parts and assess health risks.

[1083] Input: Image data sent to the server

[1084] How it works: A Python script on the server receives the image and analyzes it using a TensorFlow-based generative AI model, which detects features such as bloodshot eyes and abnormal coat condition and assesses health risks.

[1085] Output: Analysis results on pet health risks

[1086] Step 5: Generate action suggestions

[1087] Server: Based on the analysis results, the server generates specific action suggestions for the user.

[1088] Input: Analysis results on pet health risks

[1089] Specific operation: Based on the analysis results obtained by the server, the server generates specific suggestions by inserting them into a pre-prepared text template. For example, it generates a suggestion such as, "Your eyes are bloodshot. Consider using eye drops. If the symptoms persist, consult a veterinarian."

[1090] Output: Suggested actions for the user

[1091] Step 6: View suggested actions

[1092] Terminal: The generated action suggestions are displayed on the user's terminal.

[1093] Input: Suggested actions for the user

[1094] Specific actions: Highlight risk areas on the device screen and provide text explaining specific ways to deal with the problem. For example, a bloodshot eye is highlighted in a red frame with suggested actions displayed below.

[1095] Output: Action suggestions displayed on the user's device

[1096] Step 7: View detailed instructions

[1097] User: When the user taps on a high-risk area, a detailed explanation and solutions for that area will be displayed.

[1098] Input: User tap operation, information on high-risk areas

[1099] Specific operation: When the user taps the red frame of the risk area, a detailed information screen will open, displaying an explanation of the eye symptoms and how to deal with them.

[1100] Output: Detailed explanation and solutions

[1101] Step 8: Book a consultation and purchase care products

[1102] Device: Links to schedule appointments and purchase care products will be displayed on the device.

[1103] Input: Relevant links based on identified health risks

[1104] Specific behavior: Link buttons to a medical appointment booking page and a care product online store are displayed on the screen. When the user taps these buttons, the respective pages open.

[1105] Output: Links to book appointments and purchase care products

[1106] (Application example 1)

[1107] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1108] Properly managing the health of pets is an important issue for pet owners. With current technology, it is difficult to grasp a pet's health condition early and take specific measures. This increases the risk that pet owners without specialized veterinary knowledge may overlook abnormalities in their pets. Furthermore, the lack of a simple and rapid means of assessing a pet's health raises concerns that pet care may be neglected in today's busy society. Therefore, to solve these problems, a system is needed that allows users to easily assess a pet's health condition and take specific measures.

[1109] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1110] In this invention, the server includes a means for capturing images of pets, a means for preprocessing the captured images, and a means for transmitting the preprocessed images to the server. This allows the pet images to be analyzed using an AI model, and specific action suggestions for the user to be automatically generated and displayed. Additionally, by adding a means for displaying detailed explanations and countermeasures by tapping on risk areas and a means for providing links to make medical appointments or purchase care products, pet owners can immediately take appropriate measures. Furthermore, the accuracy of the analysis is improved by adding a means for resizing and noise reduction to the image preprocessing and normalizing the resized images before inputting the image data into the AI ​​model. This enables early detection of pet health conditions and appropriate measures.

[1111] "Means for capturing an image of a pet" refers to a function that allows a user to take an image of a pet or select an existing image using a device such as a smartphone or tablet.

[1112] "Means for preprocessing captured images" refers to a function that processes captured or selected images, such as resizing and noise removal, to prepare the image data in a state suitable for AI analysis.

[1113] The "means for transmitting preprocessed images to a server" is a function for transmitting preprocessed images to a server via the Internet.

[1114] "Means for analyzing pet health conditions using an AI model on a server" refers to a function that inputs image data stored on a server into an AI model, analyzes the pet's health conditions, and identifies risk areas.

[1115] "Means for generating action suggestions for users based on analysis results" refers to a function for generating appropriate action suggestions for users based on the analysis results of the AI ​​model.

[1116] The "means for displaying the generated action proposal" is a function for displaying the generated action proposal in an easily understandable manner to the user.

[1117] "A means of displaying detailed explanations and solutions by tapping on risk areas" is a function that allows the user to tap on risk areas and display detailed explanations and solutions for that area.

[1118] "Means for providing links to schedule appointments or purchase care products" refers to a function that provides links for users to schedule appointments or purchase care products.

[1119] "Resizing" is a process of changing the size of an image.

[1120] "Noise removal" is a process that removes unnecessary information (noise) from an image.

[1121] "Normalization" is the process of aligning the range of data to a certain scale.

[1122] The present invention provides a system that enables pet owners to grasp the health condition of their pets at an early stage and take appropriate measures. Specific embodiments for carrying out the present invention will be described in detail below.

[1123] System configuration

[1124] 1. Image capture method

[1125] Users can take pictures of their pets using a smartphone, tablet, or other device, and the images are temporarily saved in the app.

[1126] 2. Image preprocessing methods

[1127] The device resizes and denoises the captured images. This preprocessing is performed using OpenCV. Resizing adjusts the image size appropriately, and denoising removes unnecessary data that may affect analysis.

[1128] 3. Image transmission method

[1129] After preprocessing, the image data is sent from the device to the server via Firebase or AWS.

[1130] 4. AI model for analyzing health status

[1131] The server runs an AI model using TensorFlow or PyTorch to analyze the received image data, specifically detecting pet faces and body parts and assessing their health risks.

[1132] 5. Action suggestion generation means

[1133] Based on the analysis results of the AI ​​model, the server generates specific action suggestions for the user, such as "Your eyes are bloodshot. Consider using eye drops. If the symptoms persist, consult a veterinarian."

[1134] 6. Action suggestion display means

[1135] The device displays the generated action suggestions in an easy-to-understand manner to the user, visually indicating risk areas and presenting specific countermeasures.

[1136] 7. Detailed explanation display means

[1137] When the user taps on a risk area, the device displays a detailed explanation and instructions on how to deal with that area, allowing pet owners to take appropriate measures with confidence.

[1138] 8. How to book appointments and purchase care products

[1139] The device provides links to help users easily schedule appointments and purchase care products, helping them get the care they need faster.

[1140] Specific examples

[1141] Below we introduce some specific scenarios in which this system can be used.

[1142] 1. The user launches the app and takes a full-body photo of their dog.

[1143] The user uses the app's camera function to take a full-body photo of their pet (e.g., dog). The device temporarily saves the image and displays a "Next" button.

[1144] 2. The device preprocesses the image

[1145] The device resizes the image and performs noise reduction. This preprocessing is done using OpenCV. The preprocessed image data is then sent to the server.

[1146] 3. The server analyzes the image and generates action suggestions.

[1147] The server inputs the received images into an AI model using TensorFlow or PyTorch to analyze the pet's health. For example, it identifies risk areas such as bloodshot eyes and abdominal swelling. Based on the analysis results, it then generates appropriate action suggestions for the user.

[1148] 4. The device displays the diagnosis results and suggested actions to the user.

[1149] The generated action suggestions are displayed on the device in an easy-to-understand manner to the user, visually indicating risk areas and presenting specific countermeasures.

[1150] Example prompts to input to the generative AI model

[1151] "Evaluate the condition of this dog's face and determine its health risks."

[1152] "Please use an AI model to analyze whether there is anything abnormal about this cat's abdomen."

[1153] This invention allows pet owners to easily understand the health status of their pets and quickly take necessary measures, which is expected to make pet health management more efficient and effective.

[1154] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1155] Step 1:

[1156] The user launches the app and captures an image of their pet. The user can either take a picture of their pet using the camera function of their smartphone or tablet, or import an existing image into the app. The input is a newly taken image or an existing image, which is temporarily stored on the device. The output is image data that awaits preprocessing.

[1157] Step 2:

[1158] The device performs preprocessing. Image resizing and noise removal are performed within the device. Specifically, OpenCV is used to appropriately adjust the image size and remove noise. The input is image data awaiting preprocessing, and the output is preprocessed image data suitable for AI analysis.

[1159] Step 3:

[1160] The device sends the preprocessed image to the server. The preprocessed image data is uploaded to the server via Firebase or AWS. The preprocessed image data is the input, and the image data stored on the server is the output.

[1161] Step 4:

[1162] The server analyzes the image using an AI model. Using TensorFlow or PyTorch on the server, the received image data is input into the AI ​​model. The AI ​​model detects the pet's face and body parts and evaluates health risks. The image data stored on the server is used as input, and the analysis results of the pet's health condition are obtained as output.

[1163] Step 5:

[1164] The server generates action suggestions. Based on the analysis results of the AI ​​model, the server generates specific action suggestions for the user. For example, an action suggestion might be something like, "Your eyes are bloodshot. Consider using eye drops. If the symptoms persist, consult a veterinarian." The input is the analysis result data of the pet's health condition, and the output is action suggestion data.

[1165] Step 6:

[1166] The terminal displays the action proposal. The terminal displays the generated action proposal to the user in a visually easy-to-understand manner. Specifically, it highlights risk areas and shows how to deal with them. The input is the action proposal data, and the output is the action proposal displayed to the user.

[1167] Step 7:

[1168] The user taps on a risky area. When the user taps on a risky area on the action suggestion display screen, a detailed explanation and countermeasures for that area are displayed. Specifically, a detailed screen is displayed that describes the symptoms and describes countermeasures. The user's tap operation is the input, and detailed information is displayed as the output.

[1169] Step 8:

[1170] The terminal provides links for making appointments and purchasing care products. When the user taps the links to make appointments or purchase care products as needed, they are redirected to an external site and the procedure is completed. The input is the action suggestion data and the user's link tap operation, and the output is the completion of the appointment or product purchase.

[1171] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1172] This invention combines a system that allows pet owners to quickly understand the health condition of their pets and take appropriate measures with an emotion engine that recognizes the user's emotions. Specifically, the system captures images of the pet, preprocesses the images, analyzes them based on an AI model, and evaluates the pet's health risks. In addition, the system recognizes the user's emotions and suggests actions based on the emotions.

[1173] System configuration

[1174] 1. Image capture method

[1175] User: Uses a device such as a smartphone or tablet to take a picture of their pet or select an existing image.

[1176] 2. Image preprocessing methods

[1177] Device: Performs preprocessing such as resizing and noise removal on captured or selected images, making the image data suitable for AI analysis.

[1178] 3. Image transmission method

[1179] Terminal: Sends the pre-processed image to the server.

[1180] 4. AI model for analyzing health status

[1181] Server: The received image data is input into the AI ​​model to analyze the pet's health condition. Specifically, it detects the pet's face and body parts and determines the health risk of each part (e.g., normal, requires observation, emergency).

[1182] 5. Action suggestion generation means

[1183] Server: Based on the analysis results of the AI ​​model, it generates specific action suggestions for the user (e.g., recommendations for daily care, the need for a veterinarian).

[1184] 6. Action suggestion display means

[1185] Device: The generated action proposals are displayed in an easy-to-understand manner to the user. Specifically, risk areas and countermeasures are displayed on the screen.

[1186] 7. Emotion Engine

[1187] Terminal or server: Equipped with an emotion engine that recognizes the user's emotions, it determines the user's emotions by analyzing the user's voice data and image data.

[1188] 8. Emotion-based behavioral adjustment measures

[1189] Server: Adjusts suggested actions and provides appropriate feedback based on the user's emotions recognized by the emotion engine. For example, if the user is feeling stressed, the server provides more helpful and specific explanations.

[1190] 9. Detailed explanation display means

[1191] Device: When the user taps on a high-risk area, a detailed explanation and countermeasures for that area will be displayed.

[1192] 10. How to book appointments and purchase care products

[1193] Terminal: Provides links to schedule appointments and purchase care products. This feature allows users to easily schedule veterinary appointments and purchase necessary care products.

[1194] Specific examples

[1195] 1. The user launches the app and takes a full-body photo of their dog.

[1196] User: Use the app's camera function to take a full-body photo of your pet (e.g., dog).

[1197] Device: Temporarily saves the captured image and displays the "Next" button.

[1198] 2. The device preprocesses the image

[1199] Device: Resize and denoise the image to make it optimal for analysis.

[1200] Terminal: Sends the image data after preprocessing to the server.

[1201] 3. The server analyzes the image and generates action suggestions.

[1202] Server: The received images are input into the AI ​​model to analyze the pet's health condition. For example, risk areas such as bloodshot eyes and abdominal swelling are identified.

[1203] Server: Based on the analysis results, it generates appropriate action suggestions to the user (e.g., "Your eyes are bloodshot. Consider using eye drops. If symptoms persist, consult your veterinarian").

[1204] 4. The device displays the diagnosis results and suggested actions to the user.

[1205] Terminal: The diagnostic results are displayed intuitively to the user, clearly indicating risk areas and providing specific countermeasures.

[1206] 5. Users access more information and additional features

[1207] User: Tap on high-risk areas to see detailed explanations and solutions.

[1208] Terminal: Users can access links to book appointments and purchase care products as needed.

[1209] 6. Use of Emotion Engines

[1210] Device or server: While the user is using the app, it analyzes voice data and facial expressions to recognize the user's emotions (e.g., joy, stress, anxiety).

[1211] Server: Adjust the content of the suggested action based on the user's emotions. For example, if the user expresses anxiety, add a detailed and helpful explanation.

[1212] This system not only enables pet owners to accurately and quickly understand their pet's health condition and take appropriate measures, but also enables more effective care by recognizing the user's emotions and providing support accordingly.

[1213] The processing flow will be explained below.

[1214] Step 1:

[1215] The user launches the smartphone app and takes a picture of their pet or selects an existing image, providing the pet's health status as input data.

[1216] Step 2:

[1217] The device temporarily stores the captured or selected image, which is then used in the next processing step.

[1218] Step 3:

[1219] Your device will resize the image to the appropriate resolution, reducing data usage and improving processing speed.

[1220] Step 4:

[1221] The device will then perform noise reduction to improve image quality, which will improve the accuracy of the analysis by the AI ​​model.

[1222] Step 5:

[1223] The device sends the preprocessed image data to the server, which forwards the image data to the server for analysis.

[1224] Step 6:

[1225] The server inputs the received image data into the AI ​​model, which analyzes the image and begins the process of assessing the pet's health.

[1226] Step 7:

[1227] The server uses an AI model to detect the pet's face and body parts, such as the eyes, ears, mouth, and abdomen, and assesses risk areas.

[1228] Step 8:

[1229] The server evaluates the pet's health risk and determines the risk level (e.g., normal, requires observation, emergency). Based on the analysis results, it generates action suggestions for the user.

[1230] Step 9:

[1231] The terminal displays the diagnosis results and action suggestions received from the server to the user. An interface including risk areas and solutions is displayed.

[1232] Step 10:

[1233] When a user uses the app, the device or server inputs the user's voice data and image data into the emotion engine, which analyzes this data and recognizes the user's emotions.

[1234] Step 11:

[1235] The server analyzes the user's emotional data and adjusts the suggested actions, for example, adding specific and helpful explanations if the user is feeling stressed.

[1236] Step 12:

[1237] The device displays behavior suggestions tailored to the user's emotions, providing appropriate feedback according to the user's emotional state.

[1238] Step 13:

[1239] When the user taps on a high-risk area, the device displays a detailed explanation and how to deal with it. The user can check the cause of the specific symptom and how to deal with it.

[1240] Step 14:

[1241] The device provides links to schedule appointments and purchase care products. Users can schedule appointments with veterinarians or purchase care products as needed.

[1242] This system allows pet owners to accurately understand their pet's health condition and take appropriate measures. It also recognizes the user's emotions and provides support accordingly, enabling more effective care.

[1243] Example 2

[1244] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1245] There are few systems that can quickly identify a pet's health condition and provide appropriate treatment, and the suggested actions provided by these systems often do not take the user's feelings into consideration. Furthermore, due to a lack of preprocessing functions to improve the accuracy of pet image data analysis, the reliability of analysis results is often low. This makes it difficult for pet owners to manage their pet's health with peace of mind.

[1246] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1247] In this invention, the server includes means for capturing an image of the pet, means for preprocessing the captured image, means for transmitting the preprocessed image to the server, means for analyzing the health condition of the pet using an AI model on the server, means for generating suggested actions for the user based on the analysis results, means for displaying the generated suggested actions, means for recognizing the user's emotions, and means for adjusting the suggested actions based on the recognized emotions. This makes it possible to quickly analyze the health condition of the pet with high accuracy and provide specific and appropriate suggested actions that take the user's emotions into consideration.

[1248] A "means for capturing pet images" is a device or software that allows a user to take an image of a pet or select an existing image using a device such as a smartphone or tablet, and then register that information in the system.

[1249] The "means for pre-processing captured images" refers to devices or software that perform processes such as resizing and noise removal on images captured on the terminal.

[1250] The "means for transmitting the preprocessed image to the server" refers to a communication function or software for transferring the preprocessed image to the server via a network.

[1251] "Means for analyzing a pet's health condition using an AI model on a server" refers to devices or software that use an AI model running on a server to analyze images of a pet and evaluate its health condition.

[1252] "Means for generating suggested actions for users based on analysis results" refers to devices or software that suggest specific ways of dealing with issues or actions to users based on the results of analysis by the AI ​​model.

[1253] The "means for displaying the generated action suggestions" refers to a device or software for visually displaying the action suggestions to the user on the terminal.

[1254] The "means for recognizing the user's emotions" refers to a device or software that analyzes the user's voice data or image data and identifies the user's emotional state.

[1255] The "means for adjusting suggested actions based on recognized emotions" refers to a device or software that optimizes the content of suggested actions provided based on the results of analyzing the user's emotions using an emotion engine.

[1256] The "means for displaying detailed explanations for high-risk areas" refers to a device or software that displays detailed explanations related to a particular high-risk area when a user requests detailed information about that area.

[1257] A "means for providing links to schedule an appointment or purchase a care product" is a device or software that provides an online link for a user to schedule an appointment or purchase a care product.

[1258] "Means for resizing or denoising images in preprocessing" refers to devices or software for changing the size of images to a resolution suitable for analysis or for reducing noise contained in images.

[1259] The present invention relates to a system that allows pet owners to grasp the health condition of their pets at an early stage and take appropriate measures, and further has the function of recognizing the user's emotions and providing appropriate action suggestions.

[1260] Basic system configuration

[1261] This system consists of the following components: Users use devices such as smartphones and tablets.

[1262] 1. Image capture method

[1263] User: Use the camera function on your smartphone or tablet to take a picture of your pet or select an existing image.

[1264] 2. Image preprocessing methods

[1265] On-device: Perform preprocessing such as resizing and noise reduction on the captured or selected image. For example, you can use an image processing library such as OpenCV.

[1266] 3. Image transmission method

[1267] Terminal: The preprocessed image is sent to the server using an HTTP request.

[1268] 4. AI model for analyzing health status

[1269] Server: The received image data is input into a deep learning model (e.g., a convolutional neural network) to analyze the pet's health condition. Deep learning libraries such as TensorFlow and PyTorch can be used. Specific analysis involves detecting faces and body parts and assessing the health risk of each part.

[1270] 5. Action suggestion generation means

[1271] Server: Based on the analysis results of the AI ​​model, the server generates specific action suggestions for the user. For example, an action suggestion might be, "Your eyes are bloodshot. Consider using eye drops. If the symptoms persist, consult a veterinarian."

[1272] 6. Action suggestion display means

[1273] On-device: The generated action suggestions are displayed to the user. Risk areas are marked on the image, and specific countermeasures are displayed as text. This can be implemented using a UI framework (e.g., Flutter or React Native).

[1274] 7. Emotion Engine

[1275] Terminal or server: Equipped with an emotion engine that analyzes the user's voice data and image data and recognizes the user's emotions. Watson or Azure Cognitive Services can be used for voice recognition, and OpenFace or FaceReader can be used for emotion recognition.

[1276] 8. Emotion-based behavioral adjustment measures

[1277] Server: Adjusts the suggested actions based on the user's emotions recognized by the emotion engine. Specifically, if the user is feeling stressed, it adds detailed and helpful explanations.

[1278] 9. Detailed explanation display means

[1279] Device: When the user taps on a high-risk area, detailed explanations and countermeasures for that area are displayed. For example, if the lead user taps on the eye area, specific instructions on how to use eye drops are displayed.

[1280] 10. How to book appointments and purchase care products

[1281] Terminal: Provides links for users to schedule appointments and purchase care products. With one click, users can schedule a veterinary appointment or purchase the care products they need online.

[1282] Specific examples

[1283] A specific example of use is shown below.

[1284] 1. The user launches the app and takes a full-body photo of their dog.

[1285] User: Use the app's camera function to take a full-body photo of your pet (e.g., your dog).

[1286] Device: Temporarily saves the captured image and displays the "Next" button.

[1287] 2. The device preprocesses the image

[1288] On the device: Resize the image and remove noise to optimize it for analysis. For example, resize the image to 1280x720 pixels, and use a Gaussian filter to remove noise.

[1289] Terminal: Sends the image data after preprocessing to the server.

[1290] 3. The server analyzes the image and generates action suggestions.

[1291] Server: The received images are input into the AI ​​model to analyze the pet's health condition. For example, risk areas such as bloodshot eyes and abdominal swelling are identified.

[1292] Server: Based on the analysis results, it generates appropriate action suggestions to the user (e.g., "Your eyes are bloodshot. Consider using eye drops. If symptoms persist, consult your veterinarian").

[1293] 4. The device displays the diagnosis results and suggested actions to the user.

[1294] Terminal: The diagnostic results are displayed intuitively to the user, clearly indicating risk areas and providing specific countermeasures.

[1295] 5. Users access more information and additional features

[1296] User: Tap on high-risk areas to see detailed explanations and solutions.

[1297] Terminal: Users can access links to book appointments and purchase care products as needed.

[1298] 6. Use of Emotion Engines

[1299] Device or server: While the user is using the app, it analyzes voice data and facial expressions to recognize the user's emotions (e.g., joy, stress, anxiety).

[1300] Server: Adjust the content of the suggested action based on the user's emotions. For example, if the user expresses anxiety, add a detailed and helpful explanation.

[1301] Specific prompt examples:

[1302] "Send a user a picture of their pet and have the AI ​​model analyze its health condition. Also, explain how it can recognize the user's emotions and provide appropriate behavioral suggestions."

[1303] The system of the present invention not only allows pet owners to accurately and quickly understand their pet's health condition and take appropriate measures, but also enables more effective care by the system recognizing the user's emotions and providing support accordingly.

[1304] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1305] Step 1:

[1306] Image capture

[1307] User: Use the device's camera to take a picture of your pet or select an existing image.

[1308] Input: New image data from the camera or existing image data from the photo gallery.

[1309] Output: Temporarily stored image data.

[1310] Step 2:

[1311] Image preprocessing

[1312] Terminal: Preprocess the image, such as resizing and denoising. For example, resize the image to 1280x720 pixels and denoise it using a Gaussian filter.

[1313] Input: Temporarily stored image data.

[1314] Output: Preprocessed image data.

[1315] Step 3:

[1316] Sending image data

[1317] Terminal: Send the preprocessed image to the server using an HTTP request.

[1318] Input: Preprocessed image data.

[1319] Output: Image data sent to the server.

[1320] Step 4:

[1321] Health status analysis

[1322] Server: The received image data is input into an AI model to analyze the pet's health. Specifically, it detects the face and body parts and evaluates the health risks associated with each part. For example, it identifies bloodshot eyes and swollen abdomen.

[1323] Input: Image data sent to the server.

[1324] Output: Analysis result data (risk assessment results).

[1325] Step 5:

[1326] Generate action suggestions

[1327] Server: Generates specific action suggestions for the user based on the analysis results. For example, it creates suggestions such as "Your eyes are bloodshot. Consider using eye drops. If the symptoms persist, consult a veterinarian."

[1328] Input: Analysis result data.

[1329] Output: Action suggestion data.

[1330] Step 6:

[1331] Displaying suggested actions

[1332] Device: The generated action suggestions are displayed in an easy-to-understand manner to the user. Specifically, risk areas are marked on the image and specific countermeasures are displayed in text.

[1333] Input: Action suggestion data.

[1334] Output: Action suggestions displayed to the user.

[1335] Step 7:

[1336] Emotion recognition

[1337] Terminal or server: Analyzes the user's voice data and image data and recognizes the user's emotions using an emotion engine. For example, it uses a voice recognition engine or facial expression recognition software.

[1338] Input: User's voice or image data.

[1339] Output: Recognized emotion data.

[1340] Step 8:

[1341] Adjusting behavioral suggestions according to emotions

[1342] Server: Optimizes the content of action suggestions based on the recognized emotion. For example, if the user is recognized as feeling stressed, it adds detailed and helpful explanations.

[1343] Input: Recognized emotion data and action suggestion data.

[1344] Output: Adjusted action suggestion data.

[1345] Step 9:

[1346] View detailed description

[1347] Device: When a user taps on a high-risk area, a detailed explanation and countermeasures for that area are displayed. For example, if a user taps on a high-risk area in the eyes, specific instructions on how to use eye drops are displayed.

[1348] Input: User taps on the risk area.

[1349] Output: Display detailed descriptive data.

[1350] Step 10:

[1351] Make appointments and purchase care products

[1352] Terminal: Provides links for scheduling appointments and purchasing care products. Users can use this to schedule veterinary appointments and purchase care products online.

[1353] Input: User clicks.

[1354] Output: Display of appointment booking form and care product purchase page.

[1355] (Application example 2)

[1356] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1357] There is a lack of systems that allow pet owners to effectively and quickly understand their pet's health condition and take appropriate measures. As a result, there is a risk that pet health problems will not be detected early and appropriate care will be delayed. In addition, the lack of action suggestions that take into account the pet owner's emotions may make users feel stressed, which could lead to a decline in the quality of care.

[1358] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for capturing an image of the pet, means for preprocessing the captured image, means for transmitting the preprocessed image to the server, means for analyzing the health condition of the pet using a generative AI model, means for generating action suggestions for the user based on the analysis results, means for recognizing the user's emotions, means for adjusting the action suggestions based on the recognized emotions, means for displaying the generated action suggestions, means for suggesting appropriate pet food, and means for generating a prompt message. This enables pet owners to grasp the health condition of their pets at an early stage and receive appropriate care and food suggestions based on the user's emotions.

[1359] A "means for capturing an image of a pet" is a device or software that allows a user to take or select an image of a pet.

[1360] The "means for pre-processing captured images" refers to devices or software that perform processes such as resizing and noise removal on captured or selected images.

[1361] The "means for transmitting preprocessed images to a server" refers to a device or software that transmits preprocessed image data to a remote server via a network.

[1362] A "generative AI model" is an artificial intelligence model that runs on a computer to analyze the health of pets.

[1363] "Means for analyzing the health condition of a pet" refers to a device or software that uses a generative AI model on a server to analyze image data and determine the health condition of a pet.

[1364] The "means for generating suggested actions for the user based on the analysis results" refers to a device or software that generates specific suggested actions for the pet owner based on the results of the analyzed health condition.

[1365] The "means for recognizing user's emotions" refers to a device or software that determines the user's emotions by analyzing the user's voice data and facial expression data.

[1366] The "means for adjusting suggested actions based on recognized emotions" refers to a device or software that adjusts the content of suggested actions taking into account the user's emotional state.

[1367] The "means for displaying the generated action suggestions" refers to a device or software that visually displays the generated action suggestions on the user's terminal.

[1368] The "means for suggesting appropriate pet food" refers to a device or software that suggests appropriate food for a pet, taking into consideration the pet's health condition and the user's emotions.

[1369] A "means for generating prompt sentences" is a device or software that generates questions or instructions that a user can use to assess the health of their pet.

[1370] This invention provides a system that allows pet owners to quickly understand their pet's health condition and take appropriate measures. This system includes a function that captures images of the pet, analyzes them using an AI model, recognizes the user's emotions, and adjusts behavior suggestions accordingly. The main components of this system and their functions are as follows:

[1371] 1. How to capture images of your pet

[1372] Users can take pictures of their pets using the camera function of their smartphones, tablets, or other devices. They can also select existing images. This image capture method allows users to accurately capture their pet's current state.

[1373] 2. A means of preprocessing the captured images

[1374] The device performs preprocessing such as resizing and noise removal on the captured image, making the image data suitable for AI analysis. Specifically, this preprocessing is performed using an image processing library such as OpenCV.

[1375] 3. A means to send preprocessed images to the server

[1376] Once preprocessed, the image data is sent from the device to a server over a network, where it can be analyzed using advanced AI models.

[1377] 4. A means of analyzing your pet's health

[1378] The server uses generative AI models such as TensorFlow and Keras to analyze the received image data. It detects the pet's face and body parts and determines the health risk of each part (normal, requires observation, emergency). Based on the analysis results, the appropriate countermeasures are determined.

[1379] 5. How to Recognize User Emotions

[1380] The device or server analyzes the user's voice data and facial expressions to perform emotion recognition, using emotion analysis tools such as EmotionRecognizer, to determine how the user feels about their pet's health.

[1381] 6. A way to tailor suggested actions based on emotions

[1382] The server then adjusts the suggested actions based on the user's perceived emotions: for example, if the user expresses anxiety, it adds more specific and helpful instructions, and also suggests appropriate pet food and care products.

[1383] 7. Means for displaying generated action suggestions

[1384] The device visually displays the action suggestions received from the server to the user, highlighting risk areas and providing specific solutions to help the user take action quickly.

[1385] Specific examples

[1386] The user launches the app and takes a full-body photo of their dog. The device preprocesses the image and sends it to the server. The server analyzes the image, detects "bloodshot eyes," and suggests to the user, "Consider using eye drops. If symptoms persist, consult a veterinarian." The user then uses the system's camera function to take a photo of their own face and performs emotion analysis. The analysis results identify the user's emotion as "anxiety," so the server makes a suggested course of action with a detailed and helpful explanation.

[1387] Prompt Sentence Examples

[1388] "Take a photo of your dog's face and body with this app and analyze its health condition. Then, take a photo of your face and analyze its emotions. Based on the analysis results, the app will make appropriate food and care recommendations."

[1389] This system allows pet owners to grasp their pet's health condition early on and receive appropriate care and food suggestions based on the user's emotions.

[1390] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1391] Step 1:

[1392] The user takes a picture of their pet. The user uses the camera function of their smartphone or tablet to capture the image of their pet. The input is the "pet image" and the output is the image data stored on the device.

[1393] Step 2:

[1394] The device preprocesses the image. Specifically, it resizes and removes noise using an image processing library such as OpenCV. The input is the image data obtained in step 1, and the output is the preprocessed image data. Resizing changes the resolution to a level suitable for analysis, and noise removal increases the clarity of the image.

[1395] Step 3:

[1396] Send the preprocessed image data to the server. The device sends the image data to the remote server via the network. The input is the preprocessed image data obtained in step 2, and the output is the image data sent to the server.

[1397] Step 4:

[1398] The server analyzes the pet's health condition. The image data is analyzed using a generative AI model (using TensorFlow and Keras) on the server. The input is the image data sent in step 3, and the output is the pet's health assessment results (e.g., "bloodshot eyes" or "swollen abdomen"). The AI ​​model detects specific parts of the face and body and determines the health risk of each.

[1399] Step 5:

[1400] Recognize user emotions. The user uses a device to capture their own voice or facial image. The device or server performs emotion analysis using EmotionRecognizer. The input is the user's voice data or facial image data, and the output is the user's emotional assessment result (e.g., "joy," "stress," or "anxiety").

[1401] Step 6:

[1402] The server adjusts the action suggestions based on the user's emotions. The action suggestions generated from the analysis results are adjusted taking into account the recognized user's emotions. The input is the health assessment result from step 4 and the emotion assessment result from step 5, and the output is the final action suggestions. For example, for users who show anxiety, a detailed and friendly explanation is added.

[1403] Step 7:

[1404] The server recommends the appropriate pet food based on the pet's health condition and the user's emotions. The input is the health assessment result in step 4 and the emotion assessment result in step 5, and the output is the food recommendation.

[1405] Step 8:

[1406] Display the generated action suggestions. The device visually displays the action suggestions received from the server to the user, providing specific solutions and links to care products. The input is the suggestion results from steps 6 and 7, and the output is the action suggestions displayed on the user's device.

[1407] Step 9:

[1408] The user accesses detailed information and additional features. The user taps on a high-risk area to see detailed explanations and solutions, and then accesses links to schedule an appointment or purchase care products. The input is the action suggestion link provided by the server, and the output is the user's action (e.g., schedule an appointment, purchase a product).

[1409] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1410] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1411] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1412] [Fourth embodiment]

[1413] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1414] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1415] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1416] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1417] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1418] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1419] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1420] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1421] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1422] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1423] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1424] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1425] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1426] This invention is a system that allows pet owners to grasp the health condition of their pets at an early stage and take appropriate measures. Specifically, it assesses the health risks of pets through a series of processes that involves capturing images of the pet, preprocessing the images, and analyzing them based on an AI model, and then provides users with suggested actions.

[1427] System configuration

[1428] 1. Image capture method

[1429] User: Uses a device such as a smartphone or tablet to take a picture of their pet or select an existing image.

[1430] 2. Image preprocessing methods

[1431] Device: Performs preprocessing such as resizing and noise removal on captured or selected images, making the image data suitable for AI analysis.

[1432] 3. Image transmission method

[1433] Terminal: Sends the pre-processed image to the server.

[1434] 4. AI model for analyzing health status

[1435] Server: The received image data is input into the AI ​​model to analyze the pet's health condition. Specifically, it detects the pet's face and body parts and determines the health risk of each part (e.g., normal, requires observation, emergency).

[1436] 5. Action suggestion generation means

[1437] Server: Based on the analysis results of the AI ​​model, it generates specific action suggestions for the user (e.g., recommendations for daily care, the need for a veterinarian).

[1438] 6. Action suggestion display means

[1439] Device: The generated action proposals are displayed in an easy-to-understand manner to the user. Specifically, risk areas and countermeasures are displayed on the screen.

[1440] 7. Detailed explanation display means

[1441] Device: When the user taps on a high-risk area, a detailed explanation and countermeasures for that area will be displayed.

[1442] 8. How to book appointments and purchase care products

[1443] Terminal: Provides links to schedule appointments and purchase care products. This feature allows users to easily schedule veterinary appointments and purchase necessary care products.

[1444] Specific examples

[1445] 1. The user launches the app and takes a full-body photo of their dog.

[1446] User: Use the app's camera function to take a full-body photo of your pet (e.g., dog).

[1447] Device: Temporarily saves the captured image and displays the "Next" button.

[1448] 2. The device preprocesses the image

[1449] Device: Resize and denoise the image to make it optimal for analysis.

[1450] Terminal: Sends the image data after preprocessing to the server.

[1451] 3. The server analyzes the image and generates action suggestions.

[1452] Server: The received images are input into the AI ​​model to analyze the pet's health condition. For example, risk areas such as bloodshot eyes and abdominal swelling are identified.

[1453] Server: Based on the analysis results, it generates appropriate action suggestions to the user (e.g., "Your eyes are bloodshot. Consider using eye drops. If symptoms persist, consult your veterinarian").

[1454] 4. The device displays the diagnosis results and suggested actions to the user.

[1455] Terminal: The diagnostic results are displayed intuitively to the user, clearly indicating risk areas and providing specific countermeasures.

[1456] 5. Users access more information and additional features

[1457] User: Tap on high-risk areas to see detailed explanations and solutions.

[1458] Terminal: Users can access links to book appointments and purchase care products as needed.

[1459] This system allows pet owners to easily understand their pet's health status and quickly take necessary measures, which is expected to make pet health management more efficient and effective.

[1460] The processing flow will be explained below.

[1461] Step 1:

[1462] The user launches the app on their smartphone and takes a picture of their pet or selects an existing image, providing the system with up-to-date information about their pet's health.

[1463] Step 2:

[1464] The device temporarily stores the captured or selected image, which is then preprocessed.

[1465] Step 3:

[1466] The device resizes the image to make it easier for the AI ​​model to analyze high-resolution images.

[1467] Step 4:

[1468] The device performs noise reduction, which improves image quality and increases the accuracy of analysis.

[1469] Step 5:

[1470] The device sends the preprocessed image to the server, which forwards the image to the server for analysis.

[1471] Step 6:

[1472] The server inputs the received image into the AI ​​model, which then detects each part of the pet based on the image.

[1473] Step 7:

[1474] The server uses an AI model to analyze the pet's health condition, identifying risk areas such as bloodshot eyes or abdominal swelling, and determining the risk level (e.g., normal, requires observation, or emergency).

[1475] Step 8:

[1476] The server generates action suggestions for the user based on the analysis results, such as "Your eyes are bloodshot. Consider using eye drops. If the symptoms persist, consult a veterinarian."

[1477] Step 9:

[1478] The device receives the diagnostic results and action suggestions from the server and displays them to the user. Risk areas and solutions are presented through an intuitive interface.

[1479] Step 10:

[1480] When the user taps on a high-risk area, the device will display a detailed explanation and solutions, such as the cause of "bloodshot eyes" and how to care for them at home.

[1481] Step 11:

[1482] The device provides links to schedule appointments and purchase care products, allowing users to schedule veterinary appointments and purchase medications as needed.

[1483] This series of steps creates a system that allows pet owners to accurately and quickly understand their pet's health condition and take appropriate measures.

[1484] Example 1

[1485] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1486] For modern pet owners, it is becoming increasingly important to understand their pet's health condition early and take appropriate measures. However, traditional methods often result in late detection of abnormalities in pets or in failure to take appropriate action. Furthermore, there is a lack of means to obtain specific action suggestions and detailed explanations in real time based on a pet's health condition, making it difficult for pet owners to manage their pet's health efficiently and effectively.

[1487] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1488] In this invention, the server includes means for capturing images of the pet, means for preprocessing the captured images, means for transmitting the preprocessed images to the server, means for analyzing the health condition of the pet using a generative AI model on the server, means for generating action suggestions for the user based on the analysis results, means for displaying the generated action suggestions, means for displaying detailed explanations of high-risk areas when the user taps on those areas, and means for providing links to make appointments for medical examinations and purchase care products. This enables pet owners to grasp the health condition of their pets at an early stage and take prompt and appropriate measures.

[1489] "Means for capturing images" refers to a means by which a user can take an image of a pet using a device such as a smartphone or tablet, or select an existing image.

[1490] "Preprocessing means" refers to the process of resizing and removing noise from the captured image, making the image data suitable for AI analysis.

[1491] The "means for transmitting to the server" is a means for transmitting the image preprocessed on the terminal to the server via communication.

[1492] The "analysis means using a generative AI model" is a means of analyzing received pet image data using an AI model placed on a server and evaluating the pet's health condition.

[1493] The "means for generating behavioral suggestions" refers to a means for creating specific behavioral suggestions according to the health condition of a pet based on the analysis results of the generative AI model.

[1494] The "means for displaying suggested actions" is a means for displaying the content of suggested actions on the user's terminal in an easily understandable manner.

[1495] The "means for displaying a detailed explanation" is a means for displaying a detailed explanation and a method for dealing with a high-risk area when the user taps on that area.

[1496] "Means for providing links to schedule appointments and purchase care products" refers to means for providing links that allow a user to easily schedule an appointment with a veterinarian or purchase necessary care products.

[1497] This invention is a system that allows pet owners to grasp the health condition of their pets at an early stage and take prompt and appropriate measures to efficiently and effectively manage the health of their pets. A specific implementation method of this system is described in detail below.

[1498] Hardware and software used

[1499] 1. User Device

[1500] Hardware: Smartphones, tablets

[1501] Software: Dedicated application (e.g., pet health management app)

[1502] 2. Server

[1503] Hardware: High-performance servers (e.g., servers with Intel Xeon processors)

[1504] Software: A software environment for running generative AI models (e.g., Python, TensorFlow)

[1505] System configuration and data processing

[1506] 1. Image capture method

[1507] User: Launches the application on their smartphone or tablet to take a picture of their pet. The user can use the camera function within the application to take a picture of their pet or select an existing image.

[1508] 2. Image preprocessing methods

[1509] On-device: Captured or selected images are resized and pre-processed on-device, including resizing and noise reduction. This involves scaling the image resolution to the appropriate size and applying Gaussian filters to reduce noise.

[1510] 3. Image transmission method

[1511] Terminal: After preprocessing, the image data is sent to the server using an HTTP POST request.

[1512] 4. AI model for analyzing health status

[1513] Server: The server inputs the received image data into a generative AI model to detect the pet's face and body parts and assess health risks. This analysis uses a TensorFlow-based deep learning model.

[1514] 5. Action suggestion generation means

[1515] Server: Based on the analysis results, the server generates specific action suggestions for the user. For example, if it determines that a pet's eyes are bloodshot, it generates a suggestion such as "Consider using eye drops. If the symptoms persist, consult a veterinarian."

[1516] 6. Action suggestion display means

[1517] Device: The generated action proposals are displayed on the user's device, highlighting risk areas and showing specific countermeasures.

[1518] 7. Detailed explanation display means

[1519] On the device: When the user taps on a high-risk area, a detailed explanation and solutions for that area are displayed.

[1520] 8. How to book appointments and purchase care products

[1521] Terminal: Links for booking appointments and purchasing care products are displayed on the terminal, allowing the user to book an appointment with a veterinarian or purchase any necessary care products.

[1522] Specific examples

[1523] The user launches a dedicated application and takes a full-body photo of their pet (e.g., a dog) using their smartphone's camera. The application resizes the image, removes noise, and then sends it to a server. The server uses a generative AI model to analyze the health status of the image. Based on the analysis results, the server generates a suggested action, such as "Your eyes are bloodshot, so consider using eye drops. If symptoms persist, consult a veterinarian," and displays it on the user's device. When the user taps on high-risk areas based on the results, detailed explanations and countermeasures are displayed. Links to make appointments and purchase care products are also provided.

[1524] Examples of prompt statements

[1525] Take or select an image of your pet. After pre-processing, the image will be analyzed to assess your pet's health. For example, if it determines that your dog's eyes are bloodshot, it will display specific action suggestions such as "Consider using eye drops. If symptoms persist, consult a veterinarian."

[1526] This system is expected to enable pet owners to detect health conditions in their pets early and respond quickly and accurately, making pet health management more efficient and effective.

[1527] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1528] Step 1: Importing an image

[1529] User: The user launches a dedicated application on their smartphone or tablet and takes a picture of their pet or selects an existing image.

[1530] Input: A photo of your pet (either a new or existing photo)

[1531] What happens: The user taps the app's "Take Image" button and takes a full-body photo of their pet with the camera, or taps the "Select Image" button to choose an existing image from the gallery.

[1532] Output: Image data of the pet taken or selected by the user

[1533] Step 2: Image preprocessing

[1534] Device: The device performs pre-processing such as resizing and noise reduction on the captured or selected image.

[1535] Input: Image data of the pet taken or selected by the user

[1536] What it does: Resizes the image to a resolution of 600x800 pixels and reduces noise using a Gaussian filter. The application performs these steps automatically.

[1537] Output: Preprocessed image data

[1538] Step 3: Sending images

[1539] Terminal: Sends the image data after preprocessing to the server.

[1540] Input: Preprocessed image data

[1541] Specific operation: The device uses an HTTP POST request to upload the preprocessed image to the server.

[1542] Output: Image data sent to the server

[1543] Step 4: Analyze health status using AI models

[1544] Server: The server inputs the received image data into a generative AI model to detect the pet's face and body parts and assess health risks.

[1545] Input: Image data sent to the server

[1546] How it works: A Python script on the server receives the image and analyzes it using a TensorFlow-based generative AI model, which detects features such as bloodshot eyes and abnormal coat condition and assesses health risks.

[1547] Output: Analysis results on pet health risks

[1548] Step 5: Generate action suggestions

[1549] Server: Based on the analysis results, the server generates specific action suggestions for the user.

[1550] Input: Analysis results on pet health risks

[1551] Specific operation: Based on the analysis results obtained by the server, the server generates specific suggestions by inserting them into a pre-prepared text template. For example, it generates a suggestion such as, "Your eyes are bloodshot. Consider using eye drops. If the symptoms persist, consult a veterinarian."

[1552] Output: Suggested actions for the user

[1553] Step 6: View suggested actions

[1554] Terminal: The generated action suggestions are displayed on the user's terminal.

[1555] Input: Suggested actions for the user

[1556] Specific actions: Highlight risk areas on the device screen and provide text explaining specific ways to deal with the problem. For example, a bloodshot eye is highlighted in a red frame with suggested actions displayed below.

[1557] Output: Action suggestions displayed on the user's device

[1558] Step 7: View detailed instructions

[1559] User: When the user taps on a high-risk area, a detailed explanation and solutions for that area will be displayed.

[1560] Input: User tap operation, information on high-risk areas

[1561] Specific operation: When the user taps the red frame of the risk area, a detailed information screen will open, displaying an explanation of the eye symptoms and how to deal with them.

[1562] Output: Detailed explanation and solutions

[1563] Step 8: Book a consultation and purchase care products

[1564] Device: Links to schedule appointments and purchase care products will be displayed on the device.

[1565] Input: Relevant links based on identified health risks

[1566] Specific behavior: Link buttons to a medical appointment booking page and a care product online store are displayed on the screen. When the user taps these buttons, the respective pages open.

[1567] Output: Links to book appointments and purchase care products

[1568] (Application example 1)

[1569] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1570] Properly managing the health of pets is an important issue for pet owners. With current technology, it is difficult to grasp a pet's health condition early and take specific measures. This increases the risk that pet owners without specialized veterinary knowledge may overlook abnormalities in their pets. Furthermore, the lack of a simple and rapid means of assessing a pet's health raises concerns that pet care may be neglected in today's busy society. Therefore, to solve these problems, a system is needed that allows users to easily assess a pet's health condition and take specific measures.

[1571] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1572] In this invention, the server includes a means for capturing images of pets, a means for preprocessing the captured images, and a means for transmitting the preprocessed images to the server. This allows the pet images to be analyzed using an AI model, and specific action suggestions for the user to be automatically generated and displayed. Additionally, by adding a means for displaying detailed explanations and countermeasures by tapping on risk areas and a means for providing links to make medical appointments or purchase care products, pet owners can immediately take appropriate measures. Furthermore, the accuracy of the analysis is improved by adding a means for resizing and noise reduction to the image preprocessing and normalizing the resized images before inputting the image data into the AI ​​model. This enables early detection of pet health conditions and appropriate measures.

[1573] "Means for capturing an image of a pet" refers to a function that allows a user to take an image of a pet or select an existing image using a device such as a smartphone or tablet.

[1574] "Means for preprocessing captured images" refers to a function that processes captured or selected images, such as resizing and noise removal, to prepare the image data in a state suitable for AI analysis.

[1575] The "means for transmitting preprocessed images to a server" is a function for transmitting preprocessed images to a server via the Internet.

[1576] "Means for analyzing pet health conditions using an AI model on a server" refers to a function that inputs image data stored on a server into an AI model, analyzes the pet's health conditions, and identifies risk areas.

[1577] "Means for generating action suggestions for users based on analysis results" refers to a function for generating appropriate action suggestions for users based on the analysis results of the AI ​​model.

[1578] The "means for displaying the generated action proposal" is a function for displaying the generated action proposal in an easily understandable manner to the user.

[1579] "A means of displaying detailed explanations and solutions by tapping on risk areas" is a function that allows the user to tap on risk areas and display detailed explanations and solutions for that area.

[1580] "Means for providing links to schedule appointments or purchase care products" refers to a function that provides links for users to schedule appointments or purchase care products.

[1581] "Resizing" is a process of changing the size of an image.

[1582] "Noise removal" is a process that removes unnecessary information (noise) from an image.

[1583] "Normalization" is the process of aligning the range of data to a certain scale.

[1584] The present invention provides a system that enables pet owners to grasp the health condition of their pets at an early stage and take appropriate measures. Specific embodiments for carrying out the present invention will be described in detail below.

[1585] System configuration

[1586] 1. Image capture method

[1587] Users can take pictures of their pets using a smartphone, tablet, or other device, and the images are temporarily saved in the app.

[1588] 2. Image preprocessing methods

[1589] The device resizes and denoises the captured images. This preprocessing is performed using OpenCV. Resizing adjusts the image size appropriately, and denoising removes unnecessary data that may affect analysis.

[1590] 3. Image transmission method

[1591] After preprocessing, the image data is sent from the device to the server via Firebase or AWS.

[1592] 4. AI model for analyzing health status

[1593] The server runs an AI model using TensorFlow or PyTorch to analyze the received image data, specifically detecting pet faces and body parts and assessing their health risks.

[1594] 5. Action suggestion generation means

[1595] Based on the analysis results of the AI ​​model, the server generates specific action suggestions for the user, such as "Your eyes are bloodshot. Consider using eye drops. If the symptoms persist, consult a veterinarian."

[1596] 6. Action suggestion display means

[1597] The device displays the generated action suggestions in an easy-to-understand manner to the user, visually indicating risk areas and presenting specific countermeasures.

[1598] 7. Detailed explanation display means

[1599] When the user taps on a risk area, the device displays a detailed explanation and instructions on how to deal with that area, allowing pet owners to take appropriate measures with confidence.

[1600] 8. How to book appointments and purchase care products

[1601] The device provides links to help users easily schedule appointments and purchase care products, helping them get the care they need faster.

[1602] Specific examples

[1603] Below we introduce some specific scenarios in which this system can be used.

[1604] 1. The user launches the app and takes a full-body photo of their dog.

[1605] The user uses the app's camera function to take a full-body photo of their pet (e.g., dog). The device temporarily saves the image and displays a "Next" button.

[1606] 2. The device preprocesses the image

[1607] The device resizes the image and performs noise reduction. This preprocessing is done using OpenCV. The preprocessed image data is then sent to the server.

[1608] 3. The server analyzes the image and generates action suggestions.

[1609] The server inputs the received images into an AI model using TensorFlow or PyTorch to analyze the pet's health. For example, it identifies risk areas such as bloodshot eyes and abdominal swelling. Based on the analysis results, it then generates appropriate action suggestions for the user.

[1610] 4. The device displays the diagnosis results and suggested actions to the user.

[1611] The generated action suggestions are displayed on the device in an easy-to-understand manner to the user, visually indicating risk areas and presenting specific countermeasures.

[1612] Example prompts to input to the generative AI model

[1613] "Evaluate the condition of this dog's face and determine its health risks."

[1614] "Please use an AI model to analyze whether there is anything abnormal about this cat's abdomen."

[1615] This invention allows pet owners to easily understand the health status of their pets and quickly take necessary measures, which is expected to make pet health management more efficient and effective.

[1616] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1617] Step 1:

[1618] The user launches the app and captures an image of their pet. The user can either take a picture of their pet using the camera function of their smartphone or tablet, or import an existing image into the app. The input is a newly taken image or an existing image, which is temporarily stored on the device. The output is image data that awaits preprocessing.

[1619] Step 2:

[1620] The device performs preprocessing. Image resizing and noise removal are performed within the device. Specifically, OpenCV is used to appropriately adjust the image size and remove noise. The input is image data awaiting preprocessing, and the output is preprocessed image data suitable for AI analysis.

[1621] Step 3:

[1622] The device sends the preprocessed image to the server. The preprocessed image data is uploaded to the server via Firebase or AWS. The preprocessed image data is the input, and the image data stored on the server is the output.

[1623] Step 4:

[1624] The server analyzes the image using an AI model. Using TensorFlow or PyTorch on the server, the received image data is input into the AI ​​model. The AI ​​model detects the pet's face and body parts and evaluates health risks. The image data stored on the server is used as input, and the analysis results of the pet's health condition are obtained as output.

[1625] Step 5:

[1626] The server generates action suggestions. Based on the analysis results of the AI ​​model, the server generates specific action suggestions for the user. For example, an action suggestion might be something like, "Your eyes are bloodshot. Consider using eye drops. If the symptoms persist, consult a veterinarian." The input is the analysis result data of the pet's health condition, and the output is action suggestion data.

[1627] Step 6:

[1628] The terminal displays the action proposal. The terminal displays the generated action proposal to the user in a visually easy-to-understand manner. Specifically, it highlights risk areas and shows how to deal with them. The input is the action proposal data, and the output is the action proposal displayed to the user.

[1629] Step 7:

[1630] The user taps on a risky area. When the user taps on a risky area on the action suggestion display screen, a detailed explanation and countermeasures for that area are displayed. Specifically, a detailed screen is displayed that describes the symptoms and describes countermeasures. The user's tap operation is the input, and detailed information is displayed as the output.

[1631] Step 8:

[1632] The terminal provides links for making appointments and purchasing care products. When the user taps the links to make appointments or purchase care products as needed, they are redirected to an external site and the procedure is completed. The input is the action suggestion data and the user's link tap operation, and the output is the completion of the appointment or product purchase.

[1633] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1634] This invention combines a system that allows pet owners to quickly understand the health condition of their pets and take appropriate measures with an emotion engine that recognizes the user's emotions. Specifically, the system captures images of the pet, preprocesses the images, analyzes them based on an AI model, and evaluates the pet's health risks. In addition, the system recognizes the user's emotions and suggests actions based on the emotions.

[1635] System configuration

[1636] 1. Image capture method

[1637] User: Uses a device such as a smartphone or tablet to take a picture of their pet or select an existing image.

[1638] 2. Image preprocessing methods

[1639] Device: Performs preprocessing such as resizing and noise removal on captured or selected images, making the image data suitable for AI analysis.

[1640] 3. Image transmission method

[1641] Terminal: Sends the pre-processed image to the server.

[1642] 4. AI model for analyzing health status

[1643] Server: The received image data is input into the AI ​​model to analyze the pet's health condition. Specifically, it detects the pet's face and body parts and determines the health risk of each part (e.g., normal, requires observation, emergency).

[1644] 5. Action suggestion generation means

[1645] Server: Based on the analysis results of the AI ​​model, it generates specific action suggestions for the user (e.g., recommendations for daily care, the need for a veterinarian).

[1646] 6. Action suggestion display means

[1647] Device: The generated action proposals are displayed in an easy-to-understand manner to the user. Specifically, risk areas and countermeasures are displayed on the screen.

[1648] 7. Emotion Engine

[1649] Terminal or server: Equipped with an emotion engine that recognizes the user's emotions, it determines the user's emotions by analyzing the user's voice data and image data.

[1650] 8. Emotion-based behavioral adjustment measures

[1651] Server: Adjusts suggested actions and provides appropriate feedback based on the user's emotions recognized by the emotion engine. For example, if the user is feeling stressed, the server provides more helpful and specific explanations.

[1652] 9. Detailed explanation display means

[1653] Device: When the user taps on a high-risk area, a detailed explanation and countermeasures for that area will be displayed.

[1654] 10. How to book appointments and purchase care products

[1655] Terminal: Provides links to schedule appointments and purchase care products. This feature allows users to easily schedule veterinary appointments and purchase necessary care products.

[1656] Specific examples

[1657] 1. The user launches the app and takes a full-body photo of their dog.

[1658] User: Use the app's camera function to take a full-body photo of your pet (e.g., dog).

[1659] Device: Temporarily saves the captured image and displays the "Next" button.

[1660] 2. The device preprocesses the image

[1661] Device: Resize and denoise the image to make it optimal for analysis.

[1662] Terminal: Sends the image data after preprocessing to the server.

[1663] 3. The server analyzes the image and generates action suggestions.

[1664] Server: The received images are input into the AI ​​model to analyze the pet's health condition. For example, risk areas such as bloodshot eyes and abdominal swelling are identified.

[1665] Server: Based on the analysis results, it generates appropriate action suggestions to the user (e.g., "Your eyes are bloodshot. Consider using eye drops. If symptoms persist, consult your veterinarian").

[1666] 4. The device displays the diagnosis results and suggested actions to the user.

[1667] Terminal: The diagnostic results are displayed intuitively to the user, clearly indicating risk areas and providing specific countermeasures.

[1668] 5. Users access more information and additional features

[1669] User: Tap on high-risk areas to see detailed explanations and solutions.

[1670] Terminal: Users can access links to book appointments and purchase care products as needed.

[1671] 6. Use of Emotion Engines

[1672] Device or server: While the user is using the app, it analyzes voice data and facial expressions to recognize the user's emotions (e.g., joy, stress, anxiety).

[1673] Server: Adjust the content of the suggested action based on the user's emotions. For example, if the user expresses anxiety, add a detailed and helpful explanation.

[1674] This system not only enables pet owners to accurately and quickly understand their pet's health condition and take appropriate measures, but also enables more effective care by recognizing the user's emotions and providing support accordingly.

[1675] The processing flow will be explained below.

[1676] Step 1:

[1677] The user launches the smartphone app and takes a picture of their pet or selects an existing image, providing the pet's health status as input data.

[1678] Step 2:

[1679] The device temporarily stores the captured or selected image, which is then used in the next processing step.

[1680] Step 3:

[1681] Your device will resize the image to the appropriate resolution, reducing data usage and improving processing speed.

[1682] Step 4:

[1683] The device will then perform noise reduction to improve image quality, which will improve the accuracy of the analysis by the AI ​​model.

[1684] Step 5:

[1685] The device sends the preprocessed image data to the server, which forwards the image data to the server for analysis.

[1686] Step 6:

[1687] The server inputs the received image data into the AI ​​model, which analyzes the image and begins the process of assessing the pet's health.

[1688] Step 7:

[1689] The server uses an AI model to detect the pet's face and body parts, such as the eyes, ears, mouth, and abdomen, and assesses risk areas.

[1690] Step 8:

[1691] The server evaluates the pet's health risk and determines the risk level (e.g., normal, requires observation, emergency). Based on the analysis results, it generates action suggestions for the user.

[1692] Step 9:

[1693] The terminal displays the diagnosis results and action suggestions received from the server to the user. An interface including risk areas and solutions is displayed.

[1694] Step 10:

[1695] When a user uses the app, the device or server inputs the user's voice data and image data into the emotion engine, which analyzes this data and recognizes the user's emotions.

[1696] Step 11:

[1697] The server analyzes the user's emotional data and adjusts the suggested actions, for example, adding specific and helpful explanations if the user is feeling stressed.

[1698] Step 12:

[1699] The device displays behavior suggestions tailored to the user's emotions, providing appropriate feedback according to the user's emotional state.

[1700] Step 13:

[1701] When the user taps on a high-risk area, the device displays a detailed explanation and how to deal with it. The user can check the cause of the specific symptom and how to deal with it.

[1702] Step 14:

[1703] The device provides links to schedule appointments and purchase care products. Users can schedule appointments with veterinarians or purchase care products as needed.

[1704] This system allows pet owners to accurately understand their pet's health condition and take appropriate measures. It also recognizes the user's emotions and provides support accordingly, enabling more effective care.

[1705] Example 2

[1706] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1707] There are few systems that can quickly identify a pet's health condition and provide appropriate treatment, and the suggested actions provided by these systems often do not take the user's feelings into consideration. Furthermore, due to a lack of preprocessing functions to improve the accuracy of pet image data analysis, the reliability of analysis results is often low. This makes it difficult for pet owners to manage their pet's health with peace of mind.

[1708] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1709] In this invention, the server includes means for capturing an image of the pet, means for preprocessing the captured image, means for transmitting the preprocessed image to the server, means for analyzing the health condition of the pet using an AI model on the server, means for generating suggested actions for the user based on the analysis results, means for displaying the generated suggested actions, means for recognizing the user's emotions, and means for adjusting the suggested actions based on the recognized emotions. This makes it possible to quickly analyze the health condition of the pet with high accuracy and provide specific and appropriate suggested actions that take the user's emotions into consideration.

[1710] A "means for capturing pet images" is a device or software that allows a user to take an image of a pet or select an existing image using a device such as a smartphone or tablet, and then register that information in the system.

[1711] The "means for pre-processing captured images" refers to devices or software that perform processes such as resizing and noise removal on images captured on the terminal.

[1712] The "means for transmitting the preprocessed image to the server" refers to a communication function or software for transferring the preprocessed image to the server via a network.

[1713] "Means for analyzing a pet's health condition using an AI model on a server" refers to devices or software that use an AI model running on a server to analyze images of a pet and evaluate its health condition.

[1714] "Means for generating suggested actions for users based on analysis results" refers to devices or software that suggest specific ways of dealing with issues or actions to users based on the results of analysis by the AI ​​model.

[1715] The "means for displaying the generated action suggestions" refers to a device or software for visually displaying the action suggestions to the user on the terminal.

[1716] The "means for recognizing the user's emotions" refers to a device or software that analyzes the user's voice data or image data and identifies the user's emotional state.

[1717] The "means for adjusting suggested actions based on recognized emotions" refers to a device or software that optimizes the content of suggested actions provided based on the results of analyzing the user's emotions using an emotion engine.

[1718] The "means for displaying detailed explanations for high-risk areas" refers to a device or software that displays detailed explanations related to a particular high-risk area when a user requests detailed information about that area.

[1719] A "means for providing links to schedule an appointment or purchase a care product" is a device or software that provides an online link for a user to schedule an appointment or purchase a care product.

[1720] "Means for resizing or denoising images in preprocessing" refers to devices or software for changing the size of images to a resolution suitable for analysis or for reducing noise contained in images.

[1721] The present invention relates to a system that allows pet owners to grasp the health condition of their pets at an early stage and take appropriate measures, and further has the function of recognizing the user's emotions and providing appropriate action suggestions.

[1722] Basic system configuration

[1723] This system consists of the following components: Users use devices such as smartphones and tablets.

[1724] 1. Image capture method

[1725] User: Use the camera function on your smartphone or tablet to take a picture of your pet or select an existing image.

[1726] 2. Image preprocessing methods

[1727] On-device: Perform preprocessing such as resizing and noise reduction on the captured or selected image. For example, you can use an image processing library such as OpenCV.

[1728] 3. Image transmission method

[1729] Terminal: The preprocessed image is sent to the server using an HTTP request.

[1730] 4. AI model for analyzing health status

[1731] Server: The received image data is input into a deep learning model (e.g., a convolutional neural network) to analyze the pet's health condition. Deep learning libraries such as TensorFlow and PyTorch can be used. Specific analysis involves detecting faces and body parts and assessing the health risk of each part.

[1732] 5. Action suggestion generation means

[1733] Server: Based on the analysis results of the AI ​​model, the server generates specific action suggestions for the user. For example, an action suggestion might be, "Your eyes are bloodshot. Consider using eye drops. If the symptoms persist, consult a veterinarian."

[1734] 6. Action suggestion display means

[1735] On-device: The generated action suggestions are displayed to the user. Risk areas are marked on the image, and specific countermeasures are displayed as text. This can be implemented using a UI framework (e.g., Flutter or React Native).

[1736] 7. Emotion Engine

[1737] Terminal or server: Equipped with an emotion engine that analyzes the user's voice data and image data and recognizes the user's emotions. Watson or Azure Cognitive Services can be used for voice recognition, and OpenFace or FaceReader can be used for emotion recognition.

[1738] 8. Emotion-based behavioral adjustment measures

[1739] Server: Adjusts the suggested actions based on the user's emotions recognized by the emotion engine. Specifically, if the user is feeling stressed, it adds detailed and helpful explanations.

[1740] 9. Detailed explanation display means

[1741] Device: When the user taps on a high-risk area, detailed explanations and countermeasures for that area are displayed. For example, if the lead user taps on the eye area, specific instructions on how to use eye drops are displayed.

[1742] 10. How to book appointments and purchase care products

[1743] Terminal: Provides links for users to schedule appointments and purchase care products. With one click, users can schedule a veterinary appointment or purchase the care products they need online.

[1744] Specific examples

[1745] A specific example of use is shown below.

[1746] 1. The user launches the app and takes a full-body photo of their dog.

[1747] User: Use the app's camera function to take a full-body photo of your pet (e.g., your dog).

[1748] Device: Temporarily saves the captured image and displays the "Next" button.

[1749] 2. The device preprocesses the image

[1750] On the device: Resize the image and remove noise to optimize it for analysis. For example, resize the image to 1280x720 pixels, and use a Gaussian filter to remove noise.

[1751] Terminal: Sends the image data after preprocessing to the server.

[1752] 3. The server analyzes the image and generates action suggestions.

[1753] Server: The received images are input into the AI ​​model to analyze the pet's health condition. For example, risk areas such as bloodshot eyes and abdominal swelling are identified.

[1754] Server: Based on the analysis results, it generates appropriate action suggestions to the user (e.g., "Your eyes are bloodshot. Consider using eye drops. If symptoms persist, consult your veterinarian").

[1755] 4. The device displays the diagnosis results and suggested actions to the user.

[1756] Terminal: The diagnostic results are displayed intuitively to the user, clearly indicating risk areas and providing specific countermeasures.

[1757] 5. Users access more information and additional features

[1758] User: Tap on high-risk areas to see detailed explanations and solutions.

[1759] Terminal: Users can access links to book appointments and purchase care products as needed.

[1760] 6. Use of Emotion Engines

[1761] Device or server: While the user is using the app, it analyzes voice data and facial expressions to recognize the user's emotions (e.g., joy, stress, anxiety).

[1762] Server: Adjust the content of the suggested action based on the user's emotions. For example, if the user expresses anxiety, add a detailed and helpful explanation.

[1763] Specific prompt examples:

[1764] "Send a user a picture of their pet and have the AI ​​model analyze its health condition. Also, explain how it can recognize the user's emotions and provide appropriate behavioral suggestions."

[1765] The system of the present invention not only allows pet owners to accurately and quickly understand their pet's health condition and take appropriate measures, but also enables more effective care by the system recognizing the user's emotions and providing support accordingly.

[1766] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1767] Step 1:

[1768] Image capture

[1769] User: Use the device's camera to take a picture of your pet or select an existing image.

[1770] Input: New image data from the camera or existing image data from the photo gallery.

[1771] Output: Temporarily stored image data.

[1772] Step 2:

[1773] Image preprocessing

[1774] Terminal: Preprocess the image, such as resizing and denoising. For example, resize the image to 1280x720 pixels and denoise it using a Gaussian filter.

[1775] Input: Temporarily stored image data.

[1776] Output: Preprocessed image data.

[1777] Step 3:

[1778] Sending image data

[1779] Terminal: Send the preprocessed image to the server using an HTTP request.

[1780] Input: Preprocessed image data.

[1781] Output: Image data sent to the server.

[1782] Step 4:

[1783] Health status analysis

[1784] Server: The received image data is input into an AI model to analyze the pet's health. Specifically, it detects the face and body parts and evaluates the health risks associated with each part. For example, it identifies bloodshot eyes and swollen abdomen.

[1785] Input: Image data sent to the server.

[1786] Output: Analysis result data (risk assessment results).

[1787] Step 5:

[1788] Generate action suggestions

[1789] Server: Generates specific action suggestions for the user based on the analysis results. For example, it creates suggestions such as "Your eyes are bloodshot. Consider using eye drops. If the symptoms persist, consult a veterinarian."

[1790] Input: Analysis result data.

[1791] Output: Action suggestion data.

[1792] Step 6:

[1793] Displaying suggested actions

[1794] Device: The generated action suggestions are displayed in an easy-to-understand manner to the user. Specifically, risk areas are marked on the image and specific countermeasures are displayed in text.

[1795] Input: Action suggestion data.

[1796] Output: Action suggestions displayed to the user.

[1797] Step 7:

[1798] Emotion recognition

[1799] Terminal or server: Analyzes the user's voice data and image data and recognizes the user's emotions using an emotion engine. For example, it uses a voice recognition engine or facial expression recognition software.

[1800] Input: User's voice or image data.

[1801] Output: Recognized emotion data.

[1802] Step 8:

[1803] Adjusting behavioral suggestions according to emotions

[1804] Server: Optimizes the content of action suggestions based on the recognized emotion. For example, if the user is recognized as feeling stressed, it adds detailed and helpful explanations.

[1805] Input: Recognized emotion data and action suggestion data.

[1806] Output: Adjusted action suggestion data.

[1807] Step 9:

[1808] View detailed description

[1809] Device: When a user taps on a high-risk area, a detailed explanation and countermeasures for that area are displayed. For example, if a user taps on a high-risk area in the eyes, specific instructions on how to use eye drops are displayed.

[1810] Input: User taps on the risk area.

[1811] Output: Display detailed descriptive data.

[1812] Step 10:

[1813] Make appointments and purchase care products

[1814] Terminal: Provides links for scheduling appointments and purchasing care products. Users can use this to schedule veterinary appointments and purchase care products online.

[1815] Input: User clicks.

[1816] Output: Display of appointment booking form and care product purchase page.

[1817] (Application example 2)

[1818] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1819] There is a lack of systems that allow pet owners to effectively and quickly understand their pet's health condition and take appropriate measures. As a result, there is a risk that pet health problems will not be detected early and appropriate care will be delayed. In addition, the lack of action suggestions that take into account the pet owner's emotions may make users feel stressed, which could lead to a decline in the quality of care.

[1820] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for capturing an image of the pet, means for preprocessing the captured image, means for transmitting the preprocessed image to the server, means for analyzing the health condition of the pet using a generative AI model, means for generating action suggestions for the user based on the analysis results, means for recognizing the user's emotions, means for adjusting the action suggestions based on the recognized emotions, means for displaying the generated action suggestions, means for suggesting appropriate pet food, and means for generating a prompt message. This enables pet owners to grasp the health condition of their pets at an early stage and receive appropriate care and food suggestions based on the user's emotions.

[1821] A "means for capturing an image of a pet" is a device or software that allows a user to take or select an image of a pet.

[1822] The "means for pre-processing captured images" refers to devices or software that perform processes such as resizing and noise removal on captured or selected images.

[1823] The "means for transmitting preprocessed images to a server" refers to a device or software that transmits preprocessed image data to a remote server via a network.

[1824] A "generative AI model" is an artificial intelligence model that runs on a computer to analyze the health of pets.

[1825] "Means for analyzing the health condition of a pet" refers to a device or software that uses a generative AI model on a server to analyze image data and determine the health condition of a pet.

[1826] The "means for generating suggested actions for the user based on the analysis results" refers to a device or software that generates specific suggested actions for the pet owner based on the results of the analyzed health condition.

[1827] The "means for recognizing user's emotions" refers to a device or software that determines the user's emotions by analyzing the user's voice data and facial expression data.

[1828] The "means for adjusting suggested actions based on recognized emotions" refers to a device or software that adjusts the content of suggested actions taking into account the user's emotional state.

[1829] The "means for displaying the generated action suggestions" refers to a device or software that visually displays the generated action suggestions on the user's terminal.

[1830] The "means for suggesting appropriate pet food" refers to a device or software that suggests appropriate food for a pet, taking into consideration the pet's health condition and the user's emotions.

[1831] A "means for generating prompt sentences" is a device or software that generates questions or instructions that a user can use to assess the health of their pet.

[1832] This invention provides a system that allows pet owners to quickly understand their pet's health condition and take appropriate measures. This system includes a function that captures images of the pet, analyzes them using an AI model, recognizes the user's emotions, and adjusts behavior suggestions accordingly. The main components of this system and their functions are as follows:

[1833] 1. How to capture images of your pet

[1834] Users can take pictures of their pets using the camera function of their smartphones, tablets, or other devices. They can also select existing images. This image capture method allows users to accurately capture their pet's current state.

[1835] 2. A means of preprocessing the captured images

[1836] The device performs preprocessing such as resizing and noise removal on the captured image, making the image data suitable for AI analysis. Specifically, this preprocessing is performed using an image processing library such as OpenCV.

[1837] 3. A means to send preprocessed images to the server

[1838] Once preprocessed, the image data is sent from the device to a server over a network, where it can be analyzed using advanced AI models.

[1839] 4. A means of analyzing your pet's health

[1840] The server uses generative AI models such as TensorFlow and Keras to analyze the received image data. It detects the pet's face and body parts and determines the health risk of each part (normal, requires observation, emergency). Based on the analysis results, the appropriate countermeasures are determined.

[1841] 5. How to Recognize User Emotions

[1842] The device or server analyzes the user's voice data and facial expressions to perform emotion recognition, using emotion analysis tools such as EmotionRecognizer, to determine how the user feels about their pet's health.

[1843] 6. A way to tailor suggested actions based on emotions

[1844] The server then adjusts the suggested actions based on the user's perceived emotions: for example, if the user expresses anxiety, it adds more specific and helpful instructions, and also suggests appropriate pet food and care products.

[1845] 7. Means for displaying generated action suggestions

[1846] The device visually displays the action suggestions received from the server to the user, highlighting risk areas and providing specific solutions to help the user take action quickly.

[1847] Specific examples

[1848] The user launches the app and takes a full-body photo of their dog. The device preprocesses the image and sends it to the server. The server analyzes the image, detects "bloodshot eyes," and suggests to the user, "Consider using eye drops. If symptoms persist, consult a veterinarian." The user then uses the system's camera function to take a photo of their own face and performs emotion analysis. The analysis results identify the user's emotion as "anxiety," so the server makes a suggested course of action with a detailed and helpful explanation.

[1849] Prompt Sentence Examples

[1850] "Take a photo of your dog's face and body with this app and analyze its health condition. Then, take a photo of your face and analyze its emotions. Based on the analysis results, the app will make appropriate food and care recommendations."

[1851] This system allows pet owners to grasp their pet's health condition early on and receive appropriate care and food suggestions based on the user's emotions.

[1852] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1853] Step 1:

[1854] The user takes a picture of their pet. The user uses the camera function of their smartphone or tablet to capture the image of their pet. The input is the "pet image" and the output is the image data stored on the device.

[1855] Step 2:

[1856] The device preprocesses the image. Specifically, it resizes and removes noise using an image processing library such as OpenCV. The input is the image data obtained in step 1, and the output is the preprocessed image data. Resizing changes the resolution to a level suitable for analysis, and noise removal increases the clarity of the image.

[1857] Step 3:

[1858] Send the preprocessed image data to the server. The device sends the image data to the remote server via the network. The input is the preprocessed image data obtained in step 2, and the output is the image data sent to the server.

[1859] Step 4:

[1860] The server analyzes the pet's health condition. The image data is analyzed using a generative AI model (using TensorFlow and Keras) on the server. The input is the image data sent in step 3, and the output is the pet's health assessment results (e.g., "bloodshot eyes" or "swollen abdomen"). The AI ​​model detects specific parts of the face and body and determines the health risk of each.

[1861] Step 5:

[1862] Recognize user emotions. The user uses a device to capture their own voice or facial image. The device or server performs emotion analysis using EmotionRecognizer. The input is the user's voice data or facial image data, and the output is the user's emotional assessment result (e.g., "joy," "stress," or "anxiety").

[1863] Step 6:

[1864] The server adjusts the action suggestions based on the user's emotions. The action suggestions generated from the analysis results are adjusted taking into account the recognized user's emotions. The input is the health assessment result from step 4 and the emotion assessment result from step 5, and the output is the final action suggestions. For example, for users who show anxiety, a detailed and friendly explanation is added.

[1865] Step 7:

[1866] The server recommends the appropriate pet food based on the pet's health condition and the user's emotions. The input is the health assessment result in step 4 and the emotion assessment result in step 5, and the output is the food recommendation.

[1867] Step 8:

[1868] Display the generated action suggestions. The device visually displays the action suggestions received from the server to the user, providing specific solutions and links to care products. The input is the suggestion results from steps 6 and 7, and the output is the action suggestions displayed on the user's device.

[1869] Step 9:

[1870] The user accesses detailed information and additional features. The user taps on a high-risk area to see detailed explanations and solutions, and then accesses links to schedule an appointment or purchase care products. The input is the action suggestion link provided by the server, and the output is the user's action (e.g., schedule an appointment, purchase a product).

[1871] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1872] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1873] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1874] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1875] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1876] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1877] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1878] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1879] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1880] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1881] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1882] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1883] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1884] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1885] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1886] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1887] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1888] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1889] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1890] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1891] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1892] The following is further disclosed regarding the above embodiment.

[1893] (Claim 1)

[1894] means for capturing an image of the pet;

[1895] means for pre-processing the captured image;

[1896] means for transmitting the preprocessed image to a server;

[1897] A means to analyze the health status of pets using an AI model on a server,

[1898] means for generating action suggestions for the user based on the analysis results;

[1899] means for displaying the generated action suggestions;

[1900] A system including:

[1901] (Claim 2)

[1902] A means for displaying detailed explanations for high-risk areas;

[1903] Includes a means to provide links to book appointments or purchase care products

[1904] 10. The system of claim 1.

[1905] (Claim 3)

[1906] Includes means for resizing or denoising images during preprocessing

[1907] 10. The system of claim 1.

[1908] "Example 1"

[1909] (Claim 1)

[1910] means for capturing an image of the pet;

[1911] means for pre-processing the captured image;

[1912] means for transmitting the preprocessed image to a server;

[1913] A means to analyze the health status of pets using a generative AI model on a server,

[1914] means for generating action suggestions for the user based on the analysis results;

[1915] means for displaying the generated action suggestions;

[1916] a means for displaying a detailed explanation of a high-risk area when the user taps on the high-risk area;

[1917] a means of providing links to schedule appointments and purchase care products;

[1918] A system including:

[1919] (Claim 2)

[1920] 10. The system of claim 1, further comprising means for performing image resizing and denoising during image preprocessing.

[1921] (Claim 3)

[1922] The system of claim 1, further comprising a step of transmitting the preprocessed image to a server and performing image analysis based on the generative AI model.

[1923] "Application Example 1"

[1924] (Claim 1)

[1925] means for capturing an image of the pet;

[1926] means for pre-processing the captured image;

[1927] means for transmitting the preprocessed image to a server;

[1928] A means to analyze the health status of pets using an AI model on a server,

[1929] means for generating action suggestions for the user based on the analysis results;

[1930] means for displaying the generated action suggestions;

[1931] By tapping on the risk area, detailed explanations and solutions will be displayed.

[1932] a means of providing links to schedule appointments or purchase care products;

[1933] A system including:

[1934] (Claim 2)

[1935] A method to display detailed explanations of symptoms and countermeasures when tapping on risk areas,

[1936] Includes a means to provide links to book appointments or purchase care products

[1937] 10. The system of claim 1.

[1938] (Claim 3)

[1939] means for resizing or denoising the image in preprocessing;

[1940] Includes a means to normalize resized images before feeding them into the AI ​​model.

[1941] 10. The system of claim 1.

[1942] "Example 2: Combining Emotion Engines"

[1943] (Claim 1)

[1944] means for capturing an image of the pet;

[1945] means for pre-processing the captured image;

[1946] means for transmitting the preprocessed image to a server;

[1947] A means to analyze the health status of pets using an AI model on a server,

[1948] means for generating action suggestions for the user based on the analysis results;

[1949] means for displaying the generated action suggestions;

[1950] means for recognizing a user's emotion;

[1951] a means for adjusting behavioral suggestions based on the perceived emotions;

[1952] A system including:

[1953] (Claim 2)

[1954] A means for displaying detailed explanations for high-risk areas;

[1955] Includes a means to provide links to book appointments or purchase care products

[1956] 10. The system of claim 1.

[1957] (Claim 3)

[1958] Includes means for resizing or denoising images during preprocessing

[1959] 10. The system of claim 1.

[1960] "Application example 2 when combining emotion engines"

[1961] (Claim 1)

[1962] means for capturing an image of the pet;

[1963] means for pre-processing the captured image;

[1964] means for transmitting the preprocessed image to a server;

[1965] A means to analyze the health status of pets using a generative AI model on a server,

[1966] means for generating action suggestions for the user based on the analysis results;

[1967] means for recognizing a user's emotion;

[1968] a means for adjusting behavioral suggestions based on the perceived emotions;

[1969] means for displaying the generated action suggestions;

[1970] A means of suggesting appropriate pet food;

[1971] A system including:

[1972] (Claim 2)

[1973] A means for displaying detailed explanations for high-risk areas;

[1974] Includes a means to provide links to book appointments or purchase care products

[1975] 10. The system of claim 1.

[1976] (Claim 3)

[1977] Includes means for resizing or denoising images during preprocessing

[1978] 10. The system of claim 1.

[1979] (Claim 4)

[1980] Includes a means to generate prompts

[1981] 10. The system of claim 1. [Explanation of symbols]

[1982] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for capturing an image of the pet; means for pre-processing the captured image; means for transmitting the preprocessed image to a server; A means to analyze the health status of pets using an AI model on a server, means for generating action suggestions for the user based on the analysis results; means for displaying the generated action suggestions; A system including:

2. A means for displaying detailed explanations for high-risk areas; Includes a means to provide links to book appointments or purchase care products The system of claim 1 .

3. Includes means for resizing or denoising images during preprocessing The system of claim 1 .

Images