System

Abstract

A system is provided.SOLUTION: A system comprising: a user interface means for capturing an image of a pet; a data-transmitting means for transmitting the captured image to a server; a means for receiving and pre-processing the transmitted image; a generated AI model means for performing a medical checkup with the pre-processed image as an input; a means for generating a recommended action based on a diagnostic result by the generated AI model; and a means for displaying the diagnostic result and the recommended action to a user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] When pet owners notice signs of illness or injury in their pets, it can be difficult to decide whether or not to take them to the veterinarian. Furthermore, the high cost of veterinary consultations, the time required, and the difficulty of transporting pets are significant burdens for many owners. Many owners, in particular, lack knowledge about pets, making it difficult to properly understand their pets' health conditions. [Means for solving the problem]

[0005] The present invention provides a system that includes a user interface for capturing images of a pet, a data transmission means for transmitting the captured images to a server, a means for receiving and preprocessing the transmitted image data, a generative AI model for performing a health check using the preprocessed images as input, a means for generating recommended actions based on the diagnostic results of the generative AI model, and a means for displaying the diagnostic results and recommended actions to the user. This allows the user to quickly understand the health condition of their pet and take necessary measures. Furthermore, by including a means for providing links to purchase care products and to book medical services based on the diagnostic results, the user can complete all actions in one place. Furthermore, the generative AI model is trained using a training dataset of healthy and unhealthy pets, enabling it to provide accurate diagnoses.

[0006] "User interface means" refers to an interface designed to allow pet owners to intuitively take pictures of their pets.

[0007] "Data transmission means" refers to the technical means for securely transmitting captured image data to a server.

[0008] "Pre-processing means" refers to processes and techniques for removing noise and normalizing image data received by the server.

[0009] "Generative AI model means" refers to a means for detecting abnormalities in images and diagnosing health conditions using algorithms and models trained on training data.

[0010] "Means for generating recommended actions" refers to a means for generating specific care methods and medical service suggestions based on the diagnostic results of an AI model.

[0011] The term "diagnosis result display means" refers to a means for visually displaying the analyzed diagnosis results and recommended actions to the user on the terminal.

[0012] "Care Product Purchase Link" refers to technology that includes an online link that enables a User to directly purchase a care product suggested within the App.

[0013] "Medical Service Booking Link" refers to technology that includes an online link within an app that enables a user to book an appointment with a nearby veterinarian or medical service.

[0014] "Training Data Set" refers to a data collection that includes image data and related information of healthy and unhealthy pets. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] Overall system overview

[0037] The present invention provides a system for taking images of pets and diagnosing their health condition. The system helps users monitor their pet's health condition and seek necessary care and medical examinations. Specifically, the system includes a user interface, a data transmission means, a preprocessing means, a generative AI model means, a recommended action generation means, a diagnostic result display means, a link to purchase care products, and a link to book medical services.

[0038] Explanation of program processing

[0039] 1. User Interface

[0040] The user launches the app on their smartphone or tablet and takes a picture of their pet. The user interface is simple and easy to use, making this process intuitive and easy. For example, a user can take a picture of their pet's swollen ear.

[0041] 2. Data Transmission

[0042] The device sends the captured image data to the server. This data is sent via a secure protocol. The image data is compressed and sent in the appropriate format.

[0043] 3. Pretreatment

[0044] The server decompresses the received image data and performs preprocessing, which includes noise removal, image standardization, and extraction of necessary parts, to prepare the image for accurate analysis by the generative AI model.

[0045] 4. Applying generative AI models

[0046] The server then feeds the pre-processed images into a generative AI model for health assessment. The generative AI model uses pre-trained algorithms to detect abnormalities in the images. For example, the AI ​​model might detect swelling in a pet's ear and determine that this is an early sign of infection.

[0047] 5. Generating recommended actions

[0048] The server then recommends actions to the user based on the AI ​​model's diagnosis. These recommendations include care methods and veterinary advice. For example, a message such as "There is a high possibility of an infection, so we recommend that you take your pet to the veterinarian" is generated.

[0049] 6. Display of diagnostic results

[0050] The device displays the analyzed diagnostic results and recommended actions to the user. The diagnostic results are displayed in a visually easy-to-understand format so that the user can immediately understand them. For example, a message such as "Your pet's ear is swollen. This swelling is likely an early sign of an infection" may be displayed.

[0051] 7. Proposing and implementing actions

[0052] Based on the analysis results, the device displays links to purchase care products or make appointments for medical services. By clicking these links, users can purchase the necessary care products or make appointments with a nearby veterinarian. For example, users can select "Purchase preventative medicine" or "Make an appointment with a nearby veterinarian" to take immediate action.

[0053] Specific examples

[0054] A user notices that their pet's ear is swollen, launches the app, and takes a photo of the area. The device sends the image to a server, which performs preprocessing such as noise removal. The preprocessed image is analyzed by a generative AI model, which determines that the ear swelling may indicate an infection. The server generates a message recommending immediate veterinary care along with the diagnosis "Suspected infection: High risk," and sends this to the user's device. The device displays this, and the user can easily make an appointment with a nearby veterinarian and purchase infection preventative medication within the app.

[0055] In this way, the present invention provides innovative support for pet health management, enabling users to take appropriate measures quickly and efficiently.

[0056] The processing flow will be explained below.

[0057] Step 1:

[0058] The user launches the app on their smartphone or tablet and takes a picture of their pet. The app then checks for any abnormalities in a specific area of ​​the pet's body and takes a photo focusing on that area. For example, if the pet's ear is red and swollen, the app will capture the area around the ear.

[0059] Step 2:

[0060] The device displays the captured image, and the user confirms the image and taps the send button. This operation sends the image data to the server.

[0061] Step 3:

[0062] The device compresses the captured image data and sends it to the server using a secure protocol (e.g., HTTPS). During the data transmission process, the image is compressed and encrypted.

[0063] Step 4:

[0064] The server decompresses the received image data and performs preprocessing while preserving image quality. Preprocessing includes noise removal, contrast adjustment, and resolution standardization. This process improves the accuracy of analysis.

[0065] Step 5:

[0066] The server inputs the pre-processed images into a generative AI model, which uses pre-trained algorithms to detect anomalies, identifying abnormalities in the images (e.g., swelling or color changes) and providing a health diagnosis based on the results.

[0067] Step 6:

[0068] The server generates recommended actions based on the diagnosis results from the AI ​​model, such as "We recommend a veterinary visit as there is a high possibility of an infection" or "We recommend the use of specific care products."

[0069] Step 7:

[0070] The server sends the generated diagnostic results and recommended actions to the device, and the data is formatted in a way that allows the user to understand it intuitively and quickly.

[0071] Step 8:

[0072] The device displays the diagnosis results and recommended actions to the user, such as "Your pet's ears may have an ear infection. This is a high-risk condition, so please consult a veterinarian immediately."

[0073] Step 9:

[0074] Based on the diagnosis, the device will display links to purchase care products and book medical appointments. Users can click the links within the app to purchase the necessary care products online and book appointments with a nearby veterinarian.

[0075] Step 10:

[0076] When a user clicks on the link to buy a care product, they are taken to an online store page where they can purchase the care product, and when they click on the link to book a medical service, they are taken to a nearby veterinarian's appointment page where they can enter the necessary information to complete the appointment.

[0077] This series of processes allows the user to quickly understand the health condition of their pet and efficiently take necessary measures.

[0078] Example 1

[0079] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0080] In recent years, the importance of pet health management has increased, but it is not easy to continuously monitor pet health in busy daily lives. In addition, there is often a lack of means to detect abnormalities in pets early and provide appropriate care and examinations. With conventional methods, owners themselves need to have specialized knowledge to accurately diagnose their pet's health condition, and there is a risk that the condition may worsen before they can be examined by a specialist.

[0081] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0082] In this invention, the server includes a user interface means for taking images of the pet, a data transmission means for transmitting the captured images to the server, a means for receiving and preprocessing the transmitted image data, a generative AI model means for performing a health check using the preprocessed images as input, a means for generating recommended actions based on the diagnosis results from the generative AI model, a means for displaying the diagnosis results and the recommended actions to the user, and a means for providing links to purchase care products and to book medical services based on the diagnosis results, thereby enabling the user to quickly and accurately understand the health condition of their pet and to receive the necessary care and examination.

[0083] The "user interface means" is a means for providing an intuitive and easy-to-operate interface for the user to take pictures of their pet.

[0084] The "data transmission means" is a means for compressing and encrypting the captured image data and transmitting it to the server via a secure protocol.

[0085] The "means for performing preprocessing" refers to means for removing noise from received image data, standardizing the image data, extracting necessary parts, and the like.

[0086] A "generative AI model means" is a means including an algorithm trained using a training data set that performs a medical examination using preprocessed image data as input.

[0087] "Means for generating recommended actions" refers to means for recommending actions to be taken by the user based on the diagnostic results of the generative AI model.

[0088] The "means for displaying the diagnostic results to the user" refers to a means for displaying the analyzed diagnostic results and recommended actions to the user in a visually easy-to-understand format.

[0089] The "means for providing a link for purchasing a care product" is a means for providing a link that enables a user to purchase a necessary care product based on the diagnosis result.

[0090] The "means for providing a link to book a medical service" refers to a means for providing a link that allows a user to book a required medical service (e.g., a veterinary appointment) based on the diagnosis result.

[0091] The present invention is a system for capturing images of pets and diagnosing their health condition, which helps users monitor their pet's health and provide appropriate care and medical services.

[0092] Hardware and software used

[0093] This system uses mobile devices such as smartphones and tablets, as well as a cloud server. Specifically, it uses the following hardware and software:

[0094] 1. Smartphone or tablet (device)

[0095] Camera: Used to take pictures of your pet.

[0096] Mobile application: Provides the user interface.

[0097] 2. Cloud Server

[0098] Data transmission protocol (e.g. HTTPS): Ensures secure transmission of image data.

[0099] Image processing library (e.g. OpenCV): To preprocess images.

[0100] Generative AI models (e.g., TensorFlow, PyTorch): Perform health checkups.

[0101] System Operation Overview

[0102] A user launches a mobile application and takes a picture of their pet. For example, the user takes a picture of their pet's swollen ear. The captured image data is compressed on the device and sent to the server using HTTPS protocol. The server decompresses the received image data and performs preprocessing such as noise removal and image standardization. The preprocessed image is input into a generative AI model for a health check. The generative AI model uses a pre-trained algorithm to detect abnormalities in the image. For example, the generative AI model detects the pet's swollen ear and determines that it is an early sign of an infection.

[0103] Based on the generated diagnostic results, the server generates recommended actions for the user. For example, it generates a message such as, "There is a high suspicion of infection, so we recommend that you see a veterinarian." This diagnostic result and recommended actions are sent to the device and displayed to the user. Furthermore, based on the diagnostic results, links to purchase appropriate care products and to book medical services are also provided. The user can click on these links to purchase the necessary care products or make an appointment with a nearby veterinarian.

[0104] Specific examples

[0105] For example, if a user notices that their pet's ear is swollen, they launch the application and take a photo of the area. The device sends the image to a server, which performs preprocessing such as noise removal. The preprocessed image is analyzed by a generative AI model and determines that the ear swelling may indicate an infection. The server generates a diagnosis of "Suspected infection: High risk" and sends a message to the user's device recommending immediate veterinary treatment. The user confirms this and can easily make an appointment with a nearby veterinarian and purchase infection preventative medication within the app.

[0106] Prompt Sentence Examples

[0107] "Analyze images of your pet's swollen ears, diagnose whether there is a risk of infection, and suggest appropriate care."

[0108] The present invention provides innovative support for pet health management, enabling users to take necessary measures quickly and efficiently.

[0109] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0110] Step 1:

[0111] The user launches the app on their smartphone or tablet. They then use an intuitive interface to take a picture of their pet and capture a specific area (e.g., a swollen ear). The input is the pet's image, and the output is the captured image data.

[0112] Step 2:

[0113] The device converts the captured image data into a compressed format (e.g., JPEG or PNG). This compression reduces the amount of data. The input is the image data obtained in step 1, and the output is the compressed image data.

[0114] Step 3:

[0115] The terminal sends the compressed image data to the server using the HTTPS protocol. This is done by the data transmission means, and the data is secure because it is encrypted. The input is the compressed image data, and the output is the data received by the server.

[0116] Step 4:

[0117] The server decompresses the image data received and returns it to its original image format. For example, it decompresses a compressed JPEG image. The input is the encrypted image data, and the output is the decompressed image data.

[0118] Step 5:

[0119] The server denoises the image data. It uses an image processing library such as OpenCV to remove unnecessary noise from the image. The input is the decompressed image data, and the output is the image data after noise removal.

[0120] Step 6:

[0121] The server standardizes the images to make them uniform in size and resolution, preparing them for accurate analysis by the generative AI model. The input is image data after noise removal, and the output is standardized image data.

[0122] Step 7:

[0123] The server extracts important parts from the image. For example, it cuts out only the ears of a pet and uses them for analysis. The input is the standardized image data, and the output is the image data with the important parts extracted.

[0124] Step 8:

[0125] The server inputs the preprocessed image data into a generative AI model to perform a health check. The generative AI model is built using a deep learning algorithm (e.g., TensorFlow or PyTorch). The input is image data with important parts extracted, and the output is the diagnosis result.

[0126] Step 9:

[0127] The server recommends actions to the user based on the diagnosis results from the generative AI model. This recommended action is expressed in the form of, for example, "There is a high suspicion of infection, so we recommend that you see a veterinarian." The input is the diagnosis result, and the output is the recommended action.

[0128] Step 10:

[0129] The server sends the diagnosis results and recommended actions to the user's device. The diagnosis results and recommended actions are displayed to the user in a visually easy-to-understand format. The input is the recommended actions and diagnosis results, and the output is the data displayed on the user's device.

[0130] Step 11:

[0131] The device displays the received diagnosis results and recommended actions to the user. For example, a message such as "Your pet's ear is swollen. This swelling is likely an early sign of an infection" is displayed. The input is the data sent from the server, and the output is the information the user visually confirms.

[0132] Step 12:

[0133] Based on the diagnosis results, the terminal displays links to purchase care products or to book medical services. The user can click on these links to purchase the necessary care products or make an appointment with a nearby veterinarian. For example, links such as "Link to purchase infectious disease preventative medicine" or "You can make an appointment with a nearby veterinarian here" are displayed. The input is the diagnosis results, and the output is the links displayed.

[0134] Step 13:

[0135] The user clicks on a displayed link to perform an action, such as going to a screen to purchase preventative medicine or to a screen to complete a vet appointment. The input is the user's click action, and the output is the completed purchase of a care product or appointment for medical services.

[0136] (Application example 1)

[0137] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0138] Monitoring pet health is important in modern households, but traditional methods make it difficult to conduct regular health checks, and problems are often only noticed after they occur. Furthermore, there are limited means for monitoring pet health in real time, making it difficult to respond quickly when an abnormality occurs. Furthermore, making veterinary appointments and purchasing care products can be a hassle.

[0139] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0140] In this invention, the server includes a user interface means for taking images of the pet, a data transmission means for transmitting the captured images to the server, a means for receiving and preprocessing the transmitted image data, a generative AI model means for performing a health check using the preprocessed image as input, a means for generating recommended actions based on the diagnosis results of the generative AI model, a means for displaying the diagnosis results and the recommended actions to the user, a means for monitoring the movements and expressions of the pet in real time using a smart device, a means for transmitting the monitoring data to the server in real time and analyzing it, and a means for notifying the user when an abnormality is detected. This enables the user to efficiently monitor and manage the health condition of their pet and quickly take appropriate measures.

[0141] The "user interface means" is an interface for the user to perform operations and has a function for taking pictures of the pet.

[0142] The "data transmission means" is a means that provides a function for transmitting captured images to a server.

[0143] The "preprocessing means" is a means for receiving the transmitted image data and performing preprocessing such as noise removal and image standardization.

[0144] "Generative AI model means" refers to a generative AI model used to conduct a health check based on preprocessed image data.

[0145] The "means for generating recommended actions" is a means for generating actions to be recommended to users based on the diagnostic results obtained by the generative AI model.

[0146] The "diagnosis result display means" is a means for visually displaying to the user the diagnosis results and recommended actions obtained by the generative AI model.

[0147] "Means for monitoring pet movements and facial expressions in real time using smart devices" refers to means for constantly monitoring pet movements and facial expressions using smart glasses or other devices and collecting data.

[0148] "Means for transmitting monitoring data to a server in real time for analysis" refers to a means for immediately transmitting collected monitoring data to a server and analyzing it using a generative AI model.

[0149] "Means for notifying the user when an abnormality is detected" refers to a means for sending an alert to the user when an abnormality is detected based on the analysis results of the generative AI model.

[0150] The present invention provides a system for monitoring the health of pets in real time using smart devices and cloud computing, and providing users with diagnostic results and recommended actions.

[0151] Overall system overview

[0152] The system includes the following elements:

[0153] 1. User interface means: The user takes a picture of their pet using a smart device (smartphone, smart glasses, etc.).

[0154] 2. Data transmission means: A means for transmitting captured images to a cloud server.

[0155] 3. Preprocessing: The cloud server preprocesses the received image data. Preprocessing includes noise removal and image standardization.

[0156] 4. Generative AI model means: A means for inputting preprocessed image data into a generative AI model to perform a health check.

[0157] 5. Means for generating recommended actions: A means for generating recommended actions for users based on the diagnostic results of the generative AI model.

[0158] 6. Diagnostic result display means: A means for visually displaying diagnostic results and recommended actions on a smart device.

[0159] 7. Real-time monitoring means: A means of monitoring pet movements and facial expressions in real time using smart devices such as smart glasses.

[0160] 8. Abnormality notification means: A means of sending monitoring data to a server in real time and notifying the user if an abnormality is detected based on the analysis results.

[0161] Hardware and software used

[0162] Smart devices: smartphones, smart glasses (e.g., Google Glass, Vuzix Blade)

[0163] Cloud Server: Cloud computing platforms such as AWS and Google Cloud

[0164] Generative AI model: A generative AI model used to perform a health check

[0165] Data processing flow

[0166] 1. User interface: The user takes a picture of their pet using a smart device.

[0167] 2. Data transmission: The captured images are sent to the cloud server.

[0168] 3. Preprocessing: The cloud server receives the image data and performs noise removal and image standardization.

[0169] 4. Application of generative AI model: The preprocessed image data is input into the generative AI model to perform a health check.

[0170] 5. Generation of recommended actions: Based on the diagnostic results of the generative AI model, actions to be recommended to the user are generated.

[0171] 6. Displaying diagnostic results: The generated diagnostic results and recommended actions are displayed on the smart device.

[0172] 7. Real-time monitoring: Use smart devices to constantly monitor and collect data on your pet's movements and facial expressions.

[0173] 8. Abnormality notification: Monitoring data is sent to the cloud server in real time, and if an abnormality is detected, a notification is sent to the user.

[0174] Specific examples

[0175] A user notices that their pet's ear is swollen and takes a photo of the area using their smart device. The image is sent to a cloud server, where it undergoes preprocessing such as noise removal and standardization. The preprocessed image is analyzed by a generative AI model, which determines that the ear swelling is an early sign of infection. The server generates a message recommending immediate veterinary care along with a diagnosis of "Suspected Infection: High Risk" and sends it to the user's smart device. The user can then review this information and make an appointment with a nearby veterinarian and purchase preventative medication directly from their smart device.

[0176] Example prompt sentence:

[0177] "It monitors pet health in real time, notifies users if anything abnormal is detected, and assists with appropriate care or veterinary appointments."

[0178] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0179] Step 1:

[0180] A user takes an image of their pet using a smart device (smartphone or smart glasses), which generates image data for monitoring specific health conditions of the pet. The input is the image of the pet, and the output is the image data stored on the smart device.

[0181] Step 2:

[0182] The device sends the captured image data to the cloud server. The data sending means compresses the image data through a secure protocol and sends it to the cloud server in an appropriate format. The input is the image data stored in the smart device, and the output is the compressed image data sent to the cloud server.

[0183] Step 3:

[0184] The server decompresses the received image data and performs preprocessing. Preprocessing includes noise removal, contrast adjustment, and image standardization. The preprocessed data is ready for the AI ​​model to perform accurate analysis. The input is the compressed image data sent to the server, and the output is the image data after preprocessing.

[0185] Step 4:

[0186] The preprocessed image data is input into a generative AI model to perform a health diagnosis. The generative AI model uses a pre-trained algorithm to detect abnormalities in the image. For example, the AI ​​model may detect swelling in a pet's ear and determine that it is an early sign of infection. The input is the preprocessed image data, and the output is the diagnosis result.

[0187] Step 5:

[0188] The server recommends actions to the user based on the diagnosis results of the AI ​​model. The server analyzes the diagnosis results and generates a message such as, "There is a high suspicion of infection, so we recommend that you see a veterinarian." The input is the diagnosis result from the generative AI model, and the output is a message containing a recommended action.

[0189] Step 6:

[0190] The device displays the analyzed diagnostic results and recommended actions to the user. The diagnostic results are displayed in a visually easy-to-understand format so that the user can immediately understand them. The input is a message containing recommended actions, and the output is the diagnostic results and recommended actions displayed to the user.

[0191] Step 7:

[0192] A smart device is used to monitor the movements and expressions of pets in real time. The camera in the smart glasses continuously captures the pet's movements and sends the data to a cloud server. The input is the video stream captured by the smart device, and the output is the monitoring data sent to the cloud server.

[0193] Step 8:

[0194] The cloud server analyzes the monitoring data and notifies the user if an abnormality is detected. The AI ​​model performs behavior analysis and sends an alert to the user if an abnormality is detected. For example, a notification such as "There is something abnormal with your pet's movements. It is dragging its right hind leg" is displayed. The input is the monitoring data sent to the cloud server, and the output is an alert message notifying the user of the abnormality.

[0195] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0196] The present invention is a system for taking pictures of pets and conducting health checkups, and is combined with an emotion engine that recognizes the user's emotions. This system is designed to enable users to accurately understand their pet's health condition and take necessary measures promptly.

[0197] Overall system overview

[0198] The system includes a user interface means, a data transmission means, a preprocessing means, a generation AI model means, a recommended action generation means, a diagnosis result display means, a care product purchase link, a medical service reservation link, and an emotion engine that recognizes the user's emotions.

[0199] Specific explanation of program processing

[0200] 1. User Interface

[0201] The user launches the app on their smartphone or tablet and takes a picture of their pet. The user interface provides a simple and intuitive operation, such as capturing a red, swollen area on the pet's ear.

[0202] 2. Emotion recognition

[0203] To recognize a user's emotions, the device uses a camera and microphone to analyze the user's facial expressions and tone of voice. An emotion engine analyzes this data to identify the user's current emotional state (e.g., relief, worry, sadness, etc.).

[0204] 3. Data Transmission

[0205] The device sends the captured image and the user's emotional data to a server, where the data is sent in compressed and encrypted form using a secure protocol.

[0206] 4. Pretreatment

[0207] The server decompresses the received pet image data and performs preprocessing such as noise reduction and standardization. At the same time, the user's emotion data is also preprocessed, preparing it for analysis.

[0208] 5. Applying generative AI models

[0209] The server inputs preprocessed images of the pet into the generative AI model for a health check, which then identifies any abnormalities in the pet and generates a diagnosis.

[0210] 6. Generating recommended actions

[0211] The server then suggests appropriate actions to the user based on the AI ​​model's diagnosis and the user's emotional data. For example, specific instructions such as "There is a high possibility of an infection, so we recommend that you see a veterinarian" are generated. If the emotional state is "worried," a particularly detailed explanation is included.

[0212] 7. Displaying the diagnostic results

[0213] The device displays the diagnosis results and recommended actions to the user, and the display is customized according to the user's emotional state. For example, if the user is in an "anxious" state, a warm and reassuring message is displayed.

[0214] 8. Proposing and implementing actions

[0215] Based on the diagnosis, the device will display links to purchase care products and book medical appointments, taking into account the user's emotional state. Users can easily purchase the necessary care products and make appointments with a nearby veterinarian within the app.

[0216] Specific examples

[0217] A user notices that their pet's ear is swollen, launches the app, and takes a photo of the area. The device simultaneously reads the user's facial expression, and the emotion engine recognizes that the user is "worried." The device then sends the pet's image and the user's emotion data to the server, which preprocesses the image and analyzes it using a generative AI model. The diagnosis is "high probability of infection," and the server generates a message for the "worried" user saying, "There is a suspicion of infection, but prompt treatment is likely to improve. We recommend that you see a veterinarian immediately." The device displays this message, along with links to make an appointment at a nearby veterinarian and to purchase infection preventative medication. The user can make an appointment within the app and purchase the necessary care products, allowing for a prompt response.

[0218] This allows users to quickly understand the health condition of their pet and take appropriate measures while receiving emotional support.

[0219] The processing flow will be explained below.

[0220] Step 1:

[0221] The user launches the app on their smartphone or tablet and takes a picture of their pet, for example, a picture of their pet's swollen ear.

[0222] Step 2:

[0223] The device displays an image of the pet, and the user confirms the image and taps the send button, which sends the image data to the server.

[0224] Step 3:

[0225] To recognize the user's emotions, the device uses a camera and microphone to capture the user's facial expressions and record the tone of voice. The emotion engine analyzes the user's facial and vocal data to identify the user's current emotional state (e.g., relief, worry, sadness, etc.).

[0226] Step 4:

[0227] The device compresses the captured image data and the user's emotional data and sends them to the server using a secure protocol (e.g., HTTPS).

[0228] Step 5:

[0229] The server decompresses the received image data and performs preprocessing such as noise removal, contrast adjustment, and resolution standardization. At the same time, the user's emotional data is also preprocessed, and the emotional data is ready for analysis.

[0230] Step 6:

[0231] The server inputs the preprocessed images into a generative AI model for health checkups, which then identifies abnormalities in the images and generates a diagnosis.

[0232] Step 7:

[0233] The server generates recommended actions based on the diagnosis results from the AI ​​model and the user's emotional data. For example, specific instructions such as "There is a high possibility of an infection, so we recommend that you see a veterinarian" are generated, and if the user is "worried," a particularly polite explanation or a reassuring message is added.

[0234] Step 8:

[0235] The server generates diagnostic results and sends them to the device, and the data is customized to allow users to understand it intuitively and quickly.

[0236] Step 9:

[0237] The device displays the received diagnosis results and recommended actions to the user. The display is customized according to the user's emotional state. For example, if the user is in an "anxious" state, a reassuring message is displayed. The message reads, "Your pet may have an ear infection, but with prompt treatment it is likely to improve. Please see a veterinarian immediately."

[0238] Step 10:

[0239] Based on the diagnosis results, the device will display links to purchase care products and book medical services. Users can easily purchase the necessary care products and book veterinary appointments within the app.

[0240] Step 11:

[0241] When a user clicks on the link to buy a care product, they are taken to an online store page where they can purchase the care product, and when they click on the link to book a medical service, they are taken to a nearby veterinarian's appointment page where they can enter the necessary information to complete the appointment.

[0242] Example 2

[0243] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0244] The ability to quickly understand a pet's health condition and take appropriate measures is extremely important for pet owners. However, typical pet health checks are time-consuming and laborious, and it is difficult to respond quickly in emergencies. Responding to the pet owner's emotional state is also important, and anxious owners require particularly careful support. The present invention aims to solve the above problems by providing a system that takes images of pets, performs health checks, and recognizes the owner's emotions to provide appropriate responses.

[0245] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes an information transmission means for transmitting the acquired image to the transmission device, a device for receiving and preprocessing the transmitted image data, a generative AI model device for performing a health check using the preprocessed image as input, a device for generating recommended actions based on the diagnosis results by the generative AI model, a display device for displaying the diagnosis results and recommended actions to the user, and an emotion recognition device for recognizing the user's emotions and collecting data. This makes it possible to quickly grasp the health condition of a pet and provide appropriate measures taking the owner's emotions into consideration.

[0246] The "user interface means" refers to a user interface for acquiring images of a pet, and is a means by which a user takes an image of a pet using a smartphone or tablet.

[0247] The "information transmission means" is a means for transmitting the acquired image to the transmission device, and a means for compressing and encrypting the captured image data and transmitting it to the server.

[0248] The "preprocessing device" refers to a device that receives transmitted image data and performs preprocessing such as noise removal and standardization.

[0249] A "generative AI model device" is a device that performs health checkups using preprocessed images as input, and is a means for analyzing a pet's health condition using a generative AI model and generating diagnostic results.

[0250] A "recommended behavior generation device" refers to a device that suggests appropriate behavior to pet owners based on the diagnostic results of a generative AI model.

[0251] The "display device" is a device for displaying diagnostic results and recommended actions to the user, and is capable of customizing the display according to the user's emotional state.

[0252] An "emotion recognition device" is a device that recognizes a user's emotions and collects data, and is a means of analyzing the user's facial expressions and tone of voice using a camera and microphone.

[0253] The "purchase link for care products" is a means for providing a link for purchasing pet care products based on the diagnosis results.

[0254] The "medical service appointment link" is a means for providing a link for making an appointment for medical service based on the diagnosis result.

[0255] "Training Dataset" refers to the collection of data used to train a generative AI model using data from healthy and unhealthy pets.

[0256] The present invention provides a system for quickly understanding the health condition of a pet and recognizing the emotions of the pet owner to provide appropriate responses. The system includes a user interface, an information transmission unit, a preprocessing unit, a generative AI model device, a recommended behavior generation device, a display device, and an emotion recognition device.

[0257] First, the user launches the app on their smartphone or tablet and captures an image of their pet. The smartphone or tablet's camera is used as the user interface. This interface is intuitive and designed to allow users to easily capture images of specific parts of their pet (e.g., ears or eyes).

[0258] The device then uses its built-in camera and microphone to analyze the user's facial expressions and tone of voice in real time. This emotion recognizer is designed to analyze and collect data from the user's facial expressions and tone of voice, and an emotional state (e.g., relief, worry, sadness) is identified.

[0259] The captured image and emotion data are sent from the device to a server using a secure protocol such as TLS (Transport Layer Security), and the data is compressed and encrypted before transmission.

[0260] The server preprocesses the pet image data it receives. The preprocessing unit removes noise and standardizes the image to make it suitable for analysis. For example, it adjusts the brightness and contrast of the image and removes unnecessary noise. At the same time, the user's emotional data is also preprocessed.

[0261] The preprocessed images are input into a generative AI model device. This generative AI model uses machine learning techniques such as CNN (convolutional neural network) to analyze the pet's health and identify abnormalities. For example, a diagnosis such as "possible infection" is generated. This generative AI model has been pre-trained using a training dataset of healthy and unhealthy pets.

[0262] The generated diagnosis results are converted into appropriate action suggestions for the user by a recommended behavior generator. For example, specific instructions such as "There is a high possibility of an infection, so we recommend that you take your pet to the veterinarian" are generated. If the emotional state is recognized as "worried," instructions including particularly careful explanations and reassurances are provided.

[0263] Finally, the device displays the diagnosis and recommended actions to the user. The display offers customized messages based on the user's emotional state, such as a reassuring message like, "Your pet may have an infection, but with prompt treatment, improvement is expected. We recommend immediate veterinary care."

[0264] Additionally, based on the diagnosis results, links to purchase care products and book medical services are displayed, allowing users to easily make reservations or purchases within the app by clicking on these links.

[0265] Specific examples

[0266] A user notices that their pet's ear is swollen, launches the app, and takes a photo of the area. The device simultaneously reads the user's facial expression, and the emotion engine recognizes that the user is "worried." The device then sends the pet's image and the user's emotion data to the server, which preprocesses the image and analyzes it using a generative AI model. The diagnosis is "high probability of infection," and the server generates a message for the "worried" user saying, "There is a suspicion of infection, but prompt treatment is likely to improve. We recommend that you see a veterinarian immediately." The device displays this message, along with links to make an appointment at a nearby veterinarian and to purchase infection preventative medication. The user can make an appointment within the app and purchase the necessary care products, allowing for a prompt response.

[0267] Prompt Sentence Examples

[0268] Describe how you would generate and display diagnostic results and recommended actions when a user is concerned about their pet's red, swollen ears.

[0269] keyword

[0270] Generative AI model, prompt sentence

[0271] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0272] Step 1:

[0273] The user launches the app on their smartphone or tablet and captures an image of their pet. When the user clicks the "photo button" in the app, the camera starts up and takes an image of a specific part of the pet (e.g., ears or eyes). The input is the pet's image data, and the output is the captured image data.

[0274] Step 2:

[0275] The device uses its built-in camera and microphone to analyze the user's facial expressions and tone of voice in real time. The camera captures the user's facial expressions and applies an expression analysis algorithm. The microphone also captures audio and uses an audio analysis algorithm to determine emotions. The input is the user's real-time facial expressions and audio data, and the output is analyzed user emotional data.

[0276] Step 3:

[0277] The device sends the captured image data and the user's emotional data to the server. TLS (Transport Layer Security) is used for data transmission, compressing and encrypting the data for secure transmission. The input is the pet's image data and the user's emotional data, and the output is the compressed and encrypted data sent to the server.

[0278] Step 4:

[0279] The server decompresses the received image data and performs preprocessing such as noise removal and standardization. The preprocessing unit adjusts the brightness and contrast of the image data and removes unnecessary noise. At the same time, the user's emotional data is also preprocessed. The input is the decompressed image data and emotional data, and the output is the preprocessed image data and emotional data.

[0280] Step 5:

[0281] The server inputs preprocessed images of the pet into a generative AI model to perform a health check. The generative AI model uses a convolutional neural network (CNN) to detect abnormalities in the pet. The input is the preprocessed image data, and the output is the generated diagnosis result.

[0282] Step 6:

[0283] The server analyzes the diagnostic results of the generative AI model and generates recommended actions based on the user's emotional data. The recommended action generator integrates the diagnostic results and emotional data to suggest appropriate actions for the user. For example, it generates instructions such as "There is a possibility of an infection, so we recommend that you see a veterinarian." The input is the diagnostic results and emotional data, and the output is the generated recommended action message.

[0284] Step 7:

[0285] The terminal displays the diagnosis results and recommended actions to the user. The display device adjusts the message content according to the user's emotional state, for example, displaying a reassuring message to an anxious user. The input is the recommended action message, and the output is a customized message displayed to the user.

[0286] Step 8:

[0287] The terminal presents links to purchase care products and to book medical services based on the diagnosis results and recommended actions. Users can easily make reservations or purchases by clicking these links. The input is the recommended action message, and the output is the presented links to purchase care products and to book medical services.

[0288] (Application example 2)

[0289] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0290] Conventional food delivery applications lack the ability to recognize users' emotions and make appropriate meal suggestions, making it difficult to provide services tailored to individual users. Furthermore, they lack the ability to suggest menus based on individual users' health conditions and preferences, which means they are unable to contribute to user satisfaction or health maintenance. Inappropriate meal choices run the risk of worsening users' health.

[0291] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes emotion recognition means for recognizing the emotions of the pet and the user, user interface means for taking images of the user and the pet, and data transmission means for sending the taken images and emotion data to the server. This makes it possible to propose individually customized food menus and care products and make reservations for medical services based on the user's emotion data and health condition.

[0292] "Emotion recognition means" refers to a device or function for recognizing a user's emotions, and includes technology that uses a camera or microphone to analyze a user's facial expression or tone of voice.

[0293] The "user interface means" is an interface that allows the user to intuitively operate the device, and provides functions that allow the user to take images and input data.

[0294] The "data transmission means" is a communication means for transmitting the captured image and emotion data to the server, and has the function of transmitting data in a secure and encrypted format.

[0295] "Preprocessing means" includes technology that analyzes the image data and emotion data received by the server and performs necessary noise removal and standardization.

[0296] "Generative AI model means" refers to an artificial intelligence model that makes a diagnosis based on preprocessed data, and includes trained models for health checkups and food menu suggestions.

[0297] "Recommended action generation means" is a technology that has the function of suggesting appropriate actions to users based on the diagnostic results of the generative AI model and the user's emotional data.

[0298] The "display means" refers to a device or function for visually presenting the diagnostic results and recommended actions to the user, and provides information in a form that is easy for the user to understand.

[0299] The "means for providing a reservation link" is a function that provides a reservation link for medical services based on the diagnosis results, and is a technology that allows the user to easily take the next action.

[0300] "Learning" is the process by which a generative AI model uses a training dataset to gain knowledge and make effective diagnoses and recommendations.

[0301] The present invention relates to a food delivery system that recognizes a user's emotions and provides food menu suggestions and medical service reservations based on those emotions. The system is designed to quickly suggest a menu that is optimal for the user's health condition.

[0302] Overall system overview

[0303] The system consists of the following main components:

[0304] 1. User Interface Methods

[0305] The user launches the smartphone app and takes a photo of themselves or their family. The user interface provides simple and intuitive operation.

[0306] 2. Emotion recognition means

[0307] It uses a camera and microphone to analyze the user's facial expressions and tone of voice to recognize the user's emotional state. This technology is realized using common emotion recognition APIs (e.g., Microsoft Azure Face API).

[0308] 3. Data Transmission Method

[0309] The captured image and emotion data are sent to a server in a secure, encrypted format using secure communications such as the TLS protocol.

[0310] 4. Pretreatment Methods

[0311] The server analyzes the received images and emotion data, removes noise, and standardizes the data. This process uses libraries such as OpenCV.

[0312] 5. Generative AI Model Means

[0313] A generative AI model for diagnosis and food menu suggestions using preprocessed image and emotion data as input. This generative AI model is implemented using an advanced natural language processing model such as GPT-4.

[0314] 6. Recommended Action Generation Method

[0315] Based on the diagnostic results of the generative AI model and the user's emotional data, the system suggests the most suitable food menu and medical service reservation links for the user.

[0316] 7. Display means

[0317] Diagnostic results and recommended actions are visually presented to the user in a reassuring and customized manner based on the user's emotional state.

[0318] What the program does

[0319] Hardware and Software Configuration

[0320] Hardware: Smartphone or tablet device (iOS or Android), camera, microphone.

[0321] Software: Azure Face API, OpenCV, GPT-4, TLS protocol.

[0322] Data processing and calculation flow

[0323] 1. Data capture and transmission: The user takes a photo through a smartphone app, and emotion analysis is performed in conjunction with a facial recognition API. The analysis data and image are encrypted and sent to a server.

[0324] 2. Data preprocessing: Image data is denoised and emotion data is standardised on the server side, allowing the AI ​​model to process the data accurately.

[0325] 3. Applying the generative AI model: The preprocessed data is fed into the GPT-4 model to generate an optimal food menu based on the user's emotions and health status.

[0326] 4. Result presentation and recommended actions: The generated menu and suggestions are displayed to the user along with messages customized according to their emotional state, and links to book medical services and purchase care products are also provided.

[0327] Examples and prompts

[0328] Specific examples

[0329] When a user is tired, they launch a food delivery app and take a photo. The system recognizes the user's emotion, such as "tired," and sends the image along with the emotion data to the server. The server then performs preprocessing and uses a generative AI model to suggest "nutritious menus to relieve fatigue." As a result, the user receives "menus effective for fatigue recovery" and a link to order them on the app.

[0330] Prompt Sentence Examples

[0331] "User's emotional state is happy. Health status: No allergies, needs energy. Please suggest the best menu for this user."

[0332] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0333] Step 1:

[0334] User Interface Operations

[0335] Subject: User

[0336] The user launches the smartphone app and takes a photo of themselves or their family.

[0337] Input: A photo taken by the user

[0338] Output: Image data

[0339] How it works: Using the app's camera function, guidelines will appear on the screen to provide easy and intuitive operation.

[0340] Step 2:

[0341] emotion recognition

[0342] Subject: Terminal

[0343] The device uses a camera and microphone to analyze the user's facial expressions and tone of voice to recognize their emotional state.

[0344] Input: Captured image and audio data

[0345] Output: Emotion data

[0346] Specific operation: Uses an emotion recognition API (e.g., Microsoft Azure Face API). The API analyzes facial and voice data to obtain emotion data (e.g., "happy" or "worried").

[0347] Step 3:

[0348] Data transmission

[0349] Subject: Terminal

[0350] The terminal encrypts the captured image and emotion data and transmits them to the server.

[0351] Input: Image data and emotion data

[0352] Output: Encrypted data

[0353] What it does: Encrypts and securely transmits data using the TLS protocol.

[0354] Step 4:

[0355] Data Preprocessing

[0356] Subject: Server

[0357] The server analyzes the received data, removes noise from the images, and standardizes the data.

[0358] Input: Encrypted data

[0359] Output: Preprocessed image data and standardized emotion data

[0360] Specific operation: Image data is denoised using OpenCV and emotion data is converted into an analyzable format.

[0361] Step 5:

[0362] Applying generative AI models

[0363] Subject: Server

[0364] The server inputs the preprocessed data into a generative AI model (e.g., GPT-4) to generate an optimal food menu based on the user's emotions and health status.

[0365] Input: Preprocessed image data and emotion data

[0366] Output: Generated food menu suggestions

[0367] Specific behavior: Provide a prompt to the generative AI model, and the AI ​​will generate an appropriate menu. Example: "User's emotional state is happy. Health status: no allergies, needs energy replenishment. Please suggest the best menu for this user."

[0368] Step 6:

[0369] Recommended Action Generation

[0370] Subject: Server

[0371] Based on the menu suggestions and diagnosis results generated by the server-generated AI model, the server generates the optimal food menu for the user, links to purchase care products, and links to book medical services.

[0372] Input: Generated food menu suggestions, emotion data

[0373] Output: Recommended actions and links

[0374] Specific behavior: Generate recommended actions and create customized messages depending on the emotional state.

[0375] Step 7:

[0376] Displaying the results

[0377] Subject: Terminal

[0378] The device visually presents the diagnostic results and recommended actions to the user.

[0379] Input: Recommended actions and links

[0380] Output: Visual presentation to the user

[0381] What it does: Displays a generated food menu and reservation link along with a customized message based on the user's emotional state.

[0382] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0383] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0384] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0385] [Second embodiment]

[0386] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0387] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0388] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0389] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0390] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0391] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0392] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0393] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0394] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0395] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0396] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0397] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0398] Overall system overview

[0399] The present invention provides a system for taking images of pets and diagnosing their health condition. The system helps users monitor their pet's health condition and seek necessary care and medical examinations. Specifically, the system includes a user interface, a data transmission means, a preprocessing means, a generative AI model means, a recommended action generation means, a diagnostic result display means, a link to purchase care products, and a link to book medical services.

[0400] Explanation of program processing

[0401] 1. User Interface

[0402] The user launches the app on their smartphone or tablet and takes a picture of their pet. The user interface is simple and easy to use, making this process intuitive and easy. For example, a user can take a picture of their pet's swollen ear.

[0403] 2. Data Transmission

[0404] The device sends the captured image data to the server. This data is sent via a secure protocol. The image data is compressed and sent in the appropriate format.

[0405] 3. Pretreatment

[0406] The server decompresses the received image data and performs preprocessing, which includes noise removal, image standardization, and extraction of necessary parts, to prepare the image for accurate analysis by the generative AI model.

[0407] 4. Applying generative AI models

[0408] The server then feeds the pre-processed images into a generative AI model for health assessment. The generative AI model uses pre-trained algorithms to detect abnormalities in the images. For example, the AI ​​model might detect swelling in a pet's ear and determine that this is an early sign of infection.

[0409] 5. Generating recommended actions

[0410] The server then recommends actions to the user based on the AI ​​model's diagnosis. These recommendations include care methods and veterinary advice. For example, a message such as "There is a high possibility of an infection, so we recommend that you take your pet to the veterinarian" is generated.

[0411] 6. Display of diagnostic results

[0412] The device displays the analyzed diagnostic results and recommended actions to the user. The diagnostic results are displayed in a visually easy-to-understand format so that the user can immediately understand them. For example, a message such as "Your pet's ear is swollen. This swelling is likely an early sign of an infection" may be displayed.

[0413] 7. Proposing and implementing actions

[0414] Based on the analysis results, the device displays links to purchase care products or make appointments for medical services. By clicking these links, users can purchase the necessary care products or make appointments with a nearby veterinarian. For example, users can select "Purchase preventative medicine" or "Make an appointment with a nearby veterinarian" to take immediate action.

[0415] Specific examples

[0416] A user notices that their pet's ear is swollen, launches the app, and takes a photo of the area. The device sends the image to a server, which performs preprocessing such as noise removal. The preprocessed image is analyzed by a generative AI model, which determines that the ear swelling may indicate an infection. The server generates a message recommending immediate veterinary care along with the diagnosis "Suspected infection: High risk," and sends this to the user's device. The device displays this, and the user can easily make an appointment with a nearby veterinarian and purchase infection preventative medication within the app.

[0417] In this way, the present invention provides innovative support for pet health management, enabling users to take appropriate measures quickly and efficiently.

[0418] The processing flow will be explained below.

[0419] Step 1:

[0420] The user launches the app on their smartphone or tablet and takes a picture of their pet. The app then checks for any abnormalities in a specific area of ​​the pet's body and takes a photo focusing on that area. For example, if the pet's ear is red and swollen, the app will capture the area around the ear.

[0421] Step 2:

[0422] The device displays the captured image, and the user confirms the image and taps the send button. This operation sends the image data to the server.

[0423] Step 3:

[0424] The device compresses the captured image data and sends it to the server using a secure protocol (e.g., HTTPS). During the data transmission process, the image is compressed and encrypted.

[0425] Step 4:

[0426] The server decompresses the received image data and performs preprocessing while preserving image quality. Preprocessing includes noise removal, contrast adjustment, and resolution standardization. This process improves the accuracy of analysis.

[0427] Step 5:

[0428] The server inputs the pre-processed images into a generative AI model, which uses pre-trained algorithms to detect anomalies, identifying abnormalities in the images (e.g., swelling or color changes) and providing a health diagnosis based on the results.

[0429] Step 6:

[0430] The server generates recommended actions based on the diagnosis results from the AI ​​model, such as "We recommend a veterinary visit as there is a high possibility of an infection" or "We recommend the use of specific care products."

[0431] Step 7:

[0432] The server sends the generated diagnostic results and recommended actions to the device, and the data is formatted in a way that allows the user to understand it intuitively and quickly.

[0433] Step 8:

[0434] The device displays the diagnosis results and recommended actions to the user, such as "Your pet's ears may have an ear infection. This is a high-risk condition, so please consult a veterinarian immediately."

[0435] Step 9:

[0436] Based on the diagnosis, the device will display links to purchase care products and book medical appointments. Users can click the links within the app to purchase the necessary care products online and book appointments with a nearby veterinarian.

[0437] Step 10:

[0438] When a user clicks on the link to buy a care product, they are taken to an online store page where they can purchase the care product, and when they click on the link to book a medical service, they are taken to a nearby veterinarian's appointment page where they can enter the necessary information to complete the appointment.

[0439] This series of processes allows the user to quickly understand the health condition of their pet and efficiently take necessary measures.

[0440] Example 1

[0441] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0442] In recent years, the importance of pet health management has increased, but it is not easy to continuously monitor pet health in busy daily lives. In addition, there is often a lack of means to detect abnormalities in pets early and provide appropriate care and examinations. With conventional methods, owners themselves need to have specialized knowledge to accurately diagnose their pet's health condition, and there is a risk that the condition may worsen before they can be examined by a specialist.

[0443] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0444] In this invention, the server includes a user interface means for taking images of the pet, a data transmission means for transmitting the captured images to the server, a means for receiving and preprocessing the transmitted image data, a generative AI model means for performing a health check using the preprocessed images as input, a means for generating recommended actions based on the diagnosis results from the generative AI model, a means for displaying the diagnosis results and the recommended actions to the user, and a means for providing links to purchase care products and to book medical services based on the diagnosis results, thereby enabling the user to quickly and accurately understand the health condition of their pet and to receive the necessary care and examination.

[0445] The "user interface means" is a means for providing an intuitive and easy-to-operate interface for the user to take pictures of their pet.

[0446] The "data transmission means" is a means for compressing and encrypting the captured image data and transmitting it to the server via a secure protocol.

[0447] The "means for performing preprocessing" refers to means for removing noise from received image data, standardizing the image data, extracting necessary parts, and the like.

[0448] A "generative AI model means" is a means including an algorithm trained using a training data set that performs a medical examination using preprocessed image data as input.

[0449] "Means for generating recommended actions" refers to means for recommending actions to be taken by the user based on the diagnostic results of the generative AI model.

[0450] The "means for displaying the diagnostic results to the user" refers to a means for displaying the analyzed diagnostic results and recommended actions to the user in a visually easy-to-understand format.

[0451] The "means for providing a link for purchasing a care product" is a means for providing a link that enables a user to purchase a necessary care product based on the diagnosis result.

[0452] The "means for providing a link to book a medical service" refers to a means for providing a link that allows a user to book a required medical service (e.g., a veterinary appointment) based on the diagnosis result.

[0453] The present invention is a system for capturing images of pets and diagnosing their health condition, which helps users monitor their pet's health and provide appropriate care and medical services.

[0454] Hardware and software used

[0455] This system uses mobile devices such as smartphones and tablets, as well as a cloud server. Specifically, it uses the following hardware and software:

[0456] 1. Smartphone or tablet (device)

[0457] Camera: Used to take pictures of your pet.

[0458] Mobile application: Provides the user interface.

[0459] 2. Cloud Server

[0460] Data transmission protocol (e.g. HTTPS): Ensures secure transmission of image data.

[0461] Image processing library (e.g. OpenCV): To preprocess images.

[0462] Generative AI models (e.g., TensorFlow, PyTorch): Perform health checkups.

[0463] System Operation Overview

[0464] A user launches a mobile application and takes a picture of their pet. For example, the user takes a picture of their pet's swollen ear. The captured image data is compressed on the device and sent to the server using HTTPS protocol. The server decompresses the received image data and performs preprocessing such as noise removal and image standardization. The preprocessed image is input into a generative AI model for a health check. The generative AI model uses a pre-trained algorithm to detect abnormalities in the image. For example, the generative AI model detects the pet's swollen ear and determines that it is an early sign of an infection.

[0465] Based on the generated diagnostic results, the server generates recommended actions for the user. For example, it generates a message such as, "There is a high suspicion of infection, so we recommend that you see a veterinarian." This diagnostic result and recommended actions are sent to the device and displayed to the user. Furthermore, based on the diagnostic results, links to purchase appropriate care products and to book medical services are also provided. The user can click on these links to purchase the necessary care products or make an appointment with a nearby veterinarian.

[0466] Specific examples

[0467] For example, if a user notices that their pet's ear is swollen, they launch the application and take a photo of the area. The device sends the image to a server, which performs preprocessing such as noise removal. The preprocessed image is analyzed by a generative AI model and determines that the ear swelling may indicate an infection. The server generates a diagnosis of "Suspected infection: High risk" and sends a message to the user's device recommending immediate veterinary treatment. The user confirms this and can easily make an appointment with a nearby veterinarian and purchase infection preventative medication within the app.

[0468] Prompt Sentence Examples

[0469] "Analyze images of your pet's swollen ears, diagnose whether there is a risk of infection, and suggest appropriate care."

[0470] The present invention provides innovative support for pet health management, enabling users to take necessary measures quickly and efficiently.

[0471] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0472] Step 1:

[0473] The user launches the app on their smartphone or tablet. They then use an intuitive interface to take a picture of their pet and capture a specific area (e.g., a swollen ear). The input is the pet's image, and the output is the captured image data.

[0474] Step 2:

[0475] The device converts the captured image data into a compressed format (e.g., JPEG or PNG). This compression reduces the amount of data. The input is the image data obtained in step 1, and the output is the compressed image data.

[0476] Step 3:

[0477] The terminal sends the compressed image data to the server using the HTTPS protocol. This is done by the data transmission means, and the data is secure because it is encrypted. The input is the compressed image data, and the output is the data received by the server.

[0478] Step 4:

[0479] The server decompresses the image data received and returns it to its original image format. For example, it decompresses a compressed JPEG image. The input is the encrypted image data, and the output is the decompressed image data.

[0480] Step 5:

[0481] The server denoises the image data. It uses an image processing library such as OpenCV to remove unnecessary noise from the image. The input is the decompressed image data, and the output is the image data after noise removal.

[0482] Step 6:

[0483] The server standardizes the images to make them uniform in size and resolution, preparing them for accurate analysis by the generative AI model. The input is image data after noise removal, and the output is standardized image data.

[0484] Step 7:

[0485] The server extracts important parts from the image. For example, it cuts out only the ears of a pet and uses them for analysis. The input is the standardized image data, and the output is the image data with the important parts extracted.

[0486] Step 8:

[0487] The server inputs the preprocessed image data into a generative AI model to perform a health check. The generative AI model is built using a deep learning algorithm (e.g., TensorFlow or PyTorch). The input is image data with important parts extracted, and the output is the diagnosis result.

[0488] Step 9:

[0489] The server recommends actions to the user based on the diagnosis results from the generative AI model. This recommended action is expressed in the form of, for example, "There is a high suspicion of infection, so we recommend that you see a veterinarian." The input is the diagnosis result, and the output is the recommended action.

[0490] Step 10:

[0491] The server sends the diagnosis results and recommended actions to the user's device. The diagnosis results and recommended actions are displayed to the user in a visually easy-to-understand format. The input is the recommended actions and diagnosis results, and the output is the data displayed on the user's device.

[0492] Step 11:

[0493] The device displays the received diagnosis results and recommended actions to the user. For example, a message such as "Your pet's ear is swollen. This swelling is likely an early sign of an infection" is displayed. The input is the data sent from the server, and the output is the information the user visually confirms.

[0494] Step 12:

[0495] Based on the diagnosis results, the terminal displays links to purchase care products or to book medical services. The user can click on these links to purchase the necessary care products or make an appointment with a nearby veterinarian. For example, links such as "Link to purchase infectious disease preventative medicine" or "You can make an appointment with a nearby veterinarian here" are displayed. The input is the diagnosis results, and the output is the links displayed.

[0496] Step 13:

[0497] The user clicks on a displayed link to perform an action, such as going to a screen to purchase preventative medicine or to a screen to complete a vet appointment. The input is the user's click action, and the output is the completed purchase of a care product or appointment for medical services.

[0498] (Application example 1)

[0499] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0500] Monitoring pet health is important in modern households, but traditional methods make it difficult to conduct regular health checks, and problems are often only noticed after they occur. Furthermore, there are limited means for monitoring pet health in real time, making it difficult to respond quickly when an abnormality occurs. Furthermore, making veterinary appointments and purchasing care products can be a hassle.

[0501] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0502] In this invention, the server includes a user interface means for taking images of the pet, a data transmission means for transmitting the captured images to the server, a means for receiving and preprocessing the transmitted image data, a generative AI model means for performing a health check using the preprocessed image as input, a means for generating recommended actions based on the diagnosis results of the generative AI model, a means for displaying the diagnosis results and the recommended actions to the user, a means for monitoring the movements and expressions of the pet in real time using a smart device, a means for transmitting the monitoring data to the server in real time and analyzing it, and a means for notifying the user when an abnormality is detected. This enables the user to efficiently monitor and manage the health condition of their pet and quickly take appropriate measures.

[0503] The "user interface means" is an interface for the user to perform operations and has a function for taking pictures of the pet.

[0504] The "data transmission means" is a means that provides a function for transmitting captured images to a server.

[0505] The "preprocessing means" is a means for receiving the transmitted image data and performing preprocessing such as noise removal and image standardization.

[0506] "Generative AI model means" refers to a generative AI model used to conduct a health check based on preprocessed image data.

[0507] The "means for generating recommended actions" is a means for generating actions to be recommended to users based on the diagnostic results obtained by the generative AI model.

[0508] The "diagnosis result display means" is a means for visually displaying to the user the diagnosis results and recommended actions obtained by the generative AI model.

[0509] "Means for monitoring pet movements and facial expressions in real time using smart devices" refers to means for constantly monitoring pet movements and facial expressions using smart glasses or other devices and collecting data.

[0510] "Means for transmitting monitoring data to a server in real time for analysis" refers to a means for immediately transmitting collected monitoring data to a server and analyzing it using a generative AI model.

[0511] "Means for notifying the user when an abnormality is detected" refers to a means for sending an alert to the user when an abnormality is detected based on the analysis results of the generative AI model.

[0512] The present invention provides a system for monitoring the health of pets in real time using smart devices and cloud computing, and providing users with diagnostic results and recommended actions.

[0513] Overall system overview

[0514] The system includes the following elements:

[0515] 1. User interface means: The user takes a picture of their pet using a smart device (smartphone, smart glasses, etc.).

[0516] 2. Data transmission means: A means for transmitting captured images to a cloud server.

[0517] 3. Preprocessing: The cloud server preprocesses the received image data. Preprocessing includes noise removal and image standardization.

[0518] 4. Generative AI model means: A means for inputting preprocessed image data into a generative AI model to perform a health check.

[0519] 5. Means for generating recommended actions: A means for generating recommended actions for users based on the diagnostic results of the generative AI model.

[0520] 6. Diagnostic result display means: A means for visually displaying diagnostic results and recommended actions on a smart device.

[0521] 7. Real-time monitoring means: A means of monitoring pet movements and facial expressions in real time using smart devices such as smart glasses.

[0522] 8. Abnormality notification means: A means of sending monitoring data to a server in real time and notifying the user if an abnormality is detected based on the analysis results.

[0523] Hardware and software used

[0524] Smart devices: smartphones, smart glasses (e.g., Google Glass, Vuzix Blade)

[0525] Cloud Server: Cloud computing platforms such as AWS and Google Cloud

[0526] Generative AI model: A generative AI model used to perform a health check

[0527] Data processing flow

[0528] 1. User interface: The user takes a picture of their pet using a smart device.

[0529] 2. Data transmission: The captured images are sent to the cloud server.

[0530] 3. Preprocessing: The cloud server receives the image data and performs noise removal and image standardization.

[0531] 4. Application of generative AI model: The preprocessed image data is input into the generative AI model to perform a health check.

[0532] 5. Generation of recommended actions: Based on the diagnostic results of the generative AI model, actions to be recommended to the user are generated.

[0533] 6. Displaying diagnostic results: The generated diagnostic results and recommended actions are displayed on the smart device.

[0534] 7. Real-time monitoring: Use smart devices to constantly monitor and collect data on your pet's movements and facial expressions.

[0535] 8. Abnormality notification: Monitoring data is sent to the cloud server in real time, and if an abnormality is detected, a notification is sent to the user.

[0536] Specific examples

[0537] A user notices that their pet's ear is swollen and takes a photo of the area using their smart device. The image is sent to a cloud server, where it undergoes preprocessing such as noise removal and standardization. The preprocessed image is analyzed by a generative AI model, which determines that the ear swelling is an early sign of infection. The server generates a message recommending immediate veterinary care along with a diagnosis of "Suspected Infection: High Risk" and sends it to the user's smart device. The user can then review this information and make an appointment with a nearby veterinarian and purchase preventative medication directly from their smart device.

[0538] Example prompt sentence:

[0539] "It monitors pet health in real time, notifies users if anything abnormal is detected, and assists with appropriate care or veterinary appointments."

[0540] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0541] Step 1:

[0542] A user takes an image of their pet using a smart device (smartphone or smart glasses), which generates image data for monitoring specific health conditions of the pet. The input is the image of the pet, and the output is the image data stored on the smart device.

[0543] Step 2:

[0544] The device sends the captured image data to the cloud server. The data sending means compresses the image data through a secure protocol and sends it to the cloud server in an appropriate format. The input is the image data stored in the smart device, and the output is the compressed image data sent to the cloud server.

[0545] Step 3:

[0546] The server decompresses the received image data and performs preprocessing. Preprocessing includes noise removal, contrast adjustment, and image standardization. The preprocessed data is ready for the AI ​​model to perform accurate analysis. The input is the compressed image data sent to the server, and the output is the image data after preprocessing.

[0547] Step 4:

[0548] The preprocessed image data is input into a generative AI model to perform a health diagnosis. The generative AI model uses a pre-trained algorithm to detect abnormalities in the image. For example, the AI ​​model may detect swelling in a pet's ear and determine that it is an early sign of infection. The input is the preprocessed image data, and the output is the diagnosis result.

[0549] Step 5:

[0550] The server recommends actions to the user based on the diagnosis results of the AI ​​model. The server analyzes the diagnosis results and generates a message such as, "There is a high suspicion of infection, so we recommend that you see a veterinarian." The input is the diagnosis result from the generative AI model, and the output is a message containing a recommended action.

[0551] Step 6:

[0552] The device displays the analyzed diagnostic results and recommended actions to the user. The diagnostic results are displayed in a visually easy-to-understand format so that the user can immediately understand them. The input is a message containing recommended actions, and the output is the diagnostic results and recommended actions displayed to the user.

[0553] Step 7:

[0554] A smart device is used to monitor the movements and expressions of pets in real time. The camera in the smart glasses continuously captures the pet's movements and sends the data to a cloud server. The input is the video stream captured by the smart device, and the output is the monitoring data sent to the cloud server.

[0555] Step 8:

[0556] The cloud server analyzes the monitoring data and notifies the user if an abnormality is detected. The AI ​​model performs behavior analysis and sends an alert to the user if an abnormality is detected. For example, a notification such as "There is something abnormal with your pet's movements. It is dragging its right hind leg" is displayed. The input is the monitoring data sent to the cloud server, and the output is an alert message notifying the user of the abnormality.

[0557] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0558] The present invention is a system for taking pictures of pets and conducting health checkups, and is combined with an emotion engine that recognizes the user's emotions. This system is designed to enable users to accurately understand their pet's health condition and take necessary measures promptly.

[0559] Overall system overview

[0560] The system includes a user interface means, a data transmission means, a preprocessing means, a generation AI model means, a recommended action generation means, a diagnosis result display means, a care product purchase link, a medical service reservation link, and an emotion engine that recognizes the user's emotions.

[0561] Specific explanation of program processing

[0562] 1. User Interface

[0563] The user launches the app on their smartphone or tablet and takes a picture of their pet. The user interface provides a simple and intuitive operation, such as capturing a red, swollen area on the pet's ear.

[0564] 2. Emotion recognition

[0565] To recognize a user's emotions, the device uses a camera and microphone to analyze the user's facial expressions and tone of voice. An emotion engine analyzes this data to identify the user's current emotional state (e.g., relief, worry, sadness, etc.).

[0566] 3. Data Transmission

[0567] The device sends the captured image and the user's emotional data to a server, where the data is sent in compressed and encrypted form using a secure protocol.

[0568] 4. Pretreatment

[0569] The server decompresses the received pet image data and performs preprocessing such as noise reduction and standardization. At the same time, the user's emotion data is also preprocessed, preparing it for analysis.

[0570] 5. Applying generative AI models

[0571] The server inputs preprocessed images of the pet into the generative AI model for a health check, which then identifies any abnormalities in the pet and generates a diagnosis.

[0572] 6. Generating recommended actions

[0573] The server then suggests appropriate actions to the user based on the AI ​​model's diagnosis and the user's emotional data. For example, specific instructions such as "There is a high possibility of an infection, so we recommend that you see a veterinarian" are generated. If the emotional state is "worried," a particularly detailed explanation is included.

[0574] 7. Displaying the diagnostic results

[0575] The device displays the diagnosis results and recommended actions to the user, and the display is customized according to the user's emotional state. For example, if the user is in an "anxious" state, a warm and reassuring message is displayed.

[0576] 8. Proposing and implementing actions

[0577] Based on the diagnosis, the device will display links to purchase care products and book medical appointments, taking into account the user's emotional state. Users can easily purchase the necessary care products and make appointments with a nearby veterinarian within the app.

[0578] Specific examples

[0579] A user notices that their pet's ear is swollen, launches the app, and takes a photo of the area. The device simultaneously reads the user's facial expression, and the emotion engine recognizes that the user is "worried." The device then sends the pet's image and the user's emotion data to the server, which preprocesses the image and analyzes it using a generative AI model. The diagnosis is "high probability of infection," and the server generates a message for the "worried" user saying, "There is a suspicion of infection, but prompt treatment is likely to improve. We recommend that you see a veterinarian immediately." The device displays this message, along with links to make an appointment at a nearby veterinarian and to purchase infection preventative medication. The user can make an appointment within the app and purchase the necessary care products, allowing for a prompt response.

[0580] This allows users to quickly understand the health condition of their pet and take appropriate measures while receiving emotional support.

[0581] The processing flow will be explained below.

[0582] Step 1:

[0583] The user launches the app on their smartphone or tablet and takes a picture of their pet, for example, a picture of their pet's swollen ear.

[0584] Step 2:

[0585] The device displays an image of the pet, and the user confirms the image and taps the send button, which sends the image data to the server.

[0586] Step 3:

[0587] To recognize the user's emotions, the device uses a camera and microphone to capture the user's facial expressions and record the tone of voice. The emotion engine analyzes the user's facial and vocal data to identify the user's current emotional state (e.g., relief, worry, sadness, etc.).

[0588] Step 4:

[0589] The device compresses the captured image data and the user's emotional data and sends them to the server using a secure protocol (e.g., HTTPS).

[0590] Step 5:

[0591] The server decompresses the received image data and performs preprocessing such as noise removal, contrast adjustment, and resolution standardization. At the same time, the user's emotional data is also preprocessed, and the emotional data is ready for analysis.

[0592] Step 6:

[0593] The server inputs the preprocessed images into a generative AI model for health checkups, which then identifies abnormalities in the images and generates a diagnosis.

[0594] Step 7:

[0595] The server generates recommended actions based on the diagnosis results from the AI ​​model and the user's emotional data. For example, specific instructions such as "There is a high possibility of an infection, so we recommend that you see a veterinarian" are generated, and if the user is "worried," a particularly polite explanation or a reassuring message is added.

[0596] Step 8:

[0597] The server generates diagnostic results and sends them to the device, and the data is customized to allow users to understand it intuitively and quickly.

[0598] Step 9:

[0599] The device displays the received diagnosis results and recommended actions to the user. The display is customized according to the user's emotional state. For example, if the user is in an "anxious" state, a reassuring message is displayed. The message reads, "Your pet may have an ear infection, but with prompt treatment it is likely to improve. Please see a veterinarian immediately."

[0600] Step 10:

[0601] Based on the diagnosis results, the device will display links to purchase care products and book medical services. Users can easily purchase the necessary care products and book veterinary appointments within the app.

[0602] Step 11:

[0603] When a user clicks on the link to buy a care product, they are taken to an online store page where they can purchase the care product, and when they click on the link to book a medical service, they are taken to a nearby veterinarian's appointment page where they can enter the necessary information to complete the appointment.

[0604] Example 2

[0605] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0606] The ability to quickly understand a pet's health condition and take appropriate measures is extremely important for pet owners. However, typical pet health checks are time-consuming and laborious, and it is difficult to respond quickly in emergencies. Responding to the pet owner's emotional state is also important, and anxious owners require particularly careful support. The present invention aims to solve the above problems by providing a system that takes images of pets, performs health checks, and recognizes the owner's emotions to provide appropriate responses.

[0607] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes an information transmission means for transmitting the acquired image to the transmission device, a device for receiving and preprocessing the transmitted image data, a generative AI model device for performing a health check using the preprocessed image as input, a device for generating recommended actions based on the diagnosis results by the generative AI model, a display device for displaying the diagnosis results and recommended actions to the user, and an emotion recognition device for recognizing the user's emotions and collecting data. This makes it possible to quickly grasp the health condition of a pet and provide appropriate measures taking the owner's emotions into consideration.

[0608] The "user interface means" refers to a user interface for acquiring images of a pet, and is a means by which a user takes an image of a pet using a smartphone or tablet.

[0609] The "information transmission means" is a means for transmitting the acquired image to the transmission device, and a means for compressing and encrypting the captured image data and transmitting it to the server.

[0610] The "preprocessing device" refers to a device that receives transmitted image data and performs preprocessing such as noise removal and standardization.

[0611] A "generative AI model device" is a device that performs health checkups using preprocessed images as input, and is a means for analyzing a pet's health condition using a generative AI model and generating diagnostic results.

[0612] A "recommended behavior generation device" refers to a device that suggests appropriate behavior to pet owners based on the diagnostic results of a generative AI model.

[0613] The "display device" is a device for displaying diagnostic results and recommended actions to the user, and is capable of customizing the display according to the user's emotional state.

[0614] An "emotion recognition device" is a device that recognizes a user's emotions and collects data, and is a means of analyzing the user's facial expressions and tone of voice using a camera and microphone.

[0615] The "purchase link for care products" is a means for providing a link for purchasing pet care products based on the diagnosis results.

[0616] The "medical service appointment link" is a means for providing a link for making an appointment for medical service based on the diagnosis result.

[0617] "Training Dataset" refers to the collection of data used to train a generative AI model using data from healthy and unhealthy pets.

[0618] The present invention provides a system for quickly understanding the health condition of a pet and recognizing the emotions of the pet owner to provide appropriate responses. The system includes a user interface, an information transmission unit, a preprocessing unit, a generative AI model device, a recommended behavior generation device, a display device, and an emotion recognition device.

[0619] First, the user launches the app on their smartphone or tablet and captures an image of their pet. The smartphone or tablet's camera is used as the user interface. This interface is intuitive and designed to allow users to easily capture images of specific parts of their pet (e.g., ears or eyes).

[0620] The device then uses its built-in camera and microphone to analyze the user's facial expressions and tone of voice in real time. This emotion recognizer is designed to analyze and collect data from the user's facial expressions and tone of voice, and an emotional state (e.g., relief, worry, sadness) is identified.

[0621] The captured image and emotion data are sent from the device to a server using a secure protocol such as TLS (Transport Layer Security), and the data is compressed and encrypted before transmission.

[0622] The server preprocesses the pet image data it receives. The preprocessing unit removes noise and standardizes the image to make it suitable for analysis. For example, it adjusts the brightness and contrast of the image and removes unnecessary noise. At the same time, the user's emotional data is also preprocessed.

[0623] The preprocessed images are input into a generative AI model device. This generative AI model uses machine learning techniques such as CNN (convolutional neural network) to analyze the pet's health and identify abnormalities. For example, a diagnosis such as "possible infection" is generated. This generative AI model has been pre-trained using a training dataset of healthy and unhealthy pets.

[0624] The generated diagnosis results are converted into appropriate action suggestions for the user by a recommended behavior generator. For example, specific instructions such as "There is a high possibility of an infection, so we recommend that you take your pet to the veterinarian" are generated. If the emotional state is recognized as "worried," instructions including particularly careful explanations and reassurances are provided.

[0625] Finally, the device displays the diagnosis and recommended actions to the user. The display offers customized messages based on the user's emotional state, such as a reassuring message like, "Your pet may have an infection, but with prompt treatment, improvement is expected. We recommend immediate veterinary care."

[0626] Additionally, based on the diagnosis results, links to purchase care products and book medical services are displayed, allowing users to easily make reservations or purchases within the app by clicking on these links.

[0627] Specific examples

[0628] A user notices that their pet's ear is swollen, launches the app, and takes a photo of the area. The device simultaneously reads the user's facial expression, and the emotion engine recognizes that the user is "worried." The device then sends the pet's image and the user's emotion data to the server, which preprocesses the image and analyzes it using a generative AI model. The diagnosis is "high probability of infection," and the server generates a message for the "worried" user saying, "There is a suspicion of infection, but prompt treatment is likely to improve. We recommend that you see a veterinarian immediately." The device displays this message, along with links to make an appointment at a nearby veterinarian and to purchase infection preventative medication. The user can make an appointment within the app and purchase the necessary care products, allowing for a prompt response.

[0629] Prompt Sentence Examples

[0630] Describe how you would generate and display diagnostic results and recommended actions when a user is concerned about their pet's red, swollen ears.

[0631] keyword

[0632] Generative AI model, prompt sentence

[0633] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0634] Step 1:

[0635] The user launches the app on their smartphone or tablet and captures an image of their pet. When the user clicks the "photo button" in the app, the camera starts up and takes an image of a specific part of the pet (e.g., ears or eyes). The input is the pet's image data, and the output is the captured image data.

[0636] Step 2:

[0637] The device uses its built-in camera and microphone to analyze the user's facial expressions and tone of voice in real time. The camera captures the user's facial expressions and applies an expression analysis algorithm. The microphone also captures audio and uses an audio analysis algorithm to determine emotions. The input is the user's real-time facial expressions and audio data, and the output is analyzed user emotional data.

[0638] Step 3:

[0639] The device sends the captured image data and the user's emotional data to the server. TLS (Transport Layer Security) is used for data transmission, compressing and encrypting the data for secure transmission. The input is the pet's image data and the user's emotional data, and the output is the compressed and encrypted data sent to the server.

[0640] Step 4:

[0641] The server decompresses the received image data and performs preprocessing such as noise removal and standardization. The preprocessing unit adjusts the brightness and contrast of the image data and removes unnecessary noise. At the same time, the user's emotional data is also preprocessed. The input is the decompressed image data and emotional data, and the output is the preprocessed image data and emotional data.

[0642] Step 5:

[0643] The server inputs preprocessed images of the pet into a generative AI model to perform a health check. The generative AI model uses a convolutional neural network (CNN) to detect abnormalities in the pet. The input is the preprocessed image data, and the output is the generated diagnosis result.

[0644] Step 6:

[0645] The server analyzes the diagnostic results of the generative AI model and generates recommended actions based on the user's emotional data. The recommended action generator integrates the diagnostic results and emotional data to suggest appropriate actions for the user. For example, it generates instructions such as "There is a possibility of an infection, so we recommend that you see a veterinarian." The input is the diagnostic results and emotional data, and the output is the generated recommended action message.

[0646] Step 7:

[0647] The terminal displays the diagnosis results and recommended actions to the user. The display device adjusts the message content according to the user's emotional state, for example, displaying a reassuring message to an anxious user. The input is the recommended action message, and the output is a customized message displayed to the user.

[0648] Step 8:

[0649] The terminal presents links to purchase care products and to book medical services based on the diagnosis results and recommended actions. Users can easily make reservations or purchases by clicking these links. The input is the recommended action message, and the output is the presented links to purchase care products and to book medical services.

[0650] (Application example 2)

[0651] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0652] Conventional food delivery applications lack the ability to recognize users' emotions and make appropriate meal suggestions, making it difficult to provide services tailored to individual users. Furthermore, they lack the ability to suggest menus based on individual users' health conditions and preferences, which means they are unable to contribute to user satisfaction or health maintenance. Inappropriate meal choices run the risk of worsening users' health.

[0653] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes emotion recognition means for recognizing the emotions of the pet and the user, user interface means for taking images of the user and the pet, and data transmission means for sending the taken images and emotion data to the server. This makes it possible to propose individually customized food menus and care products and make reservations for medical services based on the user's emotion data and health condition.

[0654] "Emotion recognition means" refers to a device or function for recognizing a user's emotions, and includes technology that uses a camera or microphone to analyze a user's facial expression or tone of voice.

[0655] The "user interface means" is an interface that allows the user to intuitively operate the device, and provides functions that allow the user to take images and input data.

[0656] The "data transmission means" is a communication means for transmitting the captured image and emotion data to the server, and has the function of transmitting data in a secure and encrypted format.

[0657] "Preprocessing means" includes technology that analyzes the image data and emotion data received by the server and performs necessary noise removal and standardization.

[0658] "Generative AI model means" refers to an artificial intelligence model that makes a diagnosis based on preprocessed data, and includes trained models for health checkups and food menu suggestions.

[0659] "Recommended action generation means" is a technology that has the function of suggesting appropriate actions to users based on the diagnostic results of the generative AI model and the user's emotional data.

[0660] The "display means" refers to a device or function for visually presenting the diagnostic results and recommended actions to the user, and provides information in a form that is easy for the user to understand.

[0661] The "means for providing a reservation link" is a function that provides a reservation link for medical services based on the diagnosis results, and is a technology that allows the user to easily take the next action.

[0662] "Learning" is the process by which a generative AI model uses a training dataset to gain knowledge and make effective diagnoses and recommendations.

[0663] The present invention relates to a food delivery system that recognizes a user's emotions and provides food menu suggestions and medical service reservations based on those emotions. The system is designed to quickly suggest a menu that is optimal for the user's health condition.

[0664] Overall system overview

[0665] The system consists of the following main components:

[0666] 1. User Interface Methods

[0667] The user launches the smartphone app and takes a photo of themselves or their family. The user interface provides simple and intuitive operation.

[0668] 2. Emotion recognition means

[0669] It uses a camera and microphone to analyze the user's facial expressions and tone of voice to recognize the user's emotional state. This technology is realized using common emotion recognition APIs (e.g., Microsoft Azure Face API).

[0670] 3. Data Transmission Method

[0671] The captured image and emotion data are sent to a server in a secure, encrypted format using secure communications such as the TLS protocol.

[0672] 4. Pretreatment Methods

[0673] The server analyzes the received images and emotion data, removes noise, and standardizes the data. This process uses libraries such as OpenCV.

[0674] 5. Generative AI Model Means

[0675] A generative AI model for diagnosis and food menu suggestions using preprocessed image and emotion data as input. This generative AI model is implemented using an advanced natural language processing model such as GPT-4.

[0676] 6. Recommended Action Generation Method

[0677] Based on the diagnostic results of the generative AI model and the user's emotional data, the system suggests the most suitable food menu and medical service reservation links for the user.

[0678] 7. Display means

[0679] Diagnostic results and recommended actions are visually presented to the user in a reassuring and customized manner based on the user's emotional state.

[0680] What the program does

[0681] Hardware and Software Configuration

[0682] Hardware: Smartphone or tablet device (iOS or Android), camera, microphone.

[0683] Software: Azure Face API, OpenCV, GPT-4, TLS protocol.

[0684] Data processing and calculation flow

[0685] 1. Data capture and transmission: The user takes a photo through a smartphone app, and emotion analysis is performed in conjunction with a facial recognition API. The analysis data and image are encrypted and sent to a server.

[0686] 2. Data preprocessing: Image data is denoised and emotion data is standardised on the server side, allowing the AI ​​model to process the data accurately.

[0687] 3. Applying the generative AI model: The preprocessed data is fed into the GPT-4 model to generate an optimal food menu based on the user's emotions and health status.

[0688] 4. Result presentation and recommended actions: The generated menu and suggestions are displayed to the user along with messages customized according to their emotional state, and links to book medical services and purchase care products are also provided.

[0689] Examples and prompts

[0690] Specific examples

[0691] When a user is tired, they launch a food delivery app and take a photo. The system recognizes the user's emotion, such as "tired," and sends the image along with the emotion data to the server. The server then performs preprocessing and uses a generative AI model to suggest "nutritious menus to relieve fatigue." As a result, the user receives "menus effective for fatigue recovery" and a link to order them on the app.

[0692] Prompt Sentence Examples

[0693] "User's emotional state is happy. Health status: No allergies, needs energy. Please suggest the best menu for this user."

[0694] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0695] Step 1:

[0696] User Interface Operations

[0697] Subject: User

[0698] The user launches the smartphone app and takes a photo of themselves or their family.

[0699] Input: A photo taken by the user

[0700] Output: Image data

[0701] How it works: Using the app's camera function, guidelines will appear on the screen to provide easy and intuitive operation.

[0702] Step 2:

[0703] emotion recognition

[0704] Subject: Terminal

[0705] The device uses a camera and microphone to analyze the user's facial expressions and tone of voice to recognize their emotional state.

[0706] Input: Captured image and audio data

[0707] Output: Emotion data

[0708] Specific operation: Uses an emotion recognition API (e.g., Microsoft Azure Face API). The API analyzes facial and voice data to obtain emotion data (e.g., "happy" or "worried").

[0709] Step 3:

[0710] Data transmission

[0711] Subject: Terminal

[0712] The terminal encrypts the captured image and emotion data and transmits them to the server.

[0713] Input: Image data and emotion data

[0714] Output: Encrypted data

[0715] What it does: Encrypts and securely transmits data using the TLS protocol.

[0716] Step 4:

[0717] Data Preprocessing

[0718] Subject: Server

[0719] The server analyzes the received data, removes noise from the images, and standardizes the data.

[0720] Input: Encrypted data

[0721] Output: Preprocessed image data and standardized emotion data

[0722] Specific operation: Image data is denoised using OpenCV and emotion data is converted into an analyzable format.

[0723] Step 5:

[0724] Applying generative AI models

[0725] Subject: Server

[0726] The server inputs the preprocessed data into a generative AI model (e.g., GPT-4) to generate an optimal food menu based on the user's emotions and health status.

[0727] Input: Preprocessed image data and emotion data

[0728] Output: Generated food menu suggestions

[0729] Specific behavior: Provide a prompt to the generative AI model, and the AI ​​will generate an appropriate menu. Example: "User's emotional state is happy. Health status: no allergies, needs energy replenishment. Please suggest the best menu for this user."

[0730] Step 6:

[0731] Recommended Action Generation

[0732] Subject: Server

[0733] Based on the menu suggestions and diagnosis results generated by the server-generated AI model, the server generates the optimal food menu for the user, links to purchase care products, and links to book medical services.

[0734] Input: Generated food menu suggestions, emotion data

[0735] Output: Recommended actions and links

[0736] Specific behavior: Generate recommended actions and create customized messages depending on the emotional state.

[0737] Step 7:

[0738] Displaying the results

[0739] Subject: Terminal

[0740] The device visually presents the diagnostic results and recommended actions to the user.

[0741] Input: Recommended actions and links

[0742] Output: Visual presentation to the user

[0743] What it does: Displays a generated food menu and reservation link along with a customized message based on the user's emotional state.

[0744] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0745] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0746] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0747] [Third embodiment]

[0748] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0749] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0750] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0751] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0752] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0753] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0754] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0755] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0756] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0757] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0758] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0759] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0760] Overall system overview

[0761] The present invention provides a system for taking images of pets and diagnosing their health condition. The system helps users monitor their pet's health condition and seek necessary care and medical examinations. Specifically, the system includes a user interface, a data transmission means, a preprocessing means, a generative AI model means, a recommended action generation means, a diagnostic result display means, a link to purchase care products, and a link to book medical services.

[0762] Explanation of program processing

[0763] 1. User Interface

[0764] The user launches the app on their smartphone or tablet and takes a picture of their pet. The user interface is simple and easy to use, making this process intuitive and easy. For example, a user can take a picture of their pet's swollen ear.

[0765] 2. Data Transmission

[0766] The device sends the captured image data to the server. This data is sent via a secure protocol. The image data is compressed and sent in the appropriate format.

[0767] 3. Pretreatment

[0768] The server decompresses the received image data and performs preprocessing, which includes noise removal, image standardization, and extraction of necessary parts, to prepare the image for accurate analysis by the generative AI model.

[0769] 4. Applying generative AI models

[0770] The server then feeds the pre-processed images into a generative AI model for health assessment. The generative AI model uses pre-trained algorithms to detect abnormalities in the images. For example, the AI ​​model might detect swelling in a pet's ear and determine that this is an early sign of infection.

[0771] 5. Generating recommended actions

[0772] The server then recommends actions to the user based on the AI ​​model's diagnosis. These recommendations include care methods and veterinary advice. For example, a message such as "There is a high possibility of an infection, so we recommend that you take your pet to the veterinarian" is generated.

[0773] 6. Display of diagnostic results

[0774] The device displays the analyzed diagnostic results and recommended actions to the user. The diagnostic results are displayed in a visually easy-to-understand format so that the user can immediately understand them. For example, a message such as "Your pet's ear is swollen. This swelling is likely an early sign of an infection" may be displayed.

[0775] 7. Proposing and implementing actions

[0776] Based on the analysis results, the device displays links to purchase care products or make appointments for medical services. By clicking these links, users can purchase the necessary care products or make appointments with a nearby veterinarian. For example, users can select "Purchase preventative medicine" or "Make an appointment with a nearby veterinarian" to take immediate action.

[0777] Specific examples

[0778] A user notices that their pet's ear is swollen, launches the app, and takes a photo of the area. The device sends the image to a server, which performs preprocessing such as noise removal. The preprocessed image is analyzed by a generative AI model, which determines that the ear swelling may indicate an infection. The server generates a message recommending immediate veterinary care along with the diagnosis "Suspected infection: High risk," and sends this to the user's device. The device displays this, and the user can easily make an appointment with a nearby veterinarian and purchase infection preventative medication within the app.

[0779] In this way, the present invention provides innovative support for pet health management, enabling users to take appropriate measures quickly and efficiently.

[0780] The processing flow will be explained below.

[0781] Step 1:

[0782] The user launches the app on their smartphone or tablet and takes a picture of their pet. The app then checks for any abnormalities in a specific area of ​​the pet's body and takes a photo focusing on that area. For example, if the pet's ear is red and swollen, the app will capture the area around the ear.

[0783] Step 2:

[0784] The device displays the captured image, and the user confirms the image and taps the send button. This operation sends the image data to the server.

[0785] Step 3:

[0786] The device compresses the captured image data and sends it to the server using a secure protocol (e.g., HTTPS). During the data transmission process, the image is compressed and encrypted.

[0787] Step 4:

[0788] The server decompresses the received image data and performs preprocessing while preserving image quality. Preprocessing includes noise removal, contrast adjustment, and resolution standardization. This process improves the accuracy of analysis.

[0789] Step 5:

[0790] The server inputs the pre-processed images into a generative AI model, which uses pre-trained algorithms to detect anomalies, identifying abnormalities in the images (e.g., swelling or color changes) and providing a health diagnosis based on the results.

[0791] Step 6:

[0792] The server generates recommended actions based on the diagnosis results from the AI ​​model, such as "We recommend a veterinary visit as there is a high possibility of an infection" or "We recommend the use of specific care products."

[0793] Step 7:

[0794] The server sends the generated diagnostic results and recommended actions to the device, and the data is formatted in a way that allows the user to understand it intuitively and quickly.

[0795] Step 8:

[0796] The device displays the diagnosis results and recommended actions to the user, such as "Your pet's ears may have an ear infection. This is a high-risk condition, so please consult a veterinarian immediately."

[0797] Step 9:

[0798] Based on the diagnosis, the device will display links to purchase care products and book medical appointments. Users can click the links within the app to purchase the necessary care products online and book appointments with a nearby veterinarian.

[0799] Step 10:

[0800] When a user clicks on the link to buy a care product, they are taken to an online store page where they can purchase the care product, and when they click on the link to book a medical service, they are taken to a nearby veterinarian's appointment page where they can enter the necessary information to complete the appointment.

[0801] This series of processes allows the user to quickly understand the health condition of their pet and efficiently take necessary measures.

[0802] Example 1

[0803] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0804] In recent years, the importance of pet health management has increased, but it is not easy to continuously monitor pet health in busy daily lives. In addition, there is often a lack of means to detect abnormalities in pets early and provide appropriate care and examinations. With conventional methods, owners themselves need to have specialized knowledge to accurately diagnose their pet's health condition, and there is a risk that the condition may worsen before they can be examined by a specialist.

[0805] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0806] In this invention, the server includes a user interface means for taking images of the pet, a data transmission means for transmitting the captured images to the server, a means for receiving and preprocessing the transmitted image data, a generative AI model means for performing a health check using the preprocessed images as input, a means for generating recommended actions based on the diagnosis results from the generative AI model, a means for displaying the diagnosis results and the recommended actions to the user, and a means for providing links to purchase care products and to book medical services based on the diagnosis results, thereby enabling the user to quickly and accurately understand the health condition of their pet and to receive the necessary care and examination.

[0807] The "user interface means" is a means for providing an intuitive and easy-to-operate interface for the user to take pictures of their pet.

[0808] The "data transmission means" is a means for compressing and encrypting the captured image data and transmitting it to the server via a secure protocol.

[0809] The "means for performing preprocessing" refers to means for removing noise from received image data, standardizing the image data, extracting necessary parts, and the like.

[0810] A "generative AI model means" is a means including an algorithm trained using a training data set that performs a medical examination using preprocessed image data as input.

[0811] "Means for generating recommended actions" refers to means for recommending actions to be taken by the user based on the diagnostic results of the generative AI model.

[0812] The "means for displaying the diagnostic results to the user" refers to a means for displaying the analyzed diagnostic results and recommended actions to the user in a visually easy-to-understand format.

[0813] The "means for providing a link for purchasing a care product" is a means for providing a link that enables a user to purchase a necessary care product based on the diagnosis result.

[0814] The "means for providing a link to book a medical service" refers to a means for providing a link that allows a user to book a required medical service (e.g., a veterinary appointment) based on the diagnosis result.

[0815] The present invention is a system for capturing images of pets and diagnosing their health condition, which helps users monitor their pet's health and provide appropriate care and medical services.

[0816] Hardware and software used

[0817] This system uses mobile devices such as smartphones and tablets, as well as a cloud server. Specifically, it uses the following hardware and software:

[0818] 1. Smartphone or tablet (device)

[0819] Camera: Used to take pictures of your pet.

[0820] Mobile application: Provides the user interface.

[0821] 2. Cloud Server

[0822] Data transmission protocol (e.g. HTTPS): Ensures secure transmission of image data.

[0823] Image processing library (e.g. OpenCV): To preprocess images.

[0824] Generative AI models (e.g., TensorFlow, PyTorch): Perform health checkups.

[0825] System Operation Overview

[0826] A user launches a mobile application and takes a picture of their pet. For example, the user takes a picture of their pet's swollen ear. The captured image data is compressed on the device and sent to the server using HTTPS protocol. The server decompresses the received image data and performs preprocessing such as noise removal and image standardization. The preprocessed image is input into a generative AI model for a health check. The generative AI model uses a pre-trained algorithm to detect abnormalities in the image. For example, the generative AI model detects the pet's swollen ear and determines that it is an early sign of an infection.

[0827] Based on the generated diagnostic results, the server generates recommended actions for the user. For example, it generates a message such as, "There is a high suspicion of infection, so we recommend that you see a veterinarian." This diagnostic result and recommended actions are sent to the device and displayed to the user. Furthermore, based on the diagnostic results, links to purchase appropriate care products and to book medical services are also provided. The user can click on these links to purchase the necessary care products or make an appointment with a nearby veterinarian.

[0828] Specific examples

[0829] For example, if a user notices that their pet's ear is swollen, they launch the application and take a photo of the area. The device sends the image to a server, which performs preprocessing such as noise removal. The preprocessed image is analyzed by a generative AI model and determines that the ear swelling may indicate an infection. The server generates a diagnosis of "Suspected infection: High risk" and sends a message to the user's device recommending immediate veterinary treatment. The user confirms this and can easily make an appointment with a nearby veterinarian and purchase infection preventative medication within the app.

[0830] Prompt Sentence Examples

[0831] "Analyze images of your pet's swollen ears, diagnose whether there is a risk of infection, and suggest appropriate care."

[0832] The present invention provides innovative support for pet health management, enabling users to take necessary measures quickly and efficiently.

[0833] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0834] Step 1:

[0835] The user launches the app on their smartphone or tablet. They then use an intuitive interface to take a picture of their pet and capture a specific area (e.g., a swollen ear). The input is the pet's image, and the output is the captured image data.

[0836] Step 2:

[0837] The device converts the captured image data into a compressed format (e.g., JPEG or PNG). This compression reduces the amount of data. The input is the image data obtained in step 1, and the output is the compressed image data.

[0838] Step 3:

[0839] The terminal sends the compressed image data to the server using the HTTPS protocol. This is done by the data transmission means, and the data is secure because it is encrypted. The input is the compressed image data, and the output is the data received by the server.

[0840] Step 4:

[0841] The server decompresses the image data received and returns it to its original image format. For example, it decompresses a compressed JPEG image. The input is the encrypted image data, and the output is the decompressed image data.

[0842] Step 5:

[0843] The server denoises the image data. It uses an image processing library such as OpenCV to remove unnecessary noise from the image. The input is the decompressed image data, and the output is the image data after noise removal.

[0844] Step 6:

[0845] The server standardizes the images to make them uniform in size and resolution, preparing them for accurate analysis by the generative AI model. The input is image data after noise removal, and the output is standardized image data.

[0846] Step 7:

[0847] The server extracts important parts from the image. For example, it cuts out only the ears of a pet and uses them for analysis. The input is the standardized image data, and the output is the image data with the important parts extracted.

[0848] Step 8:

[0849] The server inputs the preprocessed image data into a generative AI model to perform a health check. The generative AI model is built using a deep learning algorithm (e.g., TensorFlow or PyTorch). The input is image data with important parts extracted, and the output is the diagnosis result.

[0850] Step 9:

[0851] The server recommends actions to the user based on the diagnosis results from the generative AI model. This recommended action is expressed in the form of, for example, "There is a high suspicion of infection, so we recommend that you see a veterinarian." The input is the diagnosis result, and the output is the recommended action.

[0852] Step 10:

[0853] The server sends the diagnosis results and recommended actions to the user's device. The diagnosis results and recommended actions are displayed to the user in a visually easy-to-understand format. The input is the recommended actions and diagnosis results, and the output is the data displayed on the user's device.

[0854] Step 11:

[0855] The device displays the received diagnosis results and recommended actions to the user. For example, a message such as "Your pet's ear is swollen. This swelling is likely an early sign of an infection" is displayed. The input is the data sent from the server, and the output is the information the user visually confirms.

[0856] Step 12:

[0857] Based on the diagnosis results, the terminal displays links to purchase care products or to book medical services. The user can click on these links to purchase the necessary care products or make an appointment with a nearby veterinarian. For example, links such as "Link to purchase infectious disease preventative medicine" or "You can make an appointment with a nearby veterinarian here" are displayed. The input is the diagnosis results, and the output is the links displayed.

[0858] Step 13:

[0859] The user clicks on a displayed link to perform an action, such as going to a screen to purchase preventative medicine or to a screen to complete a vet appointment. The input is the user's click action, and the output is the completed purchase of a care product or appointment for medical services.

[0860] (Application example 1)

[0861] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0862] Monitoring pet health is important in modern households, but traditional methods make it difficult to conduct regular health checks, and problems are often only noticed after they occur. Furthermore, there are limited means for monitoring pet health in real time, making it difficult to respond quickly when an abnormality occurs. Furthermore, making veterinary appointments and purchasing care products can be a hassle.

[0863] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0864] In this invention, the server includes a user interface means for taking images of the pet, a data transmission means for transmitting the captured images to the server, a means for receiving and preprocessing the transmitted image data, a generative AI model means for performing a health check using the preprocessed image as input, a means for generating recommended actions based on the diagnosis results of the generative AI model, a means for displaying the diagnosis results and the recommended actions to the user, a means for monitoring the movements and expressions of the pet in real time using a smart device, a means for transmitting the monitoring data to the server in real time and analyzing it, and a means for notifying the user when an abnormality is detected. This enables the user to efficiently monitor and manage the health condition of their pet and quickly take appropriate measures.

[0865] The "user interface means" is an interface for the user to perform operations and has a function for taking pictures of the pet.

[0866] The "data transmission means" is a means that provides a function for transmitting captured images to a server.

[0867] The "preprocessing means" is a means for receiving the transmitted image data and performing preprocessing such as noise removal and image standardization.

[0868] "Generative AI model means" refers to a generative AI model used to conduct a health check based on preprocessed image data.

[0869] The "means for generating recommended actions" is a means for generating actions to be recommended to users based on the diagnostic results obtained by the generative AI model.

[0870] The "diagnosis result display means" is a means for visually displaying to the user the diagnosis results and recommended actions obtained by the generative AI model.

[0871] "Means for monitoring pet movements and facial expressions in real time using smart devices" refers to means for constantly monitoring pet movements and facial expressions using smart glasses or other devices and collecting data.

[0872] "Means for transmitting monitoring data to a server in real time for analysis" refers to a means for immediately transmitting collected monitoring data to a server and analyzing it using a generative AI model.

[0873] "Means for notifying the user when an abnormality is detected" refers to a means for sending an alert to the user when an abnormality is detected based on the analysis results of the generative AI model.

[0874] The present invention provides a system for monitoring the health of pets in real time using smart devices and cloud computing, and providing users with diagnostic results and recommended actions.

[0875] Overall system overview

[0876] The system includes the following elements:

[0877] 1. User interface means: The user takes a picture of their pet using a smart device (smartphone, smart glasses, etc.).

[0878] 2. Data transmission means: A means for transmitting captured images to a cloud server.

[0879] 3. Preprocessing: The cloud server preprocesses the received image data. Preprocessing includes noise removal and image standardization.

[0880] 4. Generative AI model means: A means for inputting preprocessed image data into a generative AI model to perform a health check.

[0881] 5. Means for generating recommended actions: A means for generating recommended actions for users based on the diagnostic results of the generative AI model.

[0882] 6. Diagnostic result display means: A means for visually displaying diagnostic results and recommended actions on a smart device.

[0883] 7. Real-time monitoring means: A means of monitoring pet movements and facial expressions in real time using smart devices such as smart glasses.

[0884] 8. Abnormality notification means: A means of sending monitoring data to a server in real time and notifying the user if an abnormality is detected based on the analysis results.

[0885] Hardware and software used

[0886] Smart devices: smartphones, smart glasses (e.g., Google Glass, Vuzix Blade)

[0887] Cloud Server: Cloud computing platforms such as AWS and Google Cloud

[0888] Generative AI model: A generative AI model used to perform a health check

[0889] Data processing flow

[0890] 1. User interface: The user takes a picture of their pet using a smart device.

[0891] 2. Data transmission: The captured images are sent to the cloud server.

[0892] 3. Preprocessing: The cloud server receives the image data and performs noise removal and image standardization.

[0893] 4. Application of generative AI model: The preprocessed image data is input into the generative AI model to perform a health check.

[0894] 5. Generation of recommended actions: Based on the diagnostic results of the generative AI model, actions to be recommended to the user are generated.

[0895] 6. Displaying diagnostic results: The generated diagnostic results and recommended actions are displayed on the smart device.

[0896] 7. Real-time monitoring: Use smart devices to constantly monitor and collect data on your pet's movements and facial expressions.

[0897] 8. Abnormality notification: Monitoring data is sent to the cloud server in real time, and if an abnormality is detected, a notification is sent to the user.

[0898] Specific examples

[0899] A user notices that their pet's ear is swollen and takes a photo of the area using their smart device. The image is sent to a cloud server, where it undergoes preprocessing such as noise removal and standardization. The preprocessed image is analyzed by a generative AI model, which determines that the ear swelling is an early sign of infection. The server generates a message recommending immediate veterinary care along with a diagnosis of "Suspected Infection: High Risk" and sends it to the user's smart device. The user can then review this information and make an appointment with a nearby veterinarian and purchase preventative medication directly from their smart device.

[0900] Example prompt sentence:

[0901] "It monitors pet health in real time, notifies users if anything abnormal is detected, and assists with appropriate care or veterinary appointments."

[0902] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0903] Step 1:

[0904] A user takes an image of their pet using a smart device (smartphone or smart glasses), which generates image data for monitoring specific health conditions of the pet. The input is the image of the pet, and the output is the image data stored on the smart device.

[0905] Step 2:

[0906] The device sends the captured image data to the cloud server. The data sending means compresses the image data through a secure protocol and sends it to the cloud server in an appropriate format. The input is the image data stored in the smart device, and the output is the compressed image data sent to the cloud server.

[0907] Step 3:

[0908] The server decompresses the received image data and performs preprocessing. Preprocessing includes noise removal, contrast adjustment, and image standardization. The preprocessed data is ready for the AI ​​model to perform accurate analysis. The input is the compressed image data sent to the server, and the output is the image data after preprocessing.

[0909] Step 4:

[0910] The preprocessed image data is input into a generative AI model to perform a health diagnosis. The generative AI model uses a pre-trained algorithm to detect abnormalities in the image. For example, the AI ​​model may detect swelling in a pet's ear and determine that it is an early sign of infection. The input is the preprocessed image data, and the output is the diagnosis result.

[0911] Step 5:

[0912] The server recommends actions to the user based on the diagnosis results of the AI ​​model. The server analyzes the diagnosis results and generates a message such as, "There is a high suspicion of infection, so we recommend that you see a veterinarian." The input is the diagnosis result from the generative AI model, and the output is a message containing a recommended action.

[0913] Step 6:

[0914] The device displays the analyzed diagnostic results and recommended actions to the user. The diagnostic results are displayed in a visually easy-to-understand format so that the user can immediately understand them. The input is a message containing recommended actions, and the output is the diagnostic results and recommended actions displayed to the user.

[0915] Step 7:

[0916] A smart device is used to monitor the movements and expressions of pets in real time. The camera in the smart glasses continuously captures the pet's movements and sends the data to a cloud server. The input is the video stream captured by the smart device, and the output is the monitoring data sent to the cloud server.

[0917] Step 8:

[0918] The cloud server analyzes the monitoring data and notifies the user if an abnormality is detected. The AI ​​model performs behavior analysis and sends an alert to the user if an abnormality is detected. For example, a notification such as "There is something abnormal with your pet's movements. It is dragging its right hind leg" is displayed. The input is the monitoring data sent to the cloud server, and the output is an alert message notifying the user of the abnormality.

[0919] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0920] The present invention is a system for taking pictures of pets and conducting health checkups, and is combined with an emotion engine that recognizes the user's emotions. This system is designed to enable users to accurately understand their pet's health condition and take necessary measures promptly.

[0921] Overall system overview

[0922] The system includes a user interface means, a data transmission means, a preprocessing means, a generation AI model means, a recommended action generation means, a diagnosis result display means, a care product purchase link, a medical service reservation link, and an emotion engine that recognizes the user's emotions.

[0923] Specific explanation of program processing

[0924] 1. User Interface

[0925] The user launches the app on their smartphone or tablet and takes a picture of their pet. The user interface provides a simple and intuitive operation, such as capturing a red, swollen area on the pet's ear.

[0926] 2. Emotion recognition

[0927] To recognize a user's emotions, the device uses a camera and microphone to analyze the user's facial expressions and tone of voice. An emotion engine analyzes this data to identify the user's current emotional state (e.g., relief, worry, sadness, etc.).

[0928] 3. Data Transmission

[0929] The device sends the captured image and the user's emotional data to a server, where the data is sent in compressed and encrypted form using a secure protocol.

[0930] 4. Pretreatment

[0931] The server decompresses the received pet image data and performs preprocessing such as noise reduction and standardization. At the same time, the user's emotion data is also preprocessed, preparing it for analysis.

[0932] 5. Applying generative AI models

[0933] The server inputs preprocessed images of the pet into the generative AI model for a health check, which then identifies any abnormalities in the pet and generates a diagnosis.

[0934] 6. Generating recommended actions

[0935] The server then suggests appropriate actions to the user based on the AI ​​model's diagnosis and the user's emotional data. For example, specific instructions such as "There is a high possibility of an infection, so we recommend that you see a veterinarian" are generated. If the emotional state is "worried," a particularly detailed explanation is included.

[0936] 7. Displaying the diagnostic results

[0937] The device displays the diagnosis results and recommended actions to the user, and the display is customized according to the user's emotional state. For example, if the user is in an "anxious" state, a warm and reassuring message is displayed.

[0938] 8. Proposing and implementing actions

[0939] Based on the diagnosis, the device will display links to purchase care products and book medical appointments, taking into account the user's emotional state. Users can easily purchase the necessary care products and make appointments with a nearby veterinarian within the app.

[0940] Specific examples

[0941] A user notices that their pet's ear is swollen, launches the app, and takes a photo of the area. The device simultaneously reads the user's facial expression, and the emotion engine recognizes that the user is "worried." The device then sends the pet's image and the user's emotion data to the server, which preprocesses the image and analyzes it using a generative AI model. The diagnosis is "high probability of infection," and the server generates a message for the "worried" user saying, "There is a suspicion of infection, but prompt treatment is likely to improve. We recommend that you see a veterinarian immediately." The device displays this message, along with links to make an appointment at a nearby veterinarian and to purchase infection preventative medication. The user can make an appointment within the app and purchase the necessary care products, allowing for a prompt response.

[0942] This allows users to quickly understand the health condition of their pet and take appropriate measures while receiving emotional support.

[0943] The processing flow will be explained below.

[0944] Step 1:

[0945] The user launches the app on their smartphone or tablet and takes a picture of their pet, for example, a picture of their pet's swollen ear.

[0946] Step 2:

[0947] The device displays an image of the pet, and the user confirms the image and taps the send button, which sends the image data to the server.

[0948] Step 3:

[0949] To recognize the user's emotions, the device uses a camera and microphone to capture the user's facial expressions and record the tone of voice. The emotion engine analyzes the user's facial and vocal data to identify the user's current emotional state (e.g., relief, worry, sadness, etc.).

[0950] Step 4:

[0951] The device compresses the captured image data and the user's emotional data and sends them to the server using a secure protocol (e.g., HTTPS).

[0952] Step 5:

[0953] The server decompresses the received image data and performs preprocessing such as noise removal, contrast adjustment, and resolution standardization. At the same time, the user's emotional data is also preprocessed, and the emotional data is ready for analysis.

[0954] Step 6:

[0955] The server inputs the preprocessed images into a generative AI model for health checkups, which then identifies abnormalities in the images and generates a diagnosis.

[0956] Step 7:

[0957] The server generates recommended actions based on the diagnosis results from the AI ​​model and the user's emotional data. For example, specific instructions such as "There is a high possibility of an infection, so we recommend that you see a veterinarian" are generated, and if the user is "worried," a particularly polite explanation or a reassuring message is added.

[0958] Step 8:

[0959] The server generates diagnostic results and sends them to the device, and the data is customized to allow users to understand it intuitively and quickly.

[0960] Step 9:

[0961] The device displays the received diagnosis results and recommended actions to the user. The display is customized according to the user's emotional state. For example, if the user is in an "anxious" state, a reassuring message is displayed. The message reads, "Your pet may have an ear infection, but with prompt treatment it is likely to improve. Please see a veterinarian immediately."

[0962] Step 10:

[0963] Based on the diagnosis results, the device will display links to purchase care products and book medical services. Users can easily purchase the necessary care products and book veterinary appointments within the app.

[0964] Step 11:

[0965] When a user clicks on the link to buy a care product, they are taken to an online store page where they can purchase the care product, and when they click on the link to book a medical service, they are taken to a nearby veterinarian's appointment page where they can enter the necessary information to complete the appointment.

[0966] Example 2

[0967] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0968] The ability to quickly understand a pet's health condition and take appropriate measures is extremely important for pet owners. However, typical pet health checks are time-consuming and laborious, and it is difficult to respond quickly in emergencies. Responding to the pet owner's emotional state is also important, and anxious owners require particularly careful support. The present invention aims to solve the above problems by providing a system that takes images of pets, performs health checks, and recognizes the owner's emotions to provide appropriate responses.

[0969] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes an information transmission means for transmitting the acquired image to the transmission device, a device for receiving and preprocessing the transmitted image data, a generative AI model device for performing a health check using the preprocessed image as input, a device for generating recommended actions based on the diagnosis results by the generative AI model, a display device for displaying the diagnosis results and recommended actions to the user, and an emotion recognition device for recognizing the user's emotions and collecting data. This makes it possible to quickly grasp the health condition of a pet and provide appropriate measures taking the owner's emotions into consideration.

[0970] The "user interface means" refers to a user interface for acquiring images of a pet, and is a means by which a user takes an image of a pet using a smartphone or tablet.

[0971] The "information transmission means" is a means for transmitting the acquired image to the transmission device, and a means for compressing and encrypting the captured image data and transmitting it to the server.

[0972] The "preprocessing device" refers to a device that receives transmitted image data and performs preprocessing such as noise removal and standardization.

[0973] A "generative AI model device" is a device that performs health checkups using preprocessed images as input, and is a means for analyzing a pet's health condition using a generative AI model and generating diagnostic results.

[0974] A "recommended behavior generation device" refers to a device that suggests appropriate behavior to pet owners based on the diagnostic results of a generative AI model.

[0975] The "display device" is a device for displaying diagnostic results and recommended actions to the user, and is capable of customizing the display according to the user's emotional state.

[0976] An "emotion recognition device" is a device that recognizes a user's emotions and collects data, and is a means of analyzing the user's facial expressions and tone of voice using a camera and microphone.

[0977] The "purchase link for care products" is a means for providing a link for purchasing pet care products based on the diagnosis results.

[0978] The "medical service appointment link" is a means for providing a link for making an appointment for medical service based on the diagnosis result.

[0979] "Training Dataset" refers to the collection of data used to train a generative AI model using data from healthy and unhealthy pets.

[0980] The present invention provides a system for quickly understanding the health condition of a pet and recognizing the emotions of the pet owner to provide appropriate responses. The system includes a user interface, an information transmission unit, a preprocessing unit, a generative AI model device, a recommended behavior generation device, a display device, and an emotion recognition device.

[0981] First, the user launches the app on their smartphone or tablet and captures an image of their pet. The smartphone or tablet's camera is used as the user interface. This interface is intuitive and designed to allow users to easily capture images of specific parts of their pet (e.g., ears or eyes).

[0982] The device then uses its built-in camera and microphone to analyze the user's facial expressions and tone of voice in real time. This emotion recognizer is designed to analyze and collect data from the user's facial expressions and tone of voice, and an emotional state (e.g., relief, worry, sadness) is identified.

[0983] The captured image and emotion data are sent from the device to a server using a secure protocol such as TLS (Transport Layer Security), and the data is compressed and encrypted before transmission.

[0984] The server preprocesses the pet image data it receives. The preprocessing unit removes noise and standardizes the image to make it suitable for analysis. For example, it adjusts the brightness and contrast of the image and removes unnecessary noise. At the same time, the user's emotional data is also preprocessed.

[0985] The preprocessed images are input into a generative AI model device. This generative AI model uses machine learning techniques such as CNN (convolutional neural network) to analyze the pet's health and identify abnormalities. For example, a diagnosis such as "possible infection" is generated. This generative AI model has been pre-trained using a training dataset of healthy and unhealthy pets.

[0986] The generated diagnosis results are converted into appropriate action suggestions for the user by a recommended behavior generator. For example, specific instructions such as "There is a high possibility of an infection, so we recommend that you take your pet to the veterinarian" are generated. If the emotional state is recognized as "worried," instructions including particularly careful explanations and reassurances are provided.

[0987] Finally, the device displays the diagnosis and recommended actions to the user. The display offers customized messages based on the user's emotional state, such as a reassuring message like, "Your pet may have an infection, but with prompt treatment, improvement is expected. We recommend immediate veterinary care."

[0988] Additionally, based on the diagnosis results, links to purchase care products and book medical services are displayed, allowing users to easily make reservations or purchases within the app by clicking on these links.

[0989] Specific examples

[0990] A user notices that their pet's ear is swollen, launches the app, and takes a photo of the area. The device simultaneously reads the user's facial expression, and the emotion engine recognizes that the user is "worried." The device then sends the pet's image and the user's emotion data to the server, which preprocesses the image and analyzes it using a generative AI model. The diagnosis is "high probability of infection," and the server generates a message for the "worried" user saying, "There is a suspicion of infection, but prompt treatment is likely to improve. We recommend that you see a veterinarian immediately." The device displays this message, along with links to make an appointment at a nearby veterinarian and to purchase infection preventative medication. The user can make an appointment within the app and purchase the necessary care products, allowing for a prompt response.

[0991] Prompt Sentence Examples

[0992] Describe how you would generate and display diagnostic results and recommended actions when a user is concerned about their pet's red, swollen ears.

[0993] keyword

[0994] Generative AI model, prompt sentence

[0995] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0996] Step 1:

[0997] The user launches the app on their smartphone or tablet and captures an image of their pet. When the user clicks the "photo button" in the app, the camera starts up and takes an image of a specific part of the pet (e.g., ears or eyes). The input is the pet's image data, and the output is the captured image data.

[0998] Step 2:

[0999] The device uses its built-in camera and microphone to analyze the user's facial expressions and tone of voice in real time. The camera captures the user's facial expressions and applies an expression analysis algorithm. The microphone also captures audio and uses an audio analysis algorithm to determine emotions. The input is the user's real-time facial expressions and audio data, and the output is analyzed user emotional data.

[1000] Step 3:

[1001] The device sends the captured image data and the user's emotional data to the server. TLS (Transport Layer Security) is used for data transmission, compressing and encrypting the data for secure transmission. The input is the pet's image data and the user's emotional data, and the output is the compressed and encrypted data sent to the server.

[1002] Step 4:

[1003] The server decompresses the received image data and performs preprocessing such as noise removal and standardization. The preprocessing unit adjusts the brightness and contrast of the image data and removes unnecessary noise. At the same time, the user's emotional data is also preprocessed. The input is the decompressed image data and emotional data, and the output is the preprocessed image data and emotional data.

[1004] Step 5:

[1005] The server inputs preprocessed images of the pet into a generative AI model to perform a health check. The generative AI model uses a convolutional neural network (CNN) to detect abnormalities in the pet. The input is the preprocessed image data, and the output is the generated diagnosis result.

[1006] Step 6:

[1007] The server analyzes the diagnostic results of the generative AI model and generates recommended actions based on the user's emotional data. The recommended action generator integrates the diagnostic results and emotional data to suggest appropriate actions for the user. For example, it generates instructions such as "There is a possibility of an infection, so we recommend that you see a veterinarian." The input is the diagnostic results and emotional data, and the output is the generated recommended action message.

[1008] Step 7:

[1009] The terminal displays the diagnosis results and recommended actions to the user. The display device adjusts the message content according to the user's emotional state, for example, displaying a reassuring message to an anxious user. The input is the recommended action message, and the output is a customized message displayed to the user.

[1010] Step 8:

[1011] The terminal presents links to purchase care products and to book medical services based on the diagnosis results and recommended actions. Users can easily make reservations or purchases by clicking these links. The input is the recommended action message, and the output is the presented links to purchase care products and to book medical services.

[1012] (Application example 2)

[1013] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1014] Conventional food delivery applications lack the ability to recognize users' emotions and make appropriate meal suggestions, making it difficult to provide services tailored to individual users. Furthermore, they lack the ability to suggest menus based on individual users' health conditions and preferences, which means they are unable to contribute to user satisfaction or health maintenance. Inappropriate meal choices run the risk of worsening users' health.

[1015] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes emotion recognition means for recognizing the emotions of the pet and the user, user interface means for taking images of the user and the pet, and data transmission means for sending the taken images and emotion data to the server. This makes it possible to propose individually customized food menus and care products and make reservations for medical services based on the user's emotion data and health condition.

[1016] "Emotion recognition means" refers to a device or function for recognizing a user's emotions, and includes technology that uses a camera or microphone to analyze a user's facial expression or tone of voice.

[1017] The "user interface means" is an interface that allows the user to intuitively operate the device, and provides functions that allow the user to take images and input data.

[1018] The "data transmission means" is a communication means for transmitting the captured image and emotion data to the server, and has the function of transmitting data in a secure and encrypted format.

[1019] "Preprocessing means" includes technology that analyzes the image data and emotion data received by the server and performs necessary noise removal and standardization.

[1020] "Generative AI model means" refers to an artificial intelligence model that makes a diagnosis based on preprocessed data, and includes trained models for health checkups and food menu suggestions.

[1021] "Recommended action generation means" is a technology that has the function of suggesting appropriate actions to users based on the diagnostic results of the generative AI model and the user's emotional data.

[1022] The "display means" refers to a device or function for visually presenting the diagnostic results and recommended actions to the user, and provides information in a form that is easy for the user to understand.

[1023] The "means for providing a reservation link" is a function that provides a reservation link for medical services based on the diagnosis results, and is a technology that allows the user to easily take the next action.

[1024] "Learning" is the process by which a generative AI model uses a training dataset to gain knowledge and make effective diagnoses and recommendations.

[1025] The present invention relates to a food delivery system that recognizes a user's emotions and provides food menu suggestions and medical service reservations based on those emotions. The system is designed to quickly suggest a menu that is optimal for the user's health condition.

[1026] Overall system overview

[1027] The system consists of the following main components:

[1028] 1. User Interface Methods

[1029] The user launches the smartphone app and takes a photo of themselves or their family. The user interface provides simple and intuitive operation.

[1030] 2. Emotion recognition means

[1031] It uses a camera and microphone to analyze the user's facial expressions and tone of voice to recognize the user's emotional state. This technology is realized using common emotion recognition APIs (e.g., Microsoft Azure Face API).

[1032] 3. Data Transmission Method

[1033] The captured image and emotion data are sent to a server in a secure, encrypted format using secure communications such as the TLS protocol.

[1034] 4. Pretreatment Methods

[1035] The server analyzes the received images and emotion data, removes noise, and standardizes the data. This process uses libraries such as OpenCV.

[1036] 5. Generative AI Model Means

[1037] A generative AI model for diagnosis and food menu suggestions using preprocessed image and emotion data as input. This generative AI model is implemented using an advanced natural language processing model such as GPT-4.

[1038] 6. Recommended Action Generation Method

[1039] Based on the diagnostic results of the generative AI model and the user's emotional data, the system suggests the most suitable food menu and medical service reservation links for the user.

[1040] 7. Display means

[1041] Diagnostic results and recommended actions are visually presented to the user in a reassuring and customized manner based on the user's emotional state.

[1042] What the program does

[1043] Hardware and Software Configuration

[1044] Hardware: Smartphone or tablet device (iOS or Android), camera, microphone.

[1045] Software: Azure Face API, OpenCV, GPT-4, TLS protocol.

[1046] Data processing and calculation flow

[1047] 1. Data capture and transmission: The user takes a photo through a smartphone app, and emotion analysis is performed in conjunction with a facial recognition API. The analysis data and image are encrypted and sent to a server.

[1048] 2. Data preprocessing: Image data is denoised and emotion data is standardised on the server side, allowing the AI ​​model to process the data accurately.

[1049] 3. Applying the generative AI model: The preprocessed data is fed into the GPT-4 model to generate an optimal food menu based on the user's emotions and health status.

[1050] 4. Result presentation and recommended actions: The generated menu and suggestions are displayed to the user along with messages customized according to their emotional state, and links to book medical services and purchase care products are also provided.

[1051] Examples and prompts

[1052] Specific examples

[1053] When a user is tired, they launch a food delivery app and take a photo. The system recognizes the user's emotion, such as "tired," and sends the image along with the emotion data to the server. The server then performs preprocessing and uses a generative AI model to suggest "nutritious menus to relieve fatigue." As a result, the user receives "menus effective for fatigue recovery" and a link to order them on the app.

[1054] Prompt Sentence Examples

[1055] "User's emotional state is happy. Health status: No allergies, needs energy. Please suggest the best menu for this user."

[1056] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1057] Step 1:

[1058] User Interface Operations

[1059] Subject: User

[1060] The user launches the smartphone app and takes a photo of themselves or their family.

[1061] Input: A photo taken by the user

[1062] Output: Image data

[1063] How it works: Using the app's camera function, guidelines will appear on the screen to provide easy and intuitive operation.

[1064] Step 2:

[1065] emotion recognition

[1066] Subject: Terminal

[1067] The device uses a camera and microphone to analyze the user's facial expressions and tone of voice to recognize their emotional state.

[1068] Input: Captured image and audio data

[1069] Output: Emotion data

[1070] Specific operation: Uses an emotion recognition API (e.g., Microsoft Azure Face API). The API analyzes facial and voice data to obtain emotion data (e.g., "happy" or "worried").

[1071] Step 3:

[1072] Data transmission

[1073] Subject: Terminal

[1074] The terminal encrypts the captured image and emotion data and transmits them to the server.

[1075] Input: Image data and emotion data

[1076] Output: Encrypted data

[1077] What it does: Encrypts and securely transmits data using the TLS protocol.

[1078] Step 4:

[1079] Data Preprocessing

[1080] Subject: Server

[1081] The server analyzes the received data, removes noise from the images, and standardizes the data.

[1082] Input: Encrypted data

[1083] Output: Preprocessed image data and standardized emotion data

[1084] Specific operation: Image data is denoised using OpenCV and emotion data is converted into an analyzable format.

[1085] Step 5:

[1086] Applying generative AI models

[1087] Subject: Server

[1088] The server inputs the preprocessed data into a generative AI model (e.g., GPT-4) to generate an optimal food menu based on the user's emotions and health status.

[1089] Input: Preprocessed image data and emotion data

[1090] Output: Generated food menu suggestions

[1091] Specific behavior: Provide a prompt to the generative AI model, and the AI ​​will generate an appropriate menu. Example: "User's emotional state is happy. Health status: no allergies, needs energy replenishment. Please suggest the best menu for this user."

[1092] Step 6:

[1093] Recommended Action Generation

[1094] Subject: Server

[1095] Based on the menu suggestions and diagnosis results generated by the server-generated AI model, the server generates the optimal food menu for the user, links to purchase care products, and links to book medical services.

[1096] Input: Generated food menu suggestions, emotion data

[1097] Output: Recommended actions and links

[1098] Specific behavior: Generate recommended actions and create customized messages depending on the emotional state.

[1099] Step 7:

[1100] Displaying the results

[1101] Subject: Terminal

[1102] The device visually presents the diagnostic results and recommended actions to the user.

[1103] Input: Recommended actions and links

[1104] Output: Visual presentation to the user

[1105] What it does: Displays a generated food menu and reservation link along with a customized message based on the user's emotional state.

[1106] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1107] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1108] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1109] [Fourth embodiment]

[1110] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1111] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1112] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1113] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1114] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1115] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1116] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1117] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1118] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1119] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1120] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1121] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1122] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1123] Overall system overview

[1124] The present invention provides a system for taking images of pets and diagnosing their health condition. The system helps users monitor their pet's health condition and seek necessary care and medical examinations. Specifically, the system includes a user interface, a data transmission means, a preprocessing means, a generative AI model means, a recommended action generation means, a diagnostic result display means, a link to purchase care products, and a link to book medical services.

[1125] Explanation of program processing

[1126] 1. User Interface

[1127] The user launches the app on their smartphone or tablet and takes a picture of their pet. The user interface is simple and easy to use, making this process intuitive and easy. For example, a user can take a picture of their pet's swollen ear.

[1128] 2. Data Transmission

[1129] The device sends the captured image data to the server. This data is sent via a secure protocol. The image data is compressed and sent in the appropriate format.

[1130] 3. Pretreatment

[1131] The server decompresses the received image data and performs preprocessing, which includes noise removal, image standardization, and extraction of necessary parts, to prepare the image for accurate analysis by the generative AI model.

[1132] 4. Applying generative AI models

[1133] The server then feeds the pre-processed images into a generative AI model for health assessment. The generative AI model uses pre-trained algorithms to detect abnormalities in the images. For example, the AI ​​model might detect swelling in a pet's ear and determine that this is an early sign of infection.

[1134] 5. Generating recommended actions

[1135] The server then recommends actions to the user based on the AI ​​model's diagnosis. These recommendations include care methods and veterinary advice. For example, a message such as "There is a high possibility of an infection, so we recommend that you take your pet to the veterinarian" is generated.

[1136] 6. Display of diagnostic results

[1137] The device displays the analyzed diagnostic results and recommended actions to the user. The diagnostic results are displayed in a visually easy-to-understand format so that the user can immediately understand them. For example, a message such as "Your pet's ear is swollen. This swelling is likely an early sign of an infection" may be displayed.

[1138] 7. Proposing and implementing actions

[1139] Based on the analysis results, the device displays links to purchase care products or make appointments for medical services. By clicking these links, users can purchase the necessary care products or make appointments with a nearby veterinarian. For example, users can select "Purchase preventative medicine" or "Make an appointment with a nearby veterinarian" to take immediate action.

[1140] Specific examples

[1141] A user notices that their pet's ear is swollen, launches the app, and takes a photo of the area. The device sends the image to a server, which performs preprocessing such as noise removal. The preprocessed image is analyzed by a generative AI model, which determines that the ear swelling may indicate an infection. The server generates a message recommending immediate veterinary care along with the diagnosis "Suspected infection: High risk," and sends this to the user's device. The device displays this, and the user can easily make an appointment with a nearby veterinarian and purchase infection preventative medication within the app.

[1142] In this way, the present invention provides innovative support for pet health management, enabling users to take appropriate measures quickly and efficiently.

[1143] The processing flow will be explained below.

[1144] Step 1:

[1145] The user launches the app on their smartphone or tablet and takes a picture of their pet. The app then checks for any abnormalities in a specific area of ​​the pet's body and takes a photo focusing on that area. For example, if the pet's ear is red and swollen, the app will capture the area around the ear.

[1146] Step 2:

[1147] The device displays the captured image, and the user confirms the image and taps the send button. This operation sends the image data to the server.

[1148] Step 3:

[1149] The device compresses the captured image data and sends it to the server using a secure protocol (e.g., HTTPS). During the data transmission process, the image is compressed and encrypted.

[1150] Step 4:

[1151] The server decompresses the received image data and performs preprocessing while preserving image quality. Preprocessing includes noise removal, contrast adjustment, and resolution standardization. This process improves the accuracy of analysis.

[1152] Step 5:

[1153] The server inputs the pre-processed images into a generative AI model, which uses pre-trained algorithms to detect anomalies, identifying abnormalities in the images (e.g., swelling or color changes) and providing a health diagnosis based on the results.

[1154] Step 6:

[1155] The server generates recommended actions based on the diagnosis results from the AI ​​model, such as "We recommend a veterinary visit as there is a high possibility of an infection" or "We recommend the use of specific care products."

[1156] Step 7:

[1157] The server sends the generated diagnostic results and recommended actions to the device, and the data is formatted in a way that allows the user to understand it intuitively and quickly.

[1158] Step 8:

[1159] The device displays the diagnosis results and recommended actions to the user, such as "Your pet's ears may have an ear infection. This is a high-risk condition, so please consult a veterinarian immediately."

[1160] Step 9:

[1161] Based on the diagnosis, the device will display links to purchase care products and book medical appointments. Users can click the links within the app to purchase the necessary care products online and book appointments with a nearby veterinarian.

[1162] Step 10:

[1163] When a user clicks on the link to buy a care product, they are taken to an online store page where they can purchase the care product, and when they click on the link to book a medical service, they are taken to a nearby veterinarian's appointment page where they can enter the necessary information to complete the appointment.

[1164] This series of processes allows the user to quickly understand the health condition of their pet and efficiently take necessary measures.

[1165] Example 1

[1166] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1167] In recent years, the importance of pet health management has increased, but it is not easy to continuously monitor pet health in busy daily lives. In addition, there is often a lack of means to detect abnormalities in pets early and provide appropriate care and examinations. With conventional methods, owners themselves need to have specialized knowledge to accurately diagnose their pet's health condition, and there is a risk that the condition may worsen before they can be examined by a specialist.

[1168] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1169] In this invention, the server includes a user interface means for taking images of the pet, a data transmission means for transmitting the captured images to the server, a means for receiving and preprocessing the transmitted image data, a generative AI model means for performing a health check using the preprocessed images as input, a means for generating recommended actions based on the diagnosis results from the generative AI model, a means for displaying the diagnosis results and the recommended actions to the user, and a means for providing links to purchase care products and to book medical services based on the diagnosis results, thereby enabling the user to quickly and accurately understand the health condition of their pet and to receive the necessary care and examination.

[1170] The "user interface means" is a means for providing an intuitive and easy-to-operate interface for the user to take pictures of their pet.

[1171] The "data transmission means" is a means for compressing and encrypting the captured image data and transmitting it to the server via a secure protocol.

[1172] The "means for performing preprocessing" refers to means for removing noise from received image data, standardizing the image data, extracting necessary parts, and the like.

[1173] A "generative AI model means" is a means including an algorithm trained using a training data set that performs a medical examination using preprocessed image data as input.

[1174] "Means for generating recommended actions" refers to means for recommending actions to be taken by the user based on the diagnostic results of the generative AI model.

[1175] The "means for displaying the diagnostic results to the user" refers to a means for displaying the analyzed diagnostic results and recommended actions to the user in a visually easy-to-understand format.

[1176] The "means for providing a link for purchasing a care product" is a means for providing a link that enables a user to purchase a necessary care product based on the diagnosis result.

[1177] The "means for providing a link to book a medical service" refers to a means for providing a link that allows a user to book a required medical service (e.g., a veterinary appointment) based on the diagnosis result.

[1178] The present invention is a system for capturing images of pets and diagnosing their health condition, which helps users monitor their pet's health and provide appropriate care and medical services.

[1179] Hardware and software used

[1180] This system uses mobile devices such as smartphones and tablets, as well as a cloud server. Specifically, it uses the following hardware and software:

[1181] 1. Smartphone or tablet (device)

[1182] Camera: Used to take pictures of your pet.

[1183] Mobile application: Provides the user interface.

[1184] 2. Cloud Server

[1185] Data transmission protocol (e.g. HTTPS): Ensures secure transmission of image data.

[1186] Image processing library (e.g. OpenCV): To preprocess images.

[1187] Generative AI models (e.g., TensorFlow, PyTorch): Perform health checkups.

[1188] System Operation Overview

[1189] A user launches a mobile application and takes a picture of their pet. For example, the user takes a picture of their pet's swollen ear. The captured image data is compressed on the device and sent to the server using HTTPS protocol. The server decompresses the received image data and performs preprocessing such as noise removal and image standardization. The preprocessed image is input into a generative AI model for a health check. The generative AI model uses a pre-trained algorithm to detect abnormalities in the image. For example, the generative AI model detects the pet's swollen ear and determines that it is an early sign of an infection.

[1190] Based on the generated diagnostic results, the server generates recommended actions for the user. For example, it generates a message such as, "There is a high suspicion of infection, so we recommend that you see a veterinarian." This diagnostic result and recommended actions are sent to the device and displayed to the user. Furthermore, based on the diagnostic results, links to purchase appropriate care products and to book medical services are also provided. The user can click on these links to purchase the necessary care products or make an appointment with a nearby veterinarian.

[1191] Specific examples

[1192] For example, if a user notices that their pet's ear is swollen, they launch the application and take a photo of the area. The device sends the image to a server, which performs preprocessing such as noise removal. The preprocessed image is analyzed by a generative AI model and determines that the ear swelling may indicate an infection. The server generates a diagnosis of "Suspected infection: High risk" and sends a message to the user's device recommending immediate veterinary treatment. The user confirms this and can easily make an appointment with a nearby veterinarian and purchase infection preventative medication within the app.

[1193] Prompt Sentence Examples

[1194] "Analyze images of your pet's swollen ears, diagnose whether there is a risk of infection, and suggest appropriate care."

[1195] The present invention provides innovative support for pet health management, enabling users to take necessary measures quickly and efficiently.

[1196] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1197] Step 1:

[1198] The user launches the app on their smartphone or tablet. They then use an intuitive interface to take a picture of their pet and capture a specific area (e.g., a swollen ear). The input is the pet's image, and the output is the captured image data.

[1199] Step 2:

[1200] The device converts the captured image data into a compressed format (e.g., JPEG or PNG). This compression reduces the amount of data. The input is the image data obtained in step 1, and the output is the compressed image data.

[1201] Step 3:

[1202] The terminal sends the compressed image data to the server using the HTTPS protocol. This is done by the data transmission means, and the data is secure because it is encrypted. The input is the compressed image data, and the output is the data received by the server.

[1203] Step 4:

[1204] The server decompresses the image data received and returns it to its original image format. For example, it decompresses a compressed JPEG image. The input is the encrypted image data, and the output is the decompressed image data.

[1205] Step 5:

[1206] The server denoises the image data. It uses an image processing library such as OpenCV to remove unnecessary noise from the image. The input is the decompressed image data, and the output is the image data after noise removal.

[1207] Step 6:

[1208] The server standardizes the images to make them uniform in size and resolution, preparing them for accurate analysis by the generative AI model. The input is image data after noise removal, and the output is standardized image data.

[1209] Step 7:

[1210] The server extracts important parts from the image. For example, it cuts out only the ears of a pet and uses them for analysis. The input is the standardized image data, and the output is the image data with the important parts extracted.

[1211] Step 8:

[1212] The server inputs the preprocessed image data into a generative AI model to perform a health check. The generative AI model is built using a deep learning algorithm (e.g., TensorFlow or PyTorch). The input is image data with important parts extracted, and the output is the diagnosis result.

[1213] Step 9:

[1214] The server recommends actions to the user based on the diagnosis results from the generative AI model. This recommended action is expressed in the form of, for example, "There is a high suspicion of infection, so we recommend that you see a veterinarian." The input is the diagnosis result, and the output is the recommended action.

[1215] Step 10:

[1216] The server sends the diagnosis results and recommended actions to the user's device. The diagnosis results and recommended actions are displayed to the user in a visually easy-to-understand format. The input is the recommended actions and diagnosis results, and the output is the data displayed on the user's device.

[1217] Step 11:

[1218] The device displays the received diagnosis results and recommended actions to the user. For example, a message such as "Your pet's ear is swollen. This swelling is likely an early sign of an infection" is displayed. The input is the data sent from the server, and the output is the information the user visually confirms.

[1219] Step 12:

[1220] Based on the diagnosis results, the terminal displays links to purchase care products or to book medical services. The user can click on these links to purchase the necessary care products or make an appointment with a nearby veterinarian. For example, links such as "Link to purchase infectious disease preventative medicine" or "You can make an appointment with a nearby veterinarian here" are displayed. The input is the diagnosis results, and the output is the links displayed.

[1221] Step 13:

[1222] The user clicks on a displayed link to perform an action, such as going to a screen to purchase preventative medicine or to a screen to complete a vet appointment. The input is the user's click action, and the output is the completed purchase of a care product or appointment for medical services.

[1223] (Application example 1)

[1224] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1225] Monitoring pet health is important in modern households, but traditional methods make it difficult to conduct regular health checks, and problems are often only noticed after they occur. Furthermore, there are limited means for monitoring pet health in real time, making it difficult to respond quickly when an abnormality occurs. Furthermore, making veterinary appointments and purchasing care products can be a hassle.

[1226] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1227] In this invention, the server includes a user interface means for taking images of the pet, a data transmission means for transmitting the captured images to the server, a means for receiving and preprocessing the transmitted image data, a generative AI model means for performing a health check using the preprocessed image as input, a means for generating recommended actions based on the diagnosis results of the generative AI model, a means for displaying the diagnosis results and the recommended actions to the user, a means for monitoring the movements and expressions of the pet in real time using a smart device, a means for transmitting the monitoring data to the server in real time and analyzing it, and a means for notifying the user when an abnormality is detected. This enables the user to efficiently monitor and manage the health condition of their pet and quickly take appropriate measures.

[1228] The "user interface means" is an interface for the user to perform operations and has a function for taking pictures of the pet.

[1229] The "data transmission means" is a means that provides a function for transmitting captured images to a server.

[1230] The "preprocessing means" is a means for receiving the transmitted image data and performing preprocessing such as noise removal and image standardization.

[1231] "Generative AI model means" refers to a generative AI model used to conduct a health check based on preprocessed image data.

[1232] The "means for generating recommended actions" is a means for generating actions to be recommended to users based on the diagnostic results obtained by the generative AI model.

[1233] The "diagnosis result display means" is a means for visually displaying to the user the diagnosis results and recommended actions obtained by the generative AI model.

[1234] "Means for monitoring pet movements and facial expressions in real time using smart devices" refers to means for constantly monitoring pet movements and facial expressions using smart glasses or other devices and collecting data.

[1235] "Means for transmitting monitoring data to a server in real time for analysis" refers to a means for immediately transmitting collected monitoring data to a server and analyzing it using a generative AI model.

[1236] "Means for notifying the user when an abnormality is detected" refers to a means for sending an alert to the user when an abnormality is detected based on the analysis results of the generative AI model.

[1237] The present invention provides a system for monitoring the health of pets in real time using smart devices and cloud computing, and providing users with diagnostic results and recommended actions.

[1238] Overall system overview

[1239] The system includes the following elements:

[1240] 1. User interface means: The user takes a picture of their pet using a smart device (smartphone, smart glasses, etc.).

[1241] 2. Data transmission means: A means for transmitting captured images to a cloud server.

[1242] 3. Preprocessing: The cloud server preprocesses the received image data. Preprocessing includes noise removal and image standardization.

[1243] 4. Generative AI model means: A means for inputting preprocessed image data into a generative AI model to perform a health check.

[1244] 5. Means for generating recommended actions: A means for generating recommended actions for users based on the diagnostic results of the generative AI model.

[1245] 6. Diagnostic result display means: A means for visually displaying diagnostic results and recommended actions on a smart device.

[1246] 7. Real-time monitoring means: A means of monitoring pet movements and facial expressions in real time using smart devices such as smart glasses.

[1247] 8. Abnormality notification means: A means of sending monitoring data to a server in real time and notifying the user if an abnormality is detected based on the analysis results.

[1248] Hardware and software used

[1249] Smart devices: smartphones, smart glasses (e.g., Google Glass, Vuzix Blade)

[1250] Cloud Server: Cloud computing platforms such as AWS and Google Cloud

[1251] Generative AI model: A generative AI model used to perform a health check

[1252] Data processing flow

[1253] 1. User interface: The user takes a picture of their pet using a smart device.

[1254] 2. Data transmission: The captured images are sent to the cloud server.

[1255] 3. Preprocessing: The cloud server receives the image data and performs noise removal and image standardization.

[1256] 4. Application of generative AI model: The preprocessed image data is input into the generative AI model to perform a health check.

[1257] 5. Generation of recommended actions: Based on the diagnostic results of the generative AI model, actions to be recommended to the user are generated.

[1258] 6. Displaying diagnostic results: The generated diagnostic results and recommended actions are displayed on the smart device.

[1259] 7. Real-time monitoring: Use smart devices to constantly monitor and collect data on your pet's movements and facial expressions.

[1260] 8. Abnormality notification: Monitoring data is sent to the cloud server in real time, and if an abnormality is detected, a notification is sent to the user.

[1261] Specific examples

[1262] A user notices that their pet's ear is swollen and takes a photo of the area using their smart device. The image is sent to a cloud server, where it undergoes preprocessing such as noise removal and standardization. The preprocessed image is analyzed by a generative AI model, which determines that the ear swelling is an early sign of infection. The server generates a message recommending immediate veterinary care along with a diagnosis of "Suspected Infection: High Risk" and sends it to the user's smart device. The user can then review this information and make an appointment with a nearby veterinarian and purchase preventative medication directly from their smart device.

[1263] Example prompt sentence:

[1264] "It monitors pet health in real time, notifies users if anything abnormal is detected, and assists with appropriate care or veterinary appointments."

[1265] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1266] Step 1:

[1267] A user takes an image of their pet using a smart device (smartphone or smart glasses), which generates image data for monitoring specific health conditions of the pet. The input is the image of the pet, and the output is the image data stored on the smart device.

[1268] Step 2:

[1269] The device sends the captured image data to the cloud server. The data sending means compresses the image data through a secure protocol and sends it to the cloud server in an appropriate format. The input is the image data stored in the smart device, and the output is the compressed image data sent to the cloud server.

[1270] Step 3:

[1271] The server decompresses the received image data and performs preprocessing. Preprocessing includes noise removal, contrast adjustment, and image standardization. The preprocessed data is ready for the AI ​​model to perform accurate analysis. The input is the compressed image data sent to the server, and the output is the image data after preprocessing.

[1272] Step 4:

[1273] The preprocessed image data is input into a generative AI model to perform a health diagnosis. The generative AI model uses a pre-trained algorithm to detect abnormalities in the image. For example, the AI ​​model may detect swelling in a pet's ear and determine that it is an early sign of infection. The input is the preprocessed image data, and the output is the diagnosis result.

[1274] Step 5:

[1275] The server recommends actions to the user based on the diagnosis results of the AI ​​model. The server analyzes the diagnosis results and generates a message such as, "There is a high suspicion of infection, so we recommend that you see a veterinarian." The input is the diagnosis result from the generative AI model, and the output is a message containing a recommended action.

[1276] Step 6:

[1277] The device displays the analyzed diagnostic results and recommended actions to the user. The diagnostic results are displayed in a visually easy-to-understand format so that the user can immediately understand them. The input is a message containing recommended actions, and the output is the diagnostic results and recommended actions displayed to the user.

[1278] Step 7:

[1279] A smart device is used to monitor the movements and expressions of pets in real time. The camera in the smart glasses continuously captures the pet's movements and sends the data to a cloud server. The input is the video stream captured by the smart device, and the output is the monitoring data sent to the cloud server.

[1280] Step 8:

[1281] The cloud server analyzes the monitoring data and notifies the user if an abnormality is detected. The AI ​​model performs behavior analysis and sends an alert to the user if an abnormality is detected. For example, a notification such as "There is something abnormal with your pet's movements. It is dragging its right hind leg" is displayed. The input is the monitoring data sent to the cloud server, and the output is an alert message notifying the user of the abnormality.

[1282] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1283] The present invention is a system for taking pictures of pets and conducting health checkups, and is combined with an emotion engine that recognizes the user's emotions. This system is designed to enable users to accurately understand their pet's health condition and take necessary measures promptly.

[1284] Overall system overview

[1285] The system includes a user interface means, a data transmission means, a preprocessing means, a generation AI model means, a recommended action generation means, a diagnosis result display means, a care product purchase link, a medical service reservation link, and an emotion engine that recognizes the user's emotions.

[1286] Specific explanation of program processing

[1287] 1. User Interface

[1288] The user launches the app on their smartphone or tablet and takes a picture of their pet. The user interface provides a simple and intuitive operation, such as capturing a red, swollen area on the pet's ear.

[1289] 2. Emotion recognition

[1290] To recognize a user's emotions, the device uses a camera and microphone to analyze the user's facial expressions and tone of voice. An emotion engine analyzes this data to identify the user's current emotional state (e.g., relief, worry, sadness, etc.).

[1291] 3. Data Transmission

[1292] The device sends the captured image and the user's emotional data to a server, where the data is sent in compressed and encrypted form using a secure protocol.

[1293] 4. Pretreatment

[1294] The server decompresses the received pet image data and performs preprocessing such as noise reduction and standardization. At the same time, the user's emotion data is also preprocessed, preparing it for analysis.

[1295] 5. Applying generative AI models

[1296] The server inputs preprocessed images of the pet into the generative AI model for a health check, which then identifies any abnormalities in the pet and generates a diagnosis.

[1297] 6. Generating recommended actions

[1298] The server then suggests appropriate actions to the user based on the AI ​​model's diagnosis and the user's emotional data. For example, specific instructions such as "There is a high possibility of an infection, so we recommend that you see a veterinarian" are generated. If the emotional state is "worried," a particularly detailed explanation is included.

[1299] 7. Displaying the diagnostic results

[1300] The device displays the diagnosis results and recommended actions to the user, and the display is customized according to the user's emotional state. For example, if the user is in an "anxious" state, a warm and reassuring message is displayed.

[1301] 8. Proposing and implementing actions

[1302] Based on the diagnosis, the device will display links to purchase care products and book medical appointments, taking into account the user's emotional state. Users can easily purchase the necessary care products and make appointments with a nearby veterinarian within the app.

[1303] Specific examples

[1304] A user notices that their pet's ear is swollen, launches the app, and takes a photo of the area. The device simultaneously reads the user's facial expression, and the emotion engine recognizes that the user is "worried." The device then sends the pet's image and the user's emotion data to the server, which preprocesses the image and analyzes it using a generative AI model. The diagnosis is "high probability of infection," and the server generates a message for the "worried" user saying, "There is a suspicion of infection, but prompt treatment is likely to improve. We recommend that you see a veterinarian immediately." The device displays this message, along with links to make an appointment at a nearby veterinarian and to purchase infection preventative medication. The user can make an appointment within the app and purchase the necessary care products, allowing for a prompt response.

[1305] This allows users to quickly understand the health condition of their pet and take appropriate measures while receiving emotional support.

[1306] The processing flow will be explained below.

[1307] Step 1:

[1308] The user launches the app on their smartphone or tablet and takes a picture of their pet, for example, a picture of their pet's swollen ear.

[1309] Step 2:

[1310] The device displays an image of the pet, and the user confirms the image and taps the send button, which sends the image data to the server.

[1311] Step 3:

[1312] To recognize the user's emotions, the device uses a camera and microphone to capture the user's facial expressions and record the tone of voice. The emotion engine analyzes the user's facial and vocal data to identify the user's current emotional state (e.g., relief, worry, sadness, etc.).

[1313] Step 4:

[1314] The device compresses the captured image data and the user's emotional data and sends them to the server using a secure protocol (e.g., HTTPS).

[1315] Step 5:

[1316] The server decompresses the received image data and performs preprocessing such as noise removal, contrast adjustment, and resolution standardization. At the same time, the user's emotional data is also preprocessed, and the emotional data is ready for analysis.

[1317] Step 6:

[1318] The server inputs the preprocessed images into a generative AI model for health checkups, which then identifies abnormalities in the images and generates a diagnosis.

[1319] Step 7:

[1320] The server generates recommended actions based on the diagnosis results from the AI ​​model and the user's emotional data. For example, specific instructions such as "There is a high possibility of an infection, so we recommend that you see a veterinarian" are generated, and if the user is "worried," a particularly polite explanation or a reassuring message is added.

[1321] Step 8:

[1322] The server generates diagnostic results and sends them to the device, and the data is customized to allow users to understand it intuitively and quickly.

[1323] Step 9:

[1324] The device displays the received diagnosis results and recommended actions to the user. The display is customized according to the user's emotional state. For example, if the user is in an "anxious" state, a reassuring message is displayed. The message reads, "Your pet may have an ear infection, but with prompt treatment it is likely to improve. Please see a veterinarian immediately."

[1325] Step 10:

[1326] Based on the diagnosis results, the device will display links to purchase care products and book medical services. Users can easily purchase the necessary care products and book veterinary appointments within the app.

[1327] Step 11:

[1328] When a user clicks on the link to buy a care product, they are taken to an online store page where they can purchase the care product, and when they click on the link to book a medical service, they are taken to a nearby veterinarian's appointment page where they can enter the necessary information to complete the appointment.

[1329] Example 2

[1330] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1331] The ability to quickly understand a pet's health condition and take appropriate measures is extremely important for pet owners. However, typical pet health checks are time-consuming and laborious, and it is difficult to respond quickly in emergencies. Responding to the pet owner's emotional state is also important, and anxious owners require particularly careful support. The present invention aims to solve the above problems by providing a system that takes images of pets, performs health checks, and recognizes the owner's emotions to provide appropriate responses.

[1332] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes an information transmission means for transmitting the acquired image to the transmission device, a device for receiving and preprocessing the transmitted image data, a generative AI model device for performing a health check using the preprocessed image as input, a device for generating recommended actions based on the diagnosis results by the generative AI model, a display device for displaying the diagnosis results and recommended actions to the user, and an emotion recognition device for recognizing the user's emotions and collecting data. This makes it possible to quickly grasp the health condition of a pet and provide appropriate measures taking the owner's emotions into consideration.

[1333] The "user interface means" refers to a user interface for acquiring images of a pet, and is a means by which a user takes an image of a pet using a smartphone or tablet.

[1334] The "information transmission means" is a means for transmitting the acquired image to the transmission device, and a means for compressing and encrypting the captured image data and transmitting it to the server.

[1335] The "preprocessing device" refers to a device that receives transmitted image data and performs preprocessing such as noise removal and standardization.

[1336] A "generative AI model device" is a device that performs health checkups using preprocessed images as input, and is a means for analyzing a pet's health condition using a generative AI model and generating diagnostic results.

[1337] A "recommended behavior generation device" refers to a device that suggests appropriate behavior to pet owners based on the diagnostic results of a generative AI model.

[1338] The "display device" is a device for displaying diagnostic results and recommended actions to the user, and is capable of customizing the display according to the user's emotional state.

[1339] An "emotion recognition device" is a device that recognizes a user's emotions and collects data, and is a means of analyzing the user's facial expressions and tone of voice using a camera and microphone.

[1340] The "purchase link for care products" is a means for providing a link for purchasing pet care products based on the diagnosis results.

[1341] The "medical service appointment link" is a means for providing a link for making an appointment for medical service based on the diagnosis result.

[1342] "Training Dataset" refers to the collection of data used to train a generative AI model using data from healthy and unhealthy pets.

[1343] The present invention provides a system for quickly understanding the health condition of a pet and recognizing the emotions of the pet owner to provide appropriate responses. The system includes a user interface, an information transmission unit, a preprocessing unit, a generative AI model device, a recommended behavior generation device, a display device, and an emotion recognition device.

[1344] First, the user launches the app on their smartphone or tablet and captures an image of their pet. The smartphone or tablet's camera is used as the user interface. This interface is intuitive and designed to allow users to easily capture images of specific parts of their pet (e.g., ears or eyes).

[1345] The device then uses its built-in camera and microphone to analyze the user's facial expressions and tone of voice in real time. This emotion recognizer is designed to analyze and collect data from the user's facial expressions and tone of voice, and an emotional state (e.g., relief, worry, sadness) is identified.

[1346] The captured image and emotion data are sent from the device to a server using a secure protocol such as TLS (Transport Layer Security), and the data is compressed and encrypted before transmission.

[1347] The server preprocesses the pet image data it receives. The preprocessing unit removes noise and standardizes the image to make it suitable for analysis. For example, it adjusts the brightness and contrast of the image and removes unnecessary noise. At the same time, the user's emotional data is also preprocessed.

[1348] The preprocessed images are input into a generative AI model device. This generative AI model uses machine learning techniques such as CNN (convolutional neural network) to analyze the pet's health and identify abnormalities. For example, a diagnosis such as "possible infection" is generated. This generative AI model has been pre-trained using a training dataset of healthy and unhealthy pets.

[1349] The generated diagnosis results are converted into appropriate action suggestions for the user by a recommended behavior generator. For example, specific instructions such as "There is a high possibility of an infection, so we recommend that you take your pet to the veterinarian" are generated. If the emotional state is recognized as "worried," instructions including particularly careful explanations and reassurances are provided.

[1350] Finally, the device displays the diagnosis and recommended actions to the user. The display offers customized messages based on the user's emotional state, such as a reassuring message like, "Your pet may have an infection, but with prompt treatment, improvement is expected. We recommend immediate veterinary care."

[1351] Additionally, based on the diagnosis results, links to purchase care products and book medical services are displayed, allowing users to easily make reservations or purchases within the app by clicking on these links.

[1352] Specific examples

[1353] A user notices that their pet's ear is swollen, launches the app, and takes a photo of the area. The device simultaneously reads the user's facial expression, and the emotion engine recognizes that the user is "worried." The device then sends the pet's image and the user's emotion data to the server, which preprocesses the image and analyzes it using a generative AI model. The diagnosis is "high probability of infection," and the server generates a message for the "worried" user saying, "There is a suspicion of infection, but prompt treatment is likely to improve. We recommend that you see a veterinarian immediately." The device displays this message, along with links to make an appointment at a nearby veterinarian and to purchase infection preventative medication. The user can make an appointment within the app and purchase the necessary care products, allowing for a prompt response.

[1354] Prompt Sentence Examples

[1355] Describe how you would generate and display diagnostic results and recommended actions when a user is concerned about their pet's red, swollen ears.

[1356] keyword

[1357] Generative AI model, prompt sentence

[1358] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1359] Step 1:

[1360] The user launches the app on their smartphone or tablet and captures an image of their pet. When the user clicks the "photo button" in the app, the camera starts up and takes an image of a specific part of the pet (e.g., ears or eyes). The input is the pet's image data, and the output is the captured image data.

[1361] Step 2:

[1362] The device uses its built-in camera and microphone to analyze the user's facial expressions and tone of voice in real time. The camera captures the user's facial expressions and applies an expression analysis algorithm. The microphone also captures audio and uses an audio analysis algorithm to determine emotions. The input is the user's real-time facial expressions and audio data, and the output is analyzed user emotional data.

[1363] Step 3:

[1364] The device sends the captured image data and the user's emotional data to the server. TLS (Transport Layer Security) is used for data transmission, compressing and encrypting the data for secure transmission. The input is the pet's image data and the user's emotional data, and the output is the compressed and encrypted data sent to the server.

[1365] Step 4:

[1366] The server decompresses the received image data and performs preprocessing such as noise removal and standardization. The preprocessing unit adjusts the brightness and contrast of the image data and removes unnecessary noise. At the same time, the user's emotional data is also preprocessed. The input is the decompressed image data and emotional data, and the output is the preprocessed image data and emotional data.

[1367] Step 5:

[1368] The server inputs preprocessed images of the pet into a generative AI model to perform a health check. The generative AI model uses a convolutional neural network (CNN) to detect abnormalities in the pet. The input is the preprocessed image data, and the output is the generated diagnosis result.

[1369] Step 6:

[1370] The server analyzes the diagnostic results of the generative AI model and generates recommended actions based on the user's emotional data. The recommended action generator integrates the diagnostic results and emotional data to suggest appropriate actions for the user. For example, it generates instructions such as "There is a possibility of an infection, so we recommend that you see a veterinarian." The input is the diagnostic results and emotional data, and the output is the generated recommended action message.

[1371] Step 7:

[1372] The terminal displays the diagnosis results and recommended actions to the user. The display device adjusts the message content according to the user's emotional state, for example, displaying a reassuring message to an anxious user. The input is the recommended action message, and the output is a customized message displayed to the user.

[1373] Step 8:

[1374] The terminal presents links to purchase care products and to book medical services based on the diagnosis results and recommended actions. Users can easily make reservations or purchases by clicking these links. The input is the recommended action message, and the output is the presented links to purchase care products and to book medical services.

[1375] (Application example 2)

[1376] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1377] Conventional food delivery applications lack the ability to recognize users' emotions and make appropriate meal suggestions, making it difficult to provide services tailored to individual users. Furthermore, they lack the ability to suggest menus based on individual users' health conditions and preferences, which means they are unable to contribute to user satisfaction or health maintenance. Inappropriate meal choices run the risk of worsening users' health.

[1378] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes emotion recognition means for recognizing the emotions of the pet and the user, user interface means for taking images of the user and the pet, and data transmission means for sending the taken images and emotion data to the server. This makes it possible to propose individually customized food menus and care products and make reservations for medical services based on the user's emotion data and health condition.

[1379] "Emotion recognition means" refers to a device or function for recognizing a user's emotions, and includes technology that uses a camera or microphone to analyze a user's facial expression or tone of voice.

[1380] The "user interface means" is an interface that allows the user to intuitively operate the device, and provides functions that allow the user to take images and input data.

[1381] The "data transmission means" is a communication means for transmitting the captured image and emotion data to the server, and has the function of transmitting data in a secure and encrypted format.

[1382] "Preprocessing means" includes technology that analyzes the image data and emotion data received by the server and performs necessary noise removal and standardization.

[1383] "Generative AI model means" refers to an artificial intelligence model that makes a diagnosis based on preprocessed data, and includes trained models for health checkups and food menu suggestions.

[1384] "Recommended action generation means" is a technology that has the function of suggesting appropriate actions to users based on the diagnostic results of the generative AI model and the user's emotional data.

[1385] The "display means" refers to a device or function for visually presenting the diagnostic results and recommended actions to the user, and provides information in a form that is easy for the user to understand.

[1386] The "means for providing a reservation link" is a function that provides a reservation link for medical services based on the diagnosis results, and is a technology that allows the user to easily take the next action.

[1387] "Learning" is the process by which a generative AI model uses a training dataset to gain knowledge and make effective diagnoses and recommendations.

[1388] The present invention relates to a food delivery system that recognizes a user's emotions and provides food menu suggestions and medical service reservations based on those emotions. The system is designed to quickly suggest a menu that is optimal for the user's health condition.

[1389] Overall system overview

[1390] The system consists of the following main components:

[1391] 1. User Interface Methods

[1392] The user launches the smartphone app and takes a photo of themselves or their family. The user interface provides simple and intuitive operation.

[1393] 2. Emotion recognition means

[1394] It uses a camera and microphone to analyze the user's facial expressions and tone of voice to recognize the user's emotional state. This technology is realized using common emotion recognition APIs (e.g., Microsoft Azure Face API).

[1395] 3. Data Transmission Method

[1396] The captured image and emotion data are sent to a server in a secure, encrypted format using secure communications such as the TLS protocol.

[1397] 4. Pretreatment Methods

[1398] The server analyzes the received images and emotion data, removes noise, and standardizes the data. This process uses libraries such as OpenCV.

[1399] 5. Generative AI Model Means

[1400] A generative AI model for diagnosis and food menu suggestions using preprocessed image and emotion data as input. This generative AI model is implemented using an advanced natural language processing model such as GPT-4.

[1401] 6. Recommended Action Generation Method

[1402] Based on the diagnostic results of the generative AI model and the user's emotional data, the system suggests the most suitable food menu and medical service reservation links for the user.

[1403] 7. Display means

[1404] Diagnostic results and recommended actions are visually presented to the user in a reassuring and customized manner based on the user's emotional state.

[1405] What the program does

[1406] Hardware and Software Configuration

[1407] Hardware: Smartphone or tablet device (iOS or Android), camera, microphone.

[1408] Software: Azure Face API, OpenCV, GPT-4, TLS protocol.

[1409] Data processing and calculation flow

[1410] 1. Data capture and transmission: The user takes a photo through a smartphone app, and emotion analysis is performed in conjunction with a facial recognition API. The analysis data and image are encrypted and sent to a server.

[1411] 2. Data preprocessing: Image data is denoised and emotion data is standardised on the server side, allowing the AI ​​model to process the data accurately.

[1412] 3. Applying the generative AI model: The preprocessed data is fed into the GPT-4 model to generate an optimal food menu based on the user's emotions and health status.

[1413] 4. Result presentation and recommended actions: The generated menu and suggestions are displayed to the user along with messages customized according to their emotional state, and links to book medical services and purchase care products are also provided.

[1414] Examples and prompts

[1415] Specific examples

[1416] When a user is tired, they launch a food delivery app and take a photo. The system recognizes the user's emotion, such as "tired," and sends the image along with the emotion data to the server. The server then performs preprocessing and uses a generative AI model to suggest "nutritious menus to relieve fatigue." As a result, the user receives "menus effective for fatigue recovery" and a link to order them on the app.

[1417] Prompt Sentence Examples

[1418] "User's emotional state is happy. Health status: No allergies, needs energy. Please suggest the best menu for this user."

[1419] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1420] Step 1:

[1421] User Interface Operations

[1422] Subject: User

[1423] The user launches the smartphone app and takes a photo of themselves or their family.

[1424] Input: A photo taken by the user

[1425] Output: Image data

[1426] How it works: Using the app's camera function, guidelines will appear on the screen to provide easy and intuitive operation.

[1427] Step 2:

[1428] emotion recognition

[1429] Subject: Terminal

[1430] The device uses a camera and microphone to analyze the user's facial expressions and tone of voice to recognize their emotional state.

[1431] Input: Captured image and audio data

[1432] Output: Emotion data

[1433] Specific operation: Uses an emotion recognition API (e.g., Microsoft Azure Face API). The API analyzes facial and voice data to obtain emotion data (e.g., "happy" or "worried").

[1434] Step 3:

[1435] Data transmission

[1436] Subject: Terminal

[1437] The terminal encrypts the captured image and emotion data and transmits them to the server.

[1438] Input: Image data and emotion data

[1439] Output: Encrypted data

[1440] What it does: Encrypts and securely transmits data using the TLS protocol.

[1441] Step 4:

[1442] Data Preprocessing

[1443] Subject: Server

[1444] The server analyzes the received data, removes noise from the images, and standardizes the data.

[1445] Input: Encrypted data

[1446] Output: Preprocessed image data and standardized emotion data

[1447] Specific operation: Image data is denoised using OpenCV and emotion data is converted into an analyzable format.

[1448] Step 5:

[1449] Applying generative AI models

[1450] Subject: Server

[1451] The server inputs the preprocessed data into a generative AI model (e.g., GPT-4) to generate an optimal food menu based on the user's emotions and health status.

[1452] Input: Preprocessed image data and emotion data

[1453] Output: Generated food menu suggestions

[1454] Specific behavior: Provide a prompt to the generative AI model, and the AI ​​will generate an appropriate menu. Example: "User's emotional state is happy. Health status: no allergies, needs energy replenishment. Please suggest the best menu for this user."

[1455] Step 6:

[1456] Recommended Action Generation

[1457] Subject: Server

[1458] Based on the menu suggestions and diagnosis results generated by the server-generated AI model, the server generates the optimal food menu for the user, links to purchase care products, and links to book medical services.

[1459] Input: Generated food menu suggestions, emotion data

[1460] Output: Recommended actions and links

[1461] Specific behavior: Generate recommended actions and create customized messages depending on the emotional state.

[1462] Step 7:

[1463] Displaying the results

[1464] Subject: Terminal

[1465] The device visually presents the diagnostic results and recommended actions to the user.

[1466] Input: Recommended actions and links

[1467] Output: Visual presentation to the user

[1468] What it does: Displays a generated food menu and reservation link along with a customized message based on the user's emotional state.

[1469] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1470] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1471] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1472] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1473] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1474] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1475] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1476] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1477] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1478] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1479] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1480] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1481] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1482] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1483] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1484] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1485] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1486] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1487] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1488] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1489] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1490] The following is further disclosed regarding the above embodiment.

[1491] (Claim 1)

[1492] a user interface means for capturing an image of the pet;

[1493] a data transmission means for transmitting the captured image to a server;

[1494] means for receiving and preprocessing the transmitted image data;

[1495] A generative AI model means for performing a health check using a preprocessed image as an input;

[1496] A means for generating recommended actions based on the diagnostic results of the generative AI model;

[1497] means for displaying diagnostic results and recommended actions to a user;

[1498] A system including:

[1499] (Claim 2)

[1500] 10. The system of claim 1, further comprising means for providing a link to purchase a care product and a link to book a medical service based on the diagnosis result.

[1501] (Claim 3)

[1502] 10. The system of claim 1, wherein the generative AI model is trained using a training dataset of healthy and unhealthy pets.

[1503] "Example 1"

[1504] (Claim 1)

[1505] a user interface means for capturing an image of the pet;

[1506] a data transmission means for transmitting the captured image to a server;

[1507] means for receiving and preprocessing the transmitted image data;

[1508] A generative AI model means for performing a health check using a preprocessed image as an input;

[1509] A means for generating recommended actions based on the diagnostic results of the generative AI model;

[1510] means for displaying diagnostic results and recommended actions to a user;

[1511] A means for providing links to purchase care products and book medical services based on the diagnosis results;

[1512] A system including:

[1513] (Claim 2)

[1514] 10. The system of claim 1, further comprising a user interface means including intuitive guidelines for a user to easily take an image.

[1515] (Claim 3)

[1516] The system of claim 1, wherein the generative AI model is trained using a training dataset of healthy and unhealthy pets, and suggests actions to the user based on the diagnosis results, and provides links to necessary care products and medical services based thereon.

[1517] "Application Example 1"

[1518] (Claim 1)

[1519] a user interface means for capturing an image of the pet;

[1520] a data transmission means for transmitting the captured image to a server;

[1521] means for receiving and preprocessing the transmitted image data;

[1522] A generative AI model means for performing a health check using a preprocessed image as an input;

[1523] A means for generating recommended actions based on the diagnostic results of the generative AI model;

[1524] means for displaying diagnostic results and recommended actions to a user;

[1525] A means to monitor pet movements and facial expressions in real time using a smart device,

[1526] A means for transmitting the monitoring data to a server in real time for analysis;

[1527] means for notifying a user when an anomaly is detected;

[1528] A system including:

[1529] (Claim 2)

[1530] 10. The system of claim 1, further comprising means for providing a link to purchase a care product and a link to book a medical service based on the diagnosis result and the recommended action.

[1531] (Claim 3)

[1532] 10. The system of claim 1, wherein the generative AI model is trained using a training dataset of healthy and unhealthy animals.

[1533] "Example 2: Combining Emotion Engines"

[1534] (Claim 1)

[1535] user interface means for acquiring an image of the pet;

[1536] an information transmitting means for transmitting the acquired image to a transmitting device;

[1537] a device for receiving and preprocessing the transmitted image data;

[1538] A generative AI model device that performs health checkups using preprocessed images as input;

[1539] A device that generates recommended actions based on the diagnosis results of the generative AI model;

[1540] a display device that displays the diagnostic results and recommended actions to the user;

[1541] an emotion recognition device that recognizes a user's emotion and collects data;

[1542] A system including:

[1543] (Claim 2)

[1544] The system of claim 1, further comprising a device that provides links to purchase care products and to book appointments for medical services based on the diagnosis results and recommended actions.

[1545] (Claim 3)

[1546] 10. The system of claim 1, wherein the generative AI model is trained using a training dataset of healthy and unhealthy pets.

[1547] "Application example 2 when combining emotion engines"

[1548] (Claim 1)

[1549] An emotion recognition means for recognizing emotions of a pet or a user;

[1550] a user interface means for capturing an image of the user and the pet;

[1551] a data transmission means for transmitting the captured image and emotion data to a server;

[1552] means for receiving and preprocessing the transmitted image data;

[1553] A generative AI model means for making a diagnosis using preprocessed images and emotion data as input;

[1554] A means for generating recommended actions based on the diagnosis results of the generative AI model and the user's emotional data;

[1555] means for displaying diagnostic results and recommended actions to a user;

[1556] A system including:

[1557] (Claim 2)

[1558] The system according to claim 1, further comprising means for providing a food menu, a link for purchasing care products, and a link for booking medical services based on the diagnosis result and the user's emotional data.

[1559] (Claim 3)

[1560] 10. The system of claim 1, wherein the generative AI model is trained using a training dataset of healthy and unhealthy pets. [Explanation of symbols]

[1561] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a user interface means for capturing an image of the pet; a data transmission means for transmitting the captured image to a server; means for receiving and preprocessing the transmitted image data; A generative AI model means for performing a health check using a preprocessed image as an input; A means for generating recommended actions based on the diagnostic results of the generative AI model; means for displaying diagnostic results and recommended actions to a user; A system including:

2. The system of claim 1 , further comprising means for providing links for purchasing care products and for booking medical services based on the diagnosis results.

3. 10. The system of claim 1, wherein the generative AI model is trained using a training dataset of healthy and unhealthy pets.

Images